Class notes
Samenvatting colleges Methods and Techniques
- Course
- Institution
Summary of all collis for the Methods and techniques course
[Show more]
Seller
Followfloorvanderheide
Reviews received
Content preview
Overview van collegesMethods and techniques for evaluation
research
Rijksuniversiteit Groningen
1
, College 1
The first topic we’re going to discuss is a general introduction. Why are we talking
about evaluation research and why is it important/what is special about it? What are
the steps which need to be followed?
Evaluation research basically starts with social problems.
What problem in education needs to be solved? What social programs are trying to
tackle educational problems?
A social problem is an issue that affects the (living/learning/health/etc)
conditions of many people. It is a problem because it affects so many people,
that’s why it’s a social problem.
Examples: ‘High chronic absenteeism rates among special needs students raise
alarms’, ‘A third of young people polled by UN, report being a victim of online
bullying’, ‘Poor reading skills of primary students’, ‘Laaggeletterdheid kost
samenleving een miljard’, ‘Studiestress onder studenten en scholieren’, ‘How
teachers can buffer student stress from COVID-19: ordinary magic’.
A social program is “an organized, planned, and usually ongoing effort
designed to ameliorate a social problem or improve social conditions”.
Example: Success for All is a program for a social problem, which tries to improve
education with focus on reading for students with problematic backgrounds or
especially low SES students.
Other examples: Passend Onderwijs, which is trying to improve the situations for
some students, for inclusive education. Students with special learning needs are
being placed in regular classes; it tackles the problem of separation, it improves the
integration.
Other example: Head Start program; in the United States, it aims to provide young
children with a good quality of childhood education.
One laptop per child program has been implemented in third world economies, low
income economies, that basically provides laptops for children in their educational
programs.
KiVa program: that is an anti-bullying program that originated in Finland.
The point of this course, why we discuss social problems and social programs, is
that those programs need to be evaluated at some point. It is difficult to take the
evaluator role. The whole point of this course is that we want you to get tools to
eventually in the future be able to perform this role of evaluator of social programs.
If you think a program is doing well, that would be an evaluation question in very
general terms. What aspects of the program could be evaluated? To check if it is
doing oké or not?
2
,Program evaluation is the use of social research methods to systematically
investigate the effectiveness of social intervention programs in ways that are
adapted to their political and organizational environments and are designed to
inform social action and to improve the social conditions (the program started
from/was intended to tackle from the beginning).
Use of social research methods:
In a program evaluation you need to collect, analyse evidence, you need to interpret
the evidence. That has to be done in a way that is credible and reliable, that is valid,
and that is defensible (you can back up your claims). For that you need the best
possible methods in order to make those claims. But also of course that has to be
feasible and adapted to circumstances. Because unlike academic research a
program evaluation has a different aim, which is finally to contribute to a social good,
to help to solve social problems. Because of that there are several domains of
effectiveness.
Another aspect is that evaluations are embedded in a political and organizational
environment. They are not academic in a sense that they are not done in an
academic setting. The goal is to improve a social condition, not to get knowledge.
That program is also embedded: political and organizational setting. It is key to know
who is asking these questions, who are the sponsors and/or stakeholders (who give
the money for the evaluation). The nature of the question is important and the nature
of the question is embedded in a political and organizational environment.
This information should go back to those who are empowered to make
decisions/continue or stop with the program. There are decisions like go/no-go,
continuation, scale-up, budget… Decisions are associated with the program
evaluation. Therefore the information that you are providing should be useful and
used, directly or indirectly, but it is information that should help decision makers to
make decisions. That is the big difference with academic research, that this should
directly or indirectly inform social action.
What can be evaluated or assessed?
From the perspective of an evaluator: there is a program going on, like an anti-
bullying program. What can be assessed? There are several steps/different areas of
evaluation that build up to each other.
1: before a program exists is to detect the need for a program.
2: program theory and design. How it should work. That can also be assessed.
3: program implementation.
4: program impact. Is this program actually reducing the social problem? Only then
we get to that question. There are several steps that need to be followed before we
get to that question.
5: program cost-efficiency. Is this program being cost efficient in terms of
resources/costs and outcomes?
3
, We always put these elements in some sort of pyramid, because they build up on
each other. In order to go to the next layer of evaluation, the previous one should be
cleared. We hope you see why in the course of the coming lectures/discussions.
1. Need for the program:
The problem is central. What is going on? What is the nature of the problem? Whom
does it affect? How does the problem affect them? We focus first on the problem.
Let’s think of an educational situation related to COVID-19; so the stress-levels of
students and teachers. We need to know first before there is any program; there
needs to be clarity on what is going on. What is the nature of this problem? We don’t
have a lot of information about it yet. Is everyone affected or some families/children?
How are these children being affected? What is the way this problem is actually
resulting in poor educational conditions for them?
2. Program theory and design:
There is an outline of a program. Sometimes there is a program already running but
you have to think about a design of a program. Look at the program in itself; what
are the steps the program needs to follow in order to solve the problem? Is the
program as intended likely to ameliorate the social problem? Are the program goals
well defined? Is there a logical theory behind it? Are all program functions properly
described? Is there enough argumentation? Do people know what they should be
doing in this program?
3. Program implementation:
There is a program running. There is a good design to compare the implementation
with. Is it actually being implemented the way it was intended in the design (step 2)?
Is the intervention reaching its target population? Is the intervention implemented in
the way it was intended? Are the intended services being provided? Satisfaction of
clients/users of a program. Only after you know the program is being implemented
the way it was intended, only then it is logical to think of evaluation if a program is
actually reaching the goals.
4. Program impact:
Is the intervention effective in reaching its goals? Does the intervention have any
side effects?
5. Program cost-efficiency:
Only when you know a program is effective in reaching goals, you can think of cost-
efficiency. Is the cost of the intervention reasonable compared to its effectiveness?
Are comparable interventions as costly or cheaper? Are we spending more money
than effectiveness that comes out of this program?
4
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller floorvanderheide. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $8.96. You're not tied to anything after your purchase.
Can Stuvia be trusted?
4.6 stars on Google & Trustpilot (+1000 reviews)
82191 documents were sold in the last 30 days
Founded in 2010, the go-to place to buy study notes for 14 years now