Class notes

Transcript of all lectures Methods and Techniques for Evaluation Research

8 purchases

Course
Methods and Techniques for Evaluation Research (PAMAOW01)

Institution
Rijksuniversiteit Groningen (RuG)

Book
Evaluation

This document consists of a transcription of all the lectures of this subject. Such documents always really help me with exams. In the lectures, the book chapters that should be studied for the exam are clearly described, plus some additional things like examples and videos. I put screenshots of th...

[Show more]

Preview 4 out of 101 pages

View example

Uploaded on June 9, 2021
Number of pages 101
Written in 2020/2021
Type Class notes
Professor(s) Anneke timmermans, mayra mascareño lara
Contains All classes

methods
techniques
evaluation research
evaluation
experiment
quasi experiment
research design
social programs
program evaluation
rossi
lipsey
henry
shadish
cook
campbell
causal inference

Book Title:Evaluation

Author(s):Mark W. Lipsey, Dr. Peter H. Rossi

Edition:Unknown
ISBN:9781506386607
Edition:8

Institution
Rijksuniversiteit Groningen (RuG)
Education
Educational Sciences / Onderwijswetenschappen
Course
Methods and Techniques for Evaluation Research (PAMAOW01)

corina1106

Member since 4 year 31 documents sold

$5.91

Add to cart

Add to wishlist

100% satisfaction guarantee
Immediately available after payment
Both online and in PDF
No strings attached

Literature
• Rossi, P. H., Lipsey, M. W., & Henry, G. T. (2004). Evaluation. A systematic approach (8th ed.).
Thousand Oaks, CA: Sage. Chapter 1 - 7, 11, and 12.
• Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental
designs for generalized casual inference. Boston, MA: Houghton Mifflin. Chapter 1 - 6, and 8.

Lecture 1
Rossi, Lipsey and Freeman
Chapters 1, 2 and 3

Evaluation research starts from social problems and the social programs that are addressing or trying to
solve those problems. If we were in a live lecture room I would ask you to think first what problems in
education you can think of that need to be solved and if you can think of some social programs that are
actually trying to tackle an educational program.

A social problem is an issue that affects the (living/learning/health/etc.) conditions of many people. It is a
problem because it affects so many. That’s why it’s called social problem. We can think of many; if you
just brainstorm you will probably think of many of them. Here’s a very random sort of sample of social
problems you can think of. For example:
1. Absenteeism rates of students in higher education.

2. Bullying. This news article is talking about online bullying in particular.

3. Poor reading skills of primary school children in this article in particular, but I think it is a more
generalized problem. It doesn’t allow them to get into even further content. It’s not only about
reading, but it enables reaching other sorts of content.

, 4. This comes from a Dutch newspaper and talks about the costs that are consequence of the low
literacy levels; the cost to society.

5. Another one from a Dutch source: the stress that students experience. In this case students on
school level.

6. And of course, more timely, stress that students are suffering now – because of the COVID-19
situation and because of the change in conditions in which students have to learn – is a very
generalized problem of this pandemic.

A social program is ‘’an organized, planned, and usually ongoing effort designed to ameliorate a social
problem or to improve social,, (living/learning/health/etc.) ,,conditions.’’ (Rossi, Lipsey & Freeman, 2004,
p. 29). So again, if you can think of problems, you can also think of programs that we might know. Here
are a few examples:
1. Again, this is a Dutch problem, but it is an implementation of a program that originated in the
US, which is the Succes for All program. It is actually developed here in the Netherlands by our
institute for educational research, which is a school reform actually that tries to improve
education with focus on reading for students with problematic backgrounds; especially for low
SES students.

, 2. Then, you also have curriculum changes that, for example, try to improve the situation for some
particular students. This is our relatively recent policy of inclusive education in the Netherlands,
in which students with special learning needs are included now in regular classrooms. That is also
a social program that aims to integrate and tackle the problem of lack of integration of students
with special needs.

3. Another very famous program, and it is one of the most studied social programs in the world, is
the Head Start program, which is also a reform implemented in the US that aims to provide
young children a good quality early childhood education.

4. There are international programs, for example this one laptop per child program that has been
implemented in third world economies; low income economies I should say. That basically
provides laptops for children to include it in their educational programs.

5. Another social program that’s very well known here in The Netherlands (and at least in Europe),
is the KiVa project/program. It is an anti-bullying program that originated in Finland.

So, this is a very generous set of examples of social programs in education that attempt to solve specific
problems that have been detected in education and affect many students/children.

So, the point of this course, why we discuss social problems and programs, is that those programs need
to be evaluated at a certain point. That is the perspective we take in this point. That’s important because
we’ve been teaching this course for years and we realized that for some students it’s difficult to take the
evaluator perspective/role into this program. Many times, they go into the role of program implementer.
The whole point of this course is that we want you to get tools in order to eventually, in the future, be
able to perform this role of evaluator of a social program. So, if you think ‘is a program doing well?’, for
example ‘is the anti-bullying program doing well?’, that would be an evaluation research in very general
terms. And now again, maybe you take another second to reflect ‘what aspects of the program could be
evaluated?’; could undergo an evaluation to check whether it is doing okay of not.

, Program evaluation
Program evaluation entails the use of social research methods, in
order to systematically investigate the effectiveness of social
intervention programs, in ways that are adapted to their political
and organizational environments, and are designed to inform social
action to improve social conditions that the program started from
basically.

Let’s try to defect this definition a little bit, because there are
several aspects that are important here.
• First is the use of social research methods. In program evaluation you
need to collect, analyze evidence, and you need to interpret that
evidence. That has to be done in a way that is credible, that is valid and
reliable, and that is defensible (that you can actually backup your claims
about whether a program is working or not. For that you need the best
possible methods in order to make those claims. But also of course, that
has to be feasible and adapted to circumstances.
• Because, unlike academic research, a program evaluation has a
different aim, which is actually to finally contribute to this social good,
to be able to help in the solving of the general social problems that
have been detected. Because of that, there are several domains of
effectiveness and we’re going to discuss that in more detail in a
minute.
• Another aspect is that these evaluations are embedded in a political
and organizational environment. They are not academic in the sense that
they are not done in an academic setting and for academic purposes of
knowledge dissemination and enlarging the body of knowledge on
something. That is not the ultimate goal; it is to improve a social condition
and that social condition/the program is also embedded in a political and
organizational situation/context. So, it is key to know who is requesting
these evaluations. Who are the sponsors? Are the sponsors the ones who
put down money for this evaluation? So, it is important to know the nature
of the question. And that nature of the question is always embedded in these political and
organizational environments.
• And as I said, this information should go back to those who are able to
make decisions, to those who are finally responsible and have the power
to e.g. continue with the program or stop it. So, there are decisions
associated with program evaluations. Sometimes go/no go decisions,
sometimes scaling up decisions (e.g. if a program is working well in
Groningen, we have to scale it up to the whole Netherlands), or
increasing budget, or cutting budget. So, decisions are associated very
concrete decisions that can be associated with the program evaluation.
Therefore, the information you are providing as an evaluator should be useful and should be
used, sometimes in the most direct way, but also indirectly, but it is information that ultimately

The benefits of buying summaries with Stuvia:

Guaranteed quality through customer reviews

Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.

Quick and easy check-out

You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.

Focus on what matters

Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!

Frequently asked questions

What do I get when I buy this document?

You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.

Satisfaction guarantee: how does it work?

Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.

Who am I buying these notes from?

Stuvia is a marketplace, so you are not buying this document from us, but from seller corina1106. Stuvia facilitates payment to the seller.

Will I be stuck with a subscription?

No, you only buy these notes for $5.91. You're not tied to anything after your purchase.

Can Stuvia be trusted?

4.6 stars on Google & Trustpilot (+1000 reviews)

69252 documents were sold in the last 30 days

Founded in 2010, the go-to place to buy study notes for 15 years now

Start selling

Class notes

Transcript of all lectures Methods and Techniques for Evaluation Research

Document information

Subjects

Connected book

Written for

Seller

Reviews received

Content preview