This document comprises very detailed lecture notes with thorough explanations. It has shown to be a great preparation resource for the LIO exam (Master in Educational Sciences).
Boonstra (2004)
Guba & Lincoln (1994)
Nielsen & Randall (2013)
Visse et al (2012)
In this clip focus on 3 questions
1. Why do we evaluate organizational change?
2. What does one focus on when evaluating change?
3. How does one evaluate change?
Why evaluate?
Nielson & Randell focus in their paper on process-evaluation which can be used to:
Provide feedback for improving interventions
Minimize pitfalls when replicating interventions in other settings
Interpret outcomes of interventions
Evaluation also identifies whether the goal of the organization has focused on in the intervention –
was the right goal.
Example: if university like to increase student number – student number can easily be
evaluated by counting them. But increasing students numbers might in itself have an
undesirable effect as well. – teachers can experience higher workload, which effects quality
of teaching / learning. So is it than sensible to evaluate student number? Other process
indicators are job satisfaction or engagement, or stakeholders within the organizational
change.
1
,Nielson and Randall
Process evaluation is after the circumstances under which an intervention will work and the
processes that facilitates that changes.
Under which circumstances will an intervention work? Which where the processes that facilitated the
change?
Process evaluation vs. effect evaluation
o Effect evaluation is used to measure immediate or long-term effects of a change.
Effect evaluation focusses on the expectations of the change. Did the organization
aim to change the knowledge skills or behaviour? Behaviour is often the hardest
outcome to measure and it can require a lot long time for effects to become
containable – and if they do- change to desired direction can also be caused by
factors other than the intervention. Its difficult to conclude how or why an
intervention worked from effect evaluation.
Answering the questions of the circumstances and the processes leads to an
understanding of the change from which other organizations can learn.
Model of process evaluation – is based on a literature review on health and well-being interventions
at the organizational level. But the factors and aspects apply to organizational change as well, in a
more general sense. This model is straight forward and the paper is extensively describing all factors.
READ It.
Nielson and Randall distinguished three factors that effect implementation and evaluation of
interventions. And at the core of this model is the intervention (light green box). What matters is
who initiated the intervention, whether the intervention activities target the problems of the
workplace and who are the drivers of the change (that aspect focuses on the roles and behaviours of
key stakeholders.)
Than Nielson describes Context which can dilute or strengthen intervention effects and processes (as
shown in the big light blue box). Still it remains unclear what the authors exactly mean with context.
Mayke takes it as the concrete organization where the intervention took place. Because the omnibus
context includes organizational culture and structure, and discrete context refers to particular events
that happened during implementation.
To conclude the mental models of people play a role in the change outcomes. This includes the
readiness for change and the perceptions of intervention activities. For the exam: you should now all
the factors by heart
2
,Paradigms - Guba & Lincoln logic of science
Meant to describe how science develops (sociology, philosophy of science, e.g., Kuhn) (paradigms
can help to understand the logic of evaluation)
Fundamental discourse questions if they exist
What is to be studied, what are relevant questions, how topics are connected, how to
interpret the results obtained and how does one conduct proper research?
For the exam: keep in mind to only study the parts of the chapter addressed in this lecture.
The worldview guiding the methodological epistemological and ontological choices made by
researchers (Guba & Lincoln)
Epistemology: what is knowledge?
Ontology: what is reality?
Methodology: How does one produce valid knowledge?
These three dimensions are related and aligned with another in a paradigm.
Ontology – example with picture squirrel
What is this?
Why is it that?
Is it real? Why (not)
If no humans were on Earth, would this still be what it is?
Epistemology – example with picture squirrel
How doe you know this is a squirrel?
If you had never seen one before, does it exist?
If someone told you this isn’t a squirrel, would you believe that person? Why (not)?
If it climbs up to a tree and there is no one to see it. Can it still be observed?
Methodology
Type of study and how it is to be done. E.g. descriptive or explanatory
Mythology > methods
o Methodology determines the methods which are the actual tool we use to collect
and analyse data. What does tools look like and how they are used depends on the
research paradigm and the theoretical/ conceptual framework
Why do paradigm matter?
Paradigms shift: radical transformation of science
Facts can only be interpreted through a particular ‘lens’
Evaluation requires to explicate the zoom lens used = description of the combination of ideas
(concepts) which guide your thinking (= theoretical framework)
Paradigms are accepted based on faith, choice needs to be substantiated. What does a
particular issue ask for?
Can coexist; your paradigm ‘of choice’ determines the options at your disposal, the
possibilities available
See Visse for how paradigms can get in the way of an evaluation (table 6.2)
3
, Positivism, the ‘received view’ (that’s how Guba & Lincoln call it, as most scientists are trained in this
paradigm and it is also the most widely accepted perspective on what counts as research)
Also called conventional paradigm
‘imparted’ from the natural sciences
Precise, measurable variables that can be quantified
The role of science is to discover new knowledge about how the real world works
o That means that the real world exists separately and apart from humans – so the
best way to study it is through an unbiased and objective researcher who is not to
involved in the research (object)
You might see here that this claim involves ontology and epistemology
Usually hypothesis-drive, allows one to make predictions
The real world is separate and apart from humans
The best way to discover new knowledge is through unbiased, objective research in which
the researcher aims to limit any influence
Guba and Lincoln explain an alternative paradigm
Constructivism
There are multiple realities. These realities are locally (and often socially) constructed
(ontology)
o Reality is grounded in a particular context
The researcher is inseparable from the research, no such thing as objectivity
Interprets instead of tests (research in this paradigm is often descriptive research – aiming to
understand a phenomenon)
o The researcher tries to sensibly interpret different meanings – looking for consensus
the researcher will find differences in perspectives which can be productive and
counterproductive depending on what the researcher tries to find out.
o As constructivists believe that researchers are inseparable from the research – the
researcher should be aware of their influence when conducting research. Being
aware differences from actively minimising the role of the researcher in the positive
paradigm
Insights from constructivists research are initially local
o It remains to be seen whether that they are generalizable to other contexts
Distinguishing cause and effect is impossible because they are always shaping each other
o True evaluation and implementation interventions change for example, this animal is
more than a squirrel – it has different meanings
IMPORTANT: Reality is constructed, no such thing as objective research, or a
world that exist apart from the researcher
The gist of it al
The ‘essential’ difference between positivism and constructivism resides in the belief about the
nature of the relationship between the researcher and the phenomenon of interest (for
constructivists, phenomenon is considered as a subject instead of an object)
This has consequences for the way in which knowledge is produced and the knowledge itself
Requires the use of methods in different ways
4
Voordelen van het kopen van samenvattingen bij Stuvia op een rij:
Verzekerd van kwaliteit door reviews
Stuvia-klanten hebben meer dan 700.000 samenvattingen beoordeeld. Zo weet je zeker dat je de beste documenten koopt!
Snel en makkelijk kopen
Je betaalt supersnel en eenmalig met iDeal, creditcard of Stuvia-tegoed voor de samenvatting. Zonder lidmaatschap.
Focus op de essentie
Samenvattingen worden geschreven voor en door anderen. Daarom zijn de samenvattingen altijd betrouwbaar en actueel. Zo kom je snel tot de kern!
Veelgestelde vragen
Wat krijg ik als ik dit document koop?
Je krijgt een PDF, die direct beschikbaar is na je aankoop. Het gekochte document is altijd, overal en oneindig toegankelijk via je profiel.
Tevredenheidsgarantie: hoe werkt dat?
Onze tevredenheidsgarantie zorgt ervoor dat je altijd een studiedocument vindt dat goed bij je past. Je vult een formulier in en onze klantenservice regelt de rest.
Van wie koop ik deze samenvatting?
Stuvia is een marktplaats, je koop dit document dus niet van ons, maar van verkoper maaiketaheij. Stuvia faciliteert de betaling aan de verkoper.
Zit ik meteen vast aan een abonnement?
Nee, je koopt alleen deze samenvatting voor €9,99. Je zit daarna nergens aan vast.
