This document comprises very detailed lecture notes with thorough explanations. It has shown to be a great preparation resource for the LIO exam (Master in Educational Sciences).
Boonstra (2004)
Guba & Lincoln (1994)
Nielsen & Randall (2013)
Visse et al (2012)
In this clip focus on 3 questions
1. Why do we evaluate organizational change?
2. What does one focus on when evaluating change?
3. How does one evaluate change?
Why evaluate?
Nielson & Randell focus in their paper on process-evaluation which can be used to:
Provide feedback for improving interventions
Minimize pitfalls when replicating interventions in other settings
Interpret outcomes of interventions
Evaluation also identifies whether the goal of the organization has focused on in the intervention –
was the right goal.
Example: if university like to increase student number – student number can easily be
evaluated by counting them. But increasing students numbers might in itself have an
undesirable effect as well. – teachers can experience higher workload, which effects quality
of teaching / learning. So is it than sensible to evaluate student number? Other process
indicators are job satisfaction or engagement, or stakeholders within the organizational
change.
1
,Nielson and Randall
Process evaluation is after the circumstances under which an intervention will work and the
processes that facilitates that changes.
Under which circumstances will an intervention work? Which where the processes that facilitated the
change?
Process evaluation vs. effect evaluation
o Effect evaluation is used to measure immediate or long-term effects of a change.
Effect evaluation focusses on the expectations of the change. Did the organization
aim to change the knowledge skills or behaviour? Behaviour is often the hardest
outcome to measure and it can require a lot long time for effects to become
containable – and if they do- change to desired direction can also be caused by
factors other than the intervention. Its difficult to conclude how or why an
intervention worked from effect evaluation.
Answering the questions of the circumstances and the processes leads to an
understanding of the change from which other organizations can learn.
Model of process evaluation – is based on a literature review on health and well-being interventions
at the organizational level. But the factors and aspects apply to organizational change as well, in a
more general sense. This model is straight forward and the paper is extensively describing all factors.
READ It.
Nielson and Randall distinguished three factors that effect implementation and evaluation of
interventions. And at the core of this model is the intervention (light green box). What matters is
who initiated the intervention, whether the intervention activities target the problems of the
workplace and who are the drivers of the change (that aspect focuses on the roles and behaviours of
key stakeholders.)
Than Nielson describes Context which can dilute or strengthen intervention effects and processes (as
shown in the big light blue box). Still it remains unclear what the authors exactly mean with context.
Mayke takes it as the concrete organization where the intervention took place. Because the omnibus
context includes organizational culture and structure, and discrete context refers to particular events
that happened during implementation.
To conclude the mental models of people play a role in the change outcomes. This includes the
readiness for change and the perceptions of intervention activities. For the exam: you should now all
the factors by heart
2
,Paradigms - Guba & Lincoln logic of science
Meant to describe how science develops (sociology, philosophy of science, e.g., Kuhn) (paradigms
can help to understand the logic of evaluation)
Fundamental discourse questions if they exist
What is to be studied, what are relevant questions, how topics are connected, how to
interpret the results obtained and how does one conduct proper research?
For the exam: keep in mind to only study the parts of the chapter addressed in this lecture.
The worldview guiding the methodological epistemological and ontological choices made by
researchers (Guba & Lincoln)
Epistemology: what is knowledge?
Ontology: what is reality?
Methodology: How does one produce valid knowledge?
These three dimensions are related and aligned with another in a paradigm.
Ontology – example with picture squirrel
What is this?
Why is it that?
Is it real? Why (not)
If no humans were on Earth, would this still be what it is?
Epistemology – example with picture squirrel
How doe you know this is a squirrel?
If you had never seen one before, does it exist?
If someone told you this isn’t a squirrel, would you believe that person? Why (not)?
If it climbs up to a tree and there is no one to see it. Can it still be observed?
Methodology
Type of study and how it is to be done. E.g. descriptive or explanatory
Mythology > methods
o Methodology determines the methods which are the actual tool we use to collect
and analyse data. What does tools look like and how they are used depends on the
research paradigm and the theoretical/ conceptual framework
Why do paradigm matter?
Paradigms shift: radical transformation of science
Facts can only be interpreted through a particular ‘lens’
Evaluation requires to explicate the zoom lens used = description of the combination of ideas
(concepts) which guide your thinking (= theoretical framework)
Paradigms are accepted based on faith, choice needs to be substantiated. What does a
particular issue ask for?
Can coexist; your paradigm ‘of choice’ determines the options at your disposal, the
possibilities available
See Visse for how paradigms can get in the way of an evaluation (table 6.2)
3
, Positivism, the ‘received view’ (that’s how Guba & Lincoln call it, as most scientists are trained in this
paradigm and it is also the most widely accepted perspective on what counts as research)
Also called conventional paradigm
‘imparted’ from the natural sciences
Precise, measurable variables that can be quantified
The role of science is to discover new knowledge about how the real world works
o That means that the real world exists separately and apart from humans – so the
best way to study it is through an unbiased and objective researcher who is not to
involved in the research (object)
You might see here that this claim involves ontology and epistemology
Usually hypothesis-drive, allows one to make predictions
The real world is separate and apart from humans
The best way to discover new knowledge is through unbiased, objective research in which
the researcher aims to limit any influence
Guba and Lincoln explain an alternative paradigm
Constructivism
There are multiple realities. These realities are locally (and often socially) constructed
(ontology)
o Reality is grounded in a particular context
The researcher is inseparable from the research, no such thing as objectivity
Interprets instead of tests (research in this paradigm is often descriptive research – aiming to
understand a phenomenon)
o The researcher tries to sensibly interpret different meanings – looking for consensus
the researcher will find differences in perspectives which can be productive and
counterproductive depending on what the researcher tries to find out.
o As constructivists believe that researchers are inseparable from the research – the
researcher should be aware of their influence when conducting research. Being
aware differences from actively minimising the role of the researcher in the positive
paradigm
Insights from constructivists research are initially local
o It remains to be seen whether that they are generalizable to other contexts
Distinguishing cause and effect is impossible because they are always shaping each other
o True evaluation and implementation interventions change for example, this animal is
more than a squirrel – it has different meanings
IMPORTANT: Reality is constructed, no such thing as objective research, or a
world that exist apart from the researcher
The gist of it al
The ‘essential’ difference between positivism and constructivism resides in the belief about the
nature of the relationship between the researcher and the phenomenon of interest (for
constructivists, phenomenon is considered as a subject instead of an object)
This has consequences for the way in which knowledge is produced and the knowledge itself
Requires the use of methods in different ways
4
Les avantages d'acheter des résumés chez Stuvia:
Qualité garantie par les avis des clients
Les clients de Stuvia ont évalués plus de 700 000 résumés. C'est comme ça que vous savez que vous achetez les meilleurs documents.
L’achat facile et rapide
Vous pouvez payer rapidement avec iDeal, carte de crédit ou Stuvia-crédit pour les résumés. Il n'y a pas d'adhésion nécessaire.
Focus sur l’essentiel
Vos camarades écrivent eux-mêmes les notes d’étude, c’est pourquoi les documents sont toujours fiables et à jour. Cela garantit que vous arrivez rapidement au coeur du matériel.
Foire aux questions
Qu'est-ce que j'obtiens en achetant ce document ?
Vous obtenez un PDF, disponible immédiatement après votre achat. Le document acheté est accessible à tout moment, n'importe où et indéfiniment via votre profil.
Garantie de remboursement : comment ça marche ?
Notre garantie de satisfaction garantit que vous trouverez toujours un document d'étude qui vous convient. Vous remplissez un formulaire et notre équipe du service client s'occupe du reste.
Auprès de qui est-ce que j'achète ce résumé ?
Stuvia est une place de marché. Alors, vous n'achetez donc pas ce document chez nous, mais auprès du vendeur maaiketaheij. Stuvia facilite les paiements au vendeur.
Est-ce que j'aurai un abonnement?
Non, vous n'achetez ce résumé que pour €9,99. Vous n'êtes lié à rien après votre achat.
