This document comprises very detailed lecture notes with thorough explanations. It has shown to be a great preparation resource for the LIO exam (Master in Educational Sciences).
Boonstra (2004)
Guba & Lincoln (1994)
Nielsen & Randall (2013)
Visse et al (2012)
In this clip focus on 3 questions
1. Why do we evaluate organizational change?
2. What does one focus on when evaluating change?
3. How does one evaluate change?
Why evaluate?
Nielson & Randell focus in their paper on process-evaluation which can be used to:
Provide feedback for improving interventions
Minimize pitfalls when replicating interventions in other settings
Interpret outcomes of interventions
Evaluation also identifies whether the goal of the organization has focused on in the intervention –
was the right goal.
Example: if university like to increase student number – student number can easily be
evaluated by counting them. But increasing students numbers might in itself have an
undesirable effect as well. – teachers can experience higher workload, which effects quality
of teaching / learning. So is it than sensible to evaluate student number? Other process
indicators are job satisfaction or engagement, or stakeholders within the organizational
change.
1
,Nielson and Randall
Process evaluation is after the circumstances under which an intervention will work and the
processes that facilitates that changes.
Under which circumstances will an intervention work? Which where the processes that facilitated the
change?
Process evaluation vs. effect evaluation
o Effect evaluation is used to measure immediate or long-term effects of a change.
Effect evaluation focusses on the expectations of the change. Did the organization
aim to change the knowledge skills or behaviour? Behaviour is often the hardest
outcome to measure and it can require a lot long time for effects to become
containable – and if they do- change to desired direction can also be caused by
factors other than the intervention. Its difficult to conclude how or why an
intervention worked from effect evaluation.
Answering the questions of the circumstances and the processes leads to an
understanding of the change from which other organizations can learn.
Model of process evaluation – is based on a literature review on health and well-being interventions
at the organizational level. But the factors and aspects apply to organizational change as well, in a
more general sense. This model is straight forward and the paper is extensively describing all factors.
READ It.
Nielson and Randall distinguished three factors that effect implementation and evaluation of
interventions. And at the core of this model is the intervention (light green box). What matters is
who initiated the intervention, whether the intervention activities target the problems of the
workplace and who are the drivers of the change (that aspect focuses on the roles and behaviours of
key stakeholders.)
Than Nielson describes Context which can dilute or strengthen intervention effects and processes (as
shown in the big light blue box). Still it remains unclear what the authors exactly mean with context.
Mayke takes it as the concrete organization where the intervention took place. Because the omnibus
context includes organizational culture and structure, and discrete context refers to particular events
that happened during implementation.
To conclude the mental models of people play a role in the change outcomes. This includes the
readiness for change and the perceptions of intervention activities. For the exam: you should now all
the factors by heart
2
,Paradigms - Guba & Lincoln logic of science
Meant to describe how science develops (sociology, philosophy of science, e.g., Kuhn) (paradigms
can help to understand the logic of evaluation)
Fundamental discourse questions if they exist
What is to be studied, what are relevant questions, how topics are connected, how to
interpret the results obtained and how does one conduct proper research?
For the exam: keep in mind to only study the parts of the chapter addressed in this lecture.
The worldview guiding the methodological epistemological and ontological choices made by
researchers (Guba & Lincoln)
Epistemology: what is knowledge?
Ontology: what is reality?
Methodology: How does one produce valid knowledge?
These three dimensions are related and aligned with another in a paradigm.
Ontology – example with picture squirrel
What is this?
Why is it that?
Is it real? Why (not)
If no humans were on Earth, would this still be what it is?
Epistemology – example with picture squirrel
How doe you know this is a squirrel?
If you had never seen one before, does it exist?
If someone told you this isn’t a squirrel, would you believe that person? Why (not)?
If it climbs up to a tree and there is no one to see it. Can it still be observed?
Methodology
Type of study and how it is to be done. E.g. descriptive or explanatory
Mythology > methods
o Methodology determines the methods which are the actual tool we use to collect
and analyse data. What does tools look like and how they are used depends on the
research paradigm and the theoretical/ conceptual framework
Why do paradigm matter?
Paradigms shift: radical transformation of science
Facts can only be interpreted through a particular ‘lens’
Evaluation requires to explicate the zoom lens used = description of the combination of ideas
(concepts) which guide your thinking (= theoretical framework)
Paradigms are accepted based on faith, choice needs to be substantiated. What does a
particular issue ask for?
Can coexist; your paradigm ‘of choice’ determines the options at your disposal, the
possibilities available
See Visse for how paradigms can get in the way of an evaluation (table 6.2)
3
, Positivism, the ‘received view’ (that’s how Guba & Lincoln call it, as most scientists are trained in this
paradigm and it is also the most widely accepted perspective on what counts as research)
Also called conventional paradigm
‘imparted’ from the natural sciences
Precise, measurable variables that can be quantified
The role of science is to discover new knowledge about how the real world works
o That means that the real world exists separately and apart from humans – so the
best way to study it is through an unbiased and objective researcher who is not to
involved in the research (object)
You might see here that this claim involves ontology and epistemology
Usually hypothesis-drive, allows one to make predictions
The real world is separate and apart from humans
The best way to discover new knowledge is through unbiased, objective research in which
the researcher aims to limit any influence
Guba and Lincoln explain an alternative paradigm
Constructivism
There are multiple realities. These realities are locally (and often socially) constructed
(ontology)
o Reality is grounded in a particular context
The researcher is inseparable from the research, no such thing as objectivity
Interprets instead of tests (research in this paradigm is often descriptive research – aiming to
understand a phenomenon)
o The researcher tries to sensibly interpret different meanings – looking for consensus
the researcher will find differences in perspectives which can be productive and
counterproductive depending on what the researcher tries to find out.
o As constructivists believe that researchers are inseparable from the research – the
researcher should be aware of their influence when conducting research. Being
aware differences from actively minimising the role of the researcher in the positive
paradigm
Insights from constructivists research are initially local
o It remains to be seen whether that they are generalizable to other contexts
Distinguishing cause and effect is impossible because they are always shaping each other
o True evaluation and implementation interventions change for example, this animal is
more than a squirrel – it has different meanings
IMPORTANT: Reality is constructed, no such thing as objective research, or a
world that exist apart from the researcher
The gist of it al
The ‘essential’ difference between positivism and constructivism resides in the belief about the
nature of the relationship between the researcher and the phenomenon of interest (for
constructivists, phenomenon is considered as a subject instead of an object)
This has consequences for the way in which knowledge is produced and the knowledge itself
Requires the use of methods in different ways
4
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller maaiketaheij. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $10.88. You're not tied to anything after your purchase.
