Human error - Streetlights and
shadows
University of Groningen
Chapter 1 - Ten surprises about how we handle ambiguous situations 1
Part 1 - Making decisions 2
Chapter 2 - A passion for procedures 2
Chapter 3 - Seeing the invisible 4
Chapter 4 - How biased is our thinking? 6
Chapter 5 - Intuition versus analysis 9
Chapter 6 - Blending intuition and analysis to make rapid decisions 11
Chapter 7 - Experts and errors 14
Chapter 8 - Automating decisions 16
Part 2 - Making sense of situations 18
Chapter 9 - More is less 18
Chapter 10 - When patience is a vice 20
Chapter 11 - The limits of feedback 21
Chapter 12 - Correcting the dots 23
Chapter 13 - Do we think like computers? 25
Part 3 - Adapting 26
Chapter 14 - Moving targets 26
Chapter 15 - The risks of risk management 29
Chapter 16 - The cognitive wavelength 32
Chapter 17 - Unlearning 34
Part 4 - Finishing 36
Chapter 18 - Reclaiming our minds 36
Chapter 19 - Getting found 39
,Chapter 1 - Ten surprises about how we handle
ambiguous situations
The Gimli Glider incident illustrates an extreme case in which plans
and intentions fall apart, the typical procedures and routines don’t work,
and people have to draw on experience.
We need both mental “gears”:
1. One for using the standard procedures
2. Other for improvising when situations become unsettled
The dual viewpoint of light and shadow (the way we see in bright
light differs from the way we see in shadows; either is the ‘‘right’’ way,
we need both) affects how we make decisions and how we make sense
of situations.
→ affects how we plan and how we manage risks and uncertainty; guides how we develop
expertise and how we use our intuition
→ in this book, explored how we think and decide in the world of shadows, the world of
ambiguity
Aspects of thinking–making decisions, making sense of events, and adapting–are related to
each other, but they create different demands on us.
This book revises 10 generally accepted claims.
1
,Part 1 - Making decisions
Chapter 2 - A passion for procedures
First claim reflects our enthusiasm for taking the guesswork out of decisions by providing
procedures to follow and clear criteria for how to move on to each step.
→ reflects our desire to break complex tasks into step-by-step procedures
→ chapter examines the conditions under which procedures work for us and the conditions
under which they aren’t so helpful
Claim 1: teaching people procedures helps them perform tasks more skilfully.
The process of transforming skills into procedures is irresistible, all we have to do is
break a complex task down into steps and provide some tips about when to start and finish
each step.
→ much of our progress in fields such as training, coaching, and safety management comes
from this strategy
→ procedures also help us evaluative performance; we can see if someone knows the
procedures, and is following them
Hubert and Stuart Dreyfus described the inherent limits of procedural accounts and offered
a model of how people develop expertise.
→ according to their model, novices are given simple procedures that don’t depend on
context, on what else might be going on.
→ Dreyfus model of expertise emphasizes intuition and tacit knowledge that can’t be
captured in rules and procedures
Procedures, including checklists, are tools. Every tool has limitations, and author is not
arguing that we should do away with procedures.
→ what are those limitations?
1. procedures alone aren’t sufficient; in complex settings in which we have to take
the context into account, we can’t codify all the work in a set of procedures;
moreover, the more comprehensive the procedures the more voluminous they
become
2. procedures are difficult to keep updated; procedures are often out of date
because work practices keep evolving; and are therefore rarely complete
procedural drift = evolution of procedures
3. procedures can lead to mindlessness and complacency; procedures can lull
people into a passive mindset of just following the steps and not really thinking about
what they’re doing; when we become passive we don’t try to improve our skills
4. procedures can erode expertise; when we get comfortable with procedures, we
may stop trying to develop more skills and the result may be an erosion of expertise
in organizations that rely too heavily on procedures; people taught to understand the
system develop richer mental models than people taught to follow procedures
5. procedures can mislead us; following procedures can lead us in the wrong
direction and that we won’t notice because the reliance on procedures has made us
so complacent
2
, → all in all, we can see that procedures are insufficient, can get in the way, can interfere with
developing and applying expertise, and can erode over time
→ procedures work best in well-ordered situations in which we don’t have to worry about
changing conditions and we don’t have to take context into account to figure out how to
apply the procedures, or when to jettison them
If we took claim 1 seriously, we might create the following unintended consequences:
● establish “adequate” performance as the new ideal; too difficult and inefficient to
continually fiddle with better procedures, which creates an endless cycle of updating
● discourage people from using their judgment; we wouldn’t want them to overrule
the procedures
● generate massive volumes of procedures; because it is too expensive to go back
and cull procedures we don’t need anymore
● save money by retaining our current ways of doing the work
● issue procedures as a way to change behavior even though there may be simples
and more effective strategies
Strengths of procedures:
● they are training tools; they help novices get started in learning a task
● they are memory aids; in many jobs they help workers overcome memory slips
● they can safeguard against interruptions
● they reduce workload and make it easier to attend to critical aspects of the task
● they are a way to compile experience and historical information
● they can help teams coordinate by imposing consistency; if people on the team
know the same procedures, they can predict one another’s next moves
● important for ad hoc teams that don’t have a chance to practice together
regularly
Aviation illustrates the ideal arrangement: skilled decision makers living in harmony with
procedures.
To put procedures into perspective, consider the difference between directions and maps.
→ we sometimes get directions—a sequence of actions
→ other times we get a map showing where we are, where we want to be, and the terrain in
between
→ directions are easier to follow, but if anything goes wrong we are stuck; a map demands
more of us but makes it easier for us to adapt and can be used for other routes in the same
area
Instead of teaching by presenting standard procedures and making everyone memorize
them, set up scenarios for various kinds of challenges and let the new workers go
through the scenarios.
3