The Human Mind in the Digital World
(Previously: Human Media Interaction Theory)
Masters course (2022)
Course code: 880014-M-6
Tilburg University
Course summary including;
Weekly lecture notes
Learning Goals
Key terms, theories, and models.
Lecture Notes...................................................................................................................................2
Week 1/Lecture 1 User Engagement...........................................................................................2
Week 2/Lecture 3 Attention & Perception...................................................................................3
Week 2/Lecture 4 Attention & cognitive load.............................................................................5
Week 3/Lecture 5 Memory..........................................................................................................7
Week 3/Lecture 6 Learning behavior..........................................................................................9
Week 6/Lecture 7 Emotion & Motivation.................................................................................11
Week 6/Lecture 8 Media Multitasking......................................................................................14
Week 7/Lecture 9 Social Cognition and embodied cognition in VR.........................................15
Week 8/Lecture 10 social media use and addiction...................................................................16
Learning Goals...............................................................................................................................17
Learning Goal 1.........................................................................................................................17
Learning Goal 2.........................................................................................................................23
Learning Goal 3.........................................................................................................................24
Learning Goal 4.........................................................................................................................25
Learning Goal 5.........................................................................................................................26
Key terms, models, and theories....................................................................................................27
, Lecture Notes
Week 1/Lecture 1 User Engagement
New media & cognition most studies are correlation (not able to draw causal conclusions).
Highly theoretical background with an approach from the user perspective in this course. The
core focus is on cognition. How people interact with new media, the role of HMIT:
1) Designers & process (innovative & creative)
2) New media landscape (= complex)
3) Users & their complex/demanding human needs and abilities
Engagement = intuitive. This can be approached in two ways:
Metrics (clicks, likes measurable)
Cognitive-psychological (“experience”, describing and understanding how does it feel).
Losing track of time and surroundings. Optimal experience of and activity engagement. A
state of mind = called different terms such as flow, involvement, engagement.
In the design world, a shift is happening in which the focus shifts from usability to UX
(understand the deeper, personal experience). A high level of experience/engagement = “flow”
(like being in a bubble). All cognitive, affective, and physical/motor systems are all dedicated to
the same task and goal. Typical features:
1) Challenge & skills are in balance they both are high
2) Clear goals
3) Unambiguous feedback
4) Action/awareness (effortlessness)
5) Concentration (top-down)
6) Sense of control
7) Loss of self
8) Transformation of time
9) Autotelic experience (intrinsic rewarding)
, antecedents (prerequired actors): what contributes to flow/the experience
nature: what does it mean, what is it
consequences: impact of flow/high experience. Engagement affects loyalty, positive attitude,
repeating consumer behavior, sharing.
Antecedents can be explored with the PAT (person, artifacts, task) model. In this model, there is
an interplay between all three but most significant between the person * task.
Person: traits (stable features, long term) and states (fluctuate, short term such as
emotions). The traits contribute to the flow. Autotelic ability, explorative behavior
Artifacts: artifact characteristics (aesthetics, vividness, responsiveness) contribute to the
likelihood of flow. These are mainly important for usability.
Task: goal, autonomy, and variety enhance the likelihood of flow. This is the most
complex part of the model because of the difficulty in identifying the task.
Nature: how to operationalize the concept of engagement? It is a fuzzy concept = not directly
measurable mostly measured with self-report and behavior questionnaires.
Stages/degrees of engagement:
Point of Engagement
o Interface assessment: natural mapping, intuitive, easy to use
o Physical interaction: observability activity with the interface usability.
Clicking. Scrolling.
Deeper engagement
o Absorption: deep involvement most fuzzy to measure
Behavioral outcome
o Digital outreach: interaction with content. Repeated use.
Objective (online) measures (psycho) physiological instead of subjective (self-report /
behavior), this indicates arousal levels! (heart rate, pupil dilation)
Flow & arousal levels show an inverted u-cure. This can be positive or negative.
Week 2/Lecture 3 Attention & Perception
, Attention
what attracts attention (selective attention > flashlight metaphor)
Attention control goes two ways:
1) Bottom-up = stimulus that attracts. Something that stands out. Not intentionally.
Physically salient.
2) Top-down = goals & expectations. Goal-driven. Voluntary.
3) Relevance history: relevant and rewarding in the past (not goal determined but also not
physically salient).
In new media, there is a tendency to bottom-up control anything that appears suddenly:
buzzing, notifications, sounds.
Physical salient is also relevant to its environment (uniqueness). If everybody does it, it does not
stand out anymore.
Top-down in new media: anything that is goal-oriented (like the search option).
New media also triggers phantom vibration (illusions because of expected bottom-up attention
triggers).
Relevance history: this can be trained making associations next to the attention itself. If
something is manipulated by using relevance history technique, people tend to slow down.
e.g. good experience with a brand brand name will stand out in future ‘shopping’ sessions.
This is trained, it keeps ‘distracting’ you (grabbing your attention).
e.g. cocktail party phenomena (when somebody mentions your name). In new media, relevance
history for example is used in personalizing newsletters/advertisements/etc.
In new media attention is the new ‘currency’. Smartphones have an impact on attention:
immediate effects but maybe also long-term effects. Visibly present smartphone creates
distraction (even without stimulation). This can be explained by relevance-history.
In the long term it might change attention control in general (has not been experimentally
demonstrated yet). Attentional systems are however malleable (reshapeable).
Attention is the first step of user engagement: full focus on the flow of activity (top-down).
