In dit document vind je een samenvatting van alle colleges. Alle stof die in de colleges behandeld worden kun je vinden in dit document. Het is geschreven in het Engels. In this document you will find a summary of all of the lectures. All material covered in the lectures can be found in this docume...
Inhoud
College 1: Introduction...........................................................................................................................2
College 2: What to measure?.................................................................................................................4
College 3: Accents and prominence........................................................................................................6
College 4: Manipulation of speech.......................................................................................................10
College 5: Chunking..............................................................................................................................12
College 6: Synthesis..............................................................................................................................16
College 7: Confidence marking.............................................................................................................19
College 8: Error handling and feedback................................................................................................24
College 9: Emotion and surprise...........................................................................................................29
College 10: Stance and Irony................................................................................................................35
College 11: Mimicry..............................................................................................................................38
College 12: Metaphors..........................................................................................................................42
,College 1: Introduction
Introduction of basic concepts
Sentence: the same sentence can have different meanings through extra
connotations (that are not expressed through words or syntax) from the context or
from the way a sentence is uttered
Non-verbal communication: how you say something, consist of
o Auditory forms of prosody
Variation in voice: features that you can hear, e.g. speech melody,
intonation, loudness, tempo, rhythm, voice quality, pauses
o Visual forms of prosody
Variation in body language: features that you can see, e.g. facial
expressions, gaze patterns, hand gestures, pointing, posture, distance
o These can be different among cultures
Regarding voice and body language
How important are voice and body language?
o Rethorics: the art to persuade an audience
An important aspect: pronunciation = various forms of nonverbal
communication: voice and body language features
Already used a long time ago
We can still recognize facial expressions on paintings from long ago,
and ideally those features should match the content of the spoken
utterances
o Now also used a lot in presidential debates
o Nonverbal features account for more than 90% of the communication
o The study of nonverbal features is a new field, and have long been hampered
by a lack of tools to record, measure or analyze
How do voice and body language interact?
o Multimodality: our perception of reality is multimodal (vision, hearing, touch,
taste)
o Visual expressions: normally the speaker and addressee see each other
o Spoken communication without contact is new
o Human beings do not only interact through auditory information, but also
through visual information (in addition to other sensory information)
o Relation between faces and speech: when we see a person talking, when we
see their face, the visual information can have an effect on the way we
perceive the spoken information
o E.g. ventriloguism (buikspreker) effect: when an auditory and visual signal are
offered simultaneously in different locations, we build a perceptual construct
suggesting that the source of the audio is spatially related to the visual signal
McGurk effect: Combine a movie clip (vision only) of someone uttering ‘ga’ with the
speech sound of someone uttering ‘ba’ Perceived as ‘da’
o In addition:
Faces have an impact on how we perceive the speech of others
Cocktail party phenomenon: je kunt je op 1 persoon focussen
, Lipreading
Compensatory effects: when there is noise on the auditory or visual
channel
Voice and body language interactions in feedback mechanisms
How do voice and body language develop with age?
o When children grow older, they improve their skills in using and interpreting
verbal and non-verbal communication
o Intonation patterns, rhythm and features of the voice acquired while in
mother’s womb
o Young infants can intimate facial gestures, like tongue protrusion and mouth
opening
o Infants quickly learn to integrate information coming from different modalities
o Younger children are largely egocentric. As the child grows older, they become
more other directed or socially aware, and nonverbal features become more
functional in nature
o Hypothesis: nonverbal features may reveal differences in social awareness
between younger children, older children and adults
How universal are aspects of voice and body language?
o How do we acquire nonverbal features? By nature or nurture?
o Everyone shows facial expressions (even animals and babies and blind people)
o So there is probably a genetic basis for the use of auditory and facial
expressions, especially related to the expression of basic emotions
Example of a study (surprise experiment)
Many studies use a game-based approach (people are unaware of the real purpose of
the study; do not reflect on their non-verbal features)
Experiment: how do speakers behave nonverbally in expected (normal) vs.
unexpected (surprise) contexts?
Participants are believed to take part in a memory experiment and that they are
interested in the effect of context and in the effect of reading aloud
o 1. Participants have to imagine words that fit in a specific context
o 2. Then they get 10 words shown on a screen one by one
o 3. They have to read those words aloud as they appear on the screen
o 4. Then they have to recall as many words as possible
2 experimental context
o Saying the word “liver” in a
Normal context: organs of the human body
Surprise context: favorite food items of Dutch kids
o Combined with other questions
Perception of participants
o Everyone can see and hear the surprise
o Speakers show these nonverbal features spontaneous and automatic
, College 2: What to measure?
Not voice language + body language, but (voice) + (body language)
Terminology
Verbal vs. nonverbal communication
o Verbal = ‘wordy’ = not what this course is about
o Non-verbal = everything that is not expresses with words
o Sign language = verbal, because it is like words
o Body language = not about words
Words = have a form (sound) and meaning
o To read: meaning = a verb takes a subject (the reader) and object (the thing
that is read)
o Words are conventional, because another sound could be paired with the
same meaning
o Sing languages/visual language are also language, they are conventional, not
mimicry, they are not the same
o Some languages have alternatives for words, like morphemes = verbal
Measurements
Speech measurements
Through vocal fold vibrations
Non-verbal information: tenseness of the vocal folds
People can have a higher or lower voice
Speakers can vary pitch within their range, but you also have a physical limit
Pitch = how often the vocal folds vibrate and how you interpret other voices
Filter (not very relevant for non-verbal part)
Vowels (klinkers): position of tongue/jaw changement in the resonance of higher
frequencies (positie van tong/kaak verandert de resonantie van hogere frequenties)
Consonants (medeklinkers): interruption of airflow (sssss of p)
So: some frequency and interruption information is verbal, some non-verbal. It adds more
details on the voice.
Pitch
We hear a frequency continuously, called pitch
Vocal fold vibration has a frequency: F0
There can be no F0, but then we still hear something
There is always pitch
Use/function of pitch
Non-verbal information
Pitch accents on new/given/contractive information
o We help the listener by giving non-verbal information (e.g. pitch rise) on new
information
Pitch rise at the end, or steady drop when asking a question or saying an assertion
o Extra intonation at the end of a question caused by vocal folds vibration
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller anoukdegroot1. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $6.80. You're not tied to anything after your purchase.
