Summary Research Methods for
Analyzing Complex Problems 2018-
2019
By: Itske Smit
Master student Biomedical Sciences, major Science in Society at the University of Amsterdam
,Index
Index....................................................................................................................................................... 1
Lecture 07: Analysing qualitative data.................................................................................................... 3
Story telling..................................................................................................................................... 3
Assumptions................................................................................................................................... 3
Qualitative researcher characteristics.............................................................................................3
When to think of analysis?.............................................................................................................. 3
Quantitative analysis principles........................................................................................................... 3
Different types of analysis................................................................................................................... 3
Codes................................................................................................................................................. 4
Coding Steps.................................................................................................................................. 4
Analysis types................................................................................................................................. 6
Major points about analysis............................................................................................................ 6
! Validity.......................................................................................................................................... 6
Data saturation............................................................................................................................... 6
General pitfalls.................................................................................................................................... 7
Lecture 08: Ethography.......................................................................................................................... 8
The origins of ethnography................................................................................................................. 8
Auto-ethnography............................................................................................................................... 8
Strengths of ethnography.................................................................................................................... 8
! Three kinds of data collection........................................................................................................... 8
! Producing three kinds of data:...................................................................................................... 8
What is studied................................................................................................................................... 9
Ethnography is................................................................................................................................ 9
Objectivity debate............................................................................................................................... 9
Role of the researcher........................................................................................................................ 9
Participant observation................................................................................................................... 9
Lecture 09: statistics............................................................................................................................. 11
What is the goal?.............................................................................................................................. 11
Based on the question.................................................................................................................. 11
Population vs sample........................................................................................................................ 11
! Variables......................................................................................................................................... 11
Data preparation............................................................................................................................... 11
Assumption................................................................................................................................... 12
Types of graphs and figures......................................................................................................... 12
Descriptive statistics......................................................................................................................... 12
Distributions.................................................................................................................................. 13
Inferential statistics........................................................................................................................... 13
Test of significance are called....................................................................................................... 13
1
, Hypothesis testing......................................................................................................................... 13
Errors............................................................................................................................................ 13
! Risk ratio......................................................................................................................................... 13
! Odds ratio....................................................................................................................................... 14
Inferential statistics (numerical)........................................................................................................ 14
Inferential statistics (categorical)....................................................................................................... 14
Lecture 10: Focus groups..................................................................................................................... 16
Focus group...................................................................................................................................... 16
Focus groups can......................................................................................................................... 16
Characteristics.............................................................................................................................. 16
Structure....................................................................................................................................... 16
! Benefits....................................................................................................................................... 16
! Limitations of focus groups......................................................................................................... 16
Participant selection...................................................................................................................... 17
Moderating skills............................................................................................................................... 17
Attitudes........................................................................................................................................ 17
Challenges abroad........................................................................................................................ 17
Other roles........................................................................................................................................ 17
Keeping notes as a secretary....................................................................................................... 17
Observer/monitor.......................................................................................................................... 18
Interpreter..................................................................................................................................... 18
Questioning strategies...................................................................................................................... 18
Type of questions.......................................................................................................................... 18
Phrasing........................................................................................................................................ 18
Sequencing questions................................................................................................................... 19
Visualisations.................................................................................................................................... 19
Associations...................................................................................................................................... 19
Exercises...................................................................................................................................... 19
2
,Lecture 07: Analysing qualitative data
Story telling
The analysis of qualitative data – Rigorous, logical process through which data is given meaning. You
are interpreting the data.
Assumptions
Reality is socially constructed
Emic (insider point of view)
Variables are complex, interwoven, and difficult to measure
The researcher is his/her own instrument
No standardised procedures
Personal involvement and partiality
Empathic understanding
Qualitative researcher characteristics
Reflexive awareness – ability to:
o Think abstractly
o Step back and critically analyse situations
o Recognise the tendency towards bias
Openness
o Be flexible and open to helpful criticism
o Appreciate inquiry
Sensitivity
o Sensitive to the words, interpretations and actions of respondents
When to think of analysis?
During design
During data collection
Desk analysis afterwards
Quantitative analysis principles
1. Noticing concepts
2. Collecting examples of these concepts
3. Analyzing these concepts in order to find
commonalities
Different types of analysis
Content analysis
o The purpose is to describe the characteristics of the document’s content by examining
who says what, to whom and with what effect and make inferences (may contain
quantitative elements)
Thematic analysis
o Thematic analysis as an independent qualitative descriptive approach is mainly
described as a method for identifying, analysing and reporting patterns (themes)
within data
Grounded theory
3
, o The construction of theory through the open analysis of data
Codes
Words/short phrases that represent the essence or key attribute of narrative/verbal information
Used to categorize data
Coding is the process of organising the data into ‘chunks’ (segments) that are alike
Summarize – Understand the narrative of your interview, what is most important (vertical
analysis)
Codes are developed into a ‘coding structure/guide’
Codes structure/guide
Compilation of emerging codes
Brief definitions or properties for each code (can also include illustrative codes)
Provides guidance for when and how to use the codes
Will evolve throughout the analysis
You can continuously have discussions with your research team
Quotations
Bring reader to reality of the situations
Support your analysis and findings
Illustrative
Range of issues
Opposing views (between stakeholders)
Anonymity?
Coding Steps
1. Transcribe
2. Collect-code-collect-code
3. Read and re-read.... suspend initial interpretation. Focussed reading and open coding.
4. Close examination, label text with keywords. Reviewing and axial coding
5. Modify codes, remove duplications, hierarchical order, intergraded theory. Generate Theory
6. Look for connections that emerge from data
Step types
Open coding
Axial coding/thematic
Selective coding
Open coding
Analytical process through which concepts are identified
4
, Their properties and dimensions are discovered in data
Deductive research is guided by theory testing hypothesis
Inductive research is open to all concepts
Pitfall deductive research: too much fitting in existing boxes (close minded)
Pitfall inductive research: too free in accepting all (messy code book)
Naming and categorising phenomena through close examination of data
Ask the data specific set of questions
Analyse the data minutely
Comparing text fragments on similarities/differences
What is the underlying concept?
Labelling fragments with keywords: concepts and categories include as many as possible
More horizontal analysis
Axial coding
Process of relating categories to their sub categories
Linking categories at level of their properties and dimensions
Deductive research is looking for relations as presented in theory
Inductive research is often difficult to find evidence for relations
Pitfall deductive research: too much looking for evidence of relation (close minded)
Pitfall inductive research: accepting too vague relations as the truth
Process of making connections between the different categories produced by open coding
Examine a phenomenon in terms of properties and dimensions
What is the underlying pattern?
Linking categories on that level
More vertical analysis
Selective coding
Process of integrating and refining theory
Deductive research is testing theory and improving it
Inductive research is designing new theories
Proces of refining categories. Defining core categories. Integration of core categories into theory.
Finding a storyline around core categories
Validate relationships categories against data
Re-iterate
5
,Analysis types
Horizontal analysis – focused on aggregation and comparison of content of data across
different interviews (or other data). Pay attention to diversity (both majority and minority of
view count)
Vertical analysis – focused on understanding the essence of individual interviews: ‘the
narrative’ (or other data). You need the unique line of arguing of individuals and their priorities.
Major points about analysis
Procedure is very focussed and analytical
Listen to what respondents are saying and how they’re saying it
o Taking into account interviewee’s interpretations
Asking a lot of questions to become more specific
Keep distance: it is the data that is relevant, not the case-specifics
Conceptualise and classify events
Making theoretical comparisons
! Validity
Is the researcher observing and identifying what he/she claims she is?
External validity
o Are the findings generalisable to other social settings?
o Similar to theory?
o Definition: degree to which findings can be generalized to other social or
organizational settings. This type of validity is difficult to achieve in qualitative
research due to the use of case studies and small samples.
Internal validity
o Strong link between data and theoretical ideas developed?
o Definition: Internal validity: refers to whether there is compelling evidence that the
researcher has achieved a strong link between their evidence and the theoretical
ideas they develop from it.
! Increasing validity
Quality and credibility starts with the data!
Training of the researcher
Systematic procedures
Group work analysis
(independent) expert check
Comparative methods (triangulation)
Analytical
Literature review
Member check, expert check
Explore rival explanations
Presentation
Provide supporting evidence
Acknowledge researchers’ perspective
Provide thick descriptions
Data saturation
Theoretical saturation
From grounded theory (intensive)
o Collected data adds nothing new to developing theory
6
, o Remember: continuous modification of data collection procedures, rather than simply
repeating standardized instrument
Code saturation
o No more emerging codes
o Repeating same data collection instrument, till no new are produced
General pitfalls
Biased transcription and interpretation
Overemphasis on positive cases
Too much focus on the exotic or unusual
The ignoring of negative cases ( these can be interesting_
Vague definitions of concepts
Inconsistent application of such concepts to the data and unwarranted generalisation
7
,Lecture 08: Ethography
Ethography is the writing of the people, the writing of society, the writing of culture. “The study of
human behaviour within their culture.”
The origins of ethnography
Ethnography has a dark past when it comes to its complicity in promoting colonialism and
colonialist agendas around the world
Anthropology emerged from the colonial expansion of Europe. Colonialism structured the
relationship between anthropologists and the people they studied and had an effect on
methodological and conceptual formulations in the discipline.
Social anthropology and colonialism were contemporaneous because colonial power made
the subjects of anthropological study accessible
The British state embraced anthropology because it allowed for the collection of information
and data on its subject territories
Anthropology and self reflexibility
o In the 1980s a postmodersn turn in anthropology challenged anthropologists to
question their own assumptions and write more reflexively... questioning the ‘narrator’
and the ‘other’
Auto-ethnography
Ethnography is a highly complex and contentious discursive field at the intersection of communication,
culture and identity.
Strengths of ethnography
1. Understanding the meaning, for participant in the study, of the events, situations, and of the
accounts that they give of their lives and experiences
2. Understanding the particular context, within which the participants acts, and the influence that
this context has on their actions
3. Identifying unanticipated phenomena and influences, and generating new grounded theories
4. Understanding the process by which events and actions take place
5. Developing causal explanations
Ethnography seeks to be both descriptive and interpretive
! Three kinds of data collection
Observation
Interviews
Documents
! Producing three kinds of data:
Quotations
Interviews
Documents
! The aim is to produce a narrative that help to tell ‘the story’
Ethnography looks at and records people’s way of life, and takes an emic (folk or inside) and etic
(analytic or outside) approach to describing communities and cultures
8
, What is studied
People’s actions and accounts studied in context
Data gather from range of sources
Data rely on senses, unstructured
Focus on a few cases
Analysis: Interpretation of meanings, functions and consequences of human actions
Ethnography is...
Inductive
Researcher is data collection instrument
Results presented in (visual) narratives
Learning about culture
Writing about culture
Objectivity debate
The greater the distance the more reliable
Or
The shorter the distance the less distorted, inaccurate and damaging
Role of the researcher
Complete observer (etic point of view (outsider))
Observer as participant
Participant observer
Complete participant (emic point of view)
Participant observation
Longterm engagement
Degree of openness
o ‘Undercover’
o Semi open
o Fully open
Why observing
Seeing interactions
Telling acting
o Over/under reporting
o Socially desirable answers
o Unconscious behaviour
Direct observations in a natural setting
! Threats to quality
Biased transcription and interpretation
Overemphasis on positive cases, ignoring of negative cases (! Was on exam)
Focus on exotic or unusual (! Was on exam)
Vague definitions of concepts, inconsistent application of such concepts on the data
Unwarranted generalisation
Too involved, subjectivity
Hawthorne effect (modification of behaviour by awareness of being observed)
Labour intensive
9
