METHODS OF COMMUNICATION RESEARCH
WHAT IS COMMUNICATION RESEARCH?
Research?
A certain look comes over the faces of some of my students when they hear the word research. Their
eyes glaze over, and their faces take on a pained expression as if they had a migraine or a bad
stomachache. They see the required course on research as some kind of an ordeal they must survive
before being allowed to take the courses they want and go on to live a normal life.
Is everyone a researcher?
• Research (ENG) /chercher (Fr) à to search, to seek
• Research means looking for informa;on about something.
• You research everyday: What phone do I buy? What is healthy to eat? Is this Ne6lix
series for me? What is this conflict about?
SCIENTIFIC RESEARCH IS DIFFERENT FROM EVERYDAY RESEARCH
• More systema;c: there are procedures.
• Based more on facts than feelings
• More cau;ous
• More focus on accuracy and veracity
• Data, numbers, and sta;s;cs as part of scien;fic research? Not necessarily (cf.
historical research within literary studies vs. econometrics)
COMMUNICATION RESEARCH
• Young discipline - °50 years ago
• An interdisciplinary field: we borrow theories and methods from other fields.
• Responded to the increasing important role of print and broadcast mass
communica;on media in everyday life, drawing on concepts and methods from both
the social sciences, the humani;es, and even the natural sciences (cf. use of
psychophysiological metrics in the social lab).
• With the development of digital media, the field is more central than ever to the
poli;cal, economic, and cultural developments in socie;es and the world.
1
, COMMUNICATION RESEARCH AS INTERDISCIPLINARY RESEARCH
Two major currents within communica6on science research (see Deuze, 2021)
• Humani;es-oriented study of media and communica;on: link with disciplines that
analyze and understand products of human culture such as language, art history
literature, history, philosophy, culture, etc.
o E.g. Media studies, cultural studies, and the study of popular culture
o Examines everything from art house cinema and fic;on to comics, television,
movies, music, digital media, and everyday life
o Usually base their analyses on the concepts, ideas, and theories of
philosophers, psychologists, social scien;sts, linguists, and others
o O\en qualita;ve research
• Social science-oriented study of media and communica;on: link with disciplines that
analyze, understand, and predict human behavior such as sociology, psychology,
poli;cal science ...
o E.g. Media impact research, strategic communica;on
o Examines influence and effects of media and communica;on on behavior of
people and society (posi;ve and/or nega;ve)
o Usually base their analyses on the concepts, ideas, and theories of social
scien;sts and (experimental) psychology
o O\en quan;ta;ve research
• No strict separa;on, lots of interdisciplinary research
QUANTITATIVE VS. QUALITATIVE?
When we think of quan;ta;ve research in media and communica;ons, we think of numbers,
effect, and measurement. Quan;ta;ve researchers are some;mes accused of being too
limited, basing their research on what they can count, measure, and observe, and neglec;ng
other things.
When we think of qualita;ve research, we think of media texts, interpreta6ons, and
ethnographic research. Qualita;ve researchers are o\en accused of "reading" things into
texts or making interpreta6ons of how people consume media that are not there.
Interpreta;ons o\en seem very personal and too much evalua;ve / value driven.
2
, RESEARCH NEEDS STRATEGIC CHOICES
Different research methods, qualita;ve or quan;ta;ve, or combina;ons are not inherently
"beeer" or "truer," but lend themselves to different research ques;ons and observa;ons.
Determining which research method is part of a strategic nego6a6on.
• What method do you use when you are interested in mass media representa<ons, and
how the mass media shape social ideas about sexual iden<ty based on these
representa<ons? QUALITATIVE
• Which method do you use when you are interested in the effec<veness of
communica<on campaigns to promote healthy behavior? QUANTITATIVE
Ques6ons of interest in evalua6ng these strategic choices made by researchers
1. What methodology was used in the research, what are the strategic choices and reasoning?
2. How important is the topic and does it cons;tute a challenging scien;fic inves;ga;on?
3. What conclusions were drawn?
4. Are the conclusions supported by the data? Are the conclusions credible?
5. Can one generalize from the research?
6. Can these findings lead to prac;cal recommenda;ons?
PART I: QUANTITATIVE METHODS
• Deals with methods with which you can quan6fy (= pugng things into numbers)
• Assump;on is that you can put all things in numbers
o How oEen something occurs (e.g. misinforma;on on Facebook)
o How things are related
§ Correla;on: more misinforma;on leads to less public trust
§ Percentage: 80% chance that a fear appeal will make public smoke less
o What is more effec6ve?
§ Means and standard devia;ons: Do social robots at home lead to less
loneliness?
• Four dis;nct methods in Part 1
o Literature review
o Experiments
o Survey
o Quan;ta;ve content analysis
3
, LITERATURE REVIEW
WHY DO WE NEED IT?
The key goal of any scien;fic domain is to build cumula6ve knowledge. In this process, it is
important to synthesize empirical findings to understand current overall effects of for
instance communica;on research.
Generate overview of current knowledge regarding a communica;on science topic for
• Knowledge: what is known about a topic at this moment?
• Discussion: how to proceed with future research on the topic?
• Policy: e.g. how to deal with misinforma;on? Guidelines for pa;ents? Etc.
• For you? Knowledge that the method exists and what you can do with it. Very likely
you will encounter a literature review during your ;me at UAntwerp.
Example: How much behavior change (in %)
occurs aMer seeing a mass-mediated
campaign (168.362 par<cipants)?
Effects of campaign exposure on
• Alcohol (r = .09): .09*.09 = 0.008% of behavior change for alcohol can be explained by campaign exp.
• Smoking (r = .05): .05*.05 = 0.003% of behavior change for smoking can be explained by campaign exp.
• Sexual health (r = .04): .04*.04 = 0.002% of behavior change for sexual health can be explained by
campaign exp.
Today: how do we get to such a conclusion?
WHAT TYPES OF LITERATURE REVIEW?
Three types, based on two criteria:
• Systema6c approach or unsystema6c approach?
o Unsystema;c: no strict criteria for the inclusion or exclusion of studies and the
weight placed on the findings of single studies (e.g. sample size or quality of
studies). They tend to be biased because they rely heavily on the subjec;ve
decisions of researchers.
o Systema;c: strict rules for
making decisions regarding the
search for studies, inclusion
criteria, and the weight that is
put on the findings of single
studies. The rules enable
researchers to review studies in a
systema;c way that can be
replicated by other researchers.
4
, • Meta-analy6cal techniques or vote-coun6ng?
o Vote-coun;ng: researchers simply count the number of studies that did find
sta;s;cally significant results and the number of studies that did not find
sta;s;cally significant results based on a specified cut-off value (typically p <
.05). Based on the number of votes, the researcher judges whether there is an
effect or not. Problem: The p-value of a single study is the product of the
observed effect, variance, and the sample size. However, without an adequate
sample size, the p-value will not indicate that an effect is sta;s;cally
significant, even if the magnitude of the observed effect is substan;al. As
such, if a large propor;on of studies have inadequate sample sizes, then a
researcher will draw the erroneous conclusion that no effect exists.
o Meta-analy;cal techniques: combine the informa;on of a series of single
studies, so that small effects may be revealed that would not be found
through vote coun;ng.
NARRATIVE REVIEW
How and where did these authors search? Unclear!
What are the results of this narra<ve review? Very
liVle quan<ta<ve informa<on.
How and where did these
authors search?
What are the results of this
narra<ve review? Abacus!
• A non-systema6c approach (no systema;c words) to literature reviews regarding
o Search strategies, and/or
o The selec;on process (which criteria were used to include / exclude a study)
• Results: unclear, abacus (vote-coun6ng) at most
• Downsides of narra;ve reviews?
o Sensi6ve to the biases of the researcher
o Liele to no transparency about the review process
o Not replicable by other researchers (if you do a narra;ve review on the same
topic, you will likely end up with very different results).
5
, SYSTEMATIC REVIEW
• A systema6c approach regarding
o The search process: databases use specific terms or Key
words that everybody has to use (for instance so-called
MESH headings)
o Repor;ng: IMRAD / PRISMA structure. If you know what one review looks like,
you know what another review looks like: they have a structure, you have to
adhere to the rules.
o Criteria for selec;on: inclusion versus exclusion
An example
• CDSS = Clinical Decision Support System
• Computer system that supports / alerts clinicians when they
have to take a clinical decision, for instance changing
medicaOons or treatment opOons.
• From previous research we know that…. these clinicians disregard 50-95% of CDSS alerts
• Research quesOon of our review: What are the barriers and facilitators of CDSS- acceptance, as
reported by clinicians? This research quesOon was the starOng point of our systemaOc review
6
,• SystemaOc search in collaboraOon with ‘informaOon specialist’
o Knows which standard search terms are used in the databases (e.g. MESH headings, such as e-
prescripOon)
o Knows relevant free search terms
o Provides recommendaOons for search terms (so you do not end up with too much, but also not
too few hits)
• RegistraOon of review protocol in Prospero
o PreregistraOon of enOre process!
o Enhances transparency
• Determine inclusion & exclusion criteria: What are the requirements studies should have to be included
in the review? Such as
o Study populaOon: paOents; young adults; etc…
o Study methodology: objecOve vs subjecOve measures; research designs (experiments only?), …
• Two disOnct phases in this process
o Screening of Otles and abstracts only
o Reading of full papers
§ Only those who pass Phase 1
§ Per excluded paper: reason(s) for exclusion
• Extensive tables
• Provides detailed informaOon on
o How oden has something been studied?
o How oden have posiOve effects been reported?
• IMRAD structure
o Summarise main findings
o Interpret: summarise in own words
o ImplicaOons of findings
o Compare with previous work
7
, • Advantages: replicable and transparent
o Systema;c searches using fixed terms
o A priori protocol registra;on
o Clear and verifiable inclusion and exclusion criteria, reported in paper
o PRISMA en IMRAD structure
• Different researchers will come to the same conclusion on the same topic
• Downsides: S;ll an abacus
Abacus / vote-coun;ng: disadvantages
• Highly dependent on the p-value of an individual effect (an effect is significant or
not): relies only on the ques;on “is it significant or not?”
• Misleading results, par;cularly when you have many included studies
• p-value is dependent on three things
o Observed effect (e.g. differences in aeen;on between two groups)
o Variance (spread/devia;on around this observed effect)
o Sample size of your study / condi;ons
• In research, there is always a chance to have a chance-finding
(= you find something that is not supposed to be there)
o type I error: study shows a significant effect, which in
reality is not there
o type II error: study shows a nonsignificant effect, which
in reality is there
• Vote-coun;ng: same weight to type I error studies as type II
error studies
Vote-coun)ng results:
7/19 (36.8%) studies: significant effect of narra)ves
12/19 (63.2%) studies: non-significant effect of narra)ves
What would be your conclusion on the effects of narra)ves, using
vote-coun)ng? NOT SIGNIFICANT
Effects of the meta-analysis?
META-ANALYSIS
The term meta-analysis was introduced by Glass (1976) to refer to “the sta<s<cal analysis of
a large collec<on of analysis results from individual studies for the purpose of integra<ng the
findings.” Meta-analyses are becoming increasingly popular in the field of health and risk
communica;on—they allow for more precise es;ma;ons of the magnitude of effects and
the robustness of those effects across empirical studies in a par;cular domain.
8
