Roadmap Methodology
First question yourself: what is the reason and objective of the research?
In short: always want to report the following
- Method → Manifest and latent: how to make a construct observable?
- Results and conclusion
- Participants/media content
- Unique aspects of study
- Discussion
Step 1: Research question and hypothesis and give the direction
Research question:
- Open-ended research question (R1) = whether there is a relationship between variables
- Closed-ended research question (R2) = focus on a direction of relationship
Hypotheses = a statement about the relationship that we expect to find between variables
- Two-tailed hypotheses (H1) = there is an relationship between two variables, but the
hypotheses does not specify the direction of the relationship.
H1: there is a relationship between level of involvement in video gaming and academic
performance
- One-tailed hypotheses (H2) = predicts the relationship between the two variables.
H2: as time spent in video gaming increases, academic performance decreases.
- Null hypotheses (H0) = there is no relationship between variables.
H0: there is no relationship between level of involvement in video gaming and academic
performance.
Step 2: Quantitative or qualitative research?
Step 3: What are the dependent variables, the independent variables? If asked and wrote down in
hypothesis, what are the moderating and mediating variables? Otherwise these are confounding
variables.
• What is the measurement level of DV?
o Nominal, ordinal, interval, ratio
• What are the levels of IV?
Categorical variables à distinct categories
- Binary or dichotomous variable
= only two categories
- Nominal variable
= more than two categories
- Ordinal variable
= same as nominal, but the categories have a logical order
Continuous variable à distinct score
- Interval variable
= equal intervals on the variable represent equal differences in the property being measured
- Ratio variable
= same as an interval, but the ratio of scores on the scale must also make sense and there is
an absolute zero
Step 4: How to test the hypothesis?
Use different methods that suit the quantitative or qualitative measurement
,“One question will be short description of a study and its hypothesis/hypotheses in which you have to
identify the variables. (The example is shorter than the real exam question. The seminar exercise
from week 3 is very much like this question). This question is worth 15 points”.
First you are going to get a question in which you have to identify the variables and the moderators
and mediators (if they are there).
Dependent variable = the effect measured by the researchers (dus wat veroorzaakt de IV bij de
DV?). The effect you are measuring, the responding variable (“then-part).
You always need to mention the measurement level (nominal(binary), ordinal, interval, ratio)
Independent variable = manipulated by the researcher (wat veroorzaakt dit bij de DV?). The cause
you are testing, the manipulated variable (“if-part).
You always need to mention the levels of the independent variable, often 2 or 3 levels.
è Be aware that it is possible that there are more than one dependent or independent variables!
Example
Video games have come to appeal to people of all ages. In this study, we examine the effect of age
on how people play games. Based on data from 10416 Battlefield 3 players, we found that gaming
performance (measured as “deaths per kill”) of older people is worse that than of younger people. In
addition, speed of play is lower for older people than younger people.
IV: Age (young vs. old), between-subjects
DV1: Deaths per kill (ratio) DV2: Speed of play (ratio)
Moderator: the effect of the independent and dependent variable depends on the level of the
moderating level (which is usually not manipulated) à if we think that the moderator has an effect on
the strength of the relationship between IV and DV.
Example
Sometimes the effect of one IV on the DV depends on the level of the other IV. If you cannot state
what the effect of one IV is, without mentioning the other IV, there is an interaction effect.
Mediation: The effect of the independent and dependent variable is indirect. The IV has an effect on
the mediating variable, which in turn has an effect on the DV. Mediation may be partial or complete.
Example
,When you are reading about a study, always ask yourself: could there be a third variable?
Third variable = explains the apparent correlation (or causation) between the independent and
dependent variable. Because the third variable actually causes changes in both variables.
è You can practice with the seminar of week 3
“The second question will be a research question for which you have to choose the most appropriate
method by discussing advantages and disadvantages of two different methods. We expect you to
describe two different research methods (i.e. not ‘unstructured interview’ and ‘structured interview’).
Your description contains information about the ‘who, what, and how’. Your evaluation and choice
should include aspects of reliability, validity, potential ethical issues, and feasibility. This question is
worth 30 points.”
“In the third question you will be presented with a research question and choice of method. Your task
is to describe how you would conduct the study, taking into account everything you know about what
is important to make this particular method work. Your answer should not be about this method in
general, but clearly applied to the particular case that is presented. This question is worth 20 points.”
“The forth question will be a short description of a study and you are asked to identify
aspects of the study and to evaluate this study. This question is worth 35 points.”
(type of method, variables, evaluate reliability, validity and ethical aspects)
Different research methods
When you are going to describe the method, describe the design, the participants, the materials and
the procedure. Be aware that some methods are more suitable than others.
For example: causal relations can really only be studies in experimental research; people’s opinions
cannot be observed (only their behavior). So if the RQ refers to attitudes and opinions, you should ask
people questions (e.g. in surveys, interviews, focus groups).
Furthermore you have to describe the who, what and how. You need to make this specific e.g. you
state that you will measure ‘perception of attractiveness’ with a set of Likert-scale items, and then give
one concrete example that shows that you think about the constructs, as well as the
operationalization.
When you need to describe a follow-up study you can write sentences such as ‘in order to achieve
….’ à this shows your reasons for certain choices you make.
Quantitative method = research approach based on measurement, counting and statistical analysis.
Designs that are built around the idea that human traits and behaviors can be translated into number
to analyze them.
- Experiments
- Surveys
- Content analysis
, - Meta-analysis
- Simulations
Qualitative method = research approach based on the use of language to understand and report
human behavior. Designs that are built around the idea that human traits and behaviors are best
understood by observing and interviewing people.
- Interviews
- Focus groups
- Case studies
- Observational studies
Triangulation = by combining methods it is possible to neutralize the weaknesses of single methods.
= researches use multiple methods providing multiple perspectives to ensure that they
have a good ‘fix’ on a problem. The use of two or more research methods for the same
research question. If results from different methods agree, researches have greater
confidence in their findings.
If chosen for quantitative research method, what kind of method can you use?
Quantitative research designs are built around the idea that human traits and behaviors can be
translated into numbers to analyze them.
1. Experiment
Is manipulating variables and measure effects. Manipulating one variable to see if another variable
thought to be related to it changes as a result.
Be aware that experimental research has potential issues when it comes to ecological validity.
Advantages:
• allows you to test causality
• identifying variables that have significant causal relationships
• and that have no significant effect on other variables.
Disadvantages:
• they rarely capture the natural environment (ecological isomorphism)
• more sophisticated designs may require large number of participants for extended periods of
time.
Challenges: random distribution/external validity/confounds
Specify the type of experiment
• Between subject design: exposed to only ONE experimental condition. An experimental
design in which subjects participates under only one set of experimental conditions.
• Within subject design: exposed to MORE than one experimental condition. An experimental
design in which subjects participates under more than one set of experimental conditions.
• Pretest-posttest: mostly to measure the degree of change as a result of treatment or
inventions.
o One- group pre-posttest disadvantage: many other variables may not be in the
experimental design.
o Two-Group Pretest-Posttest: to be more certain there is an effect by measuring
changes.
o Two-Group Random Assignment Pretest-Posttest: all attributes have the same
probability of occurring in each group.
• Ex Post Facto design/natural experiment = an ‘after the fact’ experimental design in which
there is no control over experimental conditions.
