Samenvatting Correlationeel & Experimenteel onderzoek
Literatuur Correlationeel week 1
3. Three Claims, Four Validities: Interrogation Tools for Consumers of Research
Variables
- A variable, as the word implies, is something that varies, so it must have at least two levels, or
values.
- A constant is something that could potentially vary but that has only one level in the study in
question
- A measured variable is one whose levels are simply observed and recorded
- A manipulated variable is a variable a researcher controls, usually by assigning study participants to
the different levels of that variable
Each variable in a study can be referred to in two ways. When researchers are discussing their
theories and when journalists write about their research, they use more abstract names, called
constructs or conceptual variables. When testing hypotheses with empirical research, they create
operational definitions of variables, also known as operational variables. To operationalize a concept
of interest means to turn it into a measured or manipulated variable.
Construct, conceptual variable (theoretisch concept): the name of the concept being studied
‘Satisfaction with life’
Conceptual definition: a careful, theoretical definition of the construct ‘A person’s cognitive
evaluation of his or her life’
Operational definition, operationalization: how the construct is measured or manipulated in an
actual study Five questionnaire items on the Satisfaction with Life scale
Variable
Three claims
A claim is an argument someone is trying to make
- Frequency claims describe a particular rate or degree of a single variable. Frequency claims are
easily identified because they focus on only one variable – such as amount of texting. In addition, in
studies that support frequency claims, the variables are always measured, not manipulated.
- Association claims argues that one level of a variable is likely to be associated with a particular level
of another variable. Variables that are associated are sometimes said to correlate, meaning that
when one variable changes, the other variable tends to change, too. This type of study, in which the
variables are measured and the relationship between them is tested, is called a correlational study.
- Causal claims go even further, arguing that one of the variables is responsible for changing the other
Validity
Validity refers to the appropriateness of a conclusion or decision, and in general, a valid claim is
reasonable, accurate and justifiable
- Construct validity refers to how well a conceptual variable is operationalized
- External validity: how well the results of a study generalize to, or represent, people or contexts
besides those in the original study
- Statistical validity, also called statistical conclusion validity, is the extent to which a study’s statistical
conclusions are precise, reasonable and replicable. To evaluate statistical validity, we start with the
,point estimate, next we ask about the precision of that estimate. For a frequency claim, precision is
captured by the confidence interval, or margin of error of the estimate. The confidence interval is a
range designed to include the true population value a high proportion of the time
Three criteria for causation
- Covariance: the extent to which two variables are observed to go together, is established by the
results of a study
- Temporal precedence means that the method was designed so that the causal variable clearly
comes first in time, before the effect variable
- Internal validity: the study’s method ensures that there are no plausible alternative explanations for
the change in B; A is the only thing that changed
Usually, to support a causal claim, researchers must conduct a well-designed experiment, in which
one variable is manipulated and the other is measured. In the context of an experiment, the
manipulated variable is called the independent variable and the measured variable is called the
dependent variable
Researchers usually cannot achieve all four validities at once in an experiment, so they prioritize
them. Their interest in making a causal statement means they may sacrifice external validity to
ensure internal validity
5. Identifying Good Measurement
Three common types of measures
- A self-report measure operationalizes a variable by recording people’s answers to questions about
themselves in a questionnaire or interview
- An observational measure, sometimes called a behavioral measure, operationalizes a variable by
recording observable behaviors or physical traces of behaviors
- A physiological measure operationalizes a variable by recording biological data, such as brain
activity, hormone levels, or heart rate
Categorical vs. quantitative variables
Operational variables are primarily classified as categorical or quantitative. The levels of categorical
variables, are categories (also called nominal variables). In contrast, the levels of quantitative
variables (also called continuous variables) are coded with meaningful numbers
Three types of quantitative variables
- An ordinal scale of measurement applies when the numerals of a quantitative variable represent a
ranked order. For example ‘top 10 best-selling books’
- An interval scale of measurement applies to the numerals of a quantitative variable that meet two
conditions: first, the numerals represent equal intervals (distances) between levels, and second, there
is no ‘true zero’ (a person can get a score of 0, but the 0 does not literally mean ‘nothing’)
- A ration scale of measurement applies when the numerals of a quantitative variable have equal
intervals and when the value 0 truly means ‘none’
Introducing three types of reliability
- When a measure has test-retest reliability, a study participant will get pretty much the same score
each time they are measured with it
, - With interrater reliability, consistent scores are obtained no matter who measures the variable
- With internal reliability (also called internal consistency), a study participant gives a consistent
pattern of answers, no matter how the researchers phrase the question
Scatterplots can show interrater agreement or disagreement
Measurement validity of abstract constructs
Construct validity can be measured by two subjective ways to assess validity:
- A measure has face validity if it is subjectively considered to be a plausible operationalization of the
conceptual variable in question. Researchers might check face validity by consulting experts
- Content validity also requires knowledge of the conceptual definition: a measure must capture all
parts of a defined construct
Construct validity can also be measured by three empirical ways to assess validity
- Criterion validity evaluates whether the measure under consideration is associated with a concrete
behavioral outcome that it should be associated with, according to the conceptual definition.
Criterion validity is especially important for self-report measures because the correlation can indicate
how well people’s self-reports predict their actual behavior. Criterion validity provides some of the
strongest evidence for a measure’s construct validity. A way to gather evidence for criterion validity is
to use a known-groups paradigm, in which researchers see whether scores on the measure can
discriminate among two or more groups whose behavior is already confirmed.
- Another form of validity evidence is whether there is a meaningful pattern of similarities
(convergent validity) and differences (discriminant validity) among related measures. A self-report
measure should correlate more strongly with measures of similar constructs than it does with those
of dissimilar constructs.
6. Surveys and Observations: Describing What People Do
- Survey/poll: a method of posing questions to people online, in personal interviews, or in written
questionnaires
- Open-ended questions allow respondents to answer any way they like
- Forced-choice questions in which people give their opinion by picking the best of two or more
options
- In another question format, people are presented with a statement and are asked to use a rating
scale to indicate their degree of agreement: Likert scale
Writing well-worded questions
- Leading questions: wording leads people to a particular response
- Double-barreled questions: it asks two questions in one
- Negatively worded questions: whenever a question contains negative phrasing, it can cause
confusion, thereby reducing the construct validity of a survey or poll
- Question order: can also affect the responses to a survey. A specific type of order effects if priming
effects, where exposure to a particular image, word, or feeling shapes how we think and feel in the
immediate aftermath
Sometimes people use shortcuts
Response sets, also known as non-differentiation, are a type of shortcut people can take when
answering survey questions. People might adopt a consistent way of answering all the questions –
