Research Methodology – Video’s
Video 1.1 – The Empirical Cycle
Observation = Collect and group empirical facts.
Theory (induction) = Form a theory based on your
observations.
Prediction = Deduce a hypothesis/prediction from your
theory.
Testing = Run a study to test your hypothesis
Evaluation = What do the study outcomes (the new
observations) mean for the theory?
Video 1.2 – Falsifiability
To reach scientific insights, we follow the empirical cycle. You have to follow the entire cycle,
you can’t take shortcuts.
Some theories cannot be tested. They are in line with every possible observation.
Irrefutable theories
These theories are unfalsifiable. It is impossible to find evidence against the theory.
If a theory is irrefutable:
- We can’t determine which of multiple theories is the correct one
- And we will not be any closer to explaining a phenomenon
A theory can be falsifiable, even if no one has ever made an observation against the theory
The point is that observation against a theory should be possible (e.g. gravity)
Summary:
A good scientific theory has to be falsifiable
It has to be possible to make observations that are not in line with the theory
If a theory is irrefutable, it is unfalsifiable and with that not scientific.
Video 1.3 – Accumulating knowledge
To be able to follow the entire empirical cycle, theories have to lead to falsifiable predictions.
With the test results (of a study) we evaluate the theory
- What if a prediction is confirmed?
- What if a prediction is not confirmed?
Formulate a prediction
- A theory is general and not immediately testable
- We need a concrete prediction to perform a test.
Prediction is confirmed It is virtually impossible to prove a theory because;
- You can never rule out all alternative explanations
- You tested only one concrete prediction and a theory makes an infinite number of
predictions.
Prediction is not confirmed It is virtually impossible to refute a theory with a single study
because;
- Something could have gone wrong
You cannot keep fixing your theory infinitely. At some point you’ll run the risk that your theory
becomes unfalsifiable.
Knowledge accumulates slowly
- One study is no study
, - Evidence accumulates over longer periods of time
- A theory is never ‘proven’: we just hold on to it until something better comes along
Video 2.1 – Personal Experience
Personal experience is not a good source of information because;
1) There is no comparison group
2) There are possible alternative explanations (confounds)
In scientific research we make systematic comparisons under controlled circumstances to
avoid these problems.
Video 2.2 – Judging Claims
3 Claims
1) Frequency claims: Describe a particular rate or degree of a single variable
2) Association claim: Association claims argue that one level of a variable is likely to be
associated with a particular level of another variable
3) Causal claim: Argue that one of the variables is responsible for changing the other.
4 Validities
Validity = Whether a claim is reasonable, accurate and justifiable
1) Construct validity: How well is a conceptual variable (or a construct) operationalized?
2) External validity: How well do the results of the study generalize to other
people/context?
3) Statistical validity: To what extend are a study’s statistical conclusion as accurate and
reasonable?
4) Internal validity: Can we eliminate alternative explanations for an association?
Third-variable criterion = Something else that explains the association
Video 3.1 – Ethical treatment of participants
Participating in scientific research should be voluntary.
Personal data can’t be shared/stored without permission Anonymity/confidentiality
Prior to the study participants sign an informed consent.
After the study, the researchers describe the nature of the deception and why it was
necessary.
Debriefing.
IRB The Institutional Review Board weighs pros and cons based on the relevant ethical
principles in rules.
Video 3.2 – Research Misconduct
Plagiarism representing the ideas or words of other as one’s own
Falsification Influencing a study’s result by changing data or influencing participant to act
in line with the hypothesis
Fabrication Making up data that fit the hypothesis
Grey area?
- When is something an innocent mistake and when is something misconduct?
Has to do with intention, which is hard to determine.
Sometimes it suffices to conclude that the research (not the researcher) is unreliable
Scientific misconduct can sometimes be spotted in articles.
It is hard to distinguish between misconduct and grey areas.
It takes a lot of time to clean up the consequences of scientific misconduct.
Video 4.1 – Operationalizing
To create a concrete variable to measure
Constructing a variable:
1. Choose a structure (continuous or categorical)
2. Think about what the theoretical core of the construct is, and how that can be
expressed in human behavior; how could this behavior manifest itself?
Video 1.1 – The Empirical Cycle
Observation = Collect and group empirical facts.
Theory (induction) = Form a theory based on your
observations.
Prediction = Deduce a hypothesis/prediction from your
theory.
Testing = Run a study to test your hypothesis
Evaluation = What do the study outcomes (the new
observations) mean for the theory?
Video 1.2 – Falsifiability
To reach scientific insights, we follow the empirical cycle. You have to follow the entire cycle,
you can’t take shortcuts.
Some theories cannot be tested. They are in line with every possible observation.
Irrefutable theories
These theories are unfalsifiable. It is impossible to find evidence against the theory.
If a theory is irrefutable:
- We can’t determine which of multiple theories is the correct one
- And we will not be any closer to explaining a phenomenon
A theory can be falsifiable, even if no one has ever made an observation against the theory
The point is that observation against a theory should be possible (e.g. gravity)
Summary:
A good scientific theory has to be falsifiable
It has to be possible to make observations that are not in line with the theory
If a theory is irrefutable, it is unfalsifiable and with that not scientific.
Video 1.3 – Accumulating knowledge
To be able to follow the entire empirical cycle, theories have to lead to falsifiable predictions.
With the test results (of a study) we evaluate the theory
- What if a prediction is confirmed?
- What if a prediction is not confirmed?
Formulate a prediction
- A theory is general and not immediately testable
- We need a concrete prediction to perform a test.
Prediction is confirmed It is virtually impossible to prove a theory because;
- You can never rule out all alternative explanations
- You tested only one concrete prediction and a theory makes an infinite number of
predictions.
Prediction is not confirmed It is virtually impossible to refute a theory with a single study
because;
- Something could have gone wrong
You cannot keep fixing your theory infinitely. At some point you’ll run the risk that your theory
becomes unfalsifiable.
Knowledge accumulates slowly
- One study is no study
, - Evidence accumulates over longer periods of time
- A theory is never ‘proven’: we just hold on to it until something better comes along
Video 2.1 – Personal Experience
Personal experience is not a good source of information because;
1) There is no comparison group
2) There are possible alternative explanations (confounds)
In scientific research we make systematic comparisons under controlled circumstances to
avoid these problems.
Video 2.2 – Judging Claims
3 Claims
1) Frequency claims: Describe a particular rate or degree of a single variable
2) Association claim: Association claims argue that one level of a variable is likely to be
associated with a particular level of another variable
3) Causal claim: Argue that one of the variables is responsible for changing the other.
4 Validities
Validity = Whether a claim is reasonable, accurate and justifiable
1) Construct validity: How well is a conceptual variable (or a construct) operationalized?
2) External validity: How well do the results of the study generalize to other
people/context?
3) Statistical validity: To what extend are a study’s statistical conclusion as accurate and
reasonable?
4) Internal validity: Can we eliminate alternative explanations for an association?
Third-variable criterion = Something else that explains the association
Video 3.1 – Ethical treatment of participants
Participating in scientific research should be voluntary.
Personal data can’t be shared/stored without permission Anonymity/confidentiality
Prior to the study participants sign an informed consent.
After the study, the researchers describe the nature of the deception and why it was
necessary.
Debriefing.
IRB The Institutional Review Board weighs pros and cons based on the relevant ethical
principles in rules.
Video 3.2 – Research Misconduct
Plagiarism representing the ideas or words of other as one’s own
Falsification Influencing a study’s result by changing data or influencing participant to act
in line with the hypothesis
Fabrication Making up data that fit the hypothesis
Grey area?
- When is something an innocent mistake and when is something misconduct?
Has to do with intention, which is hard to determine.
Sometimes it suffices to conclude that the research (not the researcher) is unreliable
Scientific misconduct can sometimes be spotted in articles.
It is hard to distinguish between misconduct and grey areas.
It takes a lot of time to clean up the consequences of scientific misconduct.
Video 4.1 – Operationalizing
To create a concrete variable to measure
Constructing a variable:
1. Choose a structure (continuous or categorical)
2. Think about what the theoretical core of the construct is, and how that can be
expressed in human behavior; how could this behavior manifest itself?
