Research Methodology and Descriptive Statistics (RMDS)
Test 1: units 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 & 23
Unit 1: ‘what is empirical research?’
Key terms:
◼ Research question ◼ Ex-post evaluation (effect/impact
◼ Design and (cycle of) decision- research)
making ◼ Wheel of science / empirical cycle
◼ Problem & need analysis ◼ Deduction
◼ Ex-ante evaluation ◼ Induction
◼ Confirmation bias
- Explain what is meant with empirical research and which steps can be taken according the
wheel of science
- Differentiate between induction and deduction
- Recognize and explain the steps that can be taken in the decision-making process
- Explain the relationship between systematic decision making and systematically answering
empirical questions
- Recognize and mention examples of confirmation bias
- Explain how systematic empirical research helps avoiding confirmation bias
Starting point: before procedure, only know the topic and a topic is not a research question
1). Find books and articles (Google scholar, Web of Science Database)
Some observations: you need to understand a lot of words (concepts) to grasp the relevance of some
articles. You need to understand words in the context of research methods (validity, reliability etc.)
2). Getting the articles
Pay walls, repositories, the UT library. Only refer to articles you have actually read. No indirect
citations.
3). Reading an article
- Always read the abstract first, sometimes you don’t need to read further
- General structure of empirical articles (intro and RQ, theory, design, data analysis,
conclusion). Knowing that structure you know what you need to read.
Empirical research = systematically answering empirical questions, using observations
◼ Systematically: excluding possibility that other answers are better than the answer we give
◼ Empirical: about things we can observe
All these things are not about your own thinking, not answering questions ourself. It is relying on
expertise of others or traditions. Empirical questions are answered by thinking & observing (= all kind
of data collecting), so we can get systematically answers that are better than other answers.
1
, Research Methodology and Descriptive Statistics (RMDS)
Test 1: units 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 & 23
Procedure thinking & observing:
Scientific literature
Thinking Theory Think about possible answers to the questions, talk to
each other, read literature
Planning Research design Decide how we can test what we think, how we
carefully study our thinking is correct. Test the idea.
Observe Data collection On base of the planning, we observe things
Analyzing Data analysis Answer to our questions
The wheel of science / empirical cycle
Not a logistical sequencing of steps, anything goes a way to defend conclusions; logic
Research strategy
Deduction = the process of starting with theory and then thinking how we can test the theory
Induction = the process of starting with data and then try to arrive to conclusions on the bases of data
Example:
We often start with ‘thinking’ (for example about possible causes of some problem). This thinking
often starts with general idea about why things happen generally and are made more specific in the
context of a specific topic. The more specific expectations are then tested using data. (= deduction)
Sometimes it is argued, we better start with ‘observing’. We observe a specific phenomenon, compare
it with several other yet similar phenomena and on the basis of this comparison arrive at some general
idea about why things happen. (= induction)
Reasons why it is sometimes better to start with observation instead of thinking:
- Because we do not have a CLUE where to start with our thinking
- Because we have so many data, why should we start with thinking, better let the computer find
out (the ‘big’ data challenge)
2
, Research Methodology and Descriptive Statistics (RMDS)
Test 1: units 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 & 23
Unit 2: ‘what are clear research questions?’
Key terms:
◼ Research questions ◼ Descriptive
◼ Normative ◼ Unit (of analysis)
◼ Conceptual ◼ Variable (attributes/values)
◼ Empirical ◼ Setting
◼ Explanatory
- Identify units, variables (with their attributes/values) and settings in clearly formulated
descriptive and explanatory empirical research question
- Distinguish empirical questions from normative questions and from questions about concepts
- Differentiate between explanatory empirical questions from descriptive empirical questions
Research questions
Al proper (empirical) research starts with a research question (= a general idea of what you hope to
learn from your research). There are three main types of research questions: normative, conceptual and
empirical questions. Conceptual and normative aspects are relevant too when answering empirical
questions, but this is not what you want to find out. After the research question we do research. All
research questions have their own methods. Research methods aim at avoiding confirmation bias and
other sources of mistakes in all types of research. On the base of that research, we come to an answer.
Three main types of research questions
Normative question (‘ought’): ‘what should be the case’
- Often start with ‘should we..’, ‘is it justifiable’
- About good/bad, right/wrong, justifiable
- Not asking for ‘legal facts’ (not ‘is it allowed to steal a car?’)
- Procedure: cannot be answered using observations only, own observations are not necessary
o ‘Should we introduce the death penalty (in order to reduce crime)?’
▪ Does the introduction of the death penalty lead to a reduction of crime
(empirical question)
▪ Is the introduction of the death penalty acceptable? (normative question)
Example: ‘should we allow euthanasia for demented people?’, ‘to what extend should we reduce the
‘autonomy’ of ‘mentally challenged’ people of their own benefit’, ‘should we prevent the effects of
climate change on future generations?’, ‘is eating fast food bad for your health?’, ‘are free markets
better for society than regulated markets?’
Conceptual question (‘meaning’): ‘what does it mean?’
- Often starts with ‘what is..’
- Often ‘just’ based on agreement
- Procedure: no observations, cannot be answered using observations
Example: ‘what is democracy?’, ‘what is intellectual efficiency?’, ‘what do we mean with the ‘death
penalty’?’
3
, Research Methodology and Descriptive Statistics (RMDS)
Test 1: units 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 & 23
Empirical question (‘is’): ‘what is or will be, and why?’
- Questions that can only and fully be answered using observations, will be called ‘empirical
questions’
- Procedure: own observations and thinking are both necessary
Example: ‘how many people support euthanasia for demented people?’, ‘is democracy creating more
economic growth than dictatorships?’, ‘which apps help people to manage their work/life balance’
There are two main types of empirical research questions:
Descriptive questions = empirical questions about description, describing a characteristic
Example: ‘what is the shape of the earth’, ‘which countries still have the death penalty?’, ‘what do
people think about cause and effect of relationships’, ‘what reasons do people give for the effect?’
Explanatory questions, or causal = empirical questions are about causes and effects, cause and effect
relationship.
Example: ‘which apps help people to manage their work balance’, ‘does the death penalty reduce
crime?'
However also predictive questions (which are not causal, but also not simply descriptive) (example:
predict number of drownings, related to weather)
Many explanatory questions do have two variables, however
→ not all questions with two variables are causal
Example: ‘do countries with a large Muslim population more frequently have the death penalty than
countries with other religious majorities?’ (descriptive)
→ other explanatory questions do not explicitly refer to two variables
Example: ‘why did some countries abolish the death penalty, while other countries still have it?
(explanatory) it is not referring to one of the possible causes, it ask for causes more generally
Structuring a research question
General guidelines for formulating ‘empirical research questions’
Most of the time you have a topic, this topic is too abroad (crisis, war, personality). You start with a
topic, but you know is too abroad. So, what you need is a question, what is relevant and answerable,
but there are a lot of questions you can ask about this specific topic. Four pieces of advice can guide
you from a topic to a question:
- Context: you have to think about, where the question will come from, where it makes sense to
answer that question, to ask that question. What is the context you working?
- Causal or not?: what type of question are you asking in this context?
- Variable/units/setting: expresses the fact that all empirical research questions include variable,
units and setting. Often it is forgotten, and the questions become vague and not answerable
- Answerable?: do we have the data, and the methods and the theories to answer the questions.
4
