Research Methods for Environmental Sciences
L1: Introduction
Lecture contents
- What is scientific research?
- Empirical cycle
- Regulative cycle
What is scientific research?
There are many ways to answer questions about the world.
- Syllogisms and logic Example of syllogism: all Chinese students are smart; Chen is Chinese; Chen is smart. Logic
consists in inferring (given some assumptions) a conclusion from propositions.
In order for logic to work correctly premises must be respected. Logic can be very powerful but also has to be handled with
care: in a scientific study you’re always assuming the truth of your assumptions. Nevertheless, if you cannot proof the truth of
your premises from empirical data, then you’re making ‘castles in the sky’. Then, assumptions have to be very well clarified
and so logic can be used.
- Belief and authority truth by faith or by accepting the judgment of esteemed others. We are embedded in cultures in which
we have authorities. In science it has to be the opposite.
- Manners Conventions that are socially accepted or assumed. Examples: in chemistry you assume a certain precision of
data (e.g. +- 0.5); To analyse e.g. social networks through the economic perspective, economists will use economic theories,
while another specialist would choose other models. Thus manners influence the way we answer to the questions about the
world.
- Traditions truth value as a function of longevity and inclusion in cherished cultural system.
Thus, when answering our questions on the world we have to handle our answers with care, because intrinsically will be based on
Logic, believes and authority and tradition. But science has to look for evidence. Many traditions cannot be tested meaningfully.
What do we need for science?
- Access to the world Otherwise you cannot collect data, so you can no longer conduct an empirical scientific study. But we
have no complete and immediate access to the world! Moreover, we also have to understand how this world is shaped by
our systems (cultural, biological, psychological…);
- A model of the world that fits our purpose You need to have an idea of how to approach your problem, how to frame it,
select the variables you are interested in. As negative as those terms sound, it’s only through reduction, categorization and
framing that we can gain scientific knowledge
Moreover, you have to ask yourself: does the model take into account all the different variables? What happens if it doesn’t?
In order to have a model you need to have access to the world and some ideas on how the world works. And it has to fit the
purpose. E.g. a scientist arrives on an island and would start to collect EVERY collectable data (n of shells, number of
trees…). But then there is no way you can make a meaningful model out of this data. So at the end you need to have an idea
on how to prioritize data.
- Instruments that are good enough How do you actually measure the things that matter? You should have appropriate,
reliable and accurate instruments. Otherwise, you still won’t be able to get useable data. And good instruments are not so
easy/always accessible.
Scientific research is:
- Systematic following systematic, recognized procedures for doing research, i.e. a methodical way of arriving at research
questions, operationalizing key constructs, collecting data, modelling those data, making inferences from the estimates
1
, produced by applying the statistical model to the data etc. Procedures are the way to compose your research and everything
needs to be specified, needs to make sense as a procedure.
- Concerned about reliability: Throughout this systematic application of the scientific method, there is a concern for the
precision/consistency of measurements (if I measure a stable characteristic of a research object with the same instrument
several times in a row, I would expect similar results)
- Concerned about Validity: Throughout the research process, there is concern for the validity/accuracy, both of the
measurements (do I really measure what I want to measure? measurement validity), conclusions (can I support my
conclusions on the research units I’ve studied, given my design? internal validity) and inferences about the population (can I
generalize my findings to my theoretical population and to other times and contexts? external validity)
- Theory-dependent: Science attempts to summarize findings in propositions/theories. And doing so, it builds on other theories.
Since you always position yourself within the existing scientific literature and are open about your contributions, science works
in a cumulative way to generate increasingly refined theories. Every theory frames and classifies and categorizes the world in a
certain way and you’ll need to be explicit about that. E.g. bored guy: the fact that he doesn’t take notes means he’s bored. Or
maybe it’s the opposite!
- Known objectivity: Truly objective research is impossible (see next slide), but we try to not let our own prejudices and norms
and values cloud our observations and conclusions and we try to reflect well on how the subjective may still have influenced
what you did.
- Humble: Science gives us great tools to reason, explore, design, explain, predict, act and communicate, but it’s also easy to
become overconfident and lose sight of the limitations of your research. We can only get new insights about the world when we
model and classify and make inferences. We always put on a certain pair of glasses and use particular tools, but although they
might be very well-designed they are still selective and reductive tools. A good scientist is aware of those limitations and
remains humble.
We must be aware of the models’ assumptions and about how definitive/objective are the results Always be explicit about
assumptions!
Objective means:
- Non biased (i.e.: not prejudiced, influenced);
- Non-normative (i.e.: results do not state values or opinions)
This is not possible, so good science is transparent!
Does bias require intent? Most often, scientists don’t distort out of malicious intent, but they unconsciously privilege certain models
over others for non-scientific reasons such as familiarity, or they are more likely to interpret results in a way that confirms their own
beliefs. Science is not only about minimizing subjective biases, but also (and more importantly) about recognizing that we’re
inescapably subjective and about constantly and systematically attempting to quantify, transparently report and critically reflect on
possible biases.
Research can be descriptive, correlational or explanatory.
- Descriptive as the term suggests, it’s about describing the level or distribution or properties of one, or a series of, thing(s).
E.g. How large is the population of storks in Netherlands?
- Correlational Investigates the strength and direction of the relationship between two (sets of) things. E.g. Is the number of
storks related to the number of babies?
- Explanatory Tries to explain why things are as they are by testing whether, to what extent, or how other things causally
influence the thing we are interested in Does something relate to something else. And how does it relate? Why does this
cause that?
Note that correlational is not about effects (one thing making another thing move) but about associations (two things moving
together (either because they move each other, or because they are both moved by something else)). Explanatory questions are
either about whether/to what extent some things make other things move (quantification of causal effects) or about how these
things make the other things move (focus on mechanisms). Note that this is not consistent with the way Kumar explains this in his
handbook! (check the document with comments about the handbook for more information!)
2
,Also note that ‘ explanatory’ figures in a second typical classification of research, namely exploratory vs explanatory. This is an
entirely different typology and research that is explanatory according to one typology is not necessarily also explanatory according
to the other one. See the document with comments on Kumar for more information.
Primary data vs. secondary data You can perform research based on data that you collect yourself (primary, from e.g. laboratory
research) – or data that has already been collected by other researchers or organisations (secondary, desk research, census
data...). In both cases you are fully responsible to understand the quality and nature of the data. Secondary data can be tempting,
since you don’t have to do the work. But actually if you have very big datasets where data have been merged, you should ask
yourself, where do these come from? How where they collected? How can I combine them? What do they measure and from which
sample?
Sources of data: empirical vs. non empirical data
- Empirical knowledge acquired through direct observation, based on measurements of variables.
- Non-empirical knowledge not directly observed but derived from logic, inferred from logic.
We focus on empirical research in this course. However, be aware that empirical research cannot solely rely on ‘bare facts’, but
necessarily also uses ‘non-empirical’ tools such as logic. Conversely, there is no purely non-empirical science either. Logic works
with premises and these are, at least at some level, informed by our observations of the world.
Hypothesis: when is it good? When you can test it. Examples:
- The population of storks has increased over the last decade good hypothesis since it can be tested;
- Parallel canals do not cross can be derived logically but is not a question for empirical science. It is true by definition, but
there are conditions under which definitions fail. It’s not imperative.
- Fresh drinking water should be available to all people in the world it is not a hypothesis, it’s a normative statement. It cannot
fruitfully be investigated using the scientific method. Of course, what you can do is explain why you think it should be available
(because it influenced people’s health positively) and then turn it into a question about the effects of availability of fresh
drinking water on health, which can be answered by using the scientific method.
- A rainforest is a forest with an annual rainfall of at least 1750 mm it’s a definition. How you define and classify things can be
informed by science, but it’s not a scientific question.
- There is a pink invisible unicorn it is a hypothesis, but we lack the means to test it. Invisible stuff cannot be observed:
empirical claims must be testable and falsifiable.
Qualitative and quantitative
Qualitative approach:
- Describe the world in words, focusing on textual descriptions rather than numerical variables;
- Open flexible and relatively unstructured, i.e. no strict standardization in measurements. Flexible in changing instrumentation or
sampling mid-way as a response to new insights. Iterative cycles of sampling, measurement and analysis instead of a pre-
programmed procedural path;
- Recognizes and tries to manage subjectivity Constructivist philosophical background, recognizing that each of us construct
our own truths and to understand reality, we need to examine how we construct our version of it. In qualitative research, critical
examination (reflexivity) of the subjectivity of the researcher him-/herself has a strong tradition.
- Identifies unique patterns only in few cases, with limited claims to generalizability: it focus on understanding a situation /
phenomenon from an insider’s point of view, focusing on the idiosyncratic specifics, without strong claims to generalizability to
other cases.
- Useful for exploring the nature of a phenomenon, and indeed more commonly used for more explorative research. Definitely
possible for theory-testing.
Quantitative approach:
- Measures characteristics of the world numerically through variables;
- Standardized, structured, rigid procedures (to manage subjectivity) instruments and procedures are fixed and there are
highly structured process of data collection and sampling. E.g. if we decide to ask people questions, we’ll ask the exact same
questions to all our respondents, formulated in the same way and with the same answer options, presented in the same order
under similar conditions
- Goes for descriptions of objective reality that are valid and reliable Focus is not on multitude of ways that subjects construct
reality, but on describing an underlying, objective reality that we can learn about by applying precise and well-calibrated
instruments;
3
, - Tries to find general patterns in large sample size studies that apply to many cases Research units with the same measured
scores are interchangeable and the goal is to find general patterns that can be generalized statistically
- Especially good for investigating the prevalence or extent of a phenomenon (how many, what percentage of cases?) and effect
sizes (how strongly, on average, does X influence Y, or are P and Q associated with each other?).
- Quantitative and qualitative research are usually taught as opposites, but actually there is no quantitative without qualitative
investigation first. They are complementary. Qualitative and quantitative are ends on a continuum and a lot of actual research
falls somewhere in moderate positions along that range. Both approaches can also be fruitfully combined.
They both:
- Start from models of the world either explicitly or implicitly/tacitly;
- Are grounded in empiricism and are analytical even though quantitative is sometimes connected to ‘analytical’ and
qualitative to ‘interpretative’, both require analytical procedures and interpretation of data and results, both are grounded in
empiricism (i.e. both acknowledge that data collected in the real world are needed to get to new scientific knowledge)
- Are concerned with reliability and validity (quality) Even though reliability and validity tend to be more formalized in
quantitative research, they are a concern in both;
- Describe, explore, explain While qualitative research is typically tied to more descriptive and exploratory studies and
quantitative is tied more to explanatory studies, they can both do all of these.
- Informed by theoretical knowledge both start from theory and produce output that refines theory. In other words, they both
share some fundamental properties that make them legitimate scientific approaches.
- Flexible While qualitative tends to be more flexible, in practice quantitative can be the same (e.g. changing aspects of data
collection during fieldwork). The same goes for measurement levels (they can all use variables of either level) and sample size
(few or many research units)
All science is about reduction world is too complex, so we have to cut it down into parts and recognise these pieces as a
representation of the world. Qual and quat. Are about the reduction of the world into manageable pieces of representations-
Why is Interdisciplinarity important? The relationship between natural and social sciences is very important at this University,
where research done here is aimed at implementing innovative ideas in society. It is not always straightforward that good ideas can
be successfully implemented.
Failure of technocratic approach: technological innovations and/or solutions do not always fit in the socio-cultural reality (and vice
versa)
Example: Vacuum toilet to separate urine from feces. This is important, because from pure urine, phosphate can be extracted,
which is an important component of fertilizer. As you may know, there is a phosphate shortage in the world, which also threatens
the world’s food supply. 30 toilets were installed and the study failed to effectively separate the feces from urine. Why? Men refused
to sit down when peeing. So, technological innovations often come from the technical sciences. However, successful
implementation of these innovations often requires social science research. Multidisciplinary or interdisciplinary research is
important for inventions to be effectively implemented.
Even when the problem seems primarily technical, your approach tends to be socially influenced. The way you frame a problem
can have far-reaching consequences. Say you are asked to find out ways to make SUVs more efficient and safer. How do you start
looking at the problem?
What is efficiency? Just fuel efficiency? How about space efficiency? Cost efficiency?
Applied vs. fundamental science Research always aims to collect knowledge. In applied research this knowledge serves the
purpose of helping to solve a practical problem. Whereas fundamental research is aimed at gaining knowledge purely to improve or
expand the existing knowledge.
By the way: both kinds of research are important. A lot of research that starts out as fundamental research in the end happens to
result in something very helpful for solving real-world practical problems. The distinction between fundamental and applied is not
about their actual usefulness for solving practical problems, but their direct goals/objectives.
(About 80% of the research performed at Wageningen University is practice-oriented and sponsored by companies like Melkunie
4