This document is an exhaustive summary of all the material provided in the 2020/2021 Data Science Research Methods course. It includes in-depth descriptions of theory from the books Experimental Design (Berger et al., 2018) and Mostly Harmless Econometrics (Angrist et al., 2009) as well as the theo...
Class notes and Summary of materials Data Science Research Methods (JBM020 2020-2021)
All for this textbook (2)
Written for
Technische Universiteit Eindhoven (TUE)
Data Science
Data Science Research Methods (JBM020)
All documents for this subject (4)
Seller
Follow
Lieve12
Reviews received
Content preview
Lieve Göbbels
DS Research Methods (JBM020)
Semester 2, 2020-2021
Data Science Research Methods
Scienti c Method and Experimentation 3
The scienti c method 3
Experimentation and experimental design 3
Important concepts 4
One-Factor Designs and the Analysis of Variance 5
One-Factor Designs 5
Analysis of Variance (ANOVA) 6
Sample Size Determination 8
Sample size determination 8
Normal distribution 8
Binomial distribution 9
ANOVA II - Power 11
One-way ANOVA and power 11
Effect size 11
Sample size determination 11
Multiple Comparisons 12
Multiple comparisons 12
Bonferroni correction 12
Fisher’s Least Signi cance Difference test (LSD) 12
Tukey’s Honest Signi cant Difference test (HSD) 13
Two-Factor Designs 14
Two-way ANOVA with replication 14
Two-factor with no replication and no interaction 15
Introduction to blocking 16
Full Factorial Designs 17
Full factorial designs 17
Estimating effects in 2 factor 2 level experiments 18
Three factors at two levels 19
Number and kinds of effects 19
Main effects with large interactions 19
Choosing levels of factors when measured along continuum 20
Errors of estimates in full factorial designs 20
Fractional Factorial Designs 21
Blocking in full factorial designs II 21
Fractional factorial designs 22
Analysis of fractional factorial designs 23
Response Surface Optimization 24
Response Surface Optimization 24
Optimization steps 24
Regression models 24
, Step 2: Improvement 25
Step 3: Determination (Response Surface Designs) 25
Finding the optimum using CCD or BB estimates 26
Introduction to Econometrics for Data Scientists 27
Econometrics 27
Independence and correlation 27
Regressions 27
Causality and Selection 29
Causality formalized 29
Average Treatment Effect (ATE) 29
Average Treatment effect on Treated (ATT) 29
Selection (bias) 29
Random assignment 30
Potential problems with experiments 31
IV estimation 31
Selection on Observables and Matching 32
Matching estimators 32
Some recaps 32
Selection on observables 33
Matching 33
Different methods 34
Differences-in-Differences Estimation 36
Differences-in-differences estimation 36
Implementation 36
Testing the parallel trends assumption 36
Group-speci c trends and dynamic effects 37
More pre-periods 37
Compositional changes 37
Generalization: synthetic control 37
Regression Discontinuity Design 38
Regression Discontinuity Design (RDD) 38
Sharp RDD 38
Fuzzy RDD 40
Speci cation testing 41
Quiz Questions and Solutions 42
Quiz questions and solutions 42
, Scienti c Method and Experimentation
In short:
• The scienti c method
• Experimentation and experimental design
• Important concepts
The scienti c method
There are three important goals of data science (and beyond):
1. description: provide insight into past events;
2. prediction: provide insight into a (possible) future;
3. explanation/prescription: advise on possible outcomes.
Basic elements of the scienti c method
1. formulate (research) question;
2. perform background research;
3. formulate hypothesis;
4. determine logical consequence of hypothesis;
5. collect observations (conduct experiment);
6. test truth of hypothesis by analyzing observations (statistics);
7. report results;
8. if the hypothesis is not con rmed, go back to 2.
Some of these steps can be linked to the Six Sigma’s DMAIC method (De ne, Measure, Analyze,
Improve, Control):
• 1 can be linked to the De ne phase;
• 4 can be linked to the Measure phase;
• 5 can be linked to the Analyze phase.
So, the Improve and Control phases do not have a direct link. The scienti c method is characterized
by its iterative method.
Experimentation and experimental design
An experiment is an investigation in which the researcher selects the values (levels) of one or more
input (independent) variables and observes the values of the output (dependent) variables. This has
the purpose to get insight in the relationship between dependent and independent variables which is
then often used to optimize the underlying process.
An experimental design is then the aggregation of independent variables, the set of amounts,
settings or magnitudes (levels) of each independent variable, and the combinations of these levels.
So, the core of experimental design is to answer the three-part question:
• which factors should we study?
• how should the levels of these factors vary?
• in what way should these levels be combined?
Sometimes, for examples when analysis is ex post facto (after the data is already collected),
the levels of independent variables cannot be speci ed, because they are already given. Then,
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller Lieve12. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $6.95. You're not tied to anything after your purchase.
