This document is a summary of the class notes that are also in this Stuvia. It is more organized and contains all main elements of the lectures and reading material.
DATA SCIENCE
RESEARCH METHODS
LECTURE 1
Topics: course introduction, Scientific Method, Sample Size Determination, and ANOVA
p-value: highest significant value for which we accept H 0. If p<α → reject H 0.
Type I error α : reject H 0 when it is True.
Type II error β : accept H 0 when it is False.
The best combination of α and β is situation-specific.
One-Factor Design: Studies the impact of a single factor, Y =f ( X , ε ) for factor X on Y .
Replicated experiment: there is more than one data point at each level of the factor.
Replicates: number of rows, different values of Y .
Levels: number of columns, different levels of X .
Total outcomes: # replicates × # levels
Types of means:
Column mean: Sum of all values in the column divided by the number of replicates.
Grand mean: Sum of all data points divided by the total outcomes, RC ór sum of all column
means divided by the number of levels, C .
Least squares: optimal estimation that minimizes the sum of the squared differences.
Total Sum of Squares (TSS): sum of the squared difference between each data point and the grand mean.
Sum of Squares Between Columns (SSBc): sum of the squared difference between each column mean and
the grand mean, multiplied by R .
Sum of Squares Within Columns (SSW): sum of the squared difference between each data point in a
column and that column mean.
TSS=SS B c + SSW
If SSW ≈TSS → factor does not explain much.
If SSW /TSS ≈ 0 → factor has big influence.
MS=SSQ/df ; Mean square is the Sum of Squares divided by the degrees-of-freedom.
Unbiased estimate of population variance → use df instead of n .
E [ MSW ]=σ under constant variance assumption.
2
E [ MS Bc ]=σ 2 +V 2 with V col =[ R / ( C−1 ) ] ∙ ∑τ 2j .
E [ MS Bc ] ≠0 → true column means might not be equal, and sample error leads to difference in
column means.
F-statistic: way to find evidence of affects.
F calc >1 → evidence that V col ≠ 0 thus X affects Y . Evidence is not a final conclusion.
, F calc ≤1 → no evidence that X affects Y .
F calc ≫1 → reject H 0. Rejection means column means are different.
SLIDES LECTURE 1
Three goals of Data Science: Description, Prediction and Explanation.
Different types of Analytics:
1. Descriptive Analytics: insight into the past
2. Predictive Analytics: understanding the future
3. Prescriptive Analytics: advice on possible outcomes
Scientific Method: has an iterative nature.
Six Sigma: disciplined, data-driven methodology for process
improvement. Uses DMAIC cycle.
Key Insights:
Identify the three data science goals.
Scientific method is an iterative process.
Not planning an experiment will not result in the wanted outcomes.
Experiment can have more factors, that can have more than 2 levels.
Six Sigma incorporated several aspects of the scientific method.
X−μ 0
Normal distribution gives test statistic T = if σ is known. Reject if |T |> z α / 2 or P H (|T|>|T 0|) <α .
σ /√ n 0
(
Confidence interval: reject if it doesn’t fall in the interval, x−z α /2
σ
√n
, x + z α /2
σ
√n ) .
Minimal sample sizes:
Normal distribution:
( )
2
z α /2 σ
o ONE SAMPLE CASE: if σ is known → n ≥ with E maximal absolute error.
E
o ONE SAMPLE CASE: if σ is unknown → same as with known but use worst case σ .
Round up to strictly satisfy the inequality.
( ) (σ + σ )
2
z α /2 2 2
o TWO SAMPLE CASE: equal sample sizes and variances known→ n ≥ 1 2
E
Binomial distribution:
( )
2
zα/ 2
o ONE SAMPLE CASE: n ≥ ^p (1−^p )use worst case ^p or upper/lower bound
E
Analyze p → p ( 1− p ) on [ 0,1 ]
( )
2
z
o TWO SAMPLE CASE: equal sample sizes n ≥ α/ 2 ( ^ p 1 ( 1− ^
p 1) + ^ p 2) )
p 2 ( 1− ^
E
Power analysis:
Normal distribution H 0 : μ=μ0 :
o (
β=Φ z α / 2−
δ √n
σ ) (
−Φ −z α /2 −
δ √n
σ )
( )
2 2
δ √n (z +z ) σ
o Φ −z α/ 2− small compared to β → n ≈ α/ 2 β
σ δ
2
Voordelen van het kopen van samenvattingen bij Stuvia op een rij:
Verzekerd van kwaliteit door reviews
Stuvia-klanten hebben meer dan 700.000 samenvattingen beoordeeld. Zo weet je zeker dat je de beste documenten koopt!
Snel en makkelijk kopen
Je betaalt supersnel en eenmalig met iDeal, creditcard of Stuvia-tegoed voor de samenvatting. Zonder lidmaatschap.
Focus op de essentie
Samenvattingen worden geschreven voor en door anderen. Daarom zijn de samenvattingen altijd betrouwbaar en actueel. Zo kom je snel tot de kern!
Veelgestelde vragen
Wat krijg ik als ik dit document koop?
Je krijgt een PDF, die direct beschikbaar is na je aankoop. Het gekochte document is altijd, overal en oneindig toegankelijk via je profiel.
Tevredenheidsgarantie: hoe werkt dat?
Onze tevredenheidsgarantie zorgt ervoor dat je altijd een studiedocument vindt dat goed bij je past. Je vult een formulier in en onze klantenservice regelt de rest.
Van wie koop ik deze samenvatting?
Stuvia is een marktplaats, je koop dit document dus niet van ons, maar van verkoper datasciencestudent. Stuvia faciliteert de betaling aan de verkoper.
Zit ik meteen vast aan een abonnement?
Nee, je koopt alleen deze samenvatting voor €2,99. Je zit daarna nergens aan vast.
