Summary of class notes on Data Science Research Methods (JBM)
17 views 0 purchase
Course
Data Science Research Methods (JBM020)
Institution
Technische Universiteit Eindhoven (TUE)
This document is a summary of the class notes that are also in this Stuvia. It is more organized and contains all main elements of the lectures and reading material.
DATA SCIENCE
RESEARCH METHODS
LECTURE 1
Topics: course introduction, Scientific Method, Sample Size Determination, and ANOVA
p-value: highest significant value for which we accept H 0. If p<α → reject H 0.
Type I error α : reject H 0 when it is True.
Type II error β : accept H 0 when it is False.
The best combination of α and β is situation-specific.
One-Factor Design: Studies the impact of a single factor, Y =f ( X , ε ) for factor X on Y .
Replicated experiment: there is more than one data point at each level of the factor.
Replicates: number of rows, different values of Y .
Levels: number of columns, different levels of X .
Total outcomes: # replicates × # levels
Types of means:
Column mean: Sum of all values in the column divided by the number of replicates.
Grand mean: Sum of all data points divided by the total outcomes, RC ór sum of all column
means divided by the number of levels, C .
Least squares: optimal estimation that minimizes the sum of the squared differences.
Total Sum of Squares (TSS): sum of the squared difference between each data point and the grand mean.
Sum of Squares Between Columns (SSBc): sum of the squared difference between each column mean and
the grand mean, multiplied by R .
Sum of Squares Within Columns (SSW): sum of the squared difference between each data point in a
column and that column mean.
TSS=SS B c + SSW
If SSW ≈TSS → factor does not explain much.
If SSW /TSS ≈ 0 → factor has big influence.
MS=SSQ/df ; Mean square is the Sum of Squares divided by the degrees-of-freedom.
Unbiased estimate of population variance → use df instead of n .
E [ MSW ]=σ under constant variance assumption.
2
E [ MS Bc ]=σ 2 +V 2 with V col =[ R / ( C−1 ) ] ∙ ∑τ 2j .
E [ MS Bc ] ≠0 → true column means might not be equal, and sample error leads to difference in
column means.
F-statistic: way to find evidence of affects.
F calc >1 → evidence that V col ≠ 0 thus X affects Y . Evidence is not a final conclusion.
, F calc ≤1 → no evidence that X affects Y .
F calc ≫1 → reject H 0. Rejection means column means are different.
SLIDES LECTURE 1
Three goals of Data Science: Description, Prediction and Explanation.
Different types of Analytics:
1. Descriptive Analytics: insight into the past
2. Predictive Analytics: understanding the future
3. Prescriptive Analytics: advice on possible outcomes
Scientific Method: has an iterative nature.
Six Sigma: disciplined, data-driven methodology for process
improvement. Uses DMAIC cycle.
Key Insights:
Identify the three data science goals.
Scientific method is an iterative process.
Not planning an experiment will not result in the wanted outcomes.
Experiment can have more factors, that can have more than 2 levels.
Six Sigma incorporated several aspects of the scientific method.
X−μ 0
Normal distribution gives test statistic T = if σ is known. Reject if |T |> z α / 2 or P H (|T|>|T 0|) <α .
σ /√ n 0
(
Confidence interval: reject if it doesn’t fall in the interval, x−z α /2
σ
√n
, x + z α /2
σ
√n ) .
Minimal sample sizes:
Normal distribution:
( )
2
z α /2 σ
o ONE SAMPLE CASE: if σ is known → n ≥ with E maximal absolute error.
E
o ONE SAMPLE CASE: if σ is unknown → same as with known but use worst case σ .
Round up to strictly satisfy the inequality.
( ) (σ + σ )
2
z α /2 2 2
o TWO SAMPLE CASE: equal sample sizes and variances known→ n ≥ 1 2
E
Binomial distribution:
( )
2
zα/ 2
o ONE SAMPLE CASE: n ≥ ^p (1−^p )use worst case ^p or upper/lower bound
E
Analyze p → p ( 1− p ) on [ 0,1 ]
( )
2
z
o TWO SAMPLE CASE: equal sample sizes n ≥ α/ 2 ( ^ p 1 ( 1− ^
p 1) + ^ p 2) )
p 2 ( 1− ^
E
Power analysis:
Normal distribution H 0 : μ=μ0 :
o (
β=Φ z α / 2−
δ √n
σ ) (
−Φ −z α /2 −
δ √n
σ )
( )
2 2
δ √n (z +z ) σ
o Φ −z α/ 2− small compared to β → n ≈ α/ 2 β
σ δ
2
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller datasciencestudent. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $3.23. You're not tied to anything after your purchase.
