Summary
Summary DATA SCEINE
- Module
- DATA SCIENE
- Institution
- DATA SCIENE
Summary of 531 pages for the course DATA SCIENE at DATA SCIENE (BOOK GOOD)
[Show more]
Seller
Follow
Content preview
Data Science and Machine LearningMathematical and Statistical Methods
Dirk P. Kroese, Zdravko I. Botev, Thomas Taimre, Radislav Vaisman
30th October 2023
,To my wife and daughters: Lesley, Elise, and Jessica
— DPK
To Sarah, Sofia, and my parents
— ZIB
To my grandparents: Arno, Harry, Juta, and Maila
— TT
To Valerie
— RV
,CONTENTS
Preface xiii
Notation xvii
1 Importing, Summarizing, and Visualizing Data 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Structuring Features According to Type . . . . . . . . . . . . . . . . . . 3
1.3 Summary Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 Summary Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.5 Visualizing Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5.1 Plotting Qualitative Variables . . . . . . . . . . . . . . . . . . . . 9
1.5.2 Plotting Quantitative Variables . . . . . . . . . . . . . . . . . . . 9
1.5.3 Data Visualization in a Bivariate Setting . . . . . . . . . . . . . . 12
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2 Statistical Learning 19
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2 Supervised and Unsupervised Learning . . . . . . . . . . . . . . . . . . . 20
2.3 Training and Test Loss . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.4 Tradeoffs in Statistical Learning . . . . . . . . . . . . . . . . . . . . . . 31
2.5 Estimating Risk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.5.1 In-Sample Risk . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.5.2 Cross-Validation . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.6 Modeling Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.7 Multivariate Normal Models . . . . . . . . . . . . . . . . . . . . . . . . 44
2.8 Normal Linear Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.9 Bayesian Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3 Monte Carlo Methods 67
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.2 Monte Carlo Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.2.1 Generating Random Numbers . . . . . . . . . . . . . . . . . . . 68
3.2.2 Simulating Random Variables . . . . . . . . . . . . . . . . . . . 69
3.2.3 Simulating Random Vectors and Processes . . . . . . . . . . . . . 74
3.2.4 Resampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.2.5 Markov Chain Monte Carlo . . . . . . . . . . . . . . . . . . . . . 78
3.3 Monte Carlo Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
vii
, viii Contents
3.3.1 Crude Monte Carlo . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.3.2 Bootstrap Method . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.3.3 Variance Reduction . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.4 Monte Carlo for Optimization . . . . . . . . . . . . . . . . . . . . . . . . 96
3.4.1 Simulated Annealing . . . . . . . . . . . . . . . . . . . . . . . . 96
3.4.2 Cross-Entropy Method . . . . . . . . . . . . . . . . . . . . . . . 100
3.4.3 Splitting for Optimization . . . . . . . . . . . . . . . . . . . . . . 103
3.4.4 Noisy Optimization . . . . . . . . . . . . . . . . . . . . . . . . . 105
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4 Unsupervised Learning 121
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
4.2 Risk and Loss in Unsupervised Learning . . . . . . . . . . . . . . . . . . 122
4.3 Expectation–Maximization (EM) Algorithm . . . . . . . . . . . . . . . . 128
4.4 Empirical Distribution and Density Estimation . . . . . . . . . . . . . . . 131
4.5 Clustering via Mixture Models . . . . . . . . . . . . . . . . . . . . . . . 135
4.5.1 Mixture Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.5.2 EM Algorithm for Mixture Models . . . . . . . . . . . . . . . . . 137
4.6 Clustering via Vector Quantization . . . . . . . . . . . . . . . . . . . . . 142
4.6.1 K-Means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4.6.2 Clustering via Continuous Multiextremal Optimization . . . . . . 146
4.7 Hierarchical Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
4.8 Principal Component Analysis (PCA) . . . . . . . . . . . . . . . . . . . 153
4.8.1 Motivation: Principal Axes of an Ellipsoid . . . . . . . . . . . . . 153
4.8.2 PCA and Singular Value Decomposition (SVD) . . . . . . . . . . 155
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
5 Regression 167
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
5.2 Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
5.3 Analysis via Linear Models . . . . . . . . . . . . . . . . . . . . . . . . . 171
5.3.1 Parameter Estimation . . . . . . . . . . . . . . . . . . . . . . . . 171
5.3.2 Model Selection and Prediction . . . . . . . . . . . . . . . . . . . 172
5.3.3 Cross-Validation and Predictive Residual Sum of Squares . . . . . 173
5.3.4 In-Sample Risk and Akaike Information Criterion . . . . . . . . . 175
5.3.5 Categorical Features . . . . . . . . . . . . . . . . . . . . . . . . 177
5.3.6 Nested Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
5.3.7 Coefficient of Determination . . . . . . . . . . . . . . . . . . . . 181
5.4 Inference for Normal Linear Models . . . . . . . . . . . . . . . . . . . . 182
5.4.1 Comparing Two Normal Linear Models . . . . . . . . . . . . . . 183
5.4.2 Confidence and Prediction Intervals . . . . . . . . . . . . . . . . 186
5.5 Nonlinear Regression Models . . . . . . . . . . . . . . . . . . . . . . . . 188
5.6 Linear Models in Python . . . . . . . . . . . . . . . . . . . . . . . . . . 191
5.6.1 Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
5.6.2 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
5.6.3 Analysis of Variance (ANOVA) . . . . . . . . . . . . . . . . . . 195
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller annisapurwanto. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for £12.57. You're not tied to anything after your purchase.
Can Stuvia be trusted?
4.6 stars on Google & Trustpilot (+1000 reviews)
62555 documents were sold in the last 30 days
Founded in 2010, the go-to place to buy revision notes and other study material for 14 years now