Summary
Samenvatting - Data Science Methods (6414M0320Y)
- Course
- Institution
Extensive summary of the course data science methods.
[Show more]
Seller
Follow
maaikekoens
Content preview
Data Science, Model evaluation
3
After having a trained model one would like to know the prediction capabilities on new, unseen, data
Model selection: comparing performance of different models, to identify best one
e
Model assessment: having chosen a final model, estimating how well it predicts on new data
&
Linear models for regression
S
example model troughout the course
Y =
A(X ,
B) +
- N(0 ga)
,
M
FoBmhm(X)
~
A(X ,
B) :
↳
Basic function/derived future
&
ex. In linear regression hm(X) =
Xm
Determine the model parameters using maximum likelihood
&
Take product/likelihood function
[(B 02) : re ,
2
Take logaritme
In 2(B 02) Nh(8)-(2)-(yn,
: -
-
Sh(xn))
3
Take derivatives
O
with respect to B e
with respect to 8
↳
In[(B 02) : (yn- Sh(xn))hm(xn) ,
=
0 so In 2(B :),
:
↓
: (Xx)"xy 02 : Z (yn-BXn)
Y
Generalization error
To determine the performance of a model, we define a loss function that measures the size of a prediction error
examples of loss functions
We want the error to be as small as possible as this
S
(Y E(x)))
squared error -
means that there is a high generalization
L(Y , (x)) Y E(X)
absolute error
=
-
Estimating the generalization error is often not possible,
3 types of errors
3
therefore we use estimate of prediction error
1 Err : E L(Y ,
E(x)) Err :
Er Err
2 Erro :
E[L(Y , (X)) It
, 3 er
: (yn , (xn))
Bias-variance decomposition
Err(xo) : EL/Y-E(X))"(X =
xo
=
(E[E(xo)] f(xo))2 E[(xo)
- + ·
ETE(xo)" -8 When we increase the polynomial,
=
Bias
&
((xo)) + Var((xo)) + 8
higher variance but lower bias
=
Bias + Varianee + G When decrease in polynomial,
&
variance of error lower variance but higher bias
S
variance of estimated model
&
squared bias of estimated model * We need to make a trade-off such
that the variance + bias are minimal
s
High variance, low bias Low variance, high bias
Data
When dealing with a data set, it is not possible to simulate additional data points to compute the
generalization errors and expected prediction error. We therefore need alternative procedures to estimate
the generalization and prediction errors. In the next three sections we will describe several of those
procedures 2
1
O
Data-rich situations: many data available
In-sample errors: calculating
errors from data on which
Training set Validation set Test set the model was trained
D Out-of-sample errors:
Used to test the model calculation errors from data
& that was excluded from the
Used to measure the performance training set
&
Used to train the data
⑨
Insufficient data: not enough data available
2
Information theoretical measures
&
The maximum log-likelihood is a measure for how well the model can describe the data, however we need
an penalty term that takes model complexity in account
3 Choose model with smallest AIC/BIC
-
Ale : - 2h([) + 2 (M 1) +
- Ble =
-2ln() + In (N) (M 1) +
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller maaikekoens. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $6.96. You're not tied to anything after your purchase.
Can Stuvia be trusted?
4.6 stars on Google & Trustpilot (+1000 reviews)
52355 documents were sold in the last 30 days
Founded in 2010, the go-to place to buy study notes for 14 years now