Summary
Data Mining 2017/2018 - Short Summary
- Course
- Institution
Short summary (samenvatting) Data Mining Data Science Regression Classification Clustering Dimensionality Reduction
[Show more]
Seller
FollowJHessels
Reviews received
Content preview
Data Mining EssentialsSupervised vs Unsupervised Learning
- Supervised learning
o Classification (cat | dog | mouse)
o Regression (24 | 3 | 32 | 10)
- Unsupervised ‘learning’
o Clustering ( a b c | k l m | x y z)
o Dimensionality reduction (X1, X2, X3, X4, X5 –X3, –X5)
Overall goal of both methods: extract from dataset with goal to generalize.
Supervised Learning
- Training set with vectors | categorised (colours)
- Flowchart: raw data collection » pre-processing » sampling » re-processing » learning
algorithm training » hyperparameter optimisation » post-processing » final classification /
regression model
Pre-processing
Feature transformation:
- Categorical variables
o Nominal (green » [0,1,0])
o Ordinal (XL » 3)
- Normalisation and outlier removal
o Z-score (mean/SD)
o Remove outliers (depends on your goal)
- Vector normalisation
o L2-norm (√∑x²) ○
o L1-norm (∑|x|) ◊
Data Exploration and Visualisation (descriptive analysis)
- Sort or rearrange your data
- Goal of thesis: how well following the guidelines?
Splitting your data
- The fundamental goal is to generalize beyond the data instances used to train models
- Never touch the test data (until the end)
- Test data must belong to the same (statistical) distribution as the training data!
1. Sequential Split: for example a time series, typically train on a period, for example one 1-6
and test on 7-8. Common pitfall is cycles in the data (on different time-scales).
2. Random Split: blindly assign instances to training…….
Sampling and splitting your data
- In the case of small data, you want to check
(stratify) your data in terms of target, or at
least check if the ratios are representative.
- In the case of unbalanced data you might
want to stratify your data.
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller JHessels. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $3.22. You're not tied to anything after your purchase.
Can Stuvia be trusted?
4.6 stars on Google & Trustpilot (+1000 reviews)
52510 documents were sold in the last 30 days
Founded in 2010, the go-to place to buy study notes for 14 years now