Readings Summary for Data Engineering for MADS (Mandatory & Optional Papers + Book Chapters)
11 views 0 purchase
Course
Data Engineering for MADS (EBM213A05)
Institution
Rijksuniversiteit Groningen (RuG)
Book
Creating Value with Big Data Analytics
The best summary of ALL READINGS for Data Engineering for MADS (EBM213A05). Includes both mandatory and optional papers and book chapters. Enhanced with a dynamic table of contents and meticulous organization for readability and easy studying. 100% of profit from this summary is donated to local Gr...
Answers for Quiz 1, Quiz 5, Quiz 6 and a Full Practice Exam for Data Engineering for MADS (EBM213A05)
Extensive Summary of Verhoef book (Chapter 1-5) + Lecture 2 for Data Engineering for MADS (EBM213A05)
Lecture Summary for Data Engineering for MADS
All for this textbook (5)
Written for
Rijksuniversiteit Groningen (RuG)
Marketing Analytics & Data Science
Data Engineering for MADS (EBM213A05)
All documents for this subject (5)
Seller
Follow
madsmadlad
Reviews received
Content preview
SUMMARY OF ALL READINGS
INCLUDING OPTIONAL AND MANDATORY READINGS .
N o t e : S o m e r e a d i n g s o m i t t e d d u e t o i r r e l e va n c e o r r e d u n d a n c e .
E n h a n c e d w i t h a d y n a m ic t a b le o f c o n t e n t s .
, MADS MADLAD |2
Note from MADS MADLAD:
Thank you for buying my summary. I sincerely hope it helps you excel and learn
from this course. When I was writing these I sometimes struggled with this
program, but there were no summaries available.
This is why I decided to write something that is truly complete with a lot of
effort put into it. It helped me and my friends get good grades, but I also
always had you in mind, the future reader. When necessary, I always went the
extra mile to make this summary, more readable, organized and complete.
If you feel like it, leave me a review of how the course is going using this
summary, it will make my day to hear your opinion good or bad!
Check out my other extensive summaries for other MADS courses:
Available on:
Contact info:
If you need help or have an inquiry, contact me: https://www.georgedreemer.com
Connect with me on LinkedIn: https://www.linkedin.com/in/georgedreemer/
Donations:
By no means am I looking for fellow students to send me money! But if you feel like sending
me some ETH or BTC, you can do so here:
--> ETH: 0x123e086c6808459e7fC6Ac7F64a77dBA1dDe0149
--> BTC: bc1qgwzc82vph5v8rmzef4ywechjf85772n7m2e22g
, MADS MADLAD |3
wishes you good luck & perseverance.
Grades Testimony:
, MADS MADLAD |4
Table of Contents
Week 1............................................................................................................. 12
Reading – HBR Article (optional) ................................................................... 12
4 Steps to Management Questions ............................................................ 12
Reading – Book: Verhoef et al. (11.6 required +11.1 optional) ..................... 13
7 Steps of Opportunity finding ................................................................... 13
Reading – Book: Business Research Methods (2.1 optional + 2.2 required) .. 16
Week 2............................................................................................................. 20
Reading – Book: Verhoef et al.(Ch.1, 2.1-2.5, 3.4, 3.9, 4, 5) .......................... 20
Chapter 1 (required): “Data science and big data”........................................ 20
Chapter 2.1-2.5 (required): “Creating value with data science” .................... 23
2.1 INTRODUCTION ................................................................................ 23
2.2 DATA SCIENCE VALUE CREATION MODEL ......................................... 23
2.3 VALUE CREATION OBJECTIVES .......................................................... 24
2.3.1 Balance between V2F and V2C: ............................................... 24
2.3.2 V2S – Value to Society ................................................................. 25
2.3.3 Metrics for V2F and V2C .......................................................... 25
2.4 DATA ASSETS .................................................................................... 26
2.5 DATA ANALYTICS .................................................................................. 26
2.5.1 The power of visualization and storytelling ............................. 27
Chapter 3.4 & 3.9 (required): “CUSTOMER METRICS”................................... 27
3.4 CUSTOMER (FEEDBACK) METRICS .................................................... 27
First Dimension: Time span - Forward- vs. Backward-looking Metrics .. 28
Second Dimension: Measurement scale............................................... 29
3.4.1 Is there a silver metric? ........................................................... 29
3.4.2 Other theoretical relationship metrics .................................... 30
3.4.3 Customer equity drivers .............................................................. 30
3.4.4 Internal data sources ................................................................... 31
3.4.5 Online sources – Customer Reviews ............................................ 31
3.9 CUSTOMER METRICS ........................................................................ 32
, MADS MADLAD |5
3.9.1 Customer acquisition metrics ...................................................... 33
3.9.2 Customer development metrics .................................................. 33
3.9.3 Customer value metrics ............................................................... 34
3.9.4 Customer equity .......................................................................... 36
3.9.5 New big data metrics................................................................... 37
Chapter 4 (required): “Data Assets”.............................................................. 39
4.2 DATA SOURCES AND THE DIFFERENT TYPES OF DATA .......................... 39
4.2.1 External data sources vs. Internal data sources ........................... 39
4.2.2 Structured vs. Unstructured data ................................................ 40
4.2.3 Market data................................................................................. 41
4.2.5 Brand data ................................................................................... 42
4.2.7 Customer data ............................................................................. 44
4.3 USING THE DIFFERENT DATA SOURCES IN THE ERA OF BIG DATA ........ 45
4.4 DATA QUALITY AND DATA CLEANSING................................................. 46
4.4.1 Data Quality ................................................................................ 46
4.4.2 Data Cleansing ............................................................................. 47
4.4.3 Missing value and data fusion ..................................................... 48
Chapter 5 (required): “Data storing and integration” ................................... 48
5.1INTRODUCTION .................................................................................... 48
5.2 STORING AND INTEGRATING DATA SOURCES IN DATA WAREHOUSES. 48
5.2.1 Storing data in the data warehouse............................................. 49
5.2.2 The data model in a data warehouse .......................................... 50
5.2.3 Data integration into the data warehouse................................... 52
5.2.3.1 Extraction ............................................................................. 53
5.2.3.2 Transformation ..................................................................... 53
5.2.3.3 Loading ................................................................................. 53
5.3 STORING AND INTEGRATING DATA SOURCES IN DATA LAKES .............. 54
5.4 CHALLENGES OF DATA INTEGRATION IN THE ERA OF BIG DATA .......... 56
5.4.1 The technical challenges of integrated data ................................ 56
5.4.1.1 Integration at the individual level ......................................... 57
, MADS MADLAD |6
5.4.1.2 Integration at the intermediate level .................................... 57
5.4.1.3 Integration at the time level ..................................................... 57
5.4.2 The analytical challenges of integrated data ............................... 58
5.4.3 The business challenges of integrated data ................................. 58
5.4.3.1 Dealing with different data types ......................................... 58
5.4.3.2 Declared data: customer descriptors .................................... 59
5.4.3.3 Appended data ..................................................................... 59
5.4.3.4 Overlaid data ........................................................................ 59
5.4.3.5 Implied data ......................................................................... 60
Week 3 – no readings ...................................................................................... 61
Week 4 – no readings ...................................................................................... 61
Week 5............................................................................................................. 62
Reading – Wickham (2014) “Tidy Data” ........................................................ 62
Section 2: Three characteristics of a Tidy dataset ...................................... 62
Section 2.1: Data structure ................................................................... 62
Section 2.2: Data Semantics ................................................................. 63
Section 2.3: Tidy data ........................................................................... 63
Section 3: Operations to make a messy dataset tidy.................................. 64
Section 3.1: Column headers are values, not variable names ............... 64
3.2: Multiple variables are stored in one column ................................. 66
3.3: Variables are stored in both rows and columns............................. 67
3.4: Multiple types of observational units are stored in the same table
............................................................................................................. 68
3.5: A single observational unit is stored in multiple tables.................. 68
Section 4: Tidy tools (tools that input & output tidy data) ......................... 69
Section 4.1: Data Manipulation ............................................................ 69
Section 4.2: Visualization ..................................................................... 69
Section 4.3: Modeling .......................................................................... 70
Reading – de Jonge, E. and van der Loo, M. (2013) "An introduction to data
cleaning with R" ............................................................................................ 71
, MADS MADLAD |7
1 Introduction ............................................................................................ 71
1.1: Statistical analysis in 5 steps.......................................................... 71
1.2 Some general background in R ....................................................... 72
1.2.1 Variable types and indexing techniques .................................. 72
1.2.2 Special values .......................................................................... 72
2 From raw data to technically correct data .............................................. 72
2.1: Technically correct data in R.......................................................... 72
2.2: Reading text data into a R data.frame ........................................... 73
2.2.1 read.table() and its cousins ...................................................... 73
2.3: Type conversion ............................................................................ 77
2.3.1 Introduction to R’s typing system ............................................ 77
2.3.2 Recoding factors ...................................................................... 77
2.3.3 Converting dates ..................................................................... 78
2.4: character manipulation ................................................................. 79
2.4.1 String normalization ................................................................ 80
2.4.2 Approximate string matching .................................................. 81
2.5 Character encoding issues .............................................................. 84
3: From technically correct data to consistent data ................................... 84
3.1 Detection and localization of errors ............................................... 85
3.1.1 Missing values ......................................................................... 85
3.1.2 Special values .......................................................................... 86
3.1.3 Outliers .................................................................................... 87
3.1.4 Obvious inconsistencies........................................................... 88
3.2 Correction ...................................................................................... 90
3.2.1 Simple transformation rules .................................................... 91
3.2.2 Deductive correction ............................................................... 93
3.2.3 Deterministic imputation......................................................... 94
3.3 Imputation ..................................................................................... 95
3.3.1 Basic numeric imputation models ........................................... 96
3.3.2 Hot deck imputation ................................................................ 96
, MADS MADLAD |8
3.3.3 kNN-imputation ....................................................................... 97
3.3.4 Minimal value adjustment ........................................................... 98
Reading – Donders, A.G.T, van der Heijden, G.J.M.G, Stijnen, T and Moons,
K.G.M. (2006) "Review: A gentle introduction to imputation of missing values"
..................................................................................................................... 99
Reading – Schafer, J.L. and Graham, J.W. (2002), "Missing Data: Our View of
the State of the Art" ..................................................................................... 99
Week 6........................................................................................................... 100
Reading – Book: Chapter 7 and 8 ................................................................ 100
Chapter 7: Data Analytics............................................................................ 100
7.2 THE POWER OF ANALYTICS ................................................................ 100
7.3 STRATEGIES FOR ANALYZING DATA ................................................... 100
7.3.1 PROBLEM SOLVING ................................................................... 101
7.3.2 DATA EXPLOITATION ................................................................. 101
7.3.3 DATA MINING ............................................................................ 102
7.3.4 COLLATERAL CATCH................................................................... 103
7.4 TYPES OF DATA ANALYTICS ................................................................ 104
7.4.1 DESCRIPTIVE ANALYTICS ............................................................ 105
7.4.2 DIAGNOSTIC ANALYTICS ............................................................ 105
7.4.3 PREDICTIVE ANALYTICS.............................................................. 106
7.4.4 PRESCRIPTIVE ANALYTICS .......................................................... 106
7.5 HOW BIG DATA AND AI CHANGE ANALYTICS ..................................... 107
7.5.1.1 DATA SCIENCE ........................................................................ 107
7.5.1.2 AI ............................................................................................ 108
7.5.1.3 MACHINE LEARNING (ML) ...................................................... 108
7.5.1.4 DEEP LEARNING (DL) .............................................................. 109
7.6 ANALYTICAL METHODS AND TECHNIQUES......................................... 109
Chapter 8: Data Exploration ........................................................................ 110
8.1 INTRODUCTION.................................................................................. 110
8.2 DESCRIPTIVE ANALYSES – REPORTING ............................................... 111
10.3.1 CHECKLIST FOR A CLEAR STORYLINE ........................................ 130
10.4 VISUALIZATION ................................................................................ 130
10.4.1 CHOOSING THE CHART TYPE ................................................... 130
10.4.1.1 SHOWING RELATIONSHIP BETWEEN DATA POINTS .......... 130
10.4.1.2. COMPARING DATA POINTS .............................................. 131
10.4.1.3 COMPOSITION .................................................................. 132
10.4.1.4 DISTRIBUTION................................................................... 132
DECISION PROCESS FOR CHARTS (ABELA, 2008) ...................................... 133
10.4 MISLEADING GRAPHS....................................................................... 134
10.4.3.1 TRUNCATED GRAPHS ........................................................ 134
10.4.3.2 ADJUSTED AXIS ................................................................. 134
10.4.3.3 INCORRECT SCALING ........................................................ 135
10.4.3.4 LOGARITHMIC SCALING .................................................... 135
10.4.3.5 OMITTING DATA ............................................................... 136
10.4.3.6 SIMULATED TRENDS ......................................................... 136
10.4.3.7 REDUNDANT 3D PERSPECTIVE .......................................... 137
10.5 TRENDS IN VISUALIZATION .............................................................. 137
10.6 CONCLUSIONS .................................................................................. 137
Reading – Berinato, S (2016) ....................................................................... 138
Conceptual or Data Driven ....................................................................... 138
Declarative or Exploratory ....................................................................... 138
The 4 Types of Visual Communication ..................................................... 138
Idea Illustration .................................................................................. 139
Idea Generation ................................................................................. 140
Visual Discovery ................................................................................. 141
Everyday Datawiz ............................................................................... 142
Reading – Cleveland et al. (1984) ................................................................ 143
Reading – Swamy, P.R. (2013): Building Logic Into Communication Using the
Minto Pyramid Principle ............................................................................. 144
Storing and Retrieving Information .......................................................... 144
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller madsmadlad. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $13.85. You're not tied to anything after your purchase.
