This summary is written for the course “Data Preparation & Workflow Management” during the semester Spring-2022 and is part of the master Marketing Analytics. The input for this summary consists of lectures, articles and tutorials.
Disclaimer: The course “Data Preparation & Workflow Manag...
Demi van de Pol | Summary | Data Preparation & Workflow Management | TISEM | Tilburg University | Spring-2022
SUMMARY DATA
PREPARATION & WORKFLOW
MANAGEMENT
Demi van de Pol || Master Marketing Analytics || Tilburg University || 2022
1
, Demi van de Pol | Summary | Data Preparation & Workflow Management | TISEM | Tilburg University | Spring-2022
CONTENT
This summary is written for the course “Data Preparation & Workflow Management” during the
semester Spring-2022 and is part of the master Marketing Analytics. The input for this summary
consists of lectures, articles and tutorials.
Disclaimer: The course “Data Preparation & Workflow Management” is mainly focused on the practical part of this subject
(i.e. working with data). This summary is by no means a substitute for the lectures and tutorials provided by the lecturer.
This summary merely provides support on the theoretical part of the course.
…
WEEK 1
READING: Professionalize you Team Work Using Scrum
The entire article can be found via this link: https://tilburgsciencehub.com/tutorials/scale-up/scrum-for-
researchers/use-scrum-in-your-team/
● Scrum is a simple framework for effective team collaboration that provides structure which leads
to commitment and motivation.
● Scrum defines three main roles for members of the team: the product owner, the Scrum master
and development team members.
● The product owner is accountable for maximizing the value of the product and for defining a clear
“task list” (called product backlog).
● The Scrum master is accountable for the team’s effectiveness by coaching and helping the team
members to focus, removing obstacles for the team and ensuring that tasks are completed in a
positive, productive and timely manner.
● The development team members are responsible for completing the tasks in the Sprint (period).
● Scrum can be seen as a structured way of working with meetings that are shorter and more
productive, and cooperating in a flexible way in-between meetings.
2
, Demi van de Pol | Summary | Data Preparation & Workflow Management | TISEM | Tilburg University | Spring-2022
WEEK 2: Project Management &
Version Control
READING: Principles of Project Setup and Workflow Management
The entire article can be found via this link: https://tilburgsciencehub.com/tutorials/reproducible-research-
and-automation/principles-of-project-setup-and-workflow-management/project-setup-overview/
PROJECT SETUP
Two major issues in managing data-intensive projects are:
● Losing sights of the project (= directory and file chaos)
● Difficult to (re)execute the project (= lack of automation)
The primary mission of managing data- and computation-intensive projects is to build a transparent
project infrastructure, that allows for easily (re)executing your code potentially many times.
PIPELINES AND PROJECT COMPONENTS
It is useful to break down a project into its most basic parts:
● A pipeline refers to the steps that are necessary to build a project (e.g., prepare dataset, run
model, produce tables and figures).
● Components refer to a project’s most nuclear building blocks (e.g., data, source code, and
generated temporary and/or output files).
The power of setting up the project in this way lies in:
● Full portability
● Reproducibility and transparency
PIPELINES
Benefits of conceiving your project like a pipeline:
● Write clearer source code: Separate the different steps in your project in smaller steps of separate
source code files.
● Obtain results faster: Because your project is separated into different pipeline stages and each of
these stages is self-contained, you can easily run “later” stages of your project (called
“downstream”), based on different input files defined earlier in your project (called “upstream”).
● Increase transparency and foster collaboration: With more transparent source code, you allow
others to more easily understand the code you use(d).
● Use multiple software packages: Due to the smaller steps you can easily use for instance R to
prepare your dataset and Python to build an algorithm based on the cleaned data.
3
Voordelen van het kopen van samenvattingen bij Stuvia op een rij:
Verzekerd van kwaliteit door reviews
Stuvia-klanten hebben meer dan 700.000 samenvattingen beoordeeld. Zo weet je zeker dat je de beste documenten koopt!
Snel en makkelijk kopen
Je betaalt supersnel en eenmalig met iDeal, creditcard of Stuvia-tegoed voor de samenvatting. Zonder lidmaatschap.
Focus op de essentie
Samenvattingen worden geschreven voor en door anderen. Daarom zijn de samenvattingen altijd betrouwbaar en actueel. Zo kom je snel tot de kern!
Veelgestelde vragen
Wat krijg ik als ik dit document koop?
Je krijgt een PDF, die direct beschikbaar is na je aankoop. Het gekochte document is altijd, overal en oneindig toegankelijk via je profiel.
Tevredenheidsgarantie: hoe werkt dat?
Onze tevredenheidsgarantie zorgt ervoor dat je altijd een studiedocument vindt dat goed bij je past. Je vult een formulier in en onze klantenservice regelt de rest.
Van wie koop ik deze samenvatting?
Stuvia is een marktplaats, je koop dit document dus niet van ons, maar van verkoper demivandepolxxx. Stuvia faciliteert de betaling aan de verkoper.
Zit ik meteen vast aan een abonnement?
Nee, je koopt alleen deze samenvatting voor €4,99. Je zit daarna nergens aan vast.
