Extensive summary for the course 'Introduction to Deep Learning'. Including all lecture content (excl practicals) with extra notes and explanations added to make it the most clear.
Introduction to Deep Learning (800883-B-6)
Summary Lectures Final Exam
CSAI year 3
Written by Saskia Kriege
,Lecture 1 – Introduction and MultiLayer Perceptron
Neural Networks
Train = changing parameters
Trying to optimize the black box by changing numbers (parameters), done by understanding
the error that the model makes
Use error to change parameters of the network, to estimate the actual function of the network
Universal Function Approximator
Approximate functions
Y = f(x) → output y, input y, figure out the function f(x)
Input x → NN (approximate an unknown function y = f(x)) → output y
History and Context – NOT EXAM MATERIAL
Ramon y Cajal → connectionist approach of how the brain works
Individual tissues doing individual ones doing computations by themselves → neurons
Those neurons were connected, connections changed how they were firing
Emerging from this is intelligence
McCulloch and Pitts
Computers were emerging, idea to build mathematical models of this idea of the brain
Logic Gates based on connectionist approach, little units putted together gives a more
complex thing
Based on Logic, input and output only 1’s and 0’s.
Rosenblatt
The Perceptron → idea we are still using for NN
Changed → how they’re trained and put together
Perceptron learning parameters (weights)
Weight + input gives a certain output
Perceptrons and the AI Winter
Minsky and Papert
You cannot solve simple problems with this perceptron → not taking us closer to what the
brain does
Basic problems cannot be solved (sort problem)
People stopped believing in AI, funding disappeared from research
The AI Winter
Some problems could not be solved, using a perceptron was not complete enough.
1980s Boom
Found out how to train network in different ways, got more interesting results
Journals, conferences appeared
Neocognitron – Fukushima → image processing for NN
Backpropagation
,What we use to train networks
Lecun → digit recognition
Another AI Winter → we didn’t have the data and computers to apply the methods we found
out
Big Data
Computers put together with a lot of data
2012 → The cat experiment → neurons learnt to respond to specific stimuli like cats
ImageNet = image database organized according to the WordNet hierarchy, in which each
node of the hierarchy is depicted by hundreds and thousands of images
→ has a lot of biases
AlexNet
Deep CNN trained on ImageNet using GPUs.
Hinton et al.
Generative Adversarial Networks
You have to generate data instead of other way around
Relevant in +- last 5 years
Deep Reinforcement Learning
Neural network
Inputs and expected output changes
How to produce behavior that’s relevant for a particular task
Feed in a frame, calculate error
Deep Learning
Inside AI we have ML (learning from data)
More narrow in ML we have Representation Learning (take the data, model has to figure out
what to do). Transform data into something else
More narrow DL → many layers, each layer trying to extract more abstract features
Practical Deep Learning
CPU vs GPU
GPUs allow for parallelism
CPU → sequential
CPU → each core can do 1 thing at a
time
GPU → you can do as much cores there
are at the same time
DL = multiplications and additions
, GPU allows us to do it more fast, CPU cores are more powerful, but GPU allows to do the
calculations at the same time
Networks require GPU, else it would be too slow
GPU can process many pieces of data at same time
The main difference between CPU and GPU architecture is that a CPU is designed to handle
a wide-range of tasks quickly (as measured by CPU clock speed), but are limited in the
concurrency of tasks that can be running. A GPU is designed to quickly render high-resolution
images and video concurrently.
The Perceptron
A model of a neuron
Dendrites (input) → body of neuron → axon (output)
Then it connects to other neurons through electrical chemical signals, interacting with other
neurons (synapse & synaptic cleft)
A simpler model of a neuron
Incoming dendrites → soma (S) (once threshold is passed), going to axon triggering the rest
Some will increase/decrease in the soma
Incoming dendrites summed together and passed on
Rosenblatt → Perceptron designed to illustrate some of the fundamental properties of
intelligent systems in general, without becoming too deeply enmeshed in the special, and
frequently unknown, conditions which hold for particular biological organisms
Inputs → black box → outputs
Inputs take signals, in the body there is a function that will accumulate those signals by a
summation
Inputs x1, x2, x3
There are weights in the arrows
Those are summed up in the body
Gives y
Linear Classifier → inputting some values and having some weights.
Linearly combining inputs and deciding if they fit given a pattern
Bigger than a threshold → belongs to a certain class
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller saskiakriege. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $9.19. You're not tied to anything after your purchase.
