College notes Deep Learning part 2 (XM_0083), Master VU AI
33 views 0 purchase
Course
Deep Learning (XM_0083)
Institution
Vrije Universiteit Amsterdam (VU)
Those are the notes I made during the whole course, it is not a summary! As long as you understand this document you do not need to watch any lecture. With those notes I have passed the course with a 7.5.
this output is the representation of the input object, and so it the embedding of it
some embedding spaces have an useful structure in which you can navigate in certain directions with
changing features according to direction = navigable space, so direction corresponds to certain semantic
features in object
Deep Learning 98
, embedding of images
embeddings of words
distributional hypothesis
"you shall know a word by the company it keeps"
"if units of text have similar vectors in a text freq matrix, then they tend to have similar meaning"
⇒ we can use context information to define embeddings for words,
one vector representation for each word
generally, we can train them (see lecture on word2vec)
illustration idea
why use graphs as input to ML?
classification, regression, clustering of nodes/edges/whole graphs
recommender systems
document modeling: entity and document similarity (use concepts from a graph)
alignment of graphs (which nodes are similar)
link prediction and error detection
linking text and semi-structured knowledge to graphs
graph embedding techniques
one big challenge with embedding graphs
we can not straightforwardly put graph into neural network because of its structure, not a linear
structure
there are traditional ML on graphs
often having problems with scalability
often need manual feature engineering: task specific
embedding knowledge graphs in vector space
for each node in graph → create vector in vector space → easily feed into ML algo
preserve info
unsupervised: task and dataset independent
efficient computation
low dimensional representation
Deep Learning 99
, 2 major visions on how this should be done
Preserve topology
keeping neighbors close in embedding space: e.g. europe-germany
Preserve similarity
keeping similar nodes close together: e.g. europe-africa
2 major targets
improve original data
knowledge graph completion
link prediction
anomaly detection
downstream ML tasks
classification/regression/clustering/K-NN
then used as part of a larger process
QA/dialog systems
Translation
image segmentation
how to go from graph to embedding? 3 major approaches for propositionalization
translation: any relation in knowledge graph is directly mapped to what should happen in the
embedded space. So we want to see the same relations in the embedded space
has a lot of approaches, but all come down to same thing
TransE is the basic one, and all the other ones are the complex version of it
= translation embedding
take every edge of graph → if this exists in knowledge graph → we want to have it in our
embedded space with edge embedding, vector as translation form that one to that one
so adding embedding to head has to give tail
we want to minimalize distance between sum of the h and l (as close as possible) to t in
knowledge graph S
Deep Learning 100
, this is not completely sufficient, seems to work but the problem is that we only get positive
information ⇒ over optimized
so not only minimizing but also penalizing when wrong relations are given: (maximize bad
triple)
become less scalable because they do more and more stuff
one hop away from each other = current entity → one hop → next entity
tensor/matrix factorization
make a 3d matrix of all relations in the graph and factorize it [factorizing = make a lower
dimensional representation of the 3d matrix also with lower matrix dimensionality]
Deep Learning 101
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller MeldaMalkoc. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $4.71. You're not tied to anything after your purchase.
