Summary for the course Information Retrieval given in the third year of Artificial Intelligence, for the Data Science minor. This document efficiently summarizes the lectures and is a great way to study for your exams. This can also be relevant material for similar degrees given in Vrije University...
partial matching exact matching
relevant items for result single failure
-
error
-
=
,
errors aoe tolerable
-
deterministic models
probabilistic models
-
, The information retrieval framework
search speed
criteria for Good Search Engines
How fast does it search ?
-
↳
Latency as a function of index size
How fast does index ?
-
it
↳ Number documents / hour
of
-
Expressiveness of query language
↳
Ability to express complex information needs
↳ speed on complex queries
Disk requirements
•
space
•
User bossiness
,Lecture 2 :
Indexing and Boolean Retrieval
Term -
Document Incidence Matrix
r
Incidence vectors
vectors with 011 for each term
Bigger collections
consider a text collection of N =
1 million documents ,
each with about 1000
words (tokens) :
On -6
bytes / word C including
-
and
average ,
we
might have spaces
punctuation)
-
.
: 6GB of data in the documents
would consist about IVI 500 000 distinct
:
vocabulary
-
=
-
our of
words (terms)
Matrix is Getting very Big
500K ✗ IM = 1 trillion
matrix Cie zeros)
belt ,
the is
very sparse .
mostly .
↳ is 1 's
per column ,
there a maximum
of 1000 compared to 500K words
i.
better representation :
only record Is !
Inverted index
-
For each chord ) t ,
we store a list
of all documents that contain t
Identify each
by document ID serial document number
-
a :
need C- lists)
we variable size lists
posting
•
-
continuous best
disk run
postings is
-
on , of
linked lists variable
-
in memory
we can use or
length arrays
-
,
, Inverted Index Construction
Indexer seep 1 : Token sequence
Exact sequence of < normalized token , document ID >
pairs
Indexer seep 2 : sort
Indexer sleep 3 :
Dictionary and Postings
Multiple term entries in a
single document are merged :
split into dictionary and postings
•
document
frequency information is added
-
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller marinamssantos. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $10.16. You're not tied to anything after your purchase.
