13 - Vector-space models of words and sentences - Repplinger et al. (2018)
In dit artikel wordt in 4 stappen beschreven hoe NLP de transitie heeft gemaakt van op logica
gebaseerde modellen naar hedendaagse vector-space modellen:
• Wat zijn de karakteristieke elementen van de stap die wordt beschreven?
• Wat zijn de sterke en zwakke punten van de benadering binnen deze stap?
- Natural Language Processing (NLP): focuses on developing mathematical and computational
models of language
- logical models played a key role in uncovering and detailing the complexity of language
structure
- vector-space models: words are represented as numerical vectors, and sentence meanings are
computed using a variety of operatioins from lineair algebra
- semantics: de studie van de betekenis van taaluitdrukkingen bedoelt, met name hoe woorden,
zinnen en taalconstructies betekenis overbrengen
Step 1: Montague semantics – celebrating the sentence, ignoring the word
- Montague semantics provided a systematic way to translate natural language to a logical
language; this system combines insights from various modeling traditions:
→ categorical grammer: system to assign syntactic categories to words
- nouns: PN, verbs: receive ‘complex category’ → e.g. love = (S\PN)/PN
→ intensional logic: higher-order typed logic, but the semantic, logical representation is derived
simultaneously with the grammatical derivation → logical expressions with lambda calculus
- e.g. love = λqλp[love(p)(q)] → semantics of the first argument ends up in the place of
the variable marked with the first λ in the expression
- principle of compositionality: the meaning of a complex expression is determined by the
meanings of its constituents and the rules used to combine them
- recursion: recursively defined processes are widely believed to underly the capacity of
speakers of a language to build and understand arbitrary expressions of the language
(- arbitrary expression is een concept dat in een bepaalde context kan worden toegepast zonder
specifieke beperkingen of definities, waardoor het een breed scala aan mogelijke interpretaties kan
hebben)
→ recursive syntactic processing: complex expressions from a set of simple ‘building blocks’
→ recursive semantic process: understand a potential infinitude of distinct meanings
- formal semantics derives the meaning of a sentence from its smaller constituents, but the
meaning of words (the lexical foundation) receives less attention
- there is a limited integration of individual semantic theories & no syntactically defined subset
of the language which can be semantically analyzed in completion
In dit artikel wordt in 4 stappen beschreven hoe NLP de transitie heeft gemaakt van op logica
gebaseerde modellen naar hedendaagse vector-space modellen:
• Wat zijn de karakteristieke elementen van de stap die wordt beschreven?
• Wat zijn de sterke en zwakke punten van de benadering binnen deze stap?
- Natural Language Processing (NLP): focuses on developing mathematical and computational
models of language
- logical models played a key role in uncovering and detailing the complexity of language
structure
- vector-space models: words are represented as numerical vectors, and sentence meanings are
computed using a variety of operatioins from lineair algebra
- semantics: de studie van de betekenis van taaluitdrukkingen bedoelt, met name hoe woorden,
zinnen en taalconstructies betekenis overbrengen
Step 1: Montague semantics – celebrating the sentence, ignoring the word
- Montague semantics provided a systematic way to translate natural language to a logical
language; this system combines insights from various modeling traditions:
→ categorical grammer: system to assign syntactic categories to words
- nouns: PN, verbs: receive ‘complex category’ → e.g. love = (S\PN)/PN
→ intensional logic: higher-order typed logic, but the semantic, logical representation is derived
simultaneously with the grammatical derivation → logical expressions with lambda calculus
- e.g. love = λqλp[love(p)(q)] → semantics of the first argument ends up in the place of
the variable marked with the first λ in the expression
- principle of compositionality: the meaning of a complex expression is determined by the
meanings of its constituents and the rules used to combine them
- recursion: recursively defined processes are widely believed to underly the capacity of
speakers of a language to build and understand arbitrary expressions of the language
(- arbitrary expression is een concept dat in een bepaalde context kan worden toegepast zonder
specifieke beperkingen of definities, waardoor het een breed scala aan mogelijke interpretaties kan
hebben)
→ recursive syntactic processing: complex expressions from a set of simple ‘building blocks’
→ recursive semantic process: understand a potential infinitude of distinct meanings
- formal semantics derives the meaning of a sentence from its smaller constituents, but the
meaning of words (the lexical foundation) receives less attention
- there is a limited integration of individual semantic theories & no syntactically defined subset
of the language which can be semantically analyzed in completion
