Language, Knowledge and Grammar in Language Processing
On this page
Language, Knowledge and Grammar in Language Processing
Recall first
What is the difference between a sentence being grammatical and a system assigning it a high probability? Use one example to show why the two ideas are related but not identical.
First principles
A language-processing system needs several kinds of knowledge:
- Lexical knowledge: words, categories, pronunciation, senses, and subword structure.
- Morphological knowledge: how roots and affixes form words and express features such as tense or number.
- Syntactic knowledge: how constituents combine and what grammatical relations are possible.
- Semantic knowledge: what expressions denote and how composition builds meaning.
- Pragmatic/discourse knowledge: what a speaker intends and how earlier context changes interpretation.
- World knowledge: facts and expectations that are rarely stated in the sentence.
A grammar is a formal specification of permitted structures. In a context-free grammar (CFG), productions such as S → NP VP say that a sentence can consist of a noun phrase followed by a verb phrase; NP → Det N and VP → V NP refine the structure. A grammar does not itself tell us which interpretation is intended when several parses are legal. A probabilistic grammar adds weights, so a parser can rank legal trees.
Grammar is not merely a list of correctness rules. It is a compact description of productive structure. The principle of compositionality says that the meaning of a complex expression depends on the meanings of its parts and how grammar combines them, while context and world knowledge resolve what grammar leaves open.
Worked example. “I saw the man with a telescope” has at least two constituency structures: the prepositional phrase can attach to saw (the instrument) or to man (the man possessing it). Both may be syntactically legal. A statistical parser uses corpus preferences; a semantic/pragmatic system may use plausibility and discourse context. Thus grammar generates candidates, while other knowledge helps select an interpretation.
A useful distinction is competence vs. processing evidence: an abstract grammar describes possible structures, whereas a real system must cope with disfluencies, spelling errors, ambiguity, and limited evidence. Modern learned models distribute knowledge across parameters rather than storing one neat grammar, but the linguistic distinctions remain useful for analysis and error diagnosis.
Exercise — reveal after answering
Which knowledge type is most directly needed to decide whether “The dogs runs” has agreement trouble, and which is needed to decide whether “The bank is open” means a financial institution or a river edge?
Answer: Morphosyntactic knowledge (syntax plus inflection/agreement) flags the first issue. Lexical semantics plus context/world knowledge disambiguates the second.
Exam lens
- Define grammar as a formal structural constraint, not as “all language knowledge.”
- Separate lexical, morphological, syntactic, semantic, pragmatic, and world knowledge.
- State that a probabilistic grammar ranks analyses; it does not guarantee truth or meaning.
Rapid revision checklist
- Define grammar and CFG production.
- List the major knowledge layers.
- Explain compositionality.
- Explain why grammar alone cannot resolve PP attachment.
Key takeaways
- NLP needs layered knowledge, even when a single neural model stores it implicitly.
- Grammar constrains structure; probability and context choose among alternatives.
- Ambiguity is evidence that syntax, semantics, pragmatics, and world knowledge interact.