CMC733/Semester 7
Natural Language Processing
26 notes written · 64% syllabus coverage (25 of 39 topics)
CMC733 — Natural Language Processing
Detailed Syllabus
Module I — Introduction to NLP (3 hours)
- 1.1 Origin & History of NLP
- 1.2 Language, Knowledge and Grammar in language processing
- 1.3 Stages in NLP
- 1.4 Ambiguities and its types in English and Indian Regional Languages
- 1.5 Challenges of NLP
- 1.6 Applications of NLP
- 1.7 Self-Learning: Tools for regional languages pre-processing and other functionalities
Module II — Word Level Analysis (9 hours)
- 2.1 Basic Terms: Tokenization, Stemming, Lemmatization
- 2.2 Survey of English Morphology
- Inflectional Morphology
- Derivational Morphology
- 2.3 Regular expression with types
- 2.4 Morphological Models
- Dictionary lookup
- Finite state morphology
- Morphological parsing with FST (Finite State Transducer)
- Lexicon free FST Porter Stemmer algorithm
- 2.5 N-grams and its variations
- Bigram, Trigram
- Simple (Unsmoothed) N-grams
- 2.6 N-gram Sensitivity to the Training Corpus
- 2.7 Unknown Words: Open versus closed vocabulary tasks
- 2.8 Evaluating N-grams: Perplexity
- 2.9 Smoothing: Laplace Smoothing, Good-Turing Discounting
- 2.10 Self-Learning: Noisy channel models, edit distance, advanced issues in Language Modelling
Module III — Syntax Analysis (10 hours)
- 3.1 Part-Of-Speech tagging (POS)
- Tag set for English (Upenn Treebank)
- Difficulties / Challenges in POS tagging
- 3.2 Rule-based, Stochastic and Transformation-based tagging
- 3.3 Generative Model: Hidden Markov Model (HMM Viterbi) for POS tagging
- Issues in HMM POS tagging
- 3.4 Discriminative Model: Maximum Entropy model, Conditional Random Field (CRF)
- 3.5 Parsers
- Top down and Bottom up
- Modelling constituency
- 3.6 Bottom Up Parsers: CYK, PCFG (Probabilistic Context Free Grammar), Shift Reduce Parser
- 3.7 Top Down Parsers: Early Parser, Predictive Parser
- 3.8 Self-Learning: Evaluating parsers, Parsers-based language modelling, Regional languages POS tree banks
Module IV — Semantic Analysis (7 hours)
- 4.1 Introduction, meaning representation
- 4.2 Lexical Semantics, Corpus study
- 4.3 Language dictionaries: WordNet, BabelNet
- 4.4 Relations among lexemes & their senses
- Homonymy, Polysemy, Synonymy, Hyponymy
- 4.5 Semantic Ambiguity
- 4.6 Word Sense Disambiguation (WSD)
- Knowledge-based approach: Lesk’s Algorithm
- Supervised: Naïve Bayes, Decision List
- Semi-supervised: Yarowsky
- Unsupervised: HyperLex
- 4.7 Self-Learning: Dictionaries for regional languages, Distributional Semantics, Topic Models
Module V — Pragmatic & Discourse Processing (5 hours)
- 5.1 Discourse: Reference Resolution
- 5.2 Reference Phenomena
- 5.3 Syntactic & Semantic constraints on coherence
- 5.4 Anaphora Resolution using Hobbs and Centering Algorithm
- 5.5 Self-Learning: Discourse segmentation, Conference resolution
Module VI — Applications of NLP (5 hours)
- 6.1 Case studies (preferably in a regional language):
- Machine Translation
- Text Summarization
- Sentiment Analysis
- Information Retrieval
- Question Answering System
- 6.2 Self-Learning: Applications based on Deep Neural Networks with NLP — LSTM, Recurrent Neural Networks, etc.
Text Books
| # | Title | Author(s) | Publisher |
|---|---|---|---|
| 1 | Speech and Language Processing (2nd Ed.) | Daniel Jurafsky, James H. Martin | Prentice Hall, 2008 |
| 2 | Foundations of Statistical Natural Language Processing | Christopher D. Manning, Hinrich Schutze | MIT Press, 1999 |
References
| # | Title | Author(s) | Publisher |
|---|---|---|---|
| 1 | Natural Language Processing and Information Retrieval | Siddiqui, Tiwary U.S. | Oxford University Press, 2008 |
| 2 | Multilingual Natural Language Processing Applications: from Theory to Practice | Daniel M Bikel, Imed Zitouni | IBM Press, 2013 |
| 3 | The Handbook of Computational Linguistics and Natural Language Processing | Alexander Clark, Chris Fox, Shalom Lappin | John Wiley and Sons, 2012 |
| 4 | Handbook of Natural Language Processing (2nd Ed.) | Nitin Indurkhya, Fred J. Damerau | Chapman and Hall/CRC Press, 2010 |
| 5 | A Step by Step Tutorial: An Introduction into R Application and Programming | Niel J le Roux, Sugnet Lubbe | — |
| 6 | Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit | Steven Bird, Ewan Klein, Edward Loper | O’Reilly Media, 2009 |
Online References
CML783 — Natural Language Processing Lab
List of Experiments
| Sr. No. | Experiment Title | Hours |
|---|---|---|
| 01 | Study various applications of NLP and formulate the Problem Statement for Mini Project based on chosen real-world NLP application (Machine Translation, Text Categorization, Text Summarization, Chat Bot, Plagiarism, Spelling & Grammar Checkers, Sentiment/Opinion Analysis, Question Answering, Personal Assistant, Tutoring Systems, etc.) | 2 |
| 02 | Apply various text preprocessing techniques for any given text: Tokenization and Filtration & Script Validation | 2 |
| 03 | Apply various other text preprocessing techniques for any given text: Stop Word Removal, Lemmatization / Stemming | 2 |
| 04 | Perform morphological analysis and word generation for any given text | 2 |
| 05 | Implement N-Gram model for the given text input | 2 |
| 06 | Study the different POS taggers and perform POS tagging on the given text | 2 |
| 07 | Perform Chunking for the given text input | 2 |
| 08 | Implement Named Entity Recognizer for the given text input | 2 |
| 09 | Implement Text Similarity Recognizer for the chosen text documents | 2 |
| 10 | Exploratory Data Analysis of a given text (Word Cloud) | 2 |
| 11 | Mini Project Report: For any one chosen real-world NLP application | 2 |
| 12 | Implementation and Presentation of Mini Project | 2 |
Assignments / Tutorials
| Sr. No. | Description | Hours |
|---|---|---|
| 01 | Assignment covering Modules I–III (limited to 3 questions) | 2 |
| 02 | Assignment covering Modules IV–VI (limited to 3 questions) | 2 |
Lab Text Books & References
(Same as theory — see Text Books and References sections above.)