Lectures

“Introduction to the course”

• Chapter 4: Naive Bayes classification and sentiment in Jurafsky and Martin (3rd edition).

“In this lecture, we will discuss language models, i.e. modelling word sequences, and part-of-speech tagging”

• Chapter 3: Language modelling with n-grams in Jurafsky and Martin (3rd edition).

• Chapter 8: Part-of-speech tagging in Jurafsky and Martin (3rd edition).

“In this lecture, we will discuss morphological processing and syntactic parsing”

• Lecture notes on moprphology are available here

• Chapter 12: Constituency grammars in Jurafsky and Martin (3rd edition).

• Chapter 13: Constituency parsing in Jurafsky and Martin (3rd edition).

“In this lecture, we will discuss lexical semantics, i.e. modelling the meaning of words.”

• Chapter 19: Word Senses and WordNet in Jurafsky and Martin (3rd edition).

“In this lecture, we will introduce statistical models of word meaning”

• Chapter 6: Vector semantics and embeddings in Jurafsky and Martin (3rd edition).

“In this lecture, we will discuss generalisation from words to semantic classes and learning dense vector representations - word embeddings.”

• Chapter 6: Vector semantics and embeddings in Jurafsky and Martin (3rd edition).

• A gentle introduction to neural networks can be found here

• The following paper provides a nice explanation of skip-gram with negative sampling: Yoav Goldberg and Omer Levy. word2vec Explained: Deriving Mikolov et al.’s Negative-Sampling Word-Embedding Method

“In this lecture, we will discuss compositional semantics, i.e. modelling the meaning of phrases and sentences, and learning neural representations of sentences.”

• Chapter 7: Neural networks and neural language models in Jurafsky and Martin (3rd edition).

• Chapter 9: Sequence processing with recurrent neural networks in Jurafsky and Martin (3rd edition).

• A good and general reference for Neural Networks in NLP: Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing

• A gentle introduction to LSTMs is available here

• This is one of the papers that have introduced tree LSTM models: Kai Sheng Tai, Richard Socher, and Christopher D. Manning. Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

“In this lecture, we will discuss discourse processing, i.e. modelling larger text fragments.”

• Chapter 22: Coreference resolution in Jurafsky and Martin (3rd edition).

• Chapter 23: Discourse coherence in Jurafsky and Martin (3rd edition).

“In these two guest lectures, we will discuss the relationship between some NLP techniques and human sentence processing, and then introduce an important NLP application – dialogue modelling.”

• Chapter 26: Dialogue systems and chatbots in Jurafsky and Martin (3rd edition).

“In this lecture, we will talk about language generation and cover a particular language generation task, text summarisation, in more detail.”

• A survey of recent summarisation techniques is available here

• An introduction to sequence-to-sequence models: Sequence to Sequence Learning with Neural Networks

“We will discuss the fundamentals of machine translation, including word-based / alignment models and phrase-based SMT”

• Explanation of IBM models 1 and 2

• Word based IBM models

• Phrase-based SMT and this book

• Neural Encoder-Decoder

• The Annotated Encoder-Decoder blog post

“We will discuss the applications of NLP in the area of social media analysis and the challenges associated with processing the language of social media.”

In this lecture, we will discuss the differences between frequentism and Bayesian modelling. We will discuss the concept of a prior and Bayesian inference. The model we will use to illustrate concepts is the Dirichlet-Multinomial model, the base for models such as Bayesian mixture models, HMM, and LDA. For approximate inference, we will discuss MCMC and in particular Gibbs sampling.

• • For a POS tagging model: A Fully Bayesian Approach to Unsupervised Part-of-Speech Tagging
  • For a PCFG model: Bayesian Inference for PCFGs via Markov chain Monte Carlo
  • A very special type of mixture model: Bayesian Word Alignment for Statistical Machine Translation
  • The classict of all times: Latent Dirichlet Allocation
    • This paper focusses on a different type of approximate inference technique (not MCMC, but rather variational inference), in ML4NLP we cover it in great detail (in particular, this is the class of algorithms we use to do probabilistic modelling with neural networks)
  • If you are interested in Bayesian non-parametric methods for NLP, check Sharon Goldwater’s thesis, it’s remarkably well written and clear!