Foundations of Bayesian NLP
Guest lecture by Wilker Aziz.
Foundations of Bayesian NLP | Abstract
In this lecture, we will discuss the differences between frequentism and Bayesian modelling. We will discuss the concept of a prior and Bayesian inference. The model we will use to illustrate concepts is the Dirichlet-Multinomial model, the base for models such as Bayesian mixture models, HMM, and LDA. For approximate inference, we will discuss MCMC and in particular Gibbs sampling.
This paper focusses on a different type of approximate inference technique (not MCMC, but rather variational inference), in ML4NLP we cover it in great detail (in particular, this is the class of algorithms we use to do probabilistic modelling with neural networks)
If you are interested in Bayesian non-parametric methods for NLP, check Sharon Goldwater’s thesis, it’s remarkably well written and clear!