Faculty of Mathematics and Physics

- Teacher: David Mareček
- Time and location: Wednesday 9:00–10:30, S7

The course has been designed for students with a deep interest in Machine Learning. The course is a flexible combination of lectures, discussions, exercises and literature reading, aimed at the following topics:

- Bayesian inference
- practising unsupervised ML (especially methods based on sampling)

Students are expected to be familiar with basic probabilistic and ML concepts, roughly in the extent of NPFL067/068 - Statistical methods in NLP I/II, and NPFL054 - Introduction to Machine Learning (in NLP).

- All students are required to actively participate in the classes.
- ~2 homeworks (programming assignments)

**October 4th:**Introduction and**Warm-up test**- let me know what you already know- most of the questions are covered by any modern ML book, my favourite is Bishop's Pattern Recognition and Machine Learning

**October 11th:**Answering questions form the Warm-up test, introduction into probability, information, and learning theories.**October 18th:**Beta-Bernouli and Dirichlet-Categorial models- slides for Beta-Bernouli and Dirichlet-Categorial models by Carl Edward Rasmussen from University of Cambridge
- how to compute expected value of the Beta distribution can be found on YouTube
- web application showing the Beta-Bernouli distribution and many others can be found at RandomServices.com

**October 25th:**Modeling document collections, Categorical Mixture models, Expectation-Maximization- slides for introduction and categorial and mixture models by Carl Edward Rasmussen from University of Cambridge
- Expectation Maximization is also very well described in Chapter 9 in the Bishop's book Pattern Recognition and Machine Learning

**November 1st:**Gibbs Sampling, Latent Dirichlet allocation- slides for gibbs sampling and Latent Dirichlet allocation by Carl Edward Rasmussen from University of Cambridge

**November 15th:**Assignment - Latent Dirichlet Allocation: lda-assignment.pdf, lda-data.zip, evaluation, document perplexity**November 22th:**Chinese segmentation, Chinese restaurant process- slides
- inspired by the Bayessian inference with Tears tutorial by Kevin Knight (2009).

**November 29th:**Pitman-Yor process, Word alignment, Word clustering**December 6th:**K-means vs. Gaussian mixture models- Slides by David Rosenberg from the New York Univeristy.
- Good and bad examples of K-means clustering
- Implementations of K-means and GMM made by João Pedro Neto from University in Lisabon

**December 13th:**Sampling Methods: Rejection Sampling, Importance Sampling, Metropolis-Hastings Sampling- see Chapter 11 in the book Christopher Bishop: Pattern Recpgnition and Machine Learning

**December 20th:**We will only discuss problems with your homework, you do not need to come.**January 10th:**Final test, homework evaluation, tree structures sampling.