SIS code: 
winter s.:3
0/2 C

Selected Problems in Machine Learning

  • Teacher: David Mareček
  • Time and location: Wednesday 9:00–10:30, S7

Course focus

The course has been designed for students with a deep interest in Machine Learning. The course is a flexible combination of lectures, discussions, exercises and literature reading, aimed at the following topics:

  1. Bayesian inference
  2. practising unsupervised ML (especially methods based on sampling)

Course prerequisities

Students are expected to be familiar with basic probabilistic and ML concepts, roughly in the extent of NPFL067/068 - Statistical methods in NLP I/II, and NPFL054 - Introduction to Machine Learning (in NLP).

Course passing requirements

  • All students are required to actively participate in the classes.
  • ~2 homeworks (programming assignments)

Course schedule

  1. October 4th: Introduction and Warm-up test - let me know what you already know
  2. October 11th: Answering questions form the Warm-up test, introduction into probability, information, and learning theories.
  3. October 18th: Beta-Bernouli and Dirichlet-Categorial models
    • slides for Beta-Bernouli and Dirichlet-Categorial models by Carl Edward Rasmussen from University of Cambridge
    • how to compute expected value of the Beta distribution can be found on YouTube
    • web application showing the Beta-Bernouli distribution and many others can be found at
  4. October 25th: Modeling document collections, Categorical Mixture models, Expectation-Maximization
  5. November 1st: Gibbs Sampling, Latent Dirichlet allocation
  6. November 15th: Assignment - Latent Dirichlet Allocation: lda-assignment.pdf,, evaluation, document perplexity
  7. November 22th: Chinese segmentation, Chinese restaurant process
  8. November 29th: Pitman-Yor process, Word alignment, Word clustering
  9. December 6th:K-means vs. Gaussian mixture models
  10. December 13th: Sampling Methods: Rejection Sampling, Importance Sampling, Metropolis-Hastings Sampling
    • see Chapter 11 in the book Christopher Bishop: Pattern Recpgnition and Machine Learning
  11. December 20th: We will only discuss problems with your homework, you do not need to come.
  12. January 10th: Final test, homework evaluation, tree structures sampling.