Selected Problems in Machine Learning

The seminar focuses on deeper understanding of selected machine learning methods for students who have already have basic knowledge of machine learning and probability models. The first half of the semester is devoted to methods of unsupervised learning using Bayesian inference (Dirichlet process, Expectation Maximization, Gibbs sampling) and implementation of these methods on selected tasks. Other two lectures will be devoted to inspecting deep neural networks. Further topics are selected according to students interest.

About

SIS code: NPFL097
Semester: winter
E-credits: 3
Examination: 0/2 C
Guarantor: David Mareček

Timespace Coordinates

The seminar is held on Thursday, 9:00 - 10:30 in S1 (fourth floor)

Course prerequisities

Students are expected to be familiar with basic probabilistic and ML concepts, roughly in the extent of:

  • NPFL067 - Statistical methods in NLP I,
  • NPFL054 - Introduction to Machine Learning (in NLP).

In the second half of the course, you should be familiar with the basics of deep-learning methods. I recommend to attend

Course passing requirements

  • There are three programming assignments during the term. For each one, you can obtain 10 points. When submitted after the deadline, you can obtain at most half of the points.
  • You can obtain 10 point for individual 30-minutes presentation about selected machine learning method or task.
  • You pass the course if you obtain at least 20 points.

Lectures

1. Introduction Slides Warm-Up test

2. Beta-Bernoulli probabilistic model Beta-Bernoulli Beta distribution

3. Dirichlet-Categorical probabilistic model Dirichlet-Categorical

3. Modeling document collections, Categorical Mixture models, Expectation-Maximization Document collections Categorial Mixture Models

4. Gibbs Sampling, Latent Dirichlet allocation Gibbs Sampling Latent Dirichlet allocation Latent Dirichlet Allocation

5. Text segmentation Bayessian inference with Tears Unuspervised text segmentation

6. Finding motifs Finding Motifs in DNA Finding motifs in DNA

7. Inspecting Neural Networks

8. Sentence Structures

1. Introduction

 Oct 4

  • Course overview Slides
  • revision of the basics of probability and machine learning theory Warm-Up test

2. Beta-Bernoulli probabilistic model

 Oct 11

  • answering questions from the warm-up test
  • slides for Beta-Bernoulli models by Carl Edward Rasmussen from University of Cambridge
  • How to compute expected value of the Beta distribution can be found here: Beta distribution

3. Dirichlet-Categorical probabilistic model

 Oct 18

  • slides for Dirichlet-Categorical by Carl Edward Rasmussen from University of Cambridge
  • Web application showing the Beta-Bernouli distribution and many others can be found at RandomServices.com. models by Carl Edward Rasmussen from University of Cambridge

3. Modeling document collections, Categorical Mixture models, Expectation-Maximization

 Oct 25

  • slides for Document collections and Categorial Mixture Models by Carl Edward Rasmussen from University of Cambridge
  • Expectation Maximization is also very well described in Chapter 9 in the Bishop's book: Pattern Recognition and Machine Learning

4. Gibbs Sampling, Latent Dirichlet allocation

 Nov 8

5. Text segmentation

 Nov 29

6. Finding motifs

 Dec 6 Finding Motifs in DNA

7. Inspecting Neural Networks

 Dec 13

  • Deep neural networks in NLP as a BlackBox
  • What is being learned in their hiden states?
  • How the attention mechanism works?

8. Sentence Structures

 Dec 20

  • Sentence structures in NLP
  • Utilization of sentence structures in downstream NLP tasks
  • Are some kinds of sentence structures learned latently inside deep networks?

Latent Dirichlet Allocation

 Deadline: Nov 14 23:59  10 points  Duration: 2h

Unuspervised text segmentation

 Deadline: Dec 5 23:59  10 points  Duration: 2h

Finding motifs in DNA

 Deadline: Dec 19 23:59  10 points  Duration: 2h

  • Christopher Bishop: Pattern Recognition and Machine Learning, Springer-Verlag New York, 2006 (read here)

  • Kevin P. Murphy: Machine Learning: A Probabilistic Perspective, The MIT Press, Cambridge, Massachusetts, 201i2 (read here)