SIS code: 
Semester: 
winter
E-credits: 
5
Examination: 
2/2 C+Ex

 

 

Úvod do strojového učení

Introduction to machine learning

Time and place

Lecture

  • Czech    Wed, 10:40 12:10, S4
  • English  Thu, 9:00  10:30, S3

Lab session

  • Czech   Mon, 17:20  18:80, SU2
                   Thu, 12:20
     13:50, SU2
  • English  Fri, 9:00  10:30, SU2

Math and programming requirements

Probability and statistics

  • The most important requirements from probability and statistics are listed here: Preliminaries.Probability-Statistics
  • Make sure that you are familiar at least with the very basics: Prob-Stat.zaklady.2014
  • As to the MFF students, we expect the knowledge covered in the obligatory course "Pravděpodobnost a statistika" (NMAI059).
  • Gentle entry test in probability and statistics – a brief evaluation.

R programming

  • You can start with a simple tutorial Tutorial-on-R.2013
  • If you are not familiar with elementary R functions, use the resources listed below.

Calendar 2019/20

  Lecture date Lecture Lab
1. 2–3/10 Introduction to Machine Learning
    – What is Machine Learning
    – Basic formal concepts
    – Entropy, its meaning and definition
    – Overview of the course  
    – Requirements for getting credits  
 

Annotation experiment and data analysis
    – Practical experience with manual annotation
    – Annotation data analysis
    – Inter-annotator agreement
    – Confusion matrices and error analysis

A gentle tutorial on elementary data analysis in R
    – with homework

2. 9–10/10 Data analysis
    – Basic data exploration
    – Association between attributes
    – K-Means clustering
    – Hierarchical agglomerative clustering
 

R script
    – Feature frequency on the MOV data set
    – K-means on the USArrests data set
    – Hierarchical clustering on the USArrests data set


Wed 9/10
    – HW #1 assignment

3. 16–17/10

Working with data, evaluation, overfitting

Intro to Decision Trees and Random Forests

Tutorial on probability distributions and entropy in R
    – Data: xy.100.csv

Hints on computing entropy in R

Tutorial on Decision Trees

4. 23–24/10 Linear regression, Logistic regression Fri 25/10
    – HW #1 early submission date
5. 30–31/10 More about evaluation, thorough statistical tests

Mon 28/10 lab canceled (State holiday)

Wed 30/10
     – HW #1 late submission date
     – HW #2  assignment   

6. 6–7/11 More supervised learning algorithms  
7. 13–14/11 Ensemble learning methods Wed 13/11
     – HW #2 early submission date
8. 20–21/11

Wed 20/11
    – Obligatory written test in the lecture time
    – Final Homework HW #3 assignment

Thu 21/11 lecture canceled (Open Door Day)

Wed 20/11
     – HW #2 late submission date

Thu 21/11 lab canceled (Open Door Day)

Fri 22/11
    – Obligatory written test in the lab time
    – Final Homework HW #3 assignment

9. 27–28/11 The curse of dimensionality and feature selection  
10. 4–5/12 Support Vector Machines  
11. 11–12/12 Regularization, PCA  
12. 18–19/12 Fundamentals of Neural Networks  
13. 8–9/1 Wed 8/1 and Thu 9/1
    – Obligatory final written test
in the lecture time
Mon 8/1
     – HW #3 hard deadline
     

 

Literature

Recommended readings

  • James, Gareth and Witten, Daniela and Hastie, Trevor and Tibshirani, Robert. An Introduction to Statistical Learning. Springer New York, 2013. (link
  • Lantz, Brett. Machine learning with R. Packt Publishing Ltd. 2013. [available  in the MFF library]
  • Barbora Hladká — Martin Holub — Vilém Zouhar: A Collection of Machine Learning Excercises

Introductory readings

  • Alpaydin, Ethem. Introduction to Machine Learning. The MIT Press. 2004, 2010. (link)
  • Domingos, Pedro. A few useful things to know about Machine learning. Communication of the ACM, vol. 55, Issue 10, October 2012, pp. 78--87, ACM, New York, USA. (link)
  • Gonick, Larry and Woollcott Smith. The Cartoon Guide to Statistics. Harper Resource. 2005.
  • Hladká Barbora, Holub Martin: A Gentle Introduction to Machine Learning for Natural Language Processing: How to start in 16 practical steps.In: Language and Linguistics Compass, vol. 9, No. 2, pp. 55-76, 2015.
  • Hladká Barbora, Holub Martin: Machine Learning in Natural Language Processing using R. Course at ESSLLI2013, 2013.
  • Kononenko, Igor and Matjaz Kukar. Machine Learning and Data Mining: Introduction to Principles and Algorithms. Horwood Publishing, 2007. (linka light survey of the whole field)

Advanced readings

  • Baayen, R. Harald. Analyzing Linguistic Data: A Practical Introduction to Statistics using R. Cambridge University Press, 2008.
  • Bishop, Christopher M. Pattern Recognition and Machine Learning. Springer, 2006.
  • Burges Christopher J. C.  A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery, 2(2):121–167, 1998. (link)
  • Cristianni, Nello and John Shawe-Taylor. An Introduction to Support Vector M​achines and other Kernel-based Learning Methods. Cambridge University Press, 2000.
  • Duda, Richard O., Peter R. Hart and David G. Stork. Pattern Classification. Second Edition. Wiley, 2001.
  • Guyon, Isabelle and Gunn, Steve and Nikravesh, Masoud and Zadeh, Lotfi A. Feature Extraction: Foundations and Applications (Studies in Fuzziness and Soft Computing). Springer-Verlag New York, Inc. 2006.
  • Hastie, Trevor, Robert Tibshirani and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2009. (link)
  • Hsu Chih-Wei, Chang Chih-Chung Chang and Chih-Jen Lin. A Practical Guide to Support Vector Classication. 2010. (link)

About the R system

  • Everitt, B.S and Hothorn, Torsten. A Handbook of Statistical Analyses using R. CRC Press. 2010.
  • Dalgaard, Peter. Introductory Statistics with R. Springer, 2008.
  • Kerns, G. Jay. Introduction to Probability and Statistics Using R. 2011. (link) ​
  • Paradis, Emmanuel. R for Beginners. 2005. (link)
  • Rodrigue, German. Introducing R -- Getting started. (link)
  • Venables, W.N, D. M. Smith and the R core team. An Introduction to R. (link)
  • Venables, W. N. and B. D. Ripley. Modern Applied Statistics with S. Springer, 2002. (link)

Sample student projects from the past

This course was originally focused on machine learning in natural language processing. To get credits for lab sessions, students needed to do experimental projects

  Default Task Default Task Description Nice Student Reports
2014/15 Native Language Identification npfl054-term-project-2014-15.pdf
CUNI report
 
2013/14

Reuters-21578 Text Categorization

text-categorization.pdf
test-collection.README.txt
3-classes.distribution.pdf
Default task:
    Luksova.report.final.2013-14.pdf
Sentiment analysis task:
    Tam.report.final.2013-14.pdf
2012/13 Word Sense Disambiguation PFL054.project.2012-13.pdf Barancikova.report.final.2012-13.pdf
Machacek.report.final.2012-13.pdf
Franky.report.final.2012-13.pdf
2011/12 Semantic Pattern Classification PFL054.project.2011-12.specification.pdf Krejcova.report.final.2011-12.pdf
Long.report.final.2011-12.pdf
Tamchyna.report.final.2011-12.pdf
2010/11 Semantic Collocation Recognition PFL054.project.2010-11.pdf,
features.description.pdf
Lauschmannova.report.final.2010-11.pdf
Hajic.report.updated.2010-11.pdf
Kriz.report.final.2010-11.pdf
2009/10 Verb Sense Disambiguation PFL054_2009_10_project.pdf ML_report_Fabian.pdf,
ML_report_Galuscakova.pdf,
ML_report_Larasati.pdf
2008/09 Coreference Resolution

PFL054_2008_09_project.pdf

ML_report_Dusek.pdf,
ML_report_LeThanhDinh.pdf,
ML_report_Novak.pdf
2007/08 Named-entity Type Classification PFL054_2007_08_project.pdf Jana.Kravalova-FinalReport.pdf,
Sergio.Duante-finalReport.pdf,
Zorana.Ratkovic-Final_Report.pdf

 

Other machine learning courses organized by UFAL

  • NPFL097 Selected problems in machine learning
  • NPFL104 Machine learning exercises 
  • NPFL114 Deep learning

MFF UK Internal Regulations