The lecture content, including references to some additional study materials. The main study material is the Pattern Recognition and Machine Learning by Christopher Bishop, referred to as PRML.

Note that the topics in italics are not required for the exam.

1. Introduction to Machine Learning

 Sep 29, Sep 30 Slides PDF Slides

Learning objectives. After the lecture you should be able to…

• Explain to a non-expert what machine learning is.
• Explain the difference between classification and regression.
• Implement a simple linear-algebra-based algorithm for training linear regression.

Covered topics and where to find more:

• Introduction to machine learning
• Basic definitions [Sections 1 and 1.1 of PRML]
• Linear regression model [Section 3.1 of PRML]

Requirements

To pass the practicals, you need to obtain at least 70 points, excluding the bonus points. Note that up to 40 points above 70 (both bonus and non-bonus) will be transferred to the exam in a conversion rate 2:1 (i.e., 40 assignment points counts as 20 exam points). In total, assignments for at least 105 points (not including the bonus points) will be available.

Please try to implement the solutions yourself. Recent studies show that blindly relying on AI assistants negatively impacts skill formation. If you use AI assistants, use them responsibly. For instance, you can use a sutiable AGENTS.md file (this one comes from the Stanford CS336 course) that prompts models help you and tutor you on how to do the assignment, rather than filling in the TODOs.

When you submit a solution, you must be able to explain the submitted code when asked to do so. Solution that the authors cannot explain might be removed and considered as not submitted.

Environment

The tasks are evaluated automatically using the ReCodEx Code Examiner.

The evaluation is performed using Python, scikit-learn, NumPy, SciPy, pandas, and Matplotlib.

Teamwork

Solving assignments in teams (of size at most 3) is encouraged, but everyone has to participate (it is forbidden not to work on an assignment and then submit a solution created by other team members). All members of the team must submit in ReCodEx individually, but can have exactly the same sources/models/results. Each such solution must explicitly list all members of the team to allow plagiarism detection using this template.

No Cheating

Cheating is strictly prohibited and any student found cheating will be punished. The punishment can involve failing the whole course, or, in grave cases, being expelled from the faculty. While discussing assignments with any classmate is fine, each team must complete the assignments themselves, without using code they did not write (unless explicitly allowed). Of course, inside a team you are allowed to share code and submit identical solutions. Note that all students involved in cheating will be punished, so if you share your source code with a friend, both you and your friend will be punished. That also means that you should never publish your solutions.

In the competitions, your goal is to train a model and then predict target values on the test set available only in ReCodEx.

Submitting to ReCodEx

When submitting a competition solution to ReCodEx, you should submit a trained model and a Python source capable of running it.

Furthermore, please also include the Python source and hyperparameters you used to train the submitted model. But be careful that there still must be exactly one Python source with a line starting with def main(.

Do not forget about the maximum allowed model size and time and memory limits.

Competition Evaluation

• Before the deadline, ReCodEx prints the exact achieved score, but only if it is worse than the baseline.

  If you surpass the baseline, the assignment is marked as solved in ReCodEx and you immediately get regular points for the assignment. However, ReCodEx does not print the reached score.

• After the competition deadline, the latest submission of every user surpassing the required baseline participates in a competition. Additional bonus points are then awarded according to the ordering of the performance of the participating submissions.

• After the competition results announcement, ReCodEx starts to show the exact performance for all the already submitted solutions and also for the solutions submitted later.

• Each competition will be scored after the first deadline.

• The bonus points will be computed in the following fashion:

  • Let $B$ be the maximal number of bonus points that can be achieved in the competition.

  • All of the solutions that surpass the baseline will be sorted and divided into $B+1$ groups of equal size.

  • Every solution in the top group gets B points, the next group gets $B-1$ points, etc., the last group gets 0 bonus points.

  • The team solution only occupies one position in the table of the competition results.

• Please, do not forget that every member of the team needs to upload the solution to ReCodEx and to submit both the training/prediction source code and the trained model itself.

Repeated Participation in Competitions

• If a participant got non-zero points for a competition task already in previous years, they are treated slightly differently. Namely, every team with one or more returning participants still get competition points, but
  • the returning team results are not shown on the slides on the practicals;
  • the returning team results are shown in italics in ReCodEx;
  • the returning team results are not used to compute the thresholds for competition points.

What Is Allowed

• You can use only the given annotated data, both for training and evaluation.
• Additionally, you can use any unannotated or manually created data for training and evaluation.
• The test set annotations must be the result of your system (so you cannot manually correct them; but your system can contain other parts than just trained models, like hand-written rules).
• Do not use test set annotations in any way, if you somehow get access to them.
• You can use any method present in numpy or scipy, anything you implement yourself, and, unless specified otherwise in assignment description, any method from sklearn. Furthermore, the solution must be created by you, and you must understand it fully. Do not use deep network frameworks like TensorFlow or PyTorch.

Git

• Is it possible to keep the solutions in a Git repository?

  Definitely. Keeping the solutions in a branch of your repository, where you merge them with the course repository, is probably a good idea. However, please keep the cloned repository with your solutions private.

• On GitHub, do not create a public fork containing your solutions.

  If you keep your solutions in a GitHub repository, please do not create a clone of the repository by using the Fork button; this way, the cloned repository would be public.

  • If you created a public fork and want to make it private, you need to start by pressing Leave fork network in the repository settings; only then you can change the visibility to private.

  Of course, if you want to create a pull request, GitHub requires a public fork and you need to create it, just do not store your solutions in it (so you might end up with two repositories, a public fork for pull requests and a private repo for your own solutions).

• How to clone the course repository?

  To clone the course repository, run

  git clone https://github.com/ufal/npfl129
  

  This creates the repository in the npfl129 subdirectory; if you want a different name, add it as an additional parameter.

  To update the repository, run git pull inside the repository directory.

• How to merge the course repository updates into a private repository with additional changes?

  It is possible to have a private repository that combines your solutions and the updates from the course repository. To do that, start by cloning your empty private repository, and then run the following commands in it:

  git remote add course_repo https://github.com/ufal/npfl129
  git fetch course_repo
  git checkout --no-track course_repo/master
  

  This creates a new remote course_repo and a clone of the master branch from it; however, git pull and git push in this branch will operate on the repository you cloned originally.

  To update your branch with the changes from the course repository, run

  git fetch course_repo
  git merge course_repo/master
  

  while in your branch (the command git pull --no-rebase course_repo master has the same effect). Of course, it might be necessary to resolve conflicts if both you and the course repository modified the same lines in the same files.

ReCodEx

• What files can be submitted to ReCodEx?

  You can submit multiple files of any type to ReCodEx. There is a limit of 20 files per submission, with a total size of 20MB.

• What file does ReCodEx execute and what arguments does it use?

  Exactly one file with py suffix must contain a line starting with def main(. Such a file is imported by ReCodEx and the main method is executed (during the import, __name__ == "__recodex__").

  The file must also export an argument parser called parser. ReCodEx uses its arguments and default values, but it overwrites some of the arguments depending on the test being executed; the template always indicates which arguments are set by ReCodEx and which are left intact.

• What are the time and memory limits?

  The memory limit during evaluation is 1.5GB. The time limit varies, but it should be at least 10 seconds and at least twice the running time of my solution. For competition assignments, the time limit is 5 minutes.

Requirements

To pass the practicals, you need to obtain at least 70 points, excluding the bonus points. Note that up to 40 points above 70 (both bonus and non-bonus) will be transferred to the exam in a conversion rate 2:1 (i.e., 40 assignment points counts as 20 exam points). In total, assignments for at least 105 points (not including the bonus points) will be available.

To pass the exam, you need to obtain at least 60, 75, or 90 points out of 100-point exam to receive a grade 3, 2, or 1, respectively. The exam consists of 100-point-worth questions from the list below (the questions are randomly generated, but in such a way that there is at least one question from every pair of lectures).

In addition, you can get at most 20 surplus points from the practicals and at most 5 points for community work (i.e., fixing slides or reporting issues) – but only the points you already have at the time of the exam count. You can take the exam without passing the practicals first.

Another way of getting additional exam points is attending the practicals. The attendance will be registered using a on online system with one-time QR codes. There will be 11 practical sessions in the semester. You will receive 5 points for attending 6 classes (parallel Czech/English classes only count as one attendance) and one additional point for attending each class after the sixth one. I.e., the maximum number of points for participating in the practicals is 10.

Exam Questions

The exam will contain 8 questions sampled from the following list. More information about how the exams are handled at the course scale are in this blog post.

Lecture 1 Questions

Related Courses

Deep Learning by Milan Straka

Course introducing deep neural networks, from the basics to the latest advances, focusing both on theory as well as on practical aspects.

Deep Reinforcement Learning by Milan Straka

Course introducing reinforcement learning, from basic tabular methods to involvement of deep neural networks, focusing both on theory as well as on practical aspects.

Unsupervised Machine Learning in NLP by David Mareček

The seminar focuses on deeper understanding of selected unsupervised machine learning methods for students who already have basic knowledge of machine learning and probability models.

Archive

Winter 2025/26

Winter 2024/25

Winter 2023/24

Winter 2022/23

Winter 2021/22

Winter 2020/21

Winter 2019/20