Lectures

1. Introduction

 1 October

Questions

• What are dialogue systems
• Common usage areas
• Task-oriented vs. non-task oriented systems
• Closed domain, multi-domain, open domain
• System vs. user initiative in dialogue
• Standard dialogue systems components
• Research forefront
• TTS audio examples: formant, concatenative, HMMs, neural

Homework Assignments

There will be 6 homework assignments + 2 bonuses, each for a maximum of 10 points. Please see details on grading and deadlines on a separate tab.

Assignments should be submitted via Git – see instructions on a separate tab.

All deadlines are 23:59:59 CET/CEST.

Note: If you don't have a faculty Gitlab account yet, please create one as soon as possible (see the instructions). Don't wait until the deadline! It takes 5 minutes, and if you don't do it, you won't have any way of submitting.

Index

• Submission instructions
• Cluster tips

Homework Submission Instructions

All homework assignments will be submitted using a Git repository on MFF GitLab.

We provide an easy recipe to set up your repository below:

Creating the repository

1. Log into your MFF gitlab account. Your username and password should be the same as in the CAS, see this.
2. You'll have a project repository created for you under the teaching/NPFL099/2025 group. The project name will be the same as your CAS username. If you don't see any repository, it might be the first time you've ever logged on to Gitlab. In that case, Ondřej first needs to run a script that creates the repository for you (please let him know on Slack/email). In any case, you can explore everything in the base repository. Your own repo will be derived from this one.
3. Clone your repository.
4. Change into the cloned directory and run

git remote show origin

You should see these two lines:

* remote origin
  Fetch URL: git@gitlab.mff.cuni.cz:teaching/NPFL099/2025/your_username.git
  Push  URL: git@gitlab.mff.cuni.cz:teaching/NPFL099/2025/your_username.git

5. Add the base repository (with our code, for everyone) as your upstream:

git remote add upstream https://gitlab.mff.cuni.cz/teaching/NPFL099/base.git

6. You're all set!

Submitting the homework assignment

1. Make sure you're on your master branch

git checkout master

2. Checkout new branch:

git checkout -b hwX

3. Solve the assignment :)

4. Add new files (if applicable) and commit your changes:

git add hwX/solution.py
git commit -am "commit message"

5. Push to your origin remote repository:

git push origin hwX

6. Create a Merge request in the web interface. Make sure you create the merge request into the master branch in your own forked repository (not into the upstream).

    Merge requests -> New merge request
   

7. Wait a bit till we check your solution, then enjoy your points :)!
8. Once approved, merge your changes into your master branch – you might need them for further homeworks (but feel free to branch out from the previous homework in your next one if we're too slow with checking).

Updating from the base repository

You'll probably need to update from the upstream base repository every once in a while (most probably before you start implementing each assignment). We'll let you know when we make changes to the base repo.

To upgrade from upstream, do the following:

1. Make sure you're on your master branch

git checkout master

2. Fetch the changes

git fetch upstream master

3. Apply the diff

git merge upstream/master master

Automated Tests

You can run some basic sanity checks for homework assignments -- they are included in your repository (make sure to upgrade from upstream first). Note that the tests require stuff from requirements.txt to be installed in your Python environment. The tests assume checking in the current directory, they assume you have the correct branches set up.

For instance, to check hw1, run:

./run_tests.py hw1

By default, this will just check your local files. If you want to check whether you have your branches set up correctly, use the --check-git parameter. Note that this will run git checkout hw1 and git pull, so be sure to save any local changes beforehand!

Always update from upstream before running tests, we're adding checks for new assignments as we go. Some may only be available at the last minute, we're sorry for that!

AIC Cluster Computation Tips

This is just a short primer for the AIC wiki – better read that one, too. But definitely read at least this text before you start working with AIC.

Login

Use the command

ssh LOGIN@aic.ufal.mff.cuni.cz

where LOGIN is your SIS username.

On the head node

When you log on to AIC, you're at the cluster head node. Do not compute here – this just for launching computation jobs, copying files and such. All of your computation jobs will run on one of the CPU/GPU nodes. (You can run the terminal multiplexing program on the head node.)

Workflow

There are two ways to compute on the cluster:

1. submitting a batch script containing the commands you wish to run,
2. executing commands in an interactive shell.

You should use a batch script for running longer computations. The interactive shell is useful for debugging.

Batch scripts

Use the sbatch command to submit your jobs (i.e. shell scripts) into a queue. For running a python command, simply create a shell script that has one line – your command with all the parameters you need.

You can either specify the parameters in the script or on the command line.

Here are two equivalent ways of specifying a GPU job with 2 CPU cores, 1 GPU and 16G system RAM (all GPUs have 11G memory):

Using SBATCH directives

1. Create a file job_script.sh:

#!/bin/bash
#SBATCH -J hello_world					  # name of job
#SBATCH -p gpu      					  # name of partition or queue (if not specified default partition is used)
#SBATCH --cpus-per-task=2                 # number of cores/threads per task (default 1)
#SBATCH --gpus=1                          # number of GPUs to request (default 0)
#SBATCH --mem=16G                         # request 16 gigabytes memory (per node, default depends on node)

# here start the actual commands
sleep 5
echo "Hello I am running on cluster!"

2. Execute the command:

sbatch job_script.sh

Using command-line parameters

1. Create a file job_script.sh:

#!/bin/bash

sleep 5
echo "Hello I am running on cluster!"

2. Execute the command:

sbatch -J hello_world -p gpu -c2 -G1 --mem 16G job_script.sh

Have a look at the AIC wiki or man sbatch for all the command-line parameters.

(Note: long / short flags can be used interchangeably for both approaches.)

Interactive jobs

You can get an interactive console using srun.

The following command will run bash with the same resources as in the previous example:

srun -J hello_world -p gpu -c2 -G1 --mem=16G --pty bash

Important notes

• Always request the resources you'll need! (both for the batch and interactive jobs).
• With interactive jobs, don't forget to exit the console after use – you're blocking the GPU and whatever you reserve as long as the console is open!
• Don't submit too many jobs at a time (don't overfill the cluster, leave space for others).
• You can't request more than 8 CPU cores or 1 GPUs.

Other commands

• Use sinfo to list the available queues.
• Use squeue --me or squeue -u LOGIN (where LOGIN is your username) to check your jobs.
• Use squeue to see every job currently running on the cluster.
• Use scancel JOB_ID to cancel a job.

Tips for efficient development

• terminal multiplexing: byobu (or tmux, screen) allows you to have multiple terminal sessions and keeping everything open even if you lose connection (see a quick primer on byobu).
• SSH connection: mosh is able to reestablish the SSH connection after the connection is interrupted (unstable internet, turning the computer off, etc.).
• file management: you can access the AIC filesystem using SFTP: sftp://LOGIN@aic.ufal.mff.cuni.cz
  • many default Linux file managers (Nautilus, Nemo, etc.) support SFTP natively (simply press Ctrl+L),
  • for Windows use WinSCP,
  • for command line access use mc (F9-“Shell link”) or rsync.
• development: the SSH plugin for VS Code enables you to code in IDE directly on the cluster.
• Python:
  • virtual environments – managing Python packages locally,
  • pdb++, pudb, ipdb – advanced Python command line debugging,
  • gpustat – overview of GPUs on the current node,
  • miniconda – if you need to use different version than Python 3.10.6 (which is the system one).

Links

• SLURM tutorial for ÚFAL cluster: https://wiki.ufal.ms.mff.cuni.cz/slurm
• SLURM cheatsheet: https://www.carc.usc.edu/user-information/user-guides/hpc-basics/slurm-cheatsheet
• sbatch manpage: https://manpages.org/sbatch

Exam Question Pool

The exam will have 10 questions from the pool below. Each question counts for 10 points. We reserve the right to make slight alterations or use variants of the same questions. Note that all of them are covered by the lectures, and they cover most of the lecture content. In general, none of them requires you to memorize formulas, but you should know the main ideas and principles. See the Grading tab for details on grading.

Introduction

• What's the difference between task-oriented and non-task-oriented systems?
• Describe the difference between closed-domain, multi-domain, and open-doman systems.
• Describe the difference between user-initiative, mixed-initiative, and system-initiative systems.
• List the main components of a modular task-oriented dialogue system (text/voice-based)
• What is the task (input/output) of speech recognition in a dialogue system?
• What is the task (input/output) of speech synthesis in a dialogue system?

Data & Evaluation

• What are the usual approaches to collecting dialogue data (name at least 2)?
• How does Wizard-of-Oz data collection work?
• What are common annotation types you need for dialogue systems and how can you obtain them?
• What’s the difference between intrinsic and extrinsic evaluation?
• What is the difference between subjective and objective evaluation?
• What are the common options of getting people to evaluate your system?
• What are some evaluation metrics for non-task-oriented systems (chatbots)?
• How would you evaluate NLU (both slots & intents)?
• Explain an NLG evaluation metric of your choice.
• Describe how BLEU works (in principle, exact formulas not needed).
• Why is word-overlap evaluation (such as BLEU) problematic?
• Describe a metric for NLG other than BLEU.
• Show at least 2 examples of subjective (human) evaluation metrics for dialogue systems.
• How can you use an LLM to evaluate dialogue system?
• What is significance testing and what is it good for?
• Assume you have dialogue systems A and B, and A performs better than B in terms of response BLEU on a dataset of 100 dialogues. Describe how you’d test for significance.
• Why do you need to evaluate on a separate test set?

Neural Nets Basics

• What's the difference between classification and regression as a machine learning problem?
• Describe the task of sequence prediction (=autoregressive generation).
• What's the difference between classification and ranking as a machine learning problem?
• What's the difference between sequence labeling and sequence prediction?
• What is an embedding, what units can it relate to, and how can you obtain it?
• What are subwords and why are they useful?
• What's an encoder-decoder model and how does it work?
• How does an attention model work?
• What's the main principle of operation for convolutional networks?
• What's the difference between LSTM/GRU-based and Transformer-based architecture?
• Describe the basics of a Transformer language model.
• What's a pretrained language model?
• Describe the typical architecture of large language models.

Training Neural Nets

• Describe the principle of stochastic gradient descent.
• Why is it important to choose an appropriate learning rate?
• Describe an approach for learning rate adjustment of your choice.
• What is dropout, what is it good for and why does it work?
• What’s a variational autoencoder and how does it differ from a “regular” autoencoder?
• What is masked language modeling?
• How do Generative Adversarial Networks work?
• Describe the principle of the pretraining+finetuning approach.
• How does clustering work?
• What is self-supervised training and why is it useful?
• Describe 2 examples of a self-supervised training task.
• Can you apply a pretrained language model for a new task without finetuning it at all?
• How can you finetune a language model without updating all the parameters?
• What's instruction tuning and what is it good for?
• How are LLMs typically trained, in addition to basic next-word prediction training?
• How does an input into an instruction-tuned “chat” LLM typically look like?

Natural Language Understanding

• Design a (sketch) of an NLU neural architecture that joins intent detection and slot tagging.
• Describe language understanding as classification and language understanding as sequence tagging.
• What is delexicalization and why is it helpful in NLU?
• Describe one of the approaches to slot tagging as sequence tagging.
• What is the IOB/BIO format for slot tagging?
• How can you use pretrained language models (e.g. BERT) for NLU?
• How can you combine rules and neural networks in NLU?
• How can an NLU system deal with noisy ASR output? Propose an example solution.
• What is a copy mechanism/pointer network?

Dialogue State Tracking

• What is the point of dialogue state tracking in a dialogue system?
• What is the difference between dialogue state and belief state?
• What's the difference between a static and a dynamic state tracker?
• What's a partially observable Markov decision process?
• Describe a viable architecture for a belief tracker.
• What is the difference between state trackers as classifiers vs. as candidate rankers?
• Describe the principle of state tracking as span selection.
• Describe a generation-based dialogue state tracker.

Dialogue Policies

• Describe the basic reinforcement learning setup (agent, environment, actions, rewards).
• Map the general reinforcement learning setup to a dialogue system situation.
• Why is reinforcement learning preferred over supervised learning for training dialogue managers?
• What are V and Q functions in a reinforcement learning scenario?
• What's the difference between actor and critic methods in reinforcement learning?
• Describe a Deep Q Network.
• Describe the REINFORCE approach.
• What’s the main principled difference between Deep Q-Networks and Policy Gradient methods?
• What are actor-critic reinforcement learning methods?
• What’s the difference between on-policy and off-policy optimization?
• Why do you typically need a user simulator to train a reinforcement learning dialogue policy?
• Give an example of possible turn-level or dialogue-level rewards for RL optimization.
• What is a user simulator? What are some common approaches to building one?
• How can you use LLM as a tool to help learn a task-oriented policy?

Natural Language Generation

• What are the main steps of a traditional NLG pipeline – describe at least 2.
• Describe a handcrafted approach to NLG.
• What are the main problems with vanilla neural NLG systems (base RNN/Transformer) in dialogue response generation?
• What is delexicalization and why is it helpful in NLG?
• Describe a possible neural approach to NLG with an approach to combat hallucination.
• How can you use pretrained language models or LLMs in NLG?
• What are the typical decoding approaches in neural language model based NLG? Explain & contrast at least 2.
• What can you do with LLMs to reduce hallucinations in NLG?

End-to-end Models

• What are some pros and cons of end-to-end task-oriented models over traditional modular ones?
• Describe an example structure of an end-to-end task-oriented dialogue system.
• Describe the Sequicity (2-step decoding) task-oriented dilaogue model.
• Describe an end-to-end task-oriented model based on pretrained language models.
• How would you adapt a pretrained language model for an end-to-end task-oriented dialogue system?
• How would you use reinforcement learning to train an end-to-end task-oriented model?
• Why is it a bad idea to train end-to-end dialogue systems only with reinforcement learning on word level?
• Describe a viable LLM-based end-to-end task-oriented approach.
• How can you represent dialogue state for end-to-end neural task-oriented models?

Chatbots

• What are the three main approaches to building non-task-oriented chatbots?
• How does the Turing test work? Does it have any weaknesses?
• What are some techniques rule-based chitchat chatbots use to convince their users that they're human-like?
• Describe how a retrieval-based chitchat chatbot works.
• Why is a vanilla seq2seq (RNN/Transformer) architecture for chatbots problematic?
• Describe an example approach to improving diversity or coherence in a seq2seq-based chitchat chatbot.
• How can you use a pretrained language model in a chitchat chatbot?
• Describe a possible architecture of a hybrid/ensemble chitchat chatbot.
• How does retrieval-augmented generation work?
• How can you enforce a consistent personality in an LLM-based chitchat chatbot?

Multimodality

• How does the structure of traditional (modular) multimodal dialogue systems differ from non-multimodal ones?
• Give an example of 3 alternative input modalities (i.e. not voice/text).
• Give an example of 3 alternative output modalities (i.e. not voice/text).
• How would you build a multimodal end-to-end neural dialogue system (e.g. for visual dialogue)?
• Explain some problems that may occur when a dialogue system talks to two people at once.
• What’s the difference between image classification and object detection?
• How would you build a neural end-to-end image-based dialogue system (consider using pretrained components)?
• What is the task of visual dialogue about?
• How do multimodal LLMs deal with image or audio inputs?

Linguistics & Ethics

• What are turn taking cues/hints in a dialogue? Name a few examples.
• What is a barge-in?
• What is grounding in dialogue?
• Give some examples of grounding signals in dialogue.
• What is entrainment/alignment/adaptation in dialogue?
• Describe the overgeneralization/overconfidence problem in data-driven NLP models.
• Describe the demographic bias problem in data-driven NLP models.
• Give an example of a user safety concern in dialogue systems.
• What's the problem with training neural models on private data?

Course Grading

To pass this course, you will need to:

1. Take an exam (a written test covering important lecture content).
2. Do lab homeworks (implementing an end-to-end dialogue system + other tasks).

Exam test

• There will be a written exam test at the end of tihe semester.
• There will be 10 questions, we expect at least 2-3 sentences as an answer, with a maximum of 10 points per question.
  • You may get bonus points for correct and more detailed answers.
  • A list of possible exam questions is available on the website.
• To pass the course, you need to get at least 50% of the total points from the test (but 60% overall, see below).
• If needed, there will be exam dates in the summer.

Homework assignments

• There will be 6 homework assignments, introduced every other week, plus some bonus assignments.
• For each assignment, you will get a maximum of 10 points.
  • You may get a bonus point for nice code.
• You need to get at least 40 assignment points (>66%) to pass the course, and they count towards your grade (see below).
• You will submit the homework assignments into your private Gitlab repository.
• All assignments will have a fixed deadline (typically 2-3 weeks).
  • You can ask for a deadline extension if you have a good reason, such as illness.
• If you submit the assignment after the deadline, you will get:
  • up to 50% of the maximum points if it is less than 2 weeks after the deadline;
  • 0 points if it is more than 2 weeks after the deadline.
  • Any bonus points you get will not be lowered.
• Note that most assignments depend on each other! That means that if you miss a deadline, you still might need to do an assignment without points in order to score on later assignments.
• Once we check the submitted assignments, you will see the points you got and the comments from us on Gitlab, in a comment on your merge request.
  • Please bear with us for the checking, it's harder than it looks.
  • If your unsure about your total points, feel free to ask.
• You can take the exam even if you don't reach 40 homework assignment points, but you need to get at least 40 points to get the final grade.

Grading

The final grade for the course will be a combination of your exam score and your homework assignment score, weighted 3:1 (i.e. the exam accounts for 75% of the grade, the assignments for 25%).

Grading:

• Grade 1: >=87% of the weighted combination
• Grade 2: >=74% of the weighted combination
• Grade 3: >=60% of the weighted combination
• An overall score of less than 60% means you did not pass.

In any case, you need >50% of points from the test and 40+ points (i.e. 66%) from the homeworks to pass. If you get less than the minimum from either, even if you get more than 60% overall, you will not pass.

No cheating

• Cheating is strictly prohibited and any student found cheating will be punished. The punishment can involve failing the whole course, or, in grave cases, being expelled from the faculty.
• Discussing homework assignments with your classmates is OK. Sharing code is not OK (unless explicitly allowed); by default, you must complete the assignments yourself.
• All students involved in cheating will be punished. E.g. if you share your assignment with a friend, both you and your friend will be punished.
• Discussing the assignment with an AI is OK, as long as you fully understand the final code. Having the AI write everything for you is not OK.

Recommended Reading

You should be able to pass the course just by following the lectures, but here are some hints on further reading. There's nothing ideal on the topic as this is a very active research area, but some of these should give you a broader overview.

Recommended, most up-to-date thing on the matter (but not the ideal focus):

• Jurafsky & Martin: Speech & Language processing. 3rd ed. draft
  • general LLM intro is in Chapters 2-8
  • intro to dialogue is in Chapter 26
  • RAG-based systems are in Chapter 11
  • component-based (pre-LLM) systems are in Appendix K
  • you may prefer the Jan. 2025 version of Chapter 15 for a more compact intro to dialogue systems

Also recommended, more detailed and focused but a bit outdated (pre-LLM):

• McTear: Conversational AI: Dialogue Systems, Conversational Agents, and Chatbots. Morgan & Claypool 2021.

  • available as an e-book from our library)

• Gao et al.: Neural Approaches to Conversational AI. arXiv:1809.08267

Further reading:

• Reiter: Natural Language Generation. Springer 2024.

  • this is specific to just language generation but up-to-date and very relevant, including evaluation
  • there's a companion website and the whole book is on arXiv.

• Current papers from the field (see individual gray links on lecture slides)