Institute of Formal and Applied Linguistics

This is the new version of the course for the '19/20 Fall semester.

Summary

This course will present advanced problems and current state-of-the-art in the field of dialogue systems, voice assistants, and conversational systems (chatbots). After a brief introduction into the topic, the course will focus mainly on the application of machine learning – especially deep learning/neural networks – in the individual components of the traditional dialogue system architecture as well as in end-to-end approaches (joining multiple components together).

This course is a follow-up to the course NPFL123 Dialogue Systems, but can be taken independently – important basics will be repeated. All required deep learning concepts will be explained, but only briefly, so some machine learning background is recommended.

Logistics

The course will be taught in English, but we're happy to explain in Czech as well.

Lectures: Thu 10:40 S1 (room changed!)
Labs: Thu 14:00 SW1
(labs will primarily take place online & in groups, with a few meetings during the semester)

To successfully finish this course, you'll need to pass a written exam (covering the lectures, especially parts mentioned in the summary) and participate in a lab project. More info on the exam is at the end of last lecture's slides.

Lectures

Slides from past lectures will appear here:

• 1st lecture: Introduction (3rd October)
• 2nd lecture: Machine learning toolkit (10th Oct)
• 3rd lecture: Data + Evaluation (17 Oct)
• 4th lecture: Language Understanding (24 Oct)
• 5th lecture: Dialogue state tracking (31 Oct)
• 6th lecture: Dialogue management/action selection (7 Nov)
• 7th lecture: Dialogue policy (2) + NLG (28 Nov)
• 8th lecture: NLG (2) + end-to-end systems (5 Dec)
• 9th lecture: End-to-end-systems (2) (12 Dec)
• 10th lecture: Chatbots (12 Dec)
• 11th lecture: Linguistics of dialogue & NLP ethics (19 Dec)
• 12th lecture: Multimodal dialogue systems (+ a bit about domain adaptation) (9 Jan)

The last lecture slides include information about the exam.

Labs

Lab assignments appear in the dedicated GitLab repository.

Topics to be covered

• Brief introduction into dialogue systems
  • dialogue systems applications
  • basic components of dialogue systems
  • knowledge representation in dialogue systems
  • data and evaluation
• Language understanding (SLU)
  • semantic representation of utterances
  • statistical methods for SLU
• Dialogue management
  • dialogue representation as a (Partially Observable) Markov Decision Process
  • dialogue state tracking
  • action selection
  • reinforcement learning
  • user simulation
  • deep reinforcement learning (using neural networks)
• Response generation (NLG)
  • introduction to NLG, basic methods (templates)
  • generation using neural networks
• Open-domain systems (chatbots)
  • generative systems (sequence-to-sequence, hierarchical models)
  • information retrieval
  • ensemble systems
• End-to-end dialogue systems
  • training based on dialogue logs in a limited domain
  • multi-task learning
• Multi-domain systems
  • one-shot learning
• Multimodal systems
  • visual dialogue

Recommended reading

• Jurafsky & Martin: Speech & Language processing. 3rd ed. draft, Chapter 24-25

• Gao et al.: Neural Approaches to Conversational AI: Question Answering, Task-oriented Dialogues and Social Chatbots. now publishers 2019.
• Lemon & Pietquin (eds.): Data-Driven Methods for Adaptive Spoken Dialogue Systems. Springer 2012.

• Rieser & Lemon: Reinforcement learning for adaptive dialogue systems. Springer 2011.

• Jokinen & McTear: Spoken dialogue systems. Morgan & Claypool 2010.

• McTear et al.: The Conversational Interface: Talking to Smart Devices. Springer 2016.
• Psutka et al.: Mluvíme s počítačem česky. Academia 2006.
• + current papers from the field