Institute of Formal and Applied Linguistics
Charles University, Czech Republic
Faculty of Mathematics and Physics
Large Language Models
Goals of the course:
- Explain how the models work
- Teach basic usage of the models
- Help students critically assess what you read about them
- Encourage thinking about the broader context of using the models
Syllabus from SIS:
- Basics of neural networks for language modeling
- Language model typology
- Data acquisition and curation, downstream tasks
- Training (self-supervised learning, reinforcement learning with human feedback)
- Finetuning & Inference
- Multilinguality and cross-lingual transfer
- Large Language Model Applications (e.g., conversational systems, robotics, code generation)
- Multimodality (CLIP, diffusion models)
- Societal impacts
- Interpretability
The course is part of the inter-university programme prg.ai Minor.
About
SIS code: NPFL140
Semester: summer
E-credits: 3
Examination: 0/2 C
Guarantors: Jindřich Helcl, Jindřich Libovický
Timespace Coordinates
The course is held on Thursdays at 12:20 in S5.
Lectures
License
Unless otherwise stated, teaching materials for this course are available under CC BY-SA 4.0.
No lectures this semester so far. See you on Thursday 19 February at 12:20 in S5.
Project work
You will work on a team project during the semester. The project will be presented during the final class. Each team will submit a report, consisting of:
- Brief method overwiev
- Summary of related work
- Experimental design
- Results
- Conclusions
- Overview of individual contributions of team members
- References
The length of the report should be maximum 4 pages plus references and contributions. You might want to use the ACL paper template.
Reading assignments
You will be asked at least once to read a paper before the class.
Final written test
You need to take part in a final written test that will not be graded.