Syllabus
* Basics of neural networks for language modeling
* Language model typology [2]
* Data acquisition and curation, downstream tasks
* Training (self-supervised learning, reinforcement learning with human feedback)
* Finetuning & Inference
* Multilinguality and cross-lingual transfer
* Large Language Model Applications (e.g., conversational systems, robotics, code generation) [2-3]
* Multimodality (CLIP, diffusion models)
* Societal impacts
* Interpretability
Annotation
he course is devoted to large neural language models. It covers the related theoretical concepts, the technical foundations of operation and the use of language models.