The lecture “Behind the Secrets of Large Language Models” is offered by Prof. Michael Färber and Prof. Simon Razniewski in the winter semester 2024/25 at TU Dresden. This course provides a practical and in-depth understanding of large language models that power modern natural language processing systems. Students will explore the architecture, training methodologies, capabilities, and ethical implications of large language models. The course combines theoretical knowledge with hands-on experience to equip students with the skills necessary to develop, analyze, and apply LLMs in various contexts.

• Lectures take place on Mondays, 11:10 – 12:40 at FOE/0244/H
• Exercises take place on Mondays, 16:40 – 18:10 at FOE/0244/H

By the end of this course, students will be able to

• Understand the architecture and key components of large language models.
• Analyze the training processes, including data collection, model optimization, and fine-tuning.
• Evaluate the performance and limitations of LLMs in different NLP tasks.
• Apply LLMs to real-world problems, such as text generation, summarization, and translation. 
• Discuss the ethical considerations and societal impacts of deploying LLMs

Prerequisites

• Introduction to Machine Learning or equivalent
• Basic understanding of neural networks
• Programming experience in Python

Tentative Schedule

Date	Lecture	Exercise
14.10.2024	L1 – Introduction to LLMs	Crash Course on NNs
21.10.2024	L2 – Neural Networks	E1 – Neural Nets Refresher
28.10.2024	L3 – Word Representation	E2 – Word Representations
04.11.2024	L4 – Attention	E3 – Implementing Attention
11.11.2024	No Lecture	E4 – Implementing Transformers
18.11.2024	L5 – LM Architectures	E5 – Implementing LMs
25.11.2025	L6 – Collecting Data	E5 continued
02.12.2024	L7 – Training LMs	E6 – Training LMs
09.12.2024	L8 – Transfer Learning	E7 – Finetuning a LM
16.12.2024	L9 – Evaluation	E8 – Model Evaluation
23.12.2024	Christmas and New Year Break
30.12.2024	Christmas and New Year Break
06.01.2024	L10 – Applications	E9 – Applications
13.01.2024	L11 – KGs / RAG / Tools	tbd
20.01.2024	L12 – Vision Models	tbd
27.01.2024	L13 – Ethics and Safety	E12 – Ethics, Safety, Philosophy