Vorlesung Advanced Deep Learning

News:

• Exam inspection (Klausureinsicht) will take place on Monday, August 4th, from 4pm to 6pm in room 25.13.02.028.
• Online lecture on Wednesday, May 21th.
• No lecture on Monday, May 19th.
• First exercise on Wednesday, April 15th.

Organization:

• Lectures:
  • Monday 14:30 - 16:00, 2611.HS 6E
  • Wednesday: 8:30 - 10:00, 2301.HS 3C
• Exercise class:
  • Tuesday 12:30 - 14:00, 2522.00.82
• Contact persons:
  • Assistant: Paweł Batorski, pawel.batorski(at)hhu.de
  • Tutors:
    • Malte Günnewig, malte.guennewig(at)hhu.de
    • Abtin Pourhadi, abtin.pourhadi(at)hhu.de
  • Secretary: Annette Peitsch, annette.peitsch(at)hhu.de

Remote:

• Lecture will be streamed via webex
• Meeting link: hhu.webex.com/meet/pswoboda

Lecture Materials:

• Intro: t3://file?uid=250495
• GPT2 model & inference:
  • Model t3://file?uid=250497
  • KV-Cache t3://file?uid=250860
• GPT2 training:
  • Basic training & improvements t3://file?uid=250863
  • Data parallelism t3://file?uid=250862
  • Full training code with eval & logging etc. t3://file?uid=250861
• Flash attention: t3://file?uid=251187
• GPT2 Fine-tuning: t3://file?uid=254780
• RL & PPO for RLHF: t3://file?uid=254230
• RoPE & GQA: t3://file?uid=255191
• RNN: t3://file?uid=255246
• LSTM: t3://file?uid=256313
• SSM: t3://file?uid=256316
• ViT, Axial Attention, Swin, MLP-Mixer: t3://file?uid=256312
• MoE: t3://file?uid=256314
• Parallelism & Large-Scale:
• LLM Prompting, Reasoning & Agentic: t3://file?uid=257141
• Attention & Transformer Models for everything? t3://file?uid=257142

Exercise Sheets:

• Exercise sheets on Ilias