Lecture Advanced Deep Learning

News:

• First exercise on Wednesday, April 15th.

Organization:

• Lectures:
  • Monday 14:30 - 16:00, 2611.HS 6E
  • Wednesday: 8:30 - 10:00, 2301.HS 3C
• Exercise class:
  • Tuesday 12:30 - 14:00, 2522.00.82
• Contact persons:
  • Assistant: Paweł Batorski, pawel.batorski(at)hhu.de
  • Tutors:
    • Malte Günnewig, malte.guennewig(at)hhu.de
    • Abtin Pourhadi, abtin.pourhadi(at)hhu.de
  • Secretary: Annette Peitsch, annette.peitsch(at)hhu.de

Remote:

• Lecture will be streamed via webex
• Meeting link: hhu.webex.com/meet/pswoboda

Lecture Materials:

• Intro: t3://file?uid=250495
• GPT2 model & inference:
  • Model t3://file?uid=250497
  • KV-Cache t3://file?uid=250860
• GPT2 training:
  • Basic training & improvements t3://file?uid=250863
  • Data parallelism t3://file?uid=250862
  • Full training code with eval & logging etc. t3://file?uid=250861
• Flash attention: t3://file?uid=251187

Exercise Sheets:

• Exercise 1, Hand-in date 25.4.2025 9am:t3://file?uid=251187