• Week1: Let’s code a GPT model
  • Video streaming: https://youtube.com/live/dy7-g8-efCg?feature=share
  • Prerecorded: https://www.youtube.com/watch?v=sT7EWrPtjxI
  • Code:
    • https://github.com/tjwei/gpt2_numpy
    • Original GPT-2 source code https://github.com/openai/gpt-2
• Week2
  • Video streaming: https://youtube.com/live/D7bxpwU-byI?feature=share
  • slides: https://www.canva.com/design/DAGCjrQRU5Q/RworPATtIG5vZbeWrpaMsw/view?utm_content=DAGCjrQRU5Q&utm_campaign=designshare&utm_medium=link&utm_source=editor
  • ref: Stanford CS 224N | Natural Language Processing with Deep Learning
    • See slides and notes in schedule section
    • We will use lots of materials from here
• Week3
  • Video streaming: https://youtube.com/live/O83F6g3p51s?feature=share

Reference data:

Outline

• Transformers and GPT
  • run it with hugging face transformers
  • compare paper with our simple gpt2 implementation
• LLM ecosystem
  • Try large language model really quick
  • Online models
  • local models
• Basic Theory
  • n-gram
  • word2vec
• Large language models

Try LLM

Try it Online

• ChatGPT
• gemini
• claude.ai
• Bard
• Bing

Running Large language model on your machine

• We use https://github.com/oobabooga/text-generation-webui for a quick evaluation