The best way to make the modules stick is to build something. These four projects follow the course from the ground up: a GPT-style language model built from scratch, a full encoder-decoder Transformer for translation, QLoRA fine-tuning of a pretrained model, and a RAG pipeline for question answering.
Each one is a code-along video, so you can pause, type it out yourself and adapt it to your own data. They are long, so pick the project that matches the module you've just finished and come back for the rest.
Andrej Karpathy builds a character-level Transformer language model from an empty file and trains it on Shakespeare, covering self-attention step by step.
Umar Jamil codes the full encoder-decoder Transformer in PyTorch, then builds the tokenizer, dataset and training loop for a translation task and visualises the attention scores.
Understand LoRA and QLoRA, then fine-tune Mistral-7B-Instruct into a custom YouTube-comment responder, with the Colab notebook and dataset provided.
Build a RAG pipeline from the ground up: indexing, retrieval, generation, then query translation (multi-query, RAG fusion, HyDE), routing, and adaptive RAG.