Training a language model from scratch takes huge amounts of text, compute and time. Transfer learning avoids that cost: a model is first pretrained on a large body of raw text, then fine-tuned on a much smaller dataset for the task you actually care about. Almost every modern NLP system is built this way.
The lessons start with the core idea of transfer learning, then look at BERT, the encoder-only model behind classification, search and RAG. A high-level tour of large language models shows how GPT-style models are pretrained, and a hands-on lesson fine-tunes a pretrained checkpoint with the Hugging Face Trainer. You then see how LoRA makes fine-tuning cheap enough to run on modest hardware, and the module closes with a long deep dive into the full pipeline from pretraining to ChatGPT-style assistants.
Watch the lessons in order, then continue on to Translation & Summarization.
A short explainer on pretraining versus fine-tuning, and why starting from a pretrained model needs less data, time and compute than training from scratch.
Video 4minStatQuest walks through how BERT-style encoder-only transformers work, from word embedding and positional encoding to attention, and where they are used for classification and RAG.
Video 18minAndrej Karpathy gives a general-audience overview of what an LLM is, how pretraining and fine-tuning produce one, and where the field is heading.
Video 1hA hands-on lesson on fine-tuning a pretrained BERT checkpoint for sentence-pair classification using the Trainer class in Hugging Face Transformers.
Video 3minAI Coffee Break explains why fine-tuning large models is costly and how LoRA trains small low-rank matrices instead of updating every weight.
Video 8minAndrej Karpathy covers the full training stack: pretraining data and tokenization, supervised fine-tuning, reinforcement learning, and how to think about what these models can and cannot do. A long watch that ties the module together.
Video 3h 31min