Most models so far have been discriminative: given an input, predict a label. Generative models flip the task. They learn what the training data looks like, its distribution, so they can produce new examples such as images, audio or text. Language models are one kind, generating one token at a time. This module covers the main families used for images.
A variational autoencoder (VAE) compresses data into a smooth latent space and decodes samples from it back into data. A generative adversarial network (GAN) trains a generator to fool a discriminator that tries to tell real from fake. Diffusion models learn to remove noise step by step, so generation starts from pure noise and gradually becomes an image; they power most modern text-to-image tools.
Watch the lessons in order, then continue to Training at Scale & Deployment.
Arxiv Insights explains how a VAE encodes data into a latent space and samples from it to generate new examples, and why the latent space is kept smooth.
Video 15minComputerphile introduces the generator and discriminator and how training them against each other produces realistic images.
Video 21minAri Seff explains the idea behind diffusion models: gradually adding noise to data, then training a network to reverse the process.
Video 15minComputerphile looks at how text-to-image systems turn a prompt into a picture using diffusion, starting from random noise.
Video 17minA visual guest video on 3Blue1Brown that builds up the intuition behind diffusion-based image and video generation, including how text guides the result.
Video 37min