A language model assigns a probability to a sequence of words, which in practice means it can predict what word is likely to come next. This one idea sits behind autocomplete, spell correction, speech recognition and, at much larger scale, today's chatbots.
The lessons start with n-grams, sequences of n words, and show how counting them in a large text gives you unigram, bigram and trigram probabilities. You will then see why raw counts break down on unseen word combinations, and how smoothing fixes that, and how perplexity measures whether one model is better than another.
The module ends by looking at where n-gram models run out of road: they only see a few words of context and need huge amounts of data as n grows. That limit is what motivates the neural approaches in later modules. Watch the lessons in order, then continue on to Word Embeddings.
A beginner-friendly introduction to what n-grams are and how unigrams and bigrams are pulled out of a sentence.
Video 5minA worked example of calculating unigram, bigram and trigram probabilities from counts, by Vidya Mahesh Huddar.
Video 8minIntroduces building a simple n-gram based language model, with code and slides linked by the creator.
Video 17minA full lecture covering n-gram language models, Laplace (add-one) smoothing for unseen n-grams, and perplexity, with numerical examples.
Video 47minAndrej Karpathy builds a character-level bigram language model from scratch, then shows the same model as a tiny neural network. A longer, hands-on watch.
Video 1h 57minReviews n-gram language models and perplexity, then shows how neural language models and RNNs overcome their short context and sparse data problems.
Video 1h 8min