The models so far either ignore word order or look at only a few words at a time. A recurrent neural network (RNN) reads a sentence one word at a time and carries a hidden state forward as a running memory, so every prediction can depend on everything it has read so far. That makes RNNs a natural fit for language modeling, tagging and translation.
The lessons begin with how an RNN works and how it reuses the same weights at every step. You then meet its critical flaw, the vanishing and exploding gradient problem, which makes plain RNNs forget the start of long sequences. LSTMs fix this with gates that decide what to remember, update and output, and GRUs offer a simpler gated alternative.
RNNs and LSTMs dominated NLP for years, and understanding them explains why attention and transformers were such a breakthrough. Watch the lessons in order, then continue on to Attention & Transformers.
StatQuest walks through how an RNN handles inputs of different lengths one step at a time, how weights and biases are shared, and the vanishing and exploding gradient problem.
Video 16minBuilds up RNNs from first principles, covering the recurrent cell, the hidden state, the weight matrices and the tanh activation.
Video 2minStatQuest shows how LSTMs avoid the gradient problems of basic RNNs, with the sigmoid and tanh functions and the three stages of an LSTM unit.
Video 21minA deep dive into the architecture of LSTM networks, an essential concept in deep learning.
Video 20minLooks at LSTM cells both architecturally and mathematically, for a closer view of what happens inside the gates.
Video 28minShows how a GRU addresses the short-term memory problem of traditional RNNs, in a similar way to LSTM but with a simpler design.
Video 8min