Text, audio and time series share a property that images do not: order matters, and the length varies. A recurrent neural network (RNN) reads a sequence one step at a time and carries a hidden state forward, a running summary of what it has seen so far. The same weights are reused at every step, so one network can process a sentence of any length.
Plain RNNs have a weakness. When training backpropagates through many steps, the gradients shrink or blow up (vanishing and exploding gradients), so the network forgets what happened early in a long sequence. LSTMs and GRUs fix much of this with gates that control what to keep, add and discard in the memory. Understanding their limits also explains why attention and transformers, the next module, took over.
Watch the lessons in order, then continue to Attention & Transformers.
StatQuest shows how an LSTM keeps a long-term and a short-term memory, with gates that decide what to forget, what to add and what to output, which lets it work with much longer sequences.
Video 20minA longer, hands-on StatQuest session that codes an LSTM in PyTorch with Lightning and trains it. Best watched after the two concept videos above.
Video 33min