Sparse representations such as bag-of-words treat every word as an unrelated symbol, so "good" and "great" look as different as "good" and "refrigerator". Word embeddings fix this by giving each word a short, dense vector of numbers, learned from text, so that words used in similar contexts end up close together.
The lessons begin with the core idea, then look at word2vec, which learns embeddings by predicting words from their neighbours using either CBOW or skip-gram. Next comes GloVe, which learns from global co-occurrence counts, and subword embeddings such as fastText, which cope with spelling variations and unseen words.
The module ends with a full Stanford lecture on word vectors. Keep one limitation in mind as you watch: these embeddings give each word a single vector whatever the context. That gap is addressed by the sequence models later in the course. Watch the lessons in order, then continue on to Text Classification, where you will put these vectors to work.
A Rasa explainer on what word embeddings are, how they work, when they are used and some common errors.
Video 6minStatQuest shows how a simple neural network learns embeddings from context, covers CBOW and skip-gram, and explains how negative sampling cuts training cost.
Video 16minA deeper comparison of the two most important embedding techniques, including the maths behind each.
Video 19minPart of the Rasa Algorithm Whiteboard series. Explores GloVe embeddings and why word embeddings are genuinely hard to interpret.
Video 19minUses fastText to show how subword embeddings help with spelling mistakes and words the model has never seen.
Video 12minThe first lecture of the Stanford NLP with Deep Learning course, covering how word meaning is represented and introducing the word2vec algorithm.
Video 1h 21min