Until now each module has covered one piece of the puzzle. This module puts them together. A RAG pipeline has two phases: an indexing phase that runs ahead of time, and a query phase that runs every time someone asks a question.
During indexing you load your documents, split them into chunks, turn each chunk into an embedding, and store the vectors in a vector database. During a query you embed the question, retrieve the most similar chunks, place them in the prompt, and let the LLM write an answer grounded in that context.
The three videos build this end to end in Python. The first uses LangChain, Chroma and OpenAI on a set of text files. The second adds PDF loading, database updates and a local model with Ollama, plus a way to test the answers. The last is a long, step-by-step build of a local pipeline that chats with a PDF textbook, so feel free to follow along in code and come back to it later.
A first end-to-end RAG app in Python: load text documents, split them into chunks, store the embeddings in Chroma, and answer questions with an OpenAI model.
Video 16minExtends the basic pipeline: loading PDFs, giving chunks stable IDs so the database can be updated, running the model locally with Ollama, and unit testing the quality of the answers.
Video 21minA long code-along by Daniel Bourke that builds a local RAG pipeline from scratch: turning a PDF textbook into chunks and embeddings, searching them, and generating answers with a local LLM. An optional deep dive.
Video 5h 40min