A RAG app that looks fine on a few test questions can still fail in production. This module is about knowing how good your system really is, and keeping it safe. Evaluation gives you numbers you can track, and guardrails stop bad inputs and bad outputs from reaching your users.
The first lessons cover how to evaluate a RAG pipeline. You will see metrics from Ragas such as faithfulness, context precision and context recall, how to turn real failures into a test dataset, and how tracing helps you find the exact step that broke. The last two lessons cover guardrails: checks that catch hallucinations, prompt injection and leaks of personal data.
Several lessons use Langfuse as the example tool. The ideas apply to any evaluation or tracing setup, so focus on the workflow rather than the specific product.
A webinar by Prof. Tom Yeh that introduces RAG, then shows how to trace a RAG app and score it with Ragas metrics such as faithfulness, answer relevancy, context recall and context precision.
Video 1h 3minBuilds a full evaluation loop around a support assistant: find failures in production traces, capture them in a dataset, score them with automated judges, and test every change with an experiment before it ships.
Video 6minFollows a confident but wrong answer from a RAG support agent back through the trace to the single broken step behind the bad retrieval, wrong tool call and hallucination.
Video 4minWhy production RAG systems need guardrails, and how to stop the model from answering what it should not and from making things up.
Video 6minBreaks down the first of three guardrail layers for AI agents, covering how a single sentence can derail an agent and how to defend against prompt injection and personal data leaks.
Video 11min