A prompt that works once is not the same as a prompt that works reliably. To improve a prompt, you need a way to tell whether a change made it better or worse. Evaluation gives you that signal, and iteration is how you use it: change one thing, test it, and keep what works.
You'll start with how to set up evals, using metrics, unit tests, and a fixed set of test cases you can run again after every change. Then you'll learn LLM-as-a-judge, where a second model scores outputs against a rubric, which helps when answers are too open-ended for a simple pass or fail check.
Once your prompts are measurably good, you'll want them to be safe and dependable in the real world. That's the focus of Safety, Security & Production, the next module in this course.
How to set up evaluations for an LLM application using metrics, unit tests, and an LLM as a judge.
What the "LLM as a judge" method is, why it works, and how to create your own judge.
How to evaluate thousands of AI outputs when traditional metrics don't fit the task and manual labeling takes too long.
A step-by-step walkthrough of using one LLM to evaluate the quality of another LLM's response.