Hands-On Tutorials for Inspect AI

TL;DR Four hands-on Jupyter notebooks offer a practical introduction to building AI evaluations with Inspect AI . As a four-person project team for the Spring 2026 Technical AI Safety Evals Course , we revised and expanded notebooks, originally written as course materials, that teach Inspect AI - from your first evaluation through benchmarks, LLM judges, and a small agent. As a natural continuation of the project, we revised the official Inspect AI tutorial and filled the gaps we found in it. We added a new Custom Solvers section and fixed several inconsistencies. This post covers: what we built, how we worked on it, why we think teaching people to build evals well matters more for safety than for capabilities. All four notebooks are on Google Colab: Notebook 1 , Notebook 2 , Notebook 3 , Notebook 4 . The full set to run locally is on GitHub . There is also a guide via Google NotebookLM . We’d appreciate any feedback on our project. What problems with the idea or its implementation do you see? What parts do you find valuable? Do you want to contribute yourself? Feel free to write here, on discord , via email , or directly in the github . What We Built The notebooks cover real-world use cases and essential theory. They take you from your first Inspect evaluation to more advanced work with benchmarks, LLM judges, and agents. Here's what each notebook covers: The basics of Inspect AI, including how to build your first evaluation The statistical elements of evaluations The LLM-a…