Shipping an AI agent is the easy part. Knowing whether it's actually working -- and catching when it quietly breaks in production -- is the hard part. Google just made that significantly easier: Agent and Model Evaluations in the Gemini Enterprise Agent Platform are now generally available.

One engine, dev to prod

Agent quality must be measured during development against the cases you wrote, and after launch against the tasks the agent actually performed. The core promise here is a single evaluation engine that runs in both contexts with identical metric definitions. When you use consistent quality scoring on local experiments and live traffic, a drift in production points to a problem with the agent rather than with the way it was measured. That's a subtle but important guarantee -- without it, you can't tell if a score drop is a regression or just a measurement artifact.

What's actually in the box

Start from more than 20 pre-built metrics spanning quality, safety, grounding, agent tool use and trajectory, and reference-based scoring for tasks like summarization and translation. The metrics break down into two families:

• Computation-based metrics -- deterministic, ground-truth comparisons: ROUGE for summarization, BLEU, MetricX, and COMET for translation, exact match for extractive QA.
• LLM-as-a-judge metrics -- model-graded scores for things that can't be reduced to a formula: task success, tool use quality, safety, hallucination, grounding, and trajectory quality.