Google's Agent102 lab, part two: evals and a self-improving loop

Try this

1. Score the steps and tools the agent used, not just its final answer. 26:15
2. Build a small 'golden' test set: inputs with the expected answers and tool steps. 32:41
3. Use an AI judge for fuzzy things like tone, but expect drift and extra cost. 33:53
4. Put your agent tests in pytest so they run on every change. 35:47
5. Prefer 'checkers', code that confirms the result follows your rules (no double booking, under budget), over exact-match comparisons. 51:00
6. Split the roles: one agent does the work, one grades it, a router decides, and a proposer rewrites the instructions. 56:30
7. Keep the router as plain if/else logic (ship it, stop after N tries, or loop again), not another AI call. 59:00
8. Remember that 'self-improving' just means one step rewrites the instructions another step uses. 1:06:58
9. Try Google's adk optimize command to search for better instructions against your test set. 1:20:52
10. Re-score real conversations from production with your checkers and add the failures to your test set. 1:24:40

Timestamps point to the moment in the source video.