Senior Forward Deployed Engineer
Many teams realize they need to evaluate their AI systems only after problems show up — an answer that invents a policy, a refund that wasn’t deserved, or a CI check that passed while production failed. By then, the system is already live, and evaluation is an afterthought rather than part of the plan.
That pattern is getting more expensive. Agentic systems take actions, call tools, and touch customer data; a soft failure is no longer just a bad paragraph. Bigger models and longer prompts don’t fix it. The same engineering habits you use elsewhere do: name the failures, measure what’s cheap first, spend judgment only where simple checks miss, and track what you still don’t know.
You’ll learn how to build that spine — failure taxonomies, threat models, fixtures, judges, calibration, and red-teaming — and how we’re applying it inside Liatrio’s AI Evaluation Bootcamp. Start with failure, not with tools.
Industry conversation still treats evaluation like a tooling choice: pick a scorer, add a dashboard, hope the green checks mean something. The harder problem is shared language. “The model hallucinated” is not a testable failure. An unauthorized action, cross-customer data leak, invented policy, misused tool, or untrusted retrieval is. Without those names, fixtures, audits, and threat models don’t line up, and every review meeting restarts the debate.
A second blind spot makes it worse. Most teams only ask how an attacker could abuse the agent — not how someone could fool the evaluation itself by poisoning a holdout set, gaming a judge, or making a gate look green when it shouldn’t. Leaving room for that second kind of attacker is uncomfortable. It’s also correct. Otherwise, you build a check that never asks whether it would miss the failure it claims to catch.
That’s why evaluation is harder than it looks: you need stable failure types, a map of where trust changes hands, and humility to treat your own gates as an attack surface — not just the agent.
The core move is simple and opinionated: evaluate AI the way engineers already evaluate software. Don’t start with a bigger model or a cleverer prompt. Start with failure modes, cheap deterministic checks, and claims you can defend out loud in review.
That beats tool-first alternatives because vocabulary, fixtures, scorers, and gates stay pointed at the same risks — instead of drifting into dashboards that look healthy while production fails. The payoff is a review conversation that can name what broke, what you measured, and what you still can’t claim — before production teaches you the hard way. What follows shows how that spine works in practice: threat models tied to goals, fixtures that save inputs, not assembled prompts, judges that need evidence, and calibration that survives base-rate scrutiny.
Threat models name goals, not jailbreak recipes
A useful threat model for an agentic system isn’t a scanner report. It maps where trust changes hands — from the user to the agent, from the corpus to the agent, from the agent to tools, from tools to data, and back from the agent to the user — and asks what an attacker would actually want: an action they aren’t entitled to, instructions that didn’t come from the user, another customer’s data, an invented policy that pays out, or an evaluation that looks away.
Attack techniques change every week. Attacker goals don’t. Goals are stable enough to write scorers against, so we map them to a published taxonomy instead of inventing our own. Where OWASP’s agentic categories fit, we use them. Where they don’t, we say so instead of forcing a match. That way, when you finally attack the system, you’re not arguing about vocabulary on day one.
Cheap checks first, then grounded judgment you can defend
A few lines of deterministic Python that validate schema, required tokens, and forbidden tokens will outlast many clever judges. Same input in, same verdict out — no fuzzy matching, embeddings, or temperature tricks that make results hard to reproduce. Run every deterministic check first. Only if those pass do you spend an LLM judge; if they fail, record which judges you skipped. Prefer replaying recorded runs. Make live model calls only with a clear reason and a budget limit.
Then ground the judge. Vague “is this good?” prompts fail predictably: ungrounded passes, uncertainty treated as soft success, and claims untied to evidence. Quote-grounded verification forces the judge to work from packets with stable evidence IDs — system prompt, generator input, observations — and to fail closed when a pass isn’t supported. Trajectories and tool calls get allowlists and expected stop reasons. The judge is a calibrated instrument, not a loose quality check with an API key.
Finally, treat calibration as a claim. “The judge agrees with us 90% of the time” means little if almost every case is an easy pass. Defend it with Cohen’s kappa and a confidence interval, a confusion matrix, how often real failures are caught, and what the judge still can’t reliably do. Don’t fold drift and regression into one number. A CI gate with searchable logs and honest exit codes beats a dashboard screenshot; if nobody trusts the gate, an advisory check with a recorded override beats a hard fail everyone ignores.
How we’re applying this inside Liatrio
We built an internal AI Evaluation Bootcamp around the same spine: a flawed-on-purpose support agent as the system under test, chapter artifacts that accumulate into a real harness, and a capstone where you change the tool and defend how far the evaluation should still be trusted.
Learners fork the reference tool into their own harness repo and do the work there — from taxonomy through production habits — against a system they can actually break and measure. The course site teaches; the harness holds the evidence.
We also shipped a “Check your progress” Action: a deterministic scorecard for structural contracts, including fixture shape, lifecycle, citations, audit envelope, and expected files. It is not a merge gate or certificate. It answers “am I on track with the artifacts?” and refuses to answer “was my judgment good?” — the same distinction we teach. That’s eating our own cooking: field habits with receipts instead of vibes.
A fair objection is that deterministic checks and structural scorecards can’t judge quality. That’s true — and limited. Machines are excellent at files, shapes, counts, schemas, and invariants. They are terrible at pretending they graded human judgment. A row-counting self-check still looks green on bulk-generated junk; a file-presence grader can’t tell thoughtful calibration from template prose. Our answer isn’t to skip automation; it’s to size it correctly and keep judgment exercises explicitly human.
Another objection: “Just use an LLM judge for everything — it’s faster to stand up.” Faster to stand up, yes. Harder to trust. Without quote-grounded evidence, fail-closed rules, and calibration you can defend, a judge becomes vibes with an API key. Cheap checks first, then grounded judgment, accounts for both objections without pretending automation is wisdom.
Write a shared failure taxonomy before you write a scorer — specific, testable types with stable IDs.
Map threat models to attacker goals and trust boundaries, not to this week’s jailbreak recipe; reuse a published taxonomy where it fits.
Save fixtures as inputs (customer, session, message, metadata), not assembled prompts, and give each fixture one lifecycle job.
Run deterministic checks first; only then spend an LLM judge, and record which judges you skipped when cheap checks fail.
Ground judges in evidence packets, fail closed without support, and report calibration with kappa, confusion, and known blind spots — not agreement vibes.
Red-team against named goals; treat evaluation itself as attack surface; prefer honest CI logs over dashboard screenshots.
Evaluating AI well is not a vendor checklist or a bigger model. It’s engineering: name failures, measure cheaply first, spend judgment where it earns its keep, and be honest about what you still can’t claim. That spine turns after-the-fact fire drills into a system you can defend — in reviews, in CI, and in production.
The challenge is straightforward. Pick one live AI path this week, write down the failures that would actually hurt, and put a cheap check in front of an expensive opinion.
If you’re building agentic systems, try the failure-first spine on one workflow and tell us where it breaks. We’re training ourselves the same way inside Liatrio’s AI Evaluation Bootcamp, and we’d rather compare receipts than dashboards.
