Field notes from the edge of agent reliability.
Papers, postmortems, and the gap between what the literature measures and what production breaks.
Agents Fail Structurally. Current Tooling Can't See It.
Why the expensive agent failures in production are structural, not linguistic — and why better prompts, LLM-judges, and firewalls don't close the gap.
Why LLM-as-Judge Fails Silently at Scale
Our first LLM-judge ensemble scored 100% agreement on a held-out benchmark. That was the bug. A structural failure mode of LLM-as-judge.
The Translation Bottleneck
Five research groups have converged on the same pipeline for agent compliance. Every one of them stalls at the first arrow — natural language to formal constraint.
A Proposal for Coverage Over Constraint Spaces
No agent-policy coverage framework is currently published. A first sketch — four dimensions from adjacent fields, plus one that isn't named anywhere.
A Field Map of Agent Policy Compliance
A reference map of the agent policy compliance landscape — seven families of approach, five research programs worth following, six gaps still open.