Commission
Research
All research areas →

Agent Evaluation and Coordination

Generated-Code and GPU-Kernel Correctness

Federated Learning and Privacy

Decentralised Systems and Protocol

Methods
Papers
Collaborate
About Contact Bring a technical claim

Research area · Agent evaluation

Evaluating agents beyond task completion

A completed benchmark task says little about whether an agent stayed within its authority, declined to invent an answer, avoided colliding with other agents or would rank the same way on a re-run. This area collects my published evidence on those questions and the open problems around them.

The situation

What evidence distinguishes an agent that completes a benchmark from one that respects authority and recovers from failure?

What you leave with

A research collaboration or evaluation study built around a measurable definition of the agent behaviour you care about, with its uncertainty reported.

For: Agent researcher; evaluation lead

What task completion leaves out

A benchmark success rate answers one question: did the agent reach an accepted end state? The failures that matter in practice are often orthogonal to it. My published work on agents addresses five of them, each with a different measurement problem.

  • Acting within authority. The bounded-autonomy harness tracks six indicators, including escalation correctness and unsafe-action rate, and separates signals by their source of supervision. Its main result is a caution: at smoke scale, configuration variance was large enough to mask tuning effects.
  • Refusing to fabricate. When a generated constraint program is unsatisfiable, handing the model a minimal unsatisfiable core instead of an error message cut fabricated solutions from 79% to 7%. A strong chain-of-thought baseline matched it on accuracy, so the gain is in recovery behaviour, not in the success rate.
  • Coordinating with other agents. With a shared append-only log, the share of work that re-did a teammate’s task fell from 78% to 0% (Before the Pull Request).
  • Trusting the judge. LLM judges for principle-based rules need four axes, not one: accuracy, paraphrase robustness, adversarial robustness and calibration. Retrieval-side detectors show the same pattern: the input-regime audit finds the best detector depends on the input distribution.
  • Trusting the ranking. A self-audit of an LLM evaluation found the small-sample ranking identified the worst model reliably but not the best.

Choosing a route

The narrower pages below this one differ by the question you bring, not by technique.

If your question is…Start with
Do our agents duplicate or overwrite each other before review?Multi-agent coordination
Does our conflict or grounding check survive a different kind of input?RAG knowledge conflicts
How do we classify agent failures so they can be counted?Agent failure taxonomies
Does a published agent result hold under our conditions?Replication studies
No existing benchmark measures the behaviour we care aboutBenchmark design

If none of these fits, the question is probably still open, which is the most useful kind to discuss as collaboration.

What I do not claim

None of this work shows that any agent is safe, compliant or production-ready. The bounded-autonomy pilots were small (n=8 in the paper); the coordination results come from one substrate built by a company I founded; the judge benchmark covers two UK FCA principles in one domain. The contribution is measurement design and bounded evidence, stated with its limits. A collaboration starts from those limits rather than from the headline numbers.

Useful if

  • Your agent benchmark reports a success rate, but the failures that worry you are overreach, fabrication or silent conflict.
  • You are designing an agent evaluation and want indicators that a third party could recompute.
  • You use an LLM as a judge and need to know how far its verdicts survive paraphrase or adversarial input.
  • You have an unresolved evaluation question and want to work on it as research, with results that may be negative.

Not the right fit if

  • You need an agent system built, integrated or monitored in production. That is delivery work on dipankar.co.
  • You want a certificate that an agent is safe or compliant. Nothing on this page can establish that.
  • You need a leaderboard position confirmed for publicity.

Limits and unfavourable results

  • Most of the cited work is preprint or workshop-poster status, and several experiments are deliberately small (smoke-scale pilots, curated probes).
  • Each result is bounded by its own models, tasks and measurement date. Hosted endpoints have already been withdrawn for one of the studies.
  • The coordination evidence was produced with grite, an open-source tool from Neul Labs, a company I founded. It has not been independently replicated.

Evidence behind this page

  • grite (GitHub, Neul Labs) — Git-native append-only coordination log for agents, released with the coordination paper's dataset, harness and mining toolkit. MIT. Neul Labs is a company I founded.
  • ebrag-vecdb-2026-paper (GitHub) — Input-Regime Audit paper, source, one JSON artefact per experiment and a runnable Python slice. MIT.

Narrower routes from here

Questions

How do you evaluate an AI agent beyond task completion?

By defining the behaviours task success hides and measuring them separately: whether the agent escalates when it should, whether it refuses to fabricate when a problem is infeasible, whether concurrent agents duplicate or overwrite each other, and whether any LLM judge in the loop holds up under paraphrase and adversarial input. Each indicator needs its own source of supervision and its own uncertainty.

How should tool-using agent performance be measured?

Separate outcome metrics from behaviour metrics, log every action with provenance so failures can be recovered after the fact, and repeat runs. In the bounded-autonomy pilots, nominally identical configurations gave task success of 0.25 versus 0.12, so a single run is not enough to compare configurations.

Is an LLM judge good enough to score agents?

Only if it has been tested on the inputs it will actually see. In the four-axis benchmark, no method dominated accuracy, paraphrase robustness, adversarial robustness and calibration together, and the judge strongest on benign inputs lost 47 accuracy points on keyword-stuffed inputs.

Are these results peer-reviewed?

Two are workshop posters (KDD 2026 SeT-LLM and IJCAI-ECAI 2026 LogiSymb), the input-regime audit is a manuscript under submission, and the rest are arXiv preprints. Each publication page states its status.

See also

Technical guides

Discuss the unresolved evaluation question

Send the claim, the decision it informs and what access exists. A short, non-confidential description is enough to start; data and code access are agreed afterwards.

Last reviewed 2026-10-07.