Commission
Research
All research areas →

Agent Evaluation and Coordination

Generated-Code and GPU-Kernel Correctness

Federated Learning and Privacy

Decentralised Systems and Protocol

Methods
Papers
Collaborate
About Contact Bring a technical claim

Sponsored research · Study brief

Write the brief before you fund the study

A sponsored AI study is only as useful as the question it starts with. A good brief names one claim, the decision it feeds, what it must beat, what access exists and when the work should stop — and it fits on a page.

The situation

How do we commission a study with a precise question, realistic access requirements and a useful decision at the end?

What you leave with

An editable research brief covering claim, decision, baseline, access and stop criteria, refined with me into a fixed-scope statement of work.

For: R&D sponsor; research procurement lead

The five parts of a commissionable brief

Use the evaluation brief builder for a first draft. Whatever tool you use, each part has to pass a simple test.

PartWhat it statesTest it must pass
ClaimThe specific statement under test, with its conditions.Could a third party design an experiment from this sentence alone?
DecisionWhat you will do differently depending on the result.Is there a result that would change the decision? If not, do not fund the study.
BaselineWhat the claim must beat: the current system, a simpler method, or a published number.Is the baseline available to run under the same conditions?
AccessModels, data, compute, code and people the study needs, and when.Has someone with authority confirmed each item can be provided?
Stop criteriaConditions under which the work halts or is re-scoped.Are they written before any results exist?

Why stop criteria and baselines matter

Most of the waste in commissioned AI research comes from studies that cannot end cleanly. Typical stop criteria are: the faithful baseline cannot be reproduced; agreed access has not arrived by an agreed date; or an early milestone shows the measurement cannot distinguish the options at the planned sample size.

That last case is common. In a self-audit of an LLM evaluation, a small-sample ranking of eight models identified the worst model reliably but not the best (How Reproducible Are Evaluation Conclusions?). A brief that asks “which is best?” should say how much separation would count, and what happens if the study cannot provide it. Baselines matter for a similar reason: the Correctness Illusion preprint shows that the check used to define “correct” can itself certify buggy outputs, so the brief should name the check as well as the target.

Illustrative example — not a client engagement. A first draft reads: “Evaluate whether our coding agent is production-ready.” The agreed brief reads: “Claim: on our internal repository tasks, the agent’s patches pass the existing test suite at least as often as the current human-reviewed baseline, over three repeated runs. Decision: extend the pilot to a second team, or not. Baseline: last quarter’s merged patches on the same tasks. Access: read-only repository mirror and a sandboxed runner, both confirmed by the platform lead. Stop: halt if the mirror is not available within the agreed window, or if run-to-run variation exceeds the difference being measured.”

Once the brief is agreed, read publication, IP and independence before signing anything. If the answer should be public, consider sponsoring an open experiment instead.

Commission this if

  • You have budget for applied AI research and need it to end in a decision, not a slide deck.
  • You are writing an AI research statement of work and want it testable before procurement signs it.
  • Several stakeholders hold different versions of the question and need one written down.

Not the right fit if

  • The question is "is our model good?" and nobody can say good for what. Start with the decision, then come back.
  • You need a build or integration project. A brief for delivery work is a different document.
  • The access the study needs cannot be granted, and nobody is able to change that.

What you receive

  • Agreed brief. The five parts below, edited until both sides read them the same way.
  • Statement of work. Deliverables, milestones and acceptance criteria for the deliverables — never for the result.
  • Access plan. What is shared, when, under which confidentiality terms, and what happens to it at the end.
  • Stop and amendment rules. Written conditions for halting early and a log format for any change of protocol.

What the study needs from you

  • A claim owner. Someone who can state the claim precisely and say which result would change their decision.
  • A draft, however rough. The evaluation brief builder produces a usable first draft in a few minutes.
  • Funding position. Whether a budget exists and roughly what scale, so the scope can be sized honestly.

How the work runs

  1. 1
    Draft. You fill in the five parts, using the builder or plain text.
  2. 2
    Scoping call. We test each part for precision and feasibility, and I disclose any conflict of interest.
  3. 3
    Rights and publication. Agree ownership, data handling and publication terms before money is committed.
  4. 4
    Fixed scope. I quote a fixed scope; the fee is set here and does not move with the outcome.
  5. 5
    Protocol. The brief becomes a pre-specified protocol before any results exist.

Limits and unfavourable results

  • A brief narrows a question; it cannot make an unanswerable question answerable. Sometimes the right output of scoping is that the study should not run.
  • Stop criteria protect the budget but can end a study before it reaches a conclusion. That outcome is reported as such.
  • Access constraints shape what can be claimed. A study run on sample data says so in every conclusion.

The brief is written so that a negative answer is useful: if the claim fails against the baseline, the decision it informs is still made. The fee does not depend on which way it goes.

Engagement terms

Model
Fixed-scope sponsored study, scoped from an agreed brief, with acceptance criteria for deliverables (not for the result).
Who does the work
Dipankar Sarkar personally designs and runs the study. Any specialist help is disclosed and agreed in advance.
Commercial basis
Fixed research scope, quoted after a scoping conversation. The fee does not depend on the result.
Availability
Checked per enquiry.

Conflicts are checked before scoping. See publication, IP and independence.

Evidence behind this page

Questions

Is there a template for a sponsored AI research brief?

Yes. The evaluation brief builder in resources produces an editable draft covering the claim, decision, baseline, access and stop criteria. This page explains what each part has to contain to survive scoping.

What makes a brief too vague to commission?

Usually one of three things: no decision is attached to the result, there is no baseline to compare against, or the access the study needs does not exist. Each can be fixed before any money is spent.

How is this different from an AI research statement of work?

The brief states the question; the statement of work states what will be delivered and when. I write the statement of work from the agreed brief, so procurement signs something tied to the question rather than to a list of activities.

Can the study conclude that our approach does not work?

Yes. That is a valid result, the report is written the same way either way, and the fee was fixed before the work began.

See also

Commission a scoped study

Send the claim, the decision it informs and what access exists. A short, non-confidential description is enough to start; data and code access are agreed afterwards.

Last reviewed 2026-10-07.