Fund an open experiment, not a conclusion

Some questions are worth answering in public: a benchmark nobody has checked, a method that has never been compared fairly, an assumption the field relies on. Sponsoring an open experiment pays for that work, with methods stated up front and results published whether or not they are what anyone hoped.

The situation

Can we fund a useful open experiment with transparent methods and no guarantee of a positive result?

What you leave with

A public question proposal, a milestone budget discussion and a disclosure and publication plan, followed by an openly released experiment.

For: R&D sponsor; open-source programme lead

What an open experiment looks like

My recent GPU-kernel work is the shape to expect. The Correctness Illusion (preprint) showed that fixed-shape allclose checks used by LLM-kernel benchmarks certify buggy kernels as correct, and released the 26-op corpus that lets anyone check it. A follow-up, Test-Input Generation for Tensor Programs (preprint), compared seven input strategies on the same corpus and reported the uncomfortable trade-off alongside the good one: adversarial values reached 99% bug recall but 94% false positives on correct controls. The RAG conflict-detection audit (Input-Regime Audit, manuscript under submission) found that which detector wins depends on the input distribution — a finding with no single winner to announce.

They are shown here as examples of the format, not as evidence of past sponsorship: a stated question, released artefacts, and results reported whichever way they fell.

The public question proposal

A proposal is short and published before any run:

Question — one sentence, testable.
Why it matters — who relies on the current assumption.
Method — data, systems, baselines and the metric, with how uncertainty is reported.
Possible answers — what each outcome would mean, including “the measurement cannot tell”.
Disclosure — the sponsor, its interest in the outcome, and any interest I have.
Release plan — licences for code and data, and where the write-up will appear.

What sponsorship does and does not buy

The sponsor gets	The sponsor does not get
A question it cares about answered in public, with methods anyone can check	A say over the conclusion, its wording or whether it is released
Acknowledgement in the funding statement	Exclusivity over the artefacts or an embargo after the agreed review window
Sight of the write-up before release, to flag factual errors	A guarantee that the result is positive, novel or accepted at a venue
The ability to stop funding after any milestone	Payment terms that vary with the result

If those terms do not work for your organisation, a confidential sponsored study is probably the better route.

Budgeting by milestone

Milestones are defined by deliverables that exist regardless of outcome: the published plan, the harness and dataset release, the results with raw runs, and the write-up. Budget is discussed per milestone so a sponsor can stop after any of them, and so no payment is ever tied to what the data say. Terms on review, licensing and conflicts follow the publication, IP and independence checklist.

Commission this if

Your organisation benefits from a question being answered credibly in public, not privately for you.
You run an open-source or ecosystem programme and want independent evidence on a benchmark, method or claim.
You can accept a null or negative result being published with your name in the funding statement.

Not the right fit if

You need the result kept confidential. That is a sponsored study, not an open experiment.
You need a positive finding for a launch or announcement.
You want to choose the conclusion, the venue or the wording of the findings.

What you receive

Public question proposal. The question, why it matters, the method, the baselines and what would count as each possible answer — published before the work starts.
Milestone plan. Each milestone with a concrete deliverable, so budget is discussed and released against delivery rather than against results.
Open artefacts. Code, data where licensing allows, stored seeds and scripts that regenerate every table.
Preprint or report. Written up with limitations and negative results, with the sponsor named in the funding statement.

What the study needs from you

A question, or a willingness to choose one. Bring your own, or pick from the open questions on the research topic pages.
A budget range. Enough to size the milestones honestly. Scope and budget are quoted after a scoping conversation.
Acceptance of disclosure. The sponsor is named, and any interest the sponsor has in the outcome is stated in the publication.

How the work runs

1
Propose. Draft the public question proposal together, and I disclose any conflicts of interest.
2
Agree milestones and terms. Milestones, budget per milestone, licences and the publication plan, settled before work starts.
3
Publish the plan. The proposal and pre-specified analysis go public before results exist.
4
Run and release. Each milestone ships its artefacts; deviations from the plan are logged publicly.
5
Write up. The paper or report is released whatever the result. The sponsor sees it before release to check factual errors, not to edit findings.

Limits and unfavourable results

There is no guarantee of a positive, novel or publishable-at-a-venue result. A preprint and artefacts are the commitment; acceptance at a peer-reviewed venue is not.
Data from third parties may carry licences that limit what can be released; the plan states this in advance.
Sponsorship creates a perceived interest even when terms are clean. Naming the sponsor and stating its interest is how readers judge that for themselves.

A null or negative result is published in full, with the same artefacts and the same care. Budget is tied to milestones delivered, not to the direction of the finding.

Engagement terms

Model: Milestone-based sponsorship of an openly published experiment.
Who does the work: Dipankar Sarkar personally designs and runs the study. Any specialist help is disclosed and agreed in advance.
Commercial basis: Budget discussed and agreed per milestone after scoping. Payment is against delivered milestones and does not depend on the result or buy any say over conclusions.
Availability: Checked per enquiry.

Conflicts are checked before scoping. See publication, IP and independence.

Evidence behind this page

The Correctness Illusion in LLM-Generated GPU Kernels
2026 · Preprint (not peer reviewed) · cited by 6 (Google Scholar, October 2026)
Test-Input Generation for Tensor Programs: What Actually Finds Kernel Bugs
2026 · Preprint (not peer reviewed)
An Input-Regime Audit of Conflict Detection for Retrieval-Augmented Generation
2026 · Manuscript under submission

gpuemu-corpus (GitHub) — The format an open experiment releases: 26 ops (16 correct controls, 10 buggy variants), drivers for four papers, stored seeds and replay scripts. MIT or Apache-2.0.
gpuemu-corpus (Hugging Face) — The same corpus as a dataset with fp64 references. MIT.
ebrag-vecdb-2026-paper (GitHub) — Paper source, one JSON artefact per experiment and a runnable slice. MIT.

Questions

Can we fund an independent AI benchmark study?

Yes, as an open experiment, provided the question, method and analysis are published before results exist and the results are released whatever they show. If I have an interest in any system being benchmarked, that is disclosed at intake and may mean declining or narrowing the study.

What does the sponsor get?

A credible public answer to a question it cares about, the open artefacts, acknowledgement in the funding statement, and sight of the write-up before release to check factual errors. It does not get exclusivity, a veto or editorial control over findings.

How is the budget structured?

By milestone: for example, the published plan, the released harness and data, the results, and the write-up. Each milestone is priced at scoping and paid when its deliverable is delivered.

How is this different from a sponsored study?

A sponsored study answers a question for your decision and is confidential unless publication is agreed. An open experiment answers a question for everyone and is always published.

Commission a scoped study

Send the question you would like answered in public, why it matters and a rough budget range. I will say whether it is answerable, what I would need to disclose, and how the milestones might look.

Commission a scoped study Draft a brief first

Last reviewed 2026-10-07.