Methods
How I investigate evaluation claims, written so you can apply the protocol yourself. Each page states what the method can establish, what it cannot, and the primary work it comes from. Using a method is not the same as a study's result: a short checklist does not prove correctness, privacy or robustness.
Method
Benchmark Leakage and Contamination Checks
Overlap audit, written contamination hypotheses and a sensitivity analysis that tests whether the conclusion survives.
Method
Uncertainty, Repeated Trials and Evaluation Claims
Repeated-trial design, cluster bootstrap over items, rank stability and sensitivity comparisons for small performance gaps.
Method
Agent Failure Taxonomies and Incident Coding
Versioned codebook, ambiguity rules, double coding with agreement statistics and annotated examples for agent traces.
Method
Evaluation Dataset Design and Provenance
Dataset specification, sampling design with controls, per-item provenance ledger, exclusion log and access restrictions.
Method
Pre-Specified Evaluation Plans
Versioned research question, primary outcomes, exclusion and analysis rules, and an amendment log, fixed before results.
Method
Research Reproducibility Packages
Environment manifest, inputs with provenance, per-table scripts, raw per-run outputs and a known-limits statement, with worked examples.
Free tools
Free tool
Evaluation Brief Builder
Turn an uncertain technical claim into a structured experiment brief in your browser, before any conversation.
Free tool
Reproducibility Readiness Checklist
Check which missing artefacts would stop another team rerunning your study, and decide what to fix before release.
Need the method applied to your claim? Compare the commissioned study types.