Two-model pilot · August 2026
AI Handoff Continuity Benchmark
A reproducible test of whether an AI model can recover the exact state required to continue interrupted work from a transcript, compressed memory, or structured handoff.
- Cases
- 3
- Conditions
- 3
- Model systems
- 2
- Scoring
- Deterministic
Pilot result
Structured handoffs recovered the most continuation state
Across one complete run from each of two model systems, the structured handoff condition scored 79.45, compared with 76.67 for the conversation transcript and 45.00 for compressed memory. This is a small authored pilot, not a model leaderboard. Raw responses, strict-parse submissions, and deterministic scores are public.
Mean score across two systems
Handoff 79.45 · transcript 76.67 · memory 45.00
Each bar averages three cases from OpenAI gpt-5.6-terra and Anthropic claude-sonnet-5. Single-value fields required exact candidate IDs; multi-value fields used set F1.
Comparison design
Three ways to carry the same work forward
Transcript
The original conversation, including superseded decisions, incidental detail, and chronological corrections.
Compressed memory
A short narrative summary that retains the broad direction but omits some exact evidence, constraints, or open questions.
Structured handoff
An explicit continuation record containing objective, current state, decisions, evidence, constraints, next action, owner, and open questions.
Method
Inspect every input and reproduce every score
Authored pilot cases
The initial dataset covers an incident response, product research, and a data-model change. Every distractor and expected answer is visible.
Constrained answers
Models choose published candidate IDs. This isolates state recovery from prose style and makes scoring repeatable.
Deterministic scoring
Single-value fields require an exact match. Multi-value fields use set F1. Missing answers score zero.
Run it
One dataset, one runner, no dependencies
Node.js 22 or newer is sufficient. Generate the nine prompts, run them with a fixed model configuration, assemble the returned JSON, and score the submission.
curl -O https://handover.sh/benchmark/v1/run.mjs
curl -O https://handover.sh/benchmark/v1/dataset.json
node run.mjs --prompts ./prompts
node run.mjs --template ./submission.json
node run.mjs --submission ./submission.jsonIndependent results
Run the frozen method, then publish every response
Independent results are welcome when the model configuration, raw responses, failed parses, retries, scored submission, relationship disclosure, and limitations are all inspectable. No independent result has been accepted yet.
Validation proves package completeness and deterministic score reproducibility. It does not prove model provenance or experimental compliance.
- 01Freeze the method
Use release
benchmark-v1.0.0, one fresh session per prompt, and no tools, browsing, memory, or retrieval. - 02Keep the complete record
Preserve every raw response and failed parse. Declare retries instead of silently repairing output.
- 03Disclose and submit
State material relationships and limitations, validate the package locally, then open the guided submission.
100-point rubric
Continuation-critical information carries the most weight
How to read the result
Evidence for a next test, not a universal ranking
The pilot shows that an explicit continuation record can retain exact state better than a short free-form memory. It does not prove that formatting alone caused the difference, or that one model is generally better than another.
- Cases are authored rather than sampled from an independent corpus.
- The candidate format measures state recovery, not free-form task quality.
- The three conditions intentionally retain different amounts of detail.
- Schema adherence affected scores, including fenced JSON and candidate text in place of IDs.
- One run per system is insufficient to estimate variance.
- Handover created the benchmark and benefits if structured handoffs perform well.
Citation package
Cite and reuse the benchmark without rebuilding the evidence
The dataset is CC BY 4.0. Cite the canonical page and version when reusing the method, result table, or individual values. Machine-readable citation files and a flat CSV are published beside the raw responses.
@dataset{handover_continuity_benchmark_2026,
author = {{44pixels Ltd}},
title = {AI Handoff Continuity Benchmark},
year = {2026},
version = {1.0.0-pilot},
publisher = {Handover},
url = {https://handover.sh/benchmark}
}Continue the experiment
Inspect a real handoff before you build one
Open the public continuation demo to see versioned context, evidence, review, and a next action move between people and agents. No account is required.
Research context
Primary sources informing the protocol
The benchmark's structured condition implements the open Handoff Continuity Record v1.0. Inspect its schema, valid example, and conformance boundary before interpreting the result.