Runbook
A calm sequence for messy model work.
The runbook is for the moment before a team asks another model to “try again.” It slows the work enough to identify the route, then speeds it up by making each handoff explicit.
Intake the request
Name the audience, risk, required freshness, source burden, and expected answer form before a model is selected.
Assign the route
Choose whether the work needs retrieval, reasoning, transformation, speed, creative alternatives, or human review before synthesis.
Compare the output
Look for disagreement, missing caveats, invented structure, stale facts, and places where the answer hides uncertainty.
Publish the handoff
Leave a concise trail: what was asked, what was checked, what changed, and what a future editor should revisit.
What the runbook prevents
Multi-model teams often lose time in invisible loops. A first model drafts too broadly, a second model rewrites too smoothly, a search step introduces a newer source, and a final answer sounds settled even though nobody recorded which claim was verified. The runbook makes those transitions visible. It is less about bureaucracy and more about preserving the reason a decision felt trustworthy at the time.
A good runbook entry is compact but not thin. It includes the question shape, the reason a route was chosen, the checks performed, and the conditions that would change the answer. When the same type of work repeats, the route becomes easier to reuse. When the facts change, the record shows where to update. When an answer is challenged, the team can retrace the path instead of reconstructing the entire session from memory.
The runbook also clarifies escalation. Some tasks should move from a fast model to a reasoning model. Some should move from a model to a source database. Some should stop and wait for a person with domain authority. The point is not to make every answer slower. The point is to spend attention where failure is expensive and keep low-risk work moving.