Methodology
The scoring rules that keep a missing field from looking like a pass.
Versioned contract
Input is { baseline: Run, candidate: Run }. A run may carry
id, model, outcome, cost,
latencyMs, and steps. Output is a v1 report:
outcome diff, cost diff, latency diff, per-step attribution, and an
unknowns[] list. Same input always yields byte-identical JSON.
Unknown is first-class
Missing, untyped, or corrupt values stay unknown. They are never coerced to zero, pass, or fail. Cost is summed only when every generation has a known cost, so the report never understates spend. Timezone-naive ISO stamps are rejected (an ambiguous instant would break determinism).
What it does not do
conediff does not call models, collectors, or billed APIs. Adapters parse a recorded export you already have. Hosted history, auth, and billing are proposed later and are not part of this local engine.