Measured capability
Agent Reliability
Recovery behavior under deterministic tool faults and injected instructions.
1 measured exam3 recorded lanesLatest measurement 2026-09-02Published campaign scopeNo category composite
Certified exam
Tool-loop fault recovery
3 measured model lanes
Model lane
cli/claude-sonnet
10/10100% certified
Lanecli/claude-sonnetDeliveryDelivery not establishedConditionsUnproven
Model failures0Lane errors0Withheld / invalid0 / 0
Elapsed median77.0 seconds · 10/10 observedSteps median11 · 10/10 observedTokens medianNot recordedCost median (USD)Not recorded
delivery equivalence is unproven; delivery is not proved; lane configuration is not recorded
Inspect measurement evidence →
Lanecli/codexDeliveryDelivery not establishedConditionsUnproven
Model failures0Lane errors0Withheld / invalid0 / 0
Elapsed median140.0 seconds · 10/10 observedSteps median16 · 10/10 observedTokens medianNot recordedCost median (USD)Not recorded
delivery equivalence is unproven; delivery is not proved; lane configuration is not recorded
Inspect measurement evidence →
Model lane
codestral-latest
0/100% certified
Lanecodestral-latestDeliveryDelivery not establishedConditionsUnproven
Model failures10Lane errors0Withheld / invalid0 / 0
Elapsed median87.9 seconds · 10/10 observedSteps median24 · 10/10 observedTokens median90792.5 · 10/10 observedCost median (USD)Not recorded
delivery equivalence is unproven; delivery is not proved; lane configuration is not recorded
Inspect measurement evidence →