Measured capability
Memory
Fact updates across sessions when newer authority supersedes stored memory.
1 measured exam4 recorded lanesLatest measurement 2026-09-01Published campaign scopeNo category composite
Certified exam
Cross-session fact update
4 measured model lanes
Model lane · condition 1 of 2
cli/claude-sonnet
Historical · retired revision10/10100% certified
Lanecli/claude-sonnetDeliveryDelivery not establishedConditionsUnproven
Model failures0Lane errors0Withheld / invalid0 / 0
Elapsed median16.1 seconds · 10/10 observedSteps median5 · 10/10 observedTokens medianNot recordedCost median (USD)Not recorded
delivery equivalence is unproven; delivery is not proved; lane configuration is not recorded; recorded engine or harness conditions differ
Inspect measurement evidence →
Model lane · condition 2 of 2
cli/claude-sonnet
Historical · retired revision1/1Certified passes · rate not published
Lanecli/claude-sonnetDeliveryDelivery not establishedConditionsUnproven
Model failures0Lane errors0Withheld / invalid0 / 0
Elapsed median18.2 seconds · 1/1 observedSteps median5 · 1/1 observedTokens medianNot recordedCost median (USD)Not recorded
delivery equivalence is unproven; delivery is not proved; lane configuration is not recorded; recorded engine or harness conditions differ
Inspect measurement evidence →
Model lane · condition 1 of 2
cli/codex
Historical · retired revision10/10100% certified
Lanecli/codexDeliveryDelivery not establishedConditionsUnproven
Model failures0Lane errors0Withheld / invalid0 / 0
Elapsed median26.3 seconds · 10/10 observedSteps median6 · 10/10 observedTokens medianNot recordedCost median (USD)Not recorded
delivery equivalence is unproven; delivery is not proved; lane configuration is not recorded; recorded engine or harness conditions differ
Inspect measurement evidence →
Model lane · condition 2 of 2
cli/codex
Historical · retired revision1/1Certified passes · rate not published
Lanecli/codexDeliveryDelivery not establishedConditionsUnproven
Model failures0Lane errors0Withheld / invalid0 / 0
Elapsed median27.8 seconds · 1/1 observedSteps median6 · 1/1 observedTokens medianNot recordedCost median (USD)Not recorded
delivery equivalence is unproven; delivery is not proved; lane configuration is not recorded; recorded engine or harness conditions differ
Inspect measurement evidence →
Model lane · condition 1 of 2
codestral-latest
Historical · retired revision2/1020% certified
Lanecodestral-latestDeliveryDelivery not establishedConditionsUnproven
Model failures8Lane errors0Withheld / invalid0 / 0
Elapsed median7.1 seconds · 10/10 observedSteps median4 · 10/10 observedTokens median5764 · 10/10 observedCost median (USD)Not recorded
delivery equivalence is unproven; delivery is not proved; lane configuration is not recorded; recorded engine or harness conditions differ
Inspect measurement evidence →
Model lane · condition 2 of 2
codestral-latest
Historical · retired revision0/1Certified passes · rate not published
Lanecodestral-latestDeliveryDelivery not establishedConditionsUnproven
Model failures1Lane errors0Withheld / invalid0 / 0
Elapsed median6.8 seconds · 1/1 observedSteps median4 · 1/1 observedTokens median5785 · 1/1 observedCost median (USD)Not recorded
delivery equivalence is unproven; delivery is not proved; lane configuration is not recorded; recorded engine or harness conditions differ
Inspect measurement evidence →
Model lane
openai/gpt-oss-120b
Historical · retired revision1/1Certified passes · rate not published
Laneopenai/gpt-oss-120bDeliveryDelivery not establishedConditionsUnproven
Model failures0Lane errors0Withheld / invalid0 / 0
Elapsed median48.4 seconds · 1/1 observedSteps median6 · 1/1 observedTokens median7980 · 1/1 observedCost median (USD)Not recorded
delivery equivalence is unproven; delivery is not proved; lane configuration is not recorded; recorded engine or harness conditions differ
Inspect measurement evidence →