Model results
Real model attempts, recorded outcomes and the limits of each measurement. A failed attempt stays in the report. A technical error is identified separately.
Latest published campaign · CURRENT
1 tasks · 1 model lanes · 12 attempts · 4 certified passes
Models: meta/muse-spark-1.3. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · CURRENT
3 tasks · 1 model lanes · 30 attempts · 20 certified passes
Models: cli/opencode-muse-spark. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · CURRENT
2 tasks · 2 model lanes · 40 attempts · 40 certified passes
Models: cli/codex, cli/claude-sonnet. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · CURRENT
2 tasks · 7 model lanes · 136 attempts · 67 certified passes
Models: openai/gpt-oss-20b, openai/gpt-oss-120b, @cf/openai/gpt-oss-20b, codestral-latest, mistral-code-latest, ministral-14b-latest, ministral-8b-latest. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · CURRENT
2 tasks · 17 model lanes · 40 attempts · 13 certified passes
Models: cli/codex, cli/claude-sonnet, cli/cursor-grok, mistral-small-latest, mistral-medium-latest, magistral-small-latest, openai/gpt-oss-20b, openai/gpt-oss-120b, codestral-latest, mistral-code-latest, ministral-14b-latest, ministral-8b-latest, @cf/openai/gpt-oss-20b, nvidia/nemotron-3-super-120b-a12b:free, cohere/north-mini-code:free, minimax/minimax-m3:free, nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · CURRENT
4 tasks · 2 model lanes · 10 attempts · 2 certified passes
Models: gemini-3.6-flash, cli/opencode-muse-spark. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · CURRENT
1 tasks · 1 model lanes · 1 attempts · 1 certified passes
Models: meta/muse-spark-1.3. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · CURRENT
1 tasks · 1 model lanes · 1 attempts · 0 certified passes
Models: gemini-3.6-flash. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · HISTORICAL
1 tasks · 1 model lanes · 1 attempts · 0 certified passes
Models: gemini-3.6-flash. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · HISTORICAL
1 tasks · 1 model lanes · 3 attempts · 0 certified passes
Models: gemini-3.8-flash. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · CURRENT
4 tasks · 1 model lanes · 40 attempts · 25 certified passes
Models: cli/codex-gpt-5.5. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · CURRENT
5 tasks · 3 model lanes · 150 attempts · 93 certified passes
Models: cli/claude-sonnet, cli/codex, codestral-latest. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · HISTORICAL
3 tasks · 3 model lanes · 90 attempts · 52 certified passes
Models: cli/claude-sonnet, cli/codex, codestral-latest. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · HISTORICAL
2 tasks · 3 model lanes · 60 attempts · 41 certified passes
Models: cli/claude-sonnet, cli/codex, codestral-latest. Read the report for individual task outcomes, exclusions and measurement limits.
Earlier published campaign · HISTORICAL
5 tasks · 4 model lanes · 20 attempts · 8 certified passes
Models: cli/claude-sonnet, cli/codex, codestral-latest, openai/gpt-oss-120b. Read the report for individual task outcomes, exclusions and measurement limits.
Certification checks whether an environment grades work reliably. It is separate from how a model performs on that environment.
Explore the five annotation environments →