AI exam result · CURRENT
Lane pilots — fourteen API lanes and three CLIs on text and structured, one attempt each
Observed outcomes from sealed evaluation records. Failed attempts and non-model errors remain visible.
Outcomes by AI exam
| AI exam | AI agent | Passed / attempts | Technical errors | Withheld | Invalid |
|---|---|---|---|---|---|
| Record normalization and duplicate QA | @cf/openai/gpt-oss-20b | 1/1 | 0 | 0 | 0 |
| Intent and entity annotation | @cf/openai/gpt-oss-20b | 1/1 | 0 | 0 | 0 |
| Record normalization and duplicate QA | cli/claude-sonnet | 1/1 | 0 | 0 | 0 |
| Intent and entity annotation | cli/claude-sonnet | 1/1 | 0 | 0 | 0 |
| Record normalization and duplicate QA | cli/codex | 1/1 | 0 | 0 | 0 |
| Intent and entity annotation | cli/codex | 1/1 | 0 | 0 | 0 |
| Record normalization and duplicate QA | openai/gpt-oss-120b | 1/1 | 0 | 0 | 0 |
| Intent and entity annotation | openai/gpt-oss-120b | 1/1 | 0 | 0 | 0 |
| Record normalization and duplicate QA | codestral-latest | 1/1 | 0 | 0 | 0 |
| Intent and entity annotation | codestral-latest | 0/1 | 0 | 0 | 0 |
| Record normalization and duplicate QA | minimax/minimax-m3:free | 1/1 | 0 | 0 | 0 |
| Intent and entity annotation | minimax/minimax-m3:free | 0/1 | 0 | 0 | 0 |
| Record normalization and duplicate QA | mistral-code-latest | 1/1 | 0 | 0 | 0 |
| Intent and entity annotation | mistral-code-latest | 0/1 | 0 | 0 | 0 |
| Record normalization and duplicate QA | nvidia/nemotron-3-super-120b-a12b:free | 0/0 + 1 withheld | 0 | 1 | 0 |
| Intent and entity annotation | nvidia/nemotron-3-super-120b-a12b:free | 1/1 | 0 | 0 | 0 |
| Record normalization and duplicate QA | openai/gpt-oss-20b | 1/1 | 0 | 0 | 0 |
| Intent and entity annotation | openai/gpt-oss-20b | 0/0 + 1 lane error | 1 | 0 | 0 |
| Record normalization and duplicate QA | cli/cursor-grok | 0/0 + 1 invalid | 0 | 0 | 1 |
| Intent and entity annotation | cli/cursor-grok | 0/0 + 1 invalid | 0 | 0 | 1 |
| Record normalization and duplicate QA | cohere/north-mini-code:free | 0/0 + 1 withheld | 0 | 1 | 0 |
| Intent and entity annotation | cohere/north-mini-code:free | 0/0 + 1 withheld | 0 | 1 | 0 |
| Record normalization and duplicate QA | magistral-small-latest | 0/0 + 2 lane errors | 2 | 0 | 0 |
| Intent and entity annotation | magistral-small-latest | 0/0 + 2 lane errors | 2 | 0 | 0 |
| Record normalization and duplicate QA | ministral-14b-latest | 0/1 | 0 | 0 | 0 |
| Intent and entity annotation | ministral-14b-latest | 0/1 | 0 | 0 | 0 |
| Record normalization and duplicate QA | ministral-8b-latest | 0/1 | 0 | 0 | 0 |
| Intent and entity annotation | ministral-8b-latest | 0/1 | 0 | 0 | 0 |
| Record normalization and duplicate QA | mistral-medium-latest | 0/0 + 2 lane errors | 2 | 0 | 0 |
| Intent and entity annotation | mistral-medium-latest | 0/0 + 2 lane errors | 2 | 0 | 0 |
| Record normalization and duplicate QA | mistral-small-latest | 0/0 + 2 lane errors | 2 | 0 | 0 |
| Intent and entity annotation | mistral-small-latest | 0/0 + 2 lane errors | 2 | 0 | 0 |
| Record normalization and duplicate QA | nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | 0/0 + 1 lane error | 1 | 0 | 0 |
| Intent and entity annotation | nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | 0/0 + 1 lane error | 1 | 0 | 0 |
Campaign coverage
40 of 34 planned attempts recorded. All planned attempts are recorded.
Structured · cli/codex
100.00/100 · Grader quality score
1 attempts; 1 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|
Structured · cli/claude-sonnet
100.00/100 · Grader quality score
1 attempts; 1 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|
Structured · cli/cursor-grok
Not measured · Grader quality score
1 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Structured · mistral-small-latest
Not measured · Grader quality score
2 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Structured · mistral-medium-latest
Not measured · Grader quality score
2 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Structured · magistral-small-latest
Not measured · Grader quality score
2 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Structured · openai/gpt-oss-20b
100.00/100 · Grader quality score
1 attempts; 1 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|
Structured · openai/gpt-oss-120b
100.00/100 · Grader quality score
1 attempts; 1 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|
Structured · codestral-latest
100.00/100 · Grader quality score
1 attempts; 1 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|
Structured · mistral-code-latest
100.00/100 · Grader quality score
1 attempts; 1 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|
Structured · ministral-14b-latest
75.00/100 · Grader quality score
1 attempts; 0 passed; 1 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|
Structured · ministral-8b-latest
75.00/100 · Grader quality score
1 attempts; 0 passed; 1 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|
Structured · @cf/openai/gpt-oss-20b
100.00/100 · Grader quality score
1 attempts; 1 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|
Structured · nvidia/nemotron-3-super-120b-a12b:free
Not measured · Grader quality score
1 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Structured · cohere/north-mini-code:free
Not measured · Grader quality score
1 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Structured · minimax/minimax-m3:free
100.00/100 · Grader quality score
1 attempts; 1 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|
Structured · nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free
Not measured · Grader quality score
1 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Text · cli/codex
100.00/100 · Grader quality score
1 attempts; 1 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|---|
| Classification accuracy | 1.00 |
Text · cli/claude-sonnet
100.00/100 · Grader quality score
1 attempts; 1 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|---|
| Classification accuracy | 1.00 |
Text · cli/cursor-grok
Not measured · Grader quality score
1 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Text · mistral-small-latest
Not measured · Grader quality score
2 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Text · mistral-medium-latest
Not measured · Grader quality score
2 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Text · magistral-small-latest
Not measured · Grader quality score
2 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Text · openai/gpt-oss-20b
Not measured · Grader quality score
1 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Text · openai/gpt-oss-120b
100.00/100 · Grader quality score
1 attempts; 1 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|---|
| Classification accuracy | 1.00 |
Text · codestral-latest
75.00/100 · Grader quality score
1 attempts; 0 passed; 1 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|---|
| Classification accuracy | 1.00 |
Text · mistral-code-latest
70.37/100 · Grader quality score
1 attempts; 0 passed; 1 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|---|
| Classification accuracy | 1.00 |
Text · ministral-14b-latest
66.67/100 · Grader quality score
1 attempts; 0 passed; 1 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|---|
| Classification accuracy | 1.00 |
Text · ministral-8b-latest
60.00/100 · Grader quality score
1 attempts; 0 passed; 1 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|---|
| Classification accuracy | 1.00 |
Text · @cf/openai/gpt-oss-20b
100.00/100 · Grader quality score
1 attempts; 1 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|---|
| Classification accuracy | 1.00 |
Text · nvidia/nemotron-3-super-120b-a12b:free
100.00/100 · Grader quality score
1 attempts; 1 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|---|
| Classification accuracy | 1.00 |
Text · cohere/north-mini-code:free
Not measured · Grader quality score
1 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Text · minimax/minimax-m3:free
75.00/100 · Grader quality score
1 attempts; 0 passed; 1 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
| Measure | Score |
|---|---|
| Classification accuracy | 1.00 |
Text · nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free
Not measured · Grader quality score
1 attempts; 0 passed; 0 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: canonical text tools.
Interpretation and limits
- Repeated attempts on these fixed tasks only; not a broad capability score.
- All outcomes are retained; lane errors, withheld and invalid attempts are not model failures.
- Quality is the existing grader measurement, separate from the pass gate.
- Video delivery is stated per model from recorded receipts; native files and sampled frames are distinct. Provider-side sampling is not independently observed.
- Provider token usage is recorded where available. Monetary usage is not an invoice, and requested seeds do not establish deterministic replay.
- Private references, submissions, exact geometry, frame timestamps and raw sessions are withheld.
Technical evidence and detailed report
Detailed technical report (PDF) · Results and record digests (JSON) · Attempt counts and commitments (JSON)
These optional files provide the full audit detail behind this result.
fc-0c7e7c67a231