AI exam result · HISTORICAL
Featured exams — ten rollouts per lane
Observed outcomes from sealed evaluation records. Failed attempts and non-model errors remain visible.
Outcomes by AI exam
| AI exam | AI agent | Passed / attempts | Technical errors | Withheld | Invalid |
|---|---|---|---|---|---|
| Streaming items must equal what the full parse would have built | cli/codex | 10/10 | 0 | 0 | 0 |
| Answering questions from a document collection with outdated and distractor sources, and refusing when the evidence is insufficient | cli/codex | 10/10 | 0 | 0 | 0 |
| Updating a cross-session fact: answer from the newer authoritative roster, not stale carried memory | cli/codex | 10/10 | 0 | 0 | 0 |
| Streaming items must equal what the full parse would have built | cli/claude-sonnet | 9/10 | 0 | 0 | 0 |
| Answering questions from a document collection with outdated and distractor sources, and refusing when the evidence is insufficient | cli/claude-sonnet | 1/10 | 0 | 0 | 0 |
| Updating a cross-session fact: answer from the newer authoritative roster, not stale carried memory | cli/claude-sonnet | 10/10 | 0 | 0 | 0 |
| Streaming items must equal what the full parse would have built | codestral-latest | 0/10 | 0 | 0 | 0 |
| Answering questions from a document collection with outdated and distractor sources, and refusing when the evidence is insufficient | codestral-latest | 0/9 + 1 lane error | 1 | 0 | 0 |
| Updating a cross-session fact: answer from the newer authoritative roster, not stale carried memory | codestral-latest | 2/10 | 0 | 0 | 0 |
Technical evidence and detailed report
Detailed technical report (PDF) · Results and record digests (JSON) · Attempt counts and commitments (JSON)
These optional files provide the full audit detail behind this result.
fc-3194f803055c