AI exam result · CURRENT
Muse Spark 1.3 native video via OpenRouter — twelve attempts, MP4 re-encode with exact frames
Observed outcomes from sealed evaluation records. Failed attempts and non-model errors remain visible.
Outcomes by AI exam
| AI exam | AI agent | Passed / attempts | Technical errors | Withheld | Invalid |
|---|---|---|---|---|---|
| Robot catch temporal and tracking annotation | meta/muse-spark-1.3 | 4/10 + 2 lane errors | 2 | 0 | 0 |
Campaign coverage
12 of 12 planned attempts recorded. All planned attempts are recorded.
Video · meta/muse-spark-1.3
74.86/100 · Grader quality score
12 attempts; 4 passed; 6 failed. Quality measures answer accuracy; passing also requires the exam’s declared minimums.
Input: native video with supplied exact-frame images; provider-side processing is not independently observed; video re-encoded to MP4 H.264 CRF 18 because the provider accepts MP4 only; frame count, frame rate and timing preserved; exact inspection frames delivered as lossless PNG from the original file.
| Measure | Score |
|---|---|
| Classification accuracy | 1.00 |
| Event matching (F1) | 0.90 |
| Segment matching (F1) | 1.00 |
| Box matching (F1) | 0.70 |
| Track consistency on matched boxes | 1.00 |
Interpretation and limits
- Repeated attempts on these fixed tasks only; not a broad capability score.
- All outcomes are retained; lane errors, withheld and invalid attempts are not model failures.
- Quality is the existing grader measurement, separate from the pass gate.
- Video delivery is stated per model from recorded receipts; native files and sampled frames are distinct. Provider-side sampling is not independently observed.
- Provider token usage is recorded where available. Monetary usage is not an invoice, and requested seeds do not establish deterministic replay.
- Private references, submissions, exact geometry, frame timestamps and raw sessions are withheld.
Technical evidence and detailed report
Detailed technical report (PDF) · Results and record digests (JSON) · Attempt counts and commitments (JSON)
These optional files provide the full audit detail behind this result.
fc-89575ed5687d