Watch a real run
Four agents. One real bug. Hidden tests decide.
A recorded attempt, replayed step by step: the files the agent looked at, the change it made, then the grading in a separate box it never saw. The bug is issue 257 of xmltodict, an open-source Python library, fixed upstream in pull request 421.
Loading the recorded attempts…
Space plays or pauses · the arrow keys step · 1×, 2×, 4×. The replay keeps its own pace; each attempt's real time is shown with its verdict.
Same bug, one clock.
The four attempts side by side at their real durations, sped up. The fastest failed; the slowest did too.
What you are looking at
Every step is recorded
Each tool the agent called, on which file, in order, whether it worked and how much came back. The files it read are the public library and the task; test output is not shown.
The change is the agent's own
The diff is exactly what the agent submitted. The exam is public, MIT-licensed code, so its changes can be shown.
The tests stay hidden
The hidden tests and the maintainers' fix never entered the sandbox. That is why a pass means something, and why their names are not published.
It can be checked
These attempts come from one sealed record, and its featured page shows the certification behind the exam.
Sealed record fr-20260902-6616b188 · digest 138425da59aabc18… · its featured page