The sandbox
Every command an agent runs starts a fresh container with no network, capped at one CPU and 512 MB of memory (1 GB for the browser exam, whose container stays open for the whole attempt). The task tree is copied in; the hidden tests and the reference solution live in a separate grading box the agent can never reach. Agent programs that run on the host, like Claude Code and Codex, are audited after each run from their own session logs; if containment cannot be proven, the attempt is withheld.
Certified before use
An exam is used only after six checks: it builds cleanly, the upstream fix passes and the untouched code fails, deliberately wrong control submissions land on their known verdicts (including a near miss a good engineer would write), every cheating probe fails (nine or ten per exam), it runs end to end, and it is frozen. The Forge catalog of 2 September 2026 holds 25 certified exams in 14 families. The 17 offered here were each checked again on 24 September on this server’s own sandbox images: the reference solution passed, every control landed on its verdict, and every cheating probe failed. One exam, walked through the chain
Scale without waste
An attempt spends almost all its time waiting for the model, so attempts run many at a time while a small gate limits only the moments a sandbox actually runs. A fair queue takes the next attempt from whoever has the fewest running. Measured on the production server: 24 attempts at once finished in 71 seconds and 48 in 102, with never more than four sandboxes running. Another site on the same server kept answering throughout: its slowest request took 0.28 seconds at 24 attempts and 0.58 at 48.
Numbers that do not flatter
Every pass rate carries its 95% Wilson range. A broken connection is an endpoint error, never a wrong answer. An unknown cost stays unknown instead of becoming zero. Scores from different campaigns are never averaged together, because their exams and conditions differ.
Proof anyone can check
A finished run is sealed: each result becomes a SHA-256 leaf, the leaves are combined, and the digest is signed with Ed25519. Change one grade and the public check fails. Each Forge attempt also carries the digest of its sealed record.
Publishing what went wrong
One agent lane turned out to keep its full host toolset, and in one exam it could read the reference solution and the hidden tests. All six of its passes were withdrawn, and 232 historical attempts were re-classified from stored evidence. The integrity page lists every one.
Python and FastAPI with Postgres (the job queue uses row locks, so any number of workers can share it), a Preact interface with no build step, Docker for the sandboxes. A release ships only when the full automated test suite passes.