Test AI models and agents on real work.

Give a model or an agent a real task: fix a bug in an open-source library, triage an incident from its logs, correct an order in a web app. It works in a sealed sandbox, and hidden tests it never sees decide the result.

Test a model See what we tested

Five exams free · no sign-in · bring your own API key, or try one on our free model · watch a real run

Loading a recorded run…

Real attempts from one sealed record, replayed. Watch all four, step by step

We tested

How it works

From your model to a result anyone can check.

01
Connect your model or agent
Through its provider’s API with your key, your own server, a model on your machine, or an agent over HTTP. A key is encrypted the moment it arrives.
02
It works in a sealed sandbox
A real task: a repository to fix, a terminal, a web app. The model reads, runs and edits with the exam’s tools only. No network, one CPU, and a step budget.
03
Hidden tests grade it, in a separate box
The tests and the reference fix never enter the sandbox. Before an exam is offered, nine or ten cheating probes try to break its grading, and all must fail.
04
You get a score anyone can check
A pass rate with its 95% range, and a signed receipt: every result hashed, the run signed. Change one grade and the public check fails.

Grading, what we record, and what you get

The product

Pick an exam. Run it. Read the result.

Real screens from the app, with real data.

The Exams screen: every exam with its type, access and how many cheating probes it blocked. — Every exam, with how it was certified

One exam: what it measures, why its result can be trusted, and the panel to run it. — One exam: what it measures, and why a pass counts

What we tested: agents and models ranked with 95% ranges, and results per exam. — Results with their ranges, per exam

Why test with us

A score is only worth the way it was made.

Real work, not a quiz

A bug fixed in a real open-source project, an incident in a server's logs, an order in a web app, a question about real public data. The model works through it with tools, step by step, as it would on the job.

Tests it cannot see

The hidden tests and the reference answer sit in a separate box the model can never reach, and we never publish them. Code decides every grade, not an AI judge.

Exams attacked before use

Before anyone is scored, nine or ten cheating probes try to read the tests, edit the grader, fake the output or leave the sandbox. All of them must fail.

Cheating fails, and is named

An attempt that reaches for what it may not see fails, and the report says so. When our own setup once let an agent see too much, we withdrew its six passes and published the list.

Numbers that hold up

Every pass rate carries its 95% range. An outage is an endpoint error, never a wrong answer. An unknown cost stays unknown instead of becoming zero.

Proof, not a promise

Each attempt is kept with its full evidence: the whole conversation with the model, every command, change and grade. Each run is signed, and anyone can check a receipt.

Your own questions too: upload a CSV of your cases and grade answers by exact match, format, similarity or a judge model, from the app, the REST API, Python, a GitHub Action or MCP. How grading works, and what we record

What we tested

Well-known agents and models went through the same exams first. The numbers, the reports and what was withdrawn are all published.

Loading the published results…

17certified exams here: 3 open to everyone, 14 on request
6checks each exam passes before use
9–10cheating probes per exam, none may succeed
How an exam is certified Methodology What was withdrawn, and why
The sealed report of that campaign (PDF, 14 pages)

Questions

Which models and agents can I test?

Any. Connect a model through its provider’s API with your own key, through any OpenAI-compatible address (your own server, a cloud deployment, a router or a company gateway), or on your own computer through a small connector.

On our exams the model works as an agent: it reads files, runs commands, changes code and submits, using the exam’s tools. An agent or AI app you already run connects over HTTP and is tested on your own questions.

What is free?

Five exams: incident triage in a terminal, a bug in an open-source library, SQL on real public data, correcting an order through a real browser, and labelling intents and entities. Also your own questions and the playground. Bring an API key from any provider. To test your model or agent on more of our certified exams, book a call: we run them for you and send a reviewed report.

Who pays for the model calls?

Your provider bills you for calls made with your key, at their prices. Nothing is added on top. Our free model costs you nothing: it takes one exam, Intent and entity annotation, up to 2 attempts a run, while the day’s capacity lasts.

Is my API key safe?

It is encrypted with AES-256-GCM the moment it arrives, used only inside a call, and never shown again or written into logs or receipts. Remove the connection to delete it.

How long does a test take?

One attempt usually takes between half a minute and two minutes. Slow models take longer, because nearly all the time is the model answering. You see an estimate before you start, and the run continues if you close the page.

Can a model cheat or memorise the answers?

The tests and the reference solution never enter the model's sandbox, and every exam was attacked with cheating probes before it was offered. A model that reaches outside its sandbox fails the attempt.

Memorising is a real risk for exams rebuilt from bugs fixed in public open-source projects: a model may have seen the upstream fix in training. Those exams say so in their family name. The others were written for these tests, and we never publish their hidden tests or answers.

What can other people see?

Nothing, unless you share a run's link or publish it. Keys and addresses are never shown.

What does it cost?

Using it is free, with no card: your model calls are billed by your provider, to your key. For evaluations on our private exams or exams built from your work, book a call. Every limit

Put your model through a real task.

Free, no sign-in. Bring an API key from any provider, or try our free model with no key.

Test a model