We test the models or agents you are choosing between on our private exams, review the results, and send you a report you can put in front of a decision.
We are taking on our first design partners: their first evaluation is free.
Why teams test with us
Hidden tests and reference answers sit in a box the model can never reach, and we never publish them. Code grades every attempt, not an AI judge.
Each exam must survive nine or ten cheating probes, and is frozen by fingerprint, so every model you compare meets exactly the same exam.
Each failed attempt is classed: a wrong result, nothing submitted, or reaching for what it may not see. An outage on your provider's side is an endpoint error, never a failure.
Every attempt is kept with its full evidence: the whole conversation with the model, each command, change and grade. We can walk through any attempt with you.
Each run is signed. Your security or procurement team can check a receipt without taking our word for it.
When our own setup let an agent see what it should not, we withdrew its passes and published the list. About us
Your candidates on our private, certified exams: code repair, terminal operations, work in a real browser, annotation and data. Several attempts each, and the report shows how certain each number is.
You get a reviewed report with a written verdict, pass rates, cost, failure patterns and receipts.
We turn your real tasks into private exams: a bug from your repository, an operations runbook, a data workflow. Each one passes the same six checks before it is used, and stays yours.
Useful when public benchmarks do not look like your work.
A check in your CI that fails a pull request when a model drops below the bar you set, from the GitHub Action or the command line. We set it up with you.
So a model or prompt change cannot quietly make things worse.
What you receive
This is a real one: the sealed campaign report of 2 September 2026. Five exams, three agents and models, 150 attempts. Every number in it can be recomputed from the sealed records.
Open the report (PDF)
A short form: what to test, why, and by when. We reply by email.
Which exams, how many attempts per model, and when the report arrives.
Real tasks in sealed sandboxes. Your model is called through its provider's API with your key, so the cost is visible to you.
Reviewed by us, with a verdict in plain words, every number and a receipt for each run.
Encrypted with AES-256-GCM when they arrive, used only inside a call, never shown again, never written into a record or receipt. Remove the connection and the key is deleted.
Our private exams reach your model only through its provider’s own API, never a server you run: anything running on your side would see the exam. The runs happen on our servers.
Your results are visible to your workspace only. Nothing is made public unless you choose to publish it. An NDA is available before we start.
Three exams are free to run yourself. For our other certified exams, book a call and we reply by email.