A frontier model wrote this code. We verified it — or refused it.
Frontier coding agents write more code than anyone can review, and they fail silently — a plausible function that breaks on the input nobody tried. The hard part isn’t generating code; it’s knowing which code is actually correct. The engine that never serves a wrong KPI runs on code too: it independently proves the model’s output against a declared spec — with math, not trust — or refuses with the exact input that breaks it.
Proven, not trusted
Three independent methods: execution-differential against an independent oracle, property / metamorphic invariants, and SMT where the problem is decidable. The model’s output is evidence — never the verdict.
It refuses — with a counterexample
When the code breaks, you get the exact input that breaks it and which method caught it. When a property is undecidable or out of scope, it refuses rather than bless it. No silent pass.
Model-independent, drop-in for agents
Any frontier agent calls one endpoint or MCP tool with code plus a spec and gets back a verdict and the evidence. The verifier is the same regardless of which model wrote the code.
The verdict is ours, not the model’s.
The model’s confidence doesn’t enter the decision. Either an independent check confirms the code against the declared spec, or it’s refused with the reason. Here is a worked example — one function the verifier proves, and one it refuses with the input that breaks it:
Refused — with a counterexample
A median() the agent wrote returns the wrong value for an even-length list. Execution-differential against an independent oracle finds the breaking input:
input [1, 2, 3, 4] → expected 2.5 · got 3
caught by execution-differential — refused, not blessed.
Proven — over the declared domain
A double(a) the agent wrote, declared equal to 2·a. SMT proves the two are equal for every integer — symbolically, not by sampling:
method SMT · proven for all inputs (symbolic)
proven for the whole domain — not a sample, not a score.
PROVEN covers the full declared domain; VERIFIED covers the inputs that were checked. Both are independent of the model — the figures above are reproducible, not a vendor score.
12/12 planted bugs caught. 0 slipped through.
We ran 25 realistic functions across 5 families — each in a correct and a buggy variant a coding agent might write — through the engine. Every bug was caught with the exact input that breaks it, and no correct function was wrongly refused.
planted bugs caught
silently passed
false alarms
correct ones proven
| family | the bug a model wrote | counterexample | caught by |
|---|---|---|---|
| arithmetic | forgot to negate negatives | n=-20 | execution-differential |
| arithmetic | a*b+c instead of a*(b+c) | a=-1, b=5, c=5 | smt |
| arithmetic | min/max swapped | x=-4 | execution-differential |
| number-theory | missing n<2 guard (1 and 0 reported prime) | n=0 | execution-differential |
| number-theory | loop n>9 drops final digit | n=1 | execution-differential |
| list-aggregate | returns first element, not the max | xs=[0, 1] | execution-differential |
| list-aggregate | counts odds (v%2==1) | xs=[0] | execution-differential |
| list-structure | returns input unchanged | xs=[0, 1] | execution-differential |
| list-structure | always returns False | xs=[0, 0] | execution-differential |
| list-structure | returns input unsorted | xs=[1, 0] | execution-differential |
| string | checks only first/last char | s='aaba' | execution-differential |
| string | counts 'b' as a vowel | s='b' | execution-differential |
Reproduce: python -m cuenta.codeverify_bench
We never say “safe” or “bug-free” — only verified against this declared spec, or refused.