Someone else's agent just shipped a pull request to your repository.
The description is confident, the diff is clean, and every function looks reasonable.
None of that is evidence that it does what you wanted, and none of it is evidence that the change works.
**A pull request is a claim about the world, and a claim arriving from a stranger's agent comes with zero evidence attached.**

When a human colleague shipped you a pull request, you could interrogate the author.
Why this approach, what did you try first, what did you test by hand.
The author held the intent, and the conversation between your understanding and theirs was part of quality control.
An agent-written pull request has no intent to consult.
There is no one to ask, so the review starts with no information, and the question is how to fill it.

The good news is that you can obtain certainty through specific actions.
I have landed on a set of demands I now make of every inbound agent pull request, and the demands convert the claim into checks I control.
**You cannot get certainty by reading harder; you get it by forcing the pull request to produce evidence.**

## Two Questions, Two Different Certainties

"Working properly" and "implementing the desired outcome" sound like one question, but they are two, and each fails in a different way.
They are not equally important, and the more important one comes first.

Does it do what was wanted is the first question.
The person who filed the issue and the agent that wrote the code never met, and most of what was wanted lived in context the agent never saw.
This is [validation](https://en.wikipedia.org/wiki/Verification_and_validation), and it responds to comparison against a stated intent.

Does it work is the second question.
Does the bug stay fixed, does the feature behave, does the build pass under the cases nobody pasted into the prompt.
This is [verification](https://en.wikipedia.org/wiki/Verification_and_validation), and it responds to checks.

A pull request that passes every test and still solves the wrong problem is the most expensive failure of the two, because it appears successful right up until production.
**Verification proves the agent built the spec right, and only you can say whether the spec was right.**

## Demand the Definition of Done Before Anything Else

The first thing to demand is not code, it is the acceptance criteria, stated as observable behavior, written before the diff existed.

An issue that says "fix the crash" produces an uncheckable pull request.
An issue that says "this script must exit zero on this input and produce this output" produces a pull request you can actually verify, because it names the condition under which the change is done.
If the pull request links to criteria like the second kind, you have something you can check against.
If it links to nothing, or to prose so vague that two engineers would disagree about whether it was met, send it back and get the criteria written.
**Certainty about a pull request is limited by the precision of the definition of done it was built against, and no amount of review effort can exceed that limit.**

Then run the traceability check in both directions.
Every acceptance criterion should map to something in the diff, and every change in the diff should trace back to a criterion.
A criterion with no code is a claim the pull request did not implement.
A change with no criterion is an edit somebody's model wanted to make, which is exactly the category of change you did not ask for.
Unmatched items on either side are findings, and the traceability pass finds them in minutes without reading a single line for style.

## Run the Behavior, Don't Read the Prose

The second demand is behavioral evidence, produced in an environment you control.

For a bug fix, reproduce the failure before applying the patch, then apply it and reproduce again.
The claim "fixed" is only meaningful if the failure you could trigger yesterday is gone today, on your machine, against cases beyond the one the agent was handed.
For a feature, exercise the acceptance criteria yourself, directly, the way the person who filed the issue would.
Ten minutes of running the code beats an hour of reading it, because reading is plausibility-matching and plausibility is exactly what the generating model optimized.

Look hard at the tests the pull request ships, because agent-written tests deserve their own suspicion.
A test suite that passes proves nothing on its own.
Ask the question that matters: would these tests fail if the change were reverted?
A test that passes both before and after the change tests nothing, and a surprising fraction of agent-written tests are exactly that, written to satisfy a coverage gate rather than to check the behavior.
The strongest pattern is a test that demonstrably failed first, and I treat evidence of red-before-green as worth more than any prose in the pull request description.
**A test you have never seen fail does not verify the behavior it claims to check.**

Machine oracles are the checks that a plausible description cannot fool.
Tests, type checkers, the build, [mutation testing](https://en.wikipedia.org/wiki/Mutation_testing) if you have it.
They operate on what the code does, not on how the description reads, and they produce the same answer every time.
This is the argument from [Verifying Code Without Reading It](../verifying-code-without-reading-it/index.md), and it applies doubly when the author is a machine: a machine wrote the claim, so only non-prose instruments can be trusted to check it.

## Use a Verifier That Does Not Share the Author's Blind Spots

If you bring in an LLM reviewer, the same problem applies in a different form: an agent reviewing an agent's work inherits that agent's blind spots.

Two models trained on overlapping corpora find the same things plausible, and the plausibility of a bug is what makes it invisible.
The measurement backs this up: in a 27-day run where a review protocol was applied to itself, reviews by [heterogeneous model pairs recorded at least one defect or requested change in 69.8% of cases, while homogeneous pairs managed it only 53.1% of the time](https://arxiv.org/abs/2606.14445), across 375 review artifacts.
The study varies the reviewing pair, not the pairing of reviewer to author, so treat it as evidence that diverse critics catch more rather than as a measurement of author-reviewer family distance.
Same-family agreement is a weak signal, and agreement between the model that wrote the code and the model that reviews it is close to no signal at all.

So the review stack should be arranged by independence, strongest first.
A check that is not an LLM, a test or a type system or a build, is the strongest verifier.
A cross-family LLM critic is second.
A same-family LLM critic is nearly worthless, and an agent grading its own pull request is not an independent check.
When I do use an LLM critic, I give it the acceptance criteria and exactly one question, phrased so the answer is either a concrete failing case or a pass, because a critic asked "is this good" answers yes confidently and teaches you nothing.
**The verifier must not share the author's perspective, and two models trained on the same corpora share enough of it.**

## Diff the Diff: Check What Should Not Have Changed

The third demand is scope evidence: proof that the pull request changed only what it claimed to change.

Agent-generated diffs show a specific failure pattern that human diffs mostly do not.
Drive-by refactors nobody asked for.
New dependencies added because they were convenient for the model.
Renames that appear to be tidying and break an interface with no obvious warning.
Code that touches files far from the stated purpose.
Each of these reads as improvement in isolation and is scope creep in context.
The checks covered the feature, not the edit added alongside it, so the pull request as a whole is unverified again.

The omissions deserve the same scan: what the pull request failed to touch that it should have.
Other call sites of the function it changed.
Documentation that now describes a system that no longer exists.
Version pins, migration files, the second place where the same logic was duplicated.
A model that found one instance of a pattern and fixed it, while three instances remain, has shipped a bug that appears to be a fix.

Size is the controllable variable here.
The claims in a two-hundred-line pull request cannot all be checked, so they will not be, and certainty decreases as size grows.
A pull request too large to verify in a single sitting should be split until it is not, and splitting is not a stylistic preference, it is the precondition for the rest of this checklist being executable.
**Every change outside the acceptance criteria is unverified work you did not ask for, and the only good number of such changes is zero.**

## Demand Provenance and Evidence, Not Confidence

The fourth demand is provenance: which model produced the change, from what task description, and what was actually run against it.

The pull request description is the author's argument, and when the author is a language model, confident prose is the default output regardless of underlying correctness.
So I want the raw evidence, not the narrative.
The model and agent that produced the diff.
The issue or prompt it worked from.
The log of the test run, not the sentence "all tests pass".
A before-and-after reproduction for a fix, a screenshot or trace for a feature.
None of this is exotic, and all of it is cheap for the sender to attach, which is precisely why its absence is informative.

Ask for it in the pull request template, the way [Rust](https://github.com/rust-lang/rust-forge/pull/1040) and SciPy now do, and the way the Linux kernel's [generated-content policy](https://kernel.org/doc/html/next/process/coding-assistants.html) makes the submitter attest, because the ecosystem does not provide this signal on its own.
A [census of 180 million repositories](https://arxiv.org/abs/2606.24429) found that bot-account detection recovers only about three percent of the commits agents actually produced, so the overwhelming majority of machine-written code arrives with no identifying mark at all.
Disclosure is not a penalty, it is routing information.
A disclosed, well-behaved agent can earn a faster review path.
An undisclosed one gets every gate, on principle.
**Confidence is the one thing a language model always has, which is why it is the one thing worth nothing as evidence.**

## Plan for Being Wrong After Merge

The last demand is the one people skip: a plan that assumes the review was wrong.

Every check before merge is evidence, not proof, and the residual risk has to go somewhere.
The cheapest place for it is a [feature flag](https://martinfowler.com/articles/feature-toggles.html), so the change can be turned off in seconds without a revert.
Then a staged rollout, watching the specific behavior the acceptance criteria named, not generic dashboards.
Then a rollback path someone has actually rehearsed.
For a bug fix, the check that guards the fix should exist before merge and keep guarding after it, because the same regression will be re-introduced by a future agent working from a stale context, and only the check will notice.

The [Merge Gate](../the-merge-gate/index.md) framing decides how much of this a change needs: a README typo and a schema migration are both pull requests, and only one of them justifies a flag and a canary.
**Merge is not the point where certainty is complete, it is the moment the cost of being wrong starts accruing.**
The detection machinery should therefore be strongest after merge.

## Keep Score on the Source

None of these demands are one-time.
They feed a calibration loop, and the loop is what turns a set of per-pull-request checks into a durable trust decision.

Track the defect escape rate per contributor and per agent: how often changes from this source reach production broken, how often they need rework, how often the tests were decoration.
After a few cycles, the data separates sources that ship verified work from sources that ship plausible work, and the gates adjust accordingly.
A source with a clean record earns a faster review path, because the checks did their job and produced evidence about the sender, not just the change.
A source with a dirty record gets stricter gates, no matter how polished its descriptions read.
This is the same risk-based tiering Cloudflare runs at scale across thousands of repositories, where [engineers override the automated gate on six tenths of one percent of changes](https://blog.cloudflare.com/ai-code-review/), and it only works because the system records what happened after every gate decision.
**Trust is not granted to a contributor, human or agent, it is measured from escape rate and revised on evidence.**

## The Certainty Pipeline

The demands form a pipeline, and each stage converts one kind of claim into one kind of evidence.

![The certainty pipeline banded by who does the checking: you define done, run the behavior, and scan scope, machines run the oracles that cannot be charmed, production is the oracle that cannot be fooled, and missing or failing evidence at any stage sends the pull request back](images/certainty-pipeline.svg)

The bands in the figure are ordered by independence on purpose.
The early stages are things only you can do, because they require knowing what you wanted.
The middle stages are machines, which never get tired and never get misled.
The late stages are production, the only oracle that cannot be fooled by a plausible diff.
Skip a stage and the certainty you lose is exactly the failure class that stage was guarding, which is why the most common mistake, skipping straight to reading the diff, fails so predictably: people judge how the code looks and never ask what the change was supposed to accomplish.

## What to Do Next

Take the next agent-written pull request that arrives in your queue and run this sequence, in order, before you read the diff.

Find the acceptance criteria and check they are precise enough that two engineers would agree on whether they are met.
If they are not, stop and get them written, because everything downstream is meaningless without them.
Run the behavior yourself against the criteria.
Interrogate the tests until you have seen one fail, or establish that none of them would.
Run your machine oracles and one cross-family critic with a single pointed question.
Scan the diff for changes no criterion asked for, and scan the codebase for changes the criteria required that never arrived.
Demand the provenance fields, and attach the evidence to the pull request so the next reviewer inherits it.
Merge behind a flag if the blast radius earns one, and watch the named behavior after.
Then record what happened in the source's trust history.

None of this is more work than the review you were already doing.
It is the same hours, spent on instruments that produce evidence instead of impressions.
**The pull request will never prove itself, because it is the claim, not the evidence.
The certainty you want is built entirely from the checks you run around it.**

## See also

- [The Acceptance Gap](../the-acceptance-gap/index.md) - the two gaps this article turns into demands: a fast iteration loop for fit, a check for correctness
- [Verifying Code Without Reading It](../verifying-code-without-reading-it/index.md) - the system that replaces reading with per-property checks, which supplies most of the oracles in the pipeline
- [Triaging Open Source Pull Requests](../triaging-open-source-pull-requests/index.md) - the queue-side companion: routing, provenance templates, and trust gates for inbound agent contributions at volume
- [You Cannot Out-Review a Machine by Hand](../you-cannot-out-review-a-machine-by-hand/index.md) - why hand-reading is the wrong certainty instrument when generation runs at machine rate and review is capped at human rate
- [The Merge Gate](../the-merge-gate/index.md) - how to decide, by blast radius, how much of this checklist a given change must pass
- [What a Senior Engineer Owes Their Reviewer](../what-a-senior-engineer-owes-their-reviewer/index.md) - the sending-side mirror: the sender's obligations that make this pipeline cheap to run

## References

- [Wikipedia, "Verification and validation"](https://en.wikipedia.org/wiki/Verification_and_validation) - the distinction between building the thing right (does it work) and building the right thing (does it match the want) that structures the whole article
- [Wikipedia, "Oracle problem"](https://en.wikipedia.org/wiki/Oracle_(software_testing)) - why plausible output cannot be judged without an independent source of truth, the root problem of reviewing machine claims
- [Wikipedia, "Mutation testing"](https://en.wikipedia.org/wiki/Mutation_testing) - the mechanical answer to "would these tests fail if the change were wrong"
- [Martin Fowler, "Feature Toggles"](https://martinfowler.com/articles/feature-toggles.html) - the mechanics of the post-merge kill switch this article treats as the cheapest place for residual risk
- [GitHub, "Agent pull requests are everywhere"](https://github.blog/ai-and-ml/generative-ai/agent-pull-requests-are-everywhere-heres-how-to-review-them/) - the scale data showing agent-authored PRs are now a normal input to review
- [Kim, "tap: A File-Based Protocol for Heterogeneous LLM Agent Collaboration"](https://arxiv.org/abs/2606.14445) - the measurement that heterogeneous reviewing pairs recorded at least one defect or requested change in 69.8% of 375 review artifacts vs 53.1% for homogeneous pairs, grounding the independence requirement
- [Detecting AI Coding Agents in Open Source (180M-repo census)](https://arxiv.org/abs/2606.24429) - the finding that bot detection recovers only ~3% of agent commits, which is why provenance must be demanded rather than inferred
- [rust-lang/rust-forge, "LLM policy" PR #1040](https://github.com/rust-lang/rust-forge/pull/1040) - a provenance disclosure policy wired into an actual project's contributing guide and PR template
- [Linux kernel, coding-assistant documentation](https://kernel.org/doc/html/next/process/coding-assistants.html) - the kernel's rule that the submitter attests to AI-generated code, an attestation-based variant of the provenance demand
- [Cloudflare, "Orchestrating AI Code Review at Scale"](https://blog.cloudflare.com/ai-code-review/) - the production tiering system with a 0.6% human override rate, the existence proof for calibrated trust records at scale