↓ Skip to main content

My AI Workflow: The Skills Are the Part That Compounds

When someone asks what my AI workflow is, the obvious answer is boring and the answer that matters is surprising. I use a couple of coding tools and a couple of models, and none of them is the thing I would miss. What I would miss is the body of skills I have been writing, collecting, and refining for the better part of a year.

This article is a snapshot of my workflow as it stands today. The tools come first because they are the visible part, but the main argument is that the tools are the part I expect to replace, and the skills are the part I expect to keep.

I run two coding surfaces and two models, and neither is the interesting part of my setup. At home I use GLM 5.2 from z.ai, and at work I use GLM 5.2 fast from Fireworks. For editing I use Cursor at work and VS Code at home, and for anything agentic, multi-step, or long-running I use OpenChamber, which is built on OpenCode. I never open OpenCode directly; OpenChamber is the surface, OpenCode is the engine underneath. The part that actually determines the quality of my work is the library of skills I have been building and sharing in tomzx/agents, and those skills run inside all four of those tools. Everything else is interchangeable.

The setup, in one paragraph
#

Cursor or VS Code is where I read, navigate, and do small edits. The moment a task turns into a sequence of steps, create an issue, reproduce a bug, review a PR, open a PR, I move it to OpenChamber, because OpenChamber is where the work can run to a natural stopping point without me holding the state. OpenChamber is built on OpenCode and is the control room on top of it: it gives me branchable sessions, worktree isolation, and a single surface to steer several sessions at once. The skills are not tied to any of these surfaces; because they follow the open Agent Skills format, the same skill runs in Cursor, VS Code, OpenCode, or OpenChamber, so I pick the surface that fits the moment and the workflow does not change. At home the editor is VS Code and the model is GLM 5.2 on z.ai, at work the editor is Cursor and the model is GLM 5.2 fast on Fireworks, and I switch between them without changing anything else about the workflow.

The model and the shell are commodities
#

A year ago I would have had a strong opinion about which model to use. Now the models are close enough that the choice is mostly about latency and price, which is why I let the environment pick: z.ai at home, Fireworks at work. The model stopped being the lever once the models got good enough to follow a well-specified workflow reliably.

The same is true of the agent shell. Cursor, VS Code, OpenCode, and Claude Code can all read a repository, edit files, and run commands, and because the skills follow an open format they run in any of them unchanged. I use several of them because they are good at different things, but I do not expect any one of them to be the source of my advantage. The shells will keep improving, the models will keep swapping in, and my workflow should not have to move when they do.

The skills are the workflow
#

Everything that matters lives in tomzx/agents, a library of composable skills written in the open Agent Skills format. A skill is a self-contained SKILL.md with step-by-step instructions for one repeatable task. A compatible agent discovers skills by name, loads them on demand, and invokes them as slash commands like /create-pr, /review-pr, or /end-day.

The backbone is the SDLC pipeline. A high-level skill like /sdlc orchestrates dozens of focused sub-skills into a full path from issue creation through requirements, specification, plan, implementation, tests, pull request, review, documentation, and learnings. Each sub-skill is small enough to read in a minute and to improve independently. When I improve the review-pr skill, every flow that calls it gets the improvement, and I never have to re-explain that step to a session again.

The versioned skill is the difference between prompting and engineering a workflow. A prompt is a one-off conversation. A skill is the same conversation, written once, versioned, reviewed, and incrementally improved. The agent performs it the same way every time, and the library gets better as the workflows mature.

Where the human time goes
#

The SDLC pipeline is long, but my time is not spread evenly across it. Almost all of my attention goes to the top: needs, requirements, and specification. The top of the pipeline is where defects fan out the widest and where fixes compound the longest, as Defects Flow Downstream lays out, and it is where the work stays irreducibly human, deciding what to build and what “done” means. A precise specification is the input every downstream skill consumes, and Software Engineering Teams in the Age of AI argues it is now the highest-leverage skill in the field.

Everything below the specification, I automate as far as I can. The plan, the implementation, the tests, the PR description, the review, the deployment, each is a skill the agent runs, and each is a place I would rather spend a token than an hour. Once judgment is encoded into the layer every change passes through, fighting over each unit of output by hand stops paying for itself.

The downstream hour that used to go into reading diffs goes somewhere else entirely: into building high-fidelity test environments that approximate the production environment as closely as I can get them. I would rather spend an hour making the test environment catch the bug than an hour reading a diff hoping to catch it myself, because the test environment runs on every future change and the diff reading runs once. Spending the hour on the environment instead of the diff is the trade Verifying Code Without Reading It and Rethinking Code Review in the Age of LLMs both argue: review is a one-time signal, verification against a production-like environment is a compounding one.

The encoding loop
#

The hardest problem with any workflow is not doing it. It is remembering to do it, every time, in the right order, without skipping the boring step.

My rule is simple. Every time I catch myself remembering to do something, that is a skill that should exist. The reminder to check for an existing PR before starting work, the instinct to link an issue to its plan, the habit of running the linter before committing, the reflex to write a learnings note after a painful bug, each of these behaviors was once tacit knowledge that lived in my head and showed up only when I was fresh. Now each one is a skill, and the agent runs it whether I am fresh or not.

The trigger is the remembering itself. If I had to remember it, the agent should not have to. Encoding it converts a forgettable step into a reliable one, and a reliable one into an improvable one, because once the step is written down I can see it, review it, and make it better instead of re-deriving it from memory every session.

What changes when the workflow is versioned
#

Once the workflow lives in a repository, three things become true at once.

First, it is reliable. The agent triages the issue, plans the work, reviews the PR, and closes the loop the same way today as it did yesterday, because the steps are written down and not reconstructed from guesswork.

Second, it is improvable. When a step is weak, I edit one file and every session that passes through it gets better. The compounding is genuine in a way that prompting never compounds, because the improvement is captured instead of lost.

Third, it is shareable. The skills are public, so anyone on my team or on the internet can run the same workflow against their own repository. The workflow became an artifact I can hand to someone else, which is the test for whether you actually have a workflow or just a habit.

What I expect to keep
#

If you asked me which parts of this setup I will still be using in two years, the answer is easy. Not Cursor, probably not OpenChamber in its current form, and almost certainly not GLM 5.2. The models and the shells are moving targets, and betting on any specific one is a bet against the field.

The skills are the part I expect to keep, but I do not expect to keep them as they are today. They will evolve, and the more interesting bet is that they will shrink. Every skill exists to encode a judgment the model cannot yet make reliably on its own: when to check for a duplicate PR, what a good acceptance criterion looks like, which steps belong in a retrospective. As the models get smarter, often smarter than me at the specific subtask, those judgments stop being mine to encode and start being the model’s to make. The skill for them becomes redundant and gets deleted.

What persists is the slower-moving part: how I decide what is worth working on, how the phases of a feature connect, where the human checkpoint belongs. The tactical skills will collapse into the model. The skills are not a fixed asset I am accumulating; they are a temporary scaffold for the gap between what the model can do today and what it will do on its own tomorrow, and a good chunk of the work is knowing which scaffold to take down next.

See also
#

References
#

  • Mathieu Larose, “My AI Coding Workflow” - the article that prompted me to write this one
  • tomzx/agents - the skills library that is the subject of this article, public and installable
  • Agent Skills format - the open skill format that makes the library portable across agents
  • OpenCode - the terminal coding agent the skills target
  • OpenChamber - the OpenCode-native control room for parallel sessions
  • Cursor - the editor I use at work for reading and small edits
  • VS Code - the editor I use at home for reading and small edits
  • z.ai - provider of GLM 5.2, my home model
  • Fireworks - provider of GLM 5.2 fast, my work model

Model Collapse: When Code Models Train on Their Own Output

Every code-generating LLM in production today was trained on code written by humans. That fact is easy to forget, because the same models are now writing most of the code. If humans stop reading, reviewing, and refining that code, the next generation of models will have to train on the output of the current generation, and the research on what happens then is not reassuring.

Models Were Trained on Human Code
#

The large code models learned from enormous corpora of human-written code: public repositories, Stack Overflow answers, patches, and documentation. None of that code was perfect. Much of it was buggy, redundant, or obsolete. But it carried something that model output cannot manufacture: the accumulated signal of millions of programmers solving actual problems under actual constraints, correcting each other in public, and converging over decades on patterns that actually work.

That signal is the source of the model’s competence. When you prompt a model and it produces a working function, it is not reasoning from first principles. It is predicting the next token from patterns absorbed out of human code, and those patterns were selected by reality, not by another model’s preferences.

The Trend Is Pulling Humans Out of the Loop
#

I have spent the last year arguing that human code review should shrink, that we should verify code instead of reading it (Verifying Code Without Reading It), and that the specification, not the diff, is where human judgment belongs (Rethinking Code Review in the Age of LLMs). I still believe that position, at the level of a single team shipping a single product.

But there is a consequence I have been slow to take seriously, because it does not show up in any one team’s metrics. It shows up in the corpus. As review becomes automated and engineers read less of what the model writes, the public record of code becomes a record of model output, lightly curated by machines. The very thing the next models need to train on, human-written and human-corrected code, is becoming a smaller and smaller fraction of what exists.

What Model Collapse Is
#

This feedback loop has a name. Researchers call it model collapse, and the clearest statement of it is Shumailov et al.’s 2024 paper, “AI models collapse when trained on recursively generated data”.

The finding is stark. When you train a model on data produced by another model, and then train the next model on the first model’s output, and repeat, the model degrades. It does not degrade in an obvious way. It loses information about the tails of the distribution first, the rare cases, the minority examples, the edge behavior, while its performance on the average case can look fine for a surprisingly long time. Model collapse is hard to notice precisely because the headline metrics keep going up while the long tail quietly disappears.

The mechanism is statistical, not mysterious. A model that fits another model’s output is fitting a smoothed, averaged version of reality. Each generation of training removes a little more of the variety, the weird inputs, the unusual-but-correct solutions, until what remains is a narrow, over-represented center.

The preprint that introduced the term, “The Curse of Recursion”, works through the mathematics, and the Wikipedia article on model collapse lays out the two stages, early and late, and the open disagreement among researchers about how severe the impact will be in practice.

Why Code Is Especially Exposed
#

General model collapse is a concern for text and images. Code is a sharper case, for three reasons that compound.

First, the useful part of code lives in the tails. The happy path is easy. The value is in error handling, concurrency, boundary conditions, security, the obscure API, and the input that should never arrive but does. These are exactly the minority examples that collapse erodes first. A model that handles the common case and breaks on the edge case is the literal definition of early model collapse, and it is also the definition of code that passes review and fails in production.

Second, code that looks correct is easy to generate and hard to filter out. A plausible-looking function that contains a subtle bug is not obviously wrong the way a garbled sentence is. If nobody reads it, and the tests do not cover the edge case, it merges into the corpus and becomes training data for the next model, carrying its bug forward as if it were a pattern worth repeating.

Third, the volume is exploding. When a model can draft a feature in minutes, the amount of model-written code committed every day is growing faster than anyone’s ability to curate it. Public repositories are filling with generated code, and that code is the raw material for the next training run. The dead internet theory, originally a half-joke about the web filling with bots, is becoming literal for code.

What Might Happen Next
#

No one knows exactly how the loop plays out, because it has not completed a full generation at scale yet. But a few hypotheses are worth stating plainly, because they are testable and they point at where to look.

The quality plateau
#

Models keep improving on benchmarks for a while, then flatten. The plateau will not look like running out of compute. It will look like running out of signal. The bottleneck moves from the size of the model to the quality of the data, and the data has stopped improving because it is no longer being written by anyone who understands it. The quality plateau follows the same pattern I described in The Shifting Bottleneck: each constraint you remove reveals the next one, one level up.

The long-tail erosion
#

While the common case improves, the edge cases get quietly worse. Security vulnerabilities, race conditions, and incorrect handling of unusual inputs become more frequent, not less, because the training distribution has thinned exactly where those lessons were learned. Benchmarks that measure average correctness will miss the long-tail erosion. Benchmarks that measure adversarial or edge-case correctness will catch it, and they will be the ones to watch.

The grounding premium
#

Code with verifiable grounding becomes valuable. What matters is not who typed it but whether it was selected by reality: it ran in production, it passed tests against inputs no one hand-picked, it survived actual failures and was corrected by them. A human-written snippet that never ran carries no signal a model could not invent. A model-generated snippet that ran under actual load and survived carries signal that no amount of recursive training can synthesize. Labels like “ran in production without rollback” or “verified against a fuzz corpus” start to mean something, the way “organic” became a label worth paying for in food. Grounding begins to matter as much as data volume, and the organizations that can prove their code carried an actual outcome get better models. Eventually the most valuable training corpus is the one you can prove was selected by reality, not merely produced by a model.

The verification-as-data loop
#

The only new ground-truth signal that does not depend on model output is execution. The most valuable training data stops being code at all, and becomes (specification, implementation, test result) triples, where the test result is measured by reality, not inferred by another model. Reinforcement learning from execution, where the reward comes from actually running the code, replaces scraping repositories as the dominant way to improve code models. A model is, at inference time, a function of its context, as I argued in The Importance of Context When Interacting with LLMs; at training time it is a function of its corpus, and the corpus is the part that is degrading.

The split
#

A gap opens between organizations that keep humans in the loop and those that do not. The fully automated shops gain short-term velocity but slowly poison their own tooling, because their internal corpus drifts toward generic code that resembles model output. The shops that keep humans reading, correcting, and rewriting code retain a source of fresh signal, and their models, or at least their use of models, stays sharper over time. The competitive advantage shifts from speed to the quality of the signal you feed back into the system, which is the same conclusion the foundation argument reaches in The Foundation Predicts the House of Cards.

The Case That Code Improves Instead
#

Everything above assumes the only training signal is “imitate the previous generation.” For text and images that assumption is roughly correct, which is why model collapse is a genuine worry there. Code is different in one decisive respect: it has an oracle.

Code runs. It passes tests or fails them, type-checks or does not, compiles or crashes, and a fuzzer can generate inputs no human would think to test. That execution result is a ground-truth signal that does not come from any model, and it is the one thing the self-consumption loop cannot synthesize. Where collapse assumes the model learns only from its own output, execution lets it learn from reality, and reality does not degrade between generations.

Execution changes the arithmetic in several ways, all of them favoring improvement.

First, reinforcement learning from execution replaces imitation as the main source of improvement. The training unit stops being “a file someone committed” and becomes a (specification, implementation, test result) triple, where the reward is measured by running the code. A model rewarded for passing tests it has never seen is being pushed toward correct behavior, not toward the average of prior outputs, and that pressure does not loop back on itself.

Second, selection effects raise the quality of the surviving corpus rather than pull it toward the mean. When generation is best-of-N, or when an adversarial verifier tries to break each candidate, the code that survives into the corpus is the code that passed actual checks. A repository built from verified survivors can be cleaner than the human corpus ever was, because GitHub and Stack Overflow were always full of bugs that nobody ran. Filtering prevents unfiltered collapse, and code is the one modality where filtering is automatic.

Third, the long tail can get better represented, not worse. Humans are lazy about edge cases. Fuzzers, property-based tests, and generated edge-case suites are not. If the training signal includes execution across millions of rare inputs, the model sees more of the tail than human-written code ever covered, and the long-tail erosion hypothesis inverts into long-tail reinforcement.

So the optimistic thesis is precise: code is the modality least susceptible to model collapse, because it alone carries an objective, automatic, infinite ground truth. The thing that makes code hard, that it has to actually work, is also the thing that protects it.

The limitation is that the oracle only checks what you ask it to check. A passing test suite proves the code does what the tests cover, not that the tests cover what matters. Let the same model write the tests and the code, or let the test suite stay shallow, and execution stops being an oracle and becomes a formality. The virtuous loop holds only while the verification is sound, separate from generation, and broader than the happy path.

What Breaks the Loop
#

Even the imitation-only loop has a way out, if real data keeps accumulating. Gerstgrasser et al. showed in “Is Model Collapse Inevitable?” that collapse is avoided when synthetic data accumulates alongside real data instead of replacing it. Their result is the weaker version of the optimistic case: as long as we keep adding fresh human code to the corpus, the model does not have to train only on its own output.

The limitation is the word “fresh.” The mitigation only works if human code keeps flowing into the training set in meaningful quantities. And that is exactly the input the current trend is diminishing. Every hour engineers spend prompting instead of writing, every diff that ships unread, every answer on Stack Overflow that goes unposted because a model answered it privately, shrinks the supply of new human signal.

So the practical question is not whether model collapse is possible. The research says it is. The question is whether we keep enough humans writing, reading, and correcting code, in public, to keep the real data flowing.

The Verdict Hinges on the Oracle
#

I have argued, and still believe, that a single team should not force a human to read every diff if its verification system is strong enough. What I no longer believe is that leaving humans out of the loop is automatically a tragedy of the commons. It is a tragedy only if the only signal the field feeds back into its models is imitation of prior output. Execution is a second signal, and for code it is the stronger one.

The two futures run on the same variable: the soundness of verification. If verification stays grounded in actual execution, adversarial, and separate from generation, the loop improves on its own, and humans reading code matters less every year. If verification is captured, shallow, or graded by the same model that wrote the code, the loop degrades, and no amount of human reading at the end will save it.

So the instruction is narrower than “keep humans writing code in public.” It is: keep the oracle trustworthy. Run the tests, fuzz the inputs, separate the verifier from the author, and treat every passing suite as a claim about coverage rather than proof of correctness. Do that, and generated code can keep getting better long after humans stop writing most of it. Fail to do it, and the degradation arrives as predicted.

See also
#

References
#


Rolling Out the Unread Review

The case for review without reading, and the system that makes it safe, can both be sound and still fail to ship. I am writing about the part that decides whether any of that work succeeds: your team.

You can be right about review, right about the system that replaces review, and still lose, because you shipped the rollout as a decree instead of as a sequence. The rollout has its own logic, and it is not the logic of the technical argument. A system people do not trust is a system that does not run, regardless of how well it verifies.

Treat The Resistance As Information
#

The easy story is that skeptics are afraid of change, and that once they see the numbers they will change their minds. That story is mostly wrong, and believing it will cost you the rollout.

The people who push back hardest on automated review are often your most careful engineers. They have seen the bugs a tired reviewer caught by instinct. They have been the person blamed when something slipped through. Their objections are not noise to filter; they show where your system is weakest. Calling their discomfort “fear” loses the argument before it starts, because they can tell you are not listening.

Three Concerns, Three Answers
#

Behind “I do not trust the bot” are usually three distinct concerns, and each needs a different answer.

The safety concern: automated review will let bad code through. You answer the safety concern with measurement, defect escape rate compared between the automated lane and the human lane.

The accountability concern: when something breaks, there is no one to point at. You answer the accountability concern with ownership that lives upstream, in the specification and in the people who own the rules.

The learning concern is the one nobody says out loud. Review is how juniors grow and how the team stays connected to its own codebase. This loss is genuine, and it is the concern that will cause the rollout to fail if you ignore it, because no defect metric addresses it.

Sequence The Exposure
#

You cannot go from “a human reads every diff” to “no human reads any diff” in one step. Decompose the automation into three things people react to very differently.

A bot that comments is basically a linter, and almost no one objects. A bot that blocks is accepted once the rules are legible and owned. A bot that approves is the change people resist most.

Ship them in that order, weeks apart, on a low-blast-radius lane first: documentation, tests, internal-only changes inside a single module. By the time you reach the approving bot, the team has weeks of evidence that the earlier layers work. You are extending trust that has been earned, not demanding it up front.

Make Every Decision Legible
#

People resist what they cannot inspect. Every automated action must state which rule fired, what it checked (the file and line, the threshold, a link to the rule definition), and who owns that rule.

“Complexity 18 over 15 in auth.py:42, rule COMPLEXITY_LIMIT, owner Priya” is something a skeptic can argue with. “Bot approved” is something a skeptic can only rage at. The first turns a black box into a reviewable system. The second turns your careful engineers into enemies.

Promote Humans To The Rules
#

This is the move that wins senior engineers over instead of displacing them. They stop reviewing code and start reviewing the gates: the security critic’s checklist, the complexity thresholds, the blast-radius classifier.

Their judgment now scales to every change instead of being spent once on a single pull request. A senior who owns the duplication rule has more leverage than a senior who reads ten diffs a day and remembers three. You are not removing them from the loop; you are promoting them to a loop where their work compounds. Give your skeptics the highest-leverage role in the new system, and most of them stop being skeptics.

Let Shadow Mode Persuade
#

The strongest argument is not yours; it is the comparison the team runs itself. Let a team keep their human review while the automated lane runs silently beside it, recording what it would have caught and what it would have flagged wrongly.

After a few weeks you have the only case that is convincing: “your reviewers missed these defects the gate caught; the gate’s false positives were these, and they are fixed.” Run it on a low-risk slice so the stakes match the trust level. And let the skeptics be the ones who present the results. Nothing converts a skeptic faster than being the person who found the data.

Measure What They Actually Care About
#

They are concerned about defects escaping, so measure defects escaping. Track defect escape rate, rollback rate, time-to-detect, and change failure rate, and compare the automated lane to the human lane on the same slice.

If the gate wins, the argument ends, and it ends in numbers rather than opinion. If the gate loses, you have a concrete gap to close, and you have earned the credibility to say so out loud. Never defend the system with “a human looked at it”; defend it with production outcomes. A human looking at it was always a proxy for those outcomes, and a weak one.

Keep The Escape Hatch, And Say So
#

Irreversible changes, trust-boundary changes, public API changes, and changes to the gating system itself still get a deliberate human read.

The message is never “you are out entirely.” It is “you are in where you add value, and out where you do not.” That framing is far easier to accept than abolition, because it is not abolition. You are reserving humans for the small, identifiable minority of changes where a careful read is still the best tool we have, and the rest of the pipeline moves at the speed the machines can sustain.

Rebuild The Learning Channel
#

The learning concern from earlier does not go away once the rollout succeeds. That loss is a genuine cost, and pretending otherwise breaks trust.

Replace the channel deliberately. Pair juniors with seniors on specifications, because that is where the judgment now lives. Rotate rule-owner duty, so learning to curate a check is itself the apprenticeship. Hold code-archaeology sessions that read well-chosen diffs as learning material, not as a gate. Move the learning off the critical path without deleting it, and name the loss plainly so people know you took it seriously.

The First Escape Is The Moment That Matters
#

When, not if, the gate lets something through, the instinct of the holdouts will be to say “see.” That is the moment the rollout is won or lost.

Run a blameless postmortem, find the gap, and add a rule. Then say the thing human review can never say: this exact defect class can never recur, because it is now checked on every change, for as long as the rule exists.

That guarantee is what removes resistance on its own schedule. The system gets strictly better over time; human review resets every morning. Each incident encoded into a rule is a permanent gain. Show people that compounding curve, and most of the holdouts change their minds, not because you argued them down, but because the system stopped being the thing they distrusted.

What to Do Next
#

Start with the commenting bot on a low-blast-radius lane, and add the blocking bot and then the approving bot weeks apart. Run shadow mode beside the human lane, and let the skeptics be the ones who present the comparison. From the first automated action, publish the rule, the threshold, the owner, and the numbers: defect escape rate, rollback rate, time-to-detect, change failure rate.

Keep the escape hatch explicit, and rebuild the learning channel deliberately. Reserve deliberate human reads for the changes with genuine blast radius. Pair juniors with seniors on specifications, and rotate rule-owner duty, so the apprenticeship continues off the critical path.

When the gate lets its first defect through, run a blameless postmortem and encode the fix as a rule. That rule is the permanent gain the rollout was for: checked on every change, for as long as the rule exists, while human review resets every morning.

See also
#


Verifying Code Without Reading It

It is a familiar observation by now that most code review happens without anyone reading the code (the case is made in You Already Review Code Without Reading It). That observation raises a question. If no human reads the diff, how do you still get correctness, maintainability, extensibility, and the rest of the things review was supposed to deliver?

The short answer is that you stop trying to read, and start trying to verify. Reading is one tool doing many jobs poorly. Verification is many tools, each doing one job well. Done right, the unread change is checked more thoroughly than the read one ever was.

Decompose What “Reading” Was Checking
#

When a reviewer reads code, they are not doing one thing. They are running a dozen checks in their head at once, and doing each of them inconsistently. Make those checks explicit, and you can see that they are separate.

Correctness: does the code do what the task actually asked for? Tested: are the behaviors that matter covered by tests that would fail if the code were wrong? Maintainable: is the complexity bounded, the duplication low, the naming consistent, the dead code absent? Extensible: can the next change plug in without a rewrite, or has this change coupled two things that should stay independent? Secure: does it cross a trust boundary, leak a secret, or open an injection path? Reversible: if this change is wrong, can we undo it in minutes, or does it mutate data we cannot get back?

A tired reviewer scans for all of these at once and catches each of them sometimes. The approach is to give each property its own check, run on every change, without getting tired.

Match Each Property To Its Best Checker
#

Every property above has a checker that is cheaper and more reliable than a human reading a diff.

Correctness is checked by tests, and by an LLM judge that compares the diff against the acceptance criteria from the issue and asks: is there a case where the criteria hold but this code fails? Tested is checked by coverage gates and by mutation testing, which mutates the code and fails the build if the tests still pass, because that means the tests were not testing anything. Maintainable is checked by complexity limits, duplication detectors, dead-code scans, and linters, all deterministic, all running every push. Extensible is checked by dependency-direction rules, boundary tests, and an LLM critic that asks where the next feature would plug in and whether this change has made that harder. Secure is checked by static analysis, secret scanning, and an adversarial LLM pass that tries to find the input the author did not think of. Reversible is checked by a blast-radius classifier: did this touch the schema, the public API, an external commitment, or a trust boundary?

None of these requires a human to read the code. Together they cover everything a reader was supposed to cover, and they cover it on every change, not on the changes a reviewer happened to be alert for.

The LLM As Critic, Not Reader
#

The LLM’s role in this system is not to read the code for you. A summary of the diff is not reading, however it is presented. The LLM’s role is to act as a critic against a single, stated property.

Give it the diff, the specification, and one question. Does this change introduce a coupling that violates the intended dependency direction? Is there an input that bypasses the authorization check the spec requires? Does this function do something the acceptance criteria never asked for? One question per critic, phrased so the answer is either a concrete failing case or a pass.

This is verification, not reading. It is adversarial by construction: the critic is rewarded for finding a problem, not for approving. And because each critic has one job, you can run many in parallel, each a different prompt, each looking for a different class of failure. A human reader tries to notice everything and notices some of it. A battery of critics tries to notice one thing each and notices it every time.

Verification Is Not Validation
#

There is a distinction worth keeping, because the system fails if you confuse the two.

Verification asks: did we build the thing right, does the code meet its specification? Validation asks: did we build the right thing, does the specification solve the actual problem?

Reading a diff does verification badly and validation not at all. No amount of staring at code tells you whether the feature should exist. That judgment has to be made somewhere, and the productive place for it is upstream, in the specification, and downstream, in production.

So split the work. Put verification in the automated gates, where machines check the code against a precise spec on every change. Put validation upstream, where humans decide which problems are worth solving and write the acceptance criteria that encode that judgment. Put a third layer of validation downstream, in canaries, monitoring, and fast rollback, where production behavior is the ground truth no diff review can match.

The human does not read the code. The human writes what the code must satisfy, and then watches what the code does in production.

The Circularity Trap
#

There is one failure mode that will quietly ruin this system if you let it go unchecked. Do not let the same model, or the same prompt, both write the code and approve it.

When the same model acts as both author and verifier, the verifier inherits the author’s blind spots. A model that wrote a subtle bug will, asked to review its own work, tend to confirm that the work is fine. This is not malice; it is the same statistical process producing both answers.

The defense is structural. Use a different model for verification than for generation, or at minimum a different agent with a different prompt and different access to the specification. Write the verification criteria before the code exists, so they describe the intended behavior rather than rationalizing whatever was built. And run an adversarial pass whose only goal is to break the change, with no incentive to approve.

Separation of concerns is not a nicety here; it is the whole reason the system can be trusted.

What The Human Does Instead
#

If the human is not reading diffs, what are they doing?

They write the specification and its acceptance criteria, because that is where validation belongs and where the LLM’s verification is anchored. They review the rules the gates enforce, not the code the gates pass, because a bad rule approved once produces bad approvals forever, while a bad line of code is caught by a good rule. They respond to gate failures, which is where their judgment adds value, instead of using it on changes that passed cleanly. And they read code only on the small, flagged minority of changes that carry significant blast radius, where a deliberate read is still the best tool we have.

The new division of labor is more work at the top of the pipeline and less at the bottom, which is the right inversion. You are trading a low-leverage activity that scaled poorly (reading every diff) for a high-leverage one that compounds (writing the rules and specs that check every diff).

How You Know It Is Working
#

The objection that will come is simple: how do you know the unread code is good enough?

You do not answer it by pointing at how much code you read. You answer it by measuring the outcomes reading was supposed to produce.

Track defect escape rate, the bugs that reach production per change. Track rollback rate, how often a merged change has to be undone. Track time-to-detect, how quickly a regression surfaces after it lands. Track change failure rate, the fraction of deployments that cause an outage.

Run the new system on a slice of changes and compare these numbers to your old, human-read pipeline. If the gates have a lower defect escape rate and a faster cycle time than your reviewers did, the unread code is provably better than the read code was. If they do not, you have a concrete gap to close, by tightening a rule or adding a critic, not by urging reviewers to read harder.

The standard is not “a human looked at it.” The standard is “the code behaves well in production, measurably, on every change.” A human looking at it was always a proxy for that standard, and a weak one. Replace the proxy with the measurement it represented.

See also
#


You Already Review Code Without Reading It

Most code review does not involve reading code. The decision to approve is made before the diff opens. We keep the practice, and quietly drop the part that is supposed to justify it.

The same gap between claim and verification exists on the author’s side of the pull request. An LLM cannot vouch for its own output, because accepting a solution as done needs information the model does not have (The Acceptance Gap). The reviewer has the same problem in reverse: vouching for code that was never read.

What I Mean By Reading
#

There is a difference between opening a pull request and reading one. Reading means reconstructing what the code does, tracing each branch, checking every assumption against the rest of the system, and forming an independent opinion about whether it is correct. That takes time, sustained focus, and more attention than a fifteen-item review queue allows.

Opening is something else. You open the PR, you notice the file list, the green check from CI, the test count, and the description the author wrote, and within a few seconds you form a feeling: this is fine, or this is not. The code itself comes after all of that. Most of the time, you never reach it.

This is not a failure of discipline. It is how the tool presents work to you, and how a busy engineer responds to that presentation. The diff is the last tab you look at, and often the one you look at least.

The Signals That Actually Decide Approval
#

If you watch what an approver actually responds to, the code is rarely the first thing.

CI passed. That matters more than any line you might have read. The tests are green, the build is green, the linter is quiet, and those signals have already provided the safety before you started.

Then the author. A pull request from someone you trust gets less scrutiny than one from a new hire. You will deny this in a meeting and do it anyway in practice. Trust is a signal, and a useful one, but it is not “reading the code”.

Then the size of the change. A two-line diff to a README merges in seconds. A four-hundred-line refactor to the auth module stalls. You are reacting to blast radius, not to semantics, and you are right to, but again, that is not reading.

Then the description and the linked issue. These tell you what the author intended. For a lot of pull requests, the intent is all you ever verify, because the code is too long to verify against it in the time you have.

The code is the excuse. The signals are the decision.

Why We Pretend To Read
#

If the code is not really what we are checking, why the process around it?

Because reading code is the thing code review is supposed to be for. Admit that you are not doing it, and you admit that the central activity of the gate is not happening. That is an uncomfortable admission, so it does not get made. Instead we keep the form, the comments, the approving review, and let the substance go unchecked.

There is also a blame function. When something breaks, we want to point at a name on the pull request and say “they signed off”. The signature has to mean “I read this and vouched for it”, or it cannot carry that blame. So we maintain the fiction that the signature means what it says, even though everyone, on both sides, knows it often does not.

This is a performance of accountability: the approve click certifies that a person was present, but rarely certifies that the code was read. The signature allocates blame after the fact. It does not prevent harm before it.

The Self-Test
#

Here is a test you can run on yourself. Approve your next five pull requests the way you normally would. The next day, try to recall a single line of code from each one.

If you cannot, be precise about what that means. It means you approved metadata. You were responding to CI, to the author, to the description, to the size of the diff, to your sense of the person. Those are signals, and some of them are good signals. But none of them are “I read this code and understood it”.

Run the same test on your team. Ask a recent approver to explain, from memory, one function that changed in the pull request they approved. The silence you get back is the most truthful data you will collect about how code review actually works in your organization.

I am not mocking reviewers here. I have failed this test more often than I have passed it. The point is that the gap between what review is supposed to be and what it is, is large, and we never measure it because measuring it would force a conclusion we have already decided not to reach.

Two Paths
#

Once you accept that most review does not involve reading, you have two defensible directions, and only two.

The first is to actually read the code, on the changes that warrant it. Not every change. The ones with large blast radius: irreversible changes, trust-boundary changes, changes to the gating system itself. On those, slow down, read deliberately, trace the logic, and form an independent opinion. This is high-leverage work, and it is rare, and it deserves the time it takes.

The second is to drop the pretense, and let the changes you were never going to read merge on the strength of the signals you were actually using. Green CI, a passing test suite, a trusted author, a small reversible diff. These already decided the outcome. Let them decide it without a human in the middle adding latency and taking credit.

What is not defensible is the current default. The default is the second path, with the performance of the first added on top. You approve without reading, but you still require a human to approve without reading, and you call the combination a quality gate.

When The Code Was Written By A Machine
#

This was already true when humans wrote the code. It is more pronounced now that machines do.

When an LLM generates the diff, reading it line by line is even less useful, for reasons explored in Rethinking Code Review in the Age of LLMs. The reviewer becomes the only mind in the loop, reconstructing intent from output, and that reconstruction is harder and slower than reviewing a human who can at least be asked what they meant.

So the metadata review does not get more rigorous by being relabeled a code review. It gets more exhausting, and less effective, for the same automatic approval at the end.

If you were not reading human-written code, you are not going to start reading machine-written code. You are going to keep approving signals, with a worse feeling about it. The productive move is to put the effort where it was always worth more: upstream, in the specification, the tests, and the gates.

What to Do Next
#

You do not need permission to start. Pick the low-blast-radius path and let it merge on green. Then be plain with your team about what the rest of the approvals were already doing.

When a change is large, irreversible, or crosses a trust boundary, read it in full. Budget the time. Treat it as the exception it is, not as the default presented as diligence.

And measure the gap. Ask your approvers what they remember from the pull requests they approved. The number will embarrass you, and that embarrassment is the beginning of a review process that is actually worth its cost.

See also
#


Issues Are Free Now: Send the Implementation, Not the Idea

For most of software’s history, filing an issue took actual work. You had to reproduce the bug, narrow it down, write a clear description, and decide whether the request was worth the maintainer’s time. That filter is gone. An AI agent can read a README, a changelog, or spend ten minutes with the tool, and produce a polished, well-formatted issue in seconds, and it can do it a hundred times before lunch. The cost of creating an issue fell to almost nothing, the cost of clearing one did not, and every issue tracker on the internet is now overflowing.

If you are the one filing those issues, the result is unpleasant but clear. The issue you just opened is no longer a contribution in any practical sense. It is a request for someone else’s work, and it is sitting in a queue next to nine hundred other requests for someone else’s work, most of them produced just as cheaply as yours. The only thing that still changes the outcome is the implementation attached to the request, because the implementation is the one input whose cost has not fallen.

The Issue Tracker Was Always a Wish List
#

It is worth being clear about what an issue tracker actually was, even before AI. A well-written bug report was a genuine gift, because it gave the maintainer a reproduction and a starting point. A feature request was almost always a wish. Someone else should do this. Someone else should want this. Someone else should spend their Saturday on the thing I would like to have.

Maintainers cleared the gifts and let the wishes pile up, and that was fine, because the wishes cost the filer enough effort to write that only the motivated ones made it through. The friction was the filter. The issue tracker was tolerable only because filing an issue was mildly annoying, and the mildly annoyed filer self-selected for “I actually care.”

Remove the friction and the wish list becomes the whole tracker. This is already true for most trackers.

The Cost Imbalance Is the Whole Problem
#

The arithmetic that overwhelms the tracker is simple. It costs a filer roughly zero seconds and zero cents to open an issue with an AI agent. It costs a maintainer anywhere from five minutes to an afternoon to triage it: read the request, check whether it duplicates something, decide whether the project wants it, find the relevant code, and either close it, schedule it, or do it. One side of that exchange is free; the other side is unpaid, and the unpaid side is the one you are asking to work.

Multiply that imbalance by a few hundred issues a week and you get the current state of every popular open source repository on the planet. The triage queue grows faster than any single human can read it, the maintainer starts ignoring the tab, the contributor waits six months for a reply, and the project slowly gains a reputation for being unresponsive. Nobody is at fault. The system is just balanced so that the cheap side produces faster than the expensive side can consume, and the buffer between them is a person.

This cost imbalance is the same dynamic I described for pull requests in The Pull Request Queue Outgrew You: the cost to create a change fell below the cost to review it, and the maintainer became the bottleneck in their own queue. What is true for code changes is now true for ideas, and ideas are even cheaper to produce.

An Implementation Rewrites the Ask
#

Here is the move that fixes the imbalance, and it is the only move that fixes it. Stop sending the description. Send the code.

A pull request with a working implementation changes the maintainer’s job from “do this work” to “evaluate this work,” and those are not the same task. Doing the work is open-ended, expensive, and unpaid. Evaluating the work is limited, fast, and something a maintainer can actually fit into a Saturday morning. The difference between “you should add a dark mode” and “here is a dark mode, here are the tests, here is the screenshot, merge or reject” is the difference between a request for a favor and an offer to work together. The implementation is what converts your issue from a cost imposed on the maintainer into an option extended to them.

This is also why the maintainer is so much more likely to act on a pull request than on an issue. A PR gives them a simple choice they can make in minutes. An issue gives them an open-ended commitment that will eat an afternoon if they engage with it at all. If you wonder why your feature request has sat untouched for two years while a stranger’s twenty-line PR landed in a week, that is the entire explanation. The PR was cheap to say yes to. Your issue was expensive to say yes to, and expensive to say no to, so it got neither, which is what happens to expensive things in a full queue.

The Cost of Implementing Fell Too
#

The obvious pushback is that not everyone can write the implementation, and that used to be true. It is not true anymore.

The same AI agent that filed the issue can write the pull request. You describe what you want, the agent reads the codebase, produces a diff, writes the tests, and opens the PR. The skill floor for “I can send a working implementation” dropped from senior engineer to motivated user with a coding agent and an afternoon. The reason to keep sending descriptions instead of implementations is no longer that you cannot produce the implementation. It is that you have not updated your habits to match the new cost structure.

This is the same shift The Shifting Bottleneck keeps arriving at. Producing code is no longer the constraint. Deciding what to produce, and accepting what got produced, is the constraint. In an issue tracker, that means the scarce resource is no longer “someone willing to file the issue” or even “someone willing to write the code.” The scarce resource is the maintainer’s attention, and the only way to earn a share of it is to bring the work to a state where saying yes costs them almost nothing.

But I Am Not Sure My Implementation Is Good
#

This is the second pushback, and it is more serious than the first. You worry that sending an AI-generated PR is dumping low-quality work on an already overloaded maintainer, and that is a genuine risk. The answer is not to hold back the PR. The answer is to do the work that makes the PR cheap to evaluate.

Run the tests. Write a new one for the behavior you added. Reproduce the bug you claim to fix, and show the test going red before your patch and green after. Keep the change small and focused on one thing. Link the issue you are closing. Write a PR description that lets the maintainer evaluate the change in thirty seconds. Every one of these is an attempt to lower the cost of saying yes, and lowering the cost of saying yes is the entire point.

A small, tested, well-described PR is a gift. A large, untested, AI-generated diff with a one-line description is just a different kind of issue, and it will be treated like one. The discipline is not “send code instead of an issue.” The discipline is “send code that is cheaper to merge than to discuss, and if you cannot produce that, the issue tracker will not save you either.”

What the Maintainer Actually Wants
#

It helps to look at the exchange from the maintainer’s perspective for a moment, because the request they are implicitly making is not unreasonable. They want to spend their scarce attention on decisions only they can make: the architecture, the direction, the boundary between what the project is and is not. They do not want to spend it turning a feature description into code, because that is the part the LLM can do now. The highest-value contribution you can make is one that arrives with every reversible decision already made and every test already green, so that the maintainer’s job is reduced to a judgment call they were going to have to make anyway.

This is the open source version of the argument in Rethinking Code Review in the Age of LLMs. When the implementation is cheap to produce, the value concentrates upstream, at the point where someone decides what the implementation should be. A good contributor in this era is not someone who writes clean code. A good contributor is someone who arrives with a concrete proposal, already implemented, already tested, already de-risked, leaving the maintainer nothing to do but approve, redirect, or reject.

What About Bugs You Cannot Reproduce
#

There is a genuine exception, and it should be named so the argument does not go too far. Some issues are genuine bug reports where the value is in the reproduction, not in the fix. A crash that happens on a specific kernel version, a regression that only shows up under load, a data corruption bug that requires a particular sequence of inputs. These are gifts, when written carefully, because the maintainer could not have produced the reproduction themselves.

But notice what makes them gifts. They carry information the maintainer did not have and could not get cheaply. The test for whether your issue is a contribution or noise is exactly this: does it contain information the maintainer could not have produced on their own in fifteen minutes with an AI agent? If yes, file it. If no, the issue is a request for someone else to do work you could have done yourself, and the queue is full of those already.

The same test separates a useful feature request from noise. “Add dark mode” is noise. “Add dark mode, here is the design, here is why it does not conflict with the theming system, here is the PR, here are the screenshots, here are the tests” is a contribution. The information that converts noise into contribution is exactly the information that comes from having tried to build the thing.

What to Do Next
#

If you file issues against projects you depend on, change one habit. Before you open the issue, try to open the PR instead. Point an agent at the codebase, describe the change you want, and let it produce a draft. Spend the time you would have spent polishing the issue description on getting the PR to a state where merging it is the obvious move: tests passing, scope small, description clear. File the issue only if the PR attempt really failed, and when you do, include what you learned from the attempt, because that is the information the maintainer actually needs.

If you maintain a project, change the default you invite. Rewrite your contributing guide to say, plainly, that feature requests without an accompanying PR will be closed, and that bug reports without a reproduction will be closed faster. Be clear that this is not hostility. It is the only way the queue stays manageable. Make the pull request template the default path, and the issue template the exception for the narrow set of things only an issue can carry. Setting a higher standard for contributions is, in this era, an act of respect for the contributors who are willing to meet it, because they are the ones whose work will otherwise be buried under the noise.

And for everyone, learn the new cost structure. Ideas are free. Descriptions are free. Issues are free. Implementations are the only thing left that earns maintainer attention, and the reason is not that maintainers are picky. It is that implementations are the one input whose cost has not fallen to zero, which makes them the one input that still signals you meant it. If you want a maintainer to take your request seriously, prove it the only way that still costs you something: by sending the code.

See also
#

  • The Pull Request Queue Outgrew You - the companion argument from the maintainer’s side: the cost to create a change fell below the cost to review it, and triage has to replace review
  • The Shifting Bottleneck - the upstream pattern: producing code is no longer the constraint, deciding and accepting are, which relocates where contribution lives
  • Rethinking Code Review in the Age of LLMs - why value concentrates upstream at the decision point when implementation becomes cheap
  • The Acceptance Gap - the maintainer’s remaining job is acceptance, and an implementation is what lets them do that job in minutes instead of afternoons
  • Feature Parity Is Not a Moat - the parallel collapse of copy-time, which made features cheap to clone the same way AI made issues cheap to file

Who Resolves the Merge Conflict? Why the Bot and the Author Are Not Interchangeable

A pull request falls behind main and conflicts. Who resolves the conflict, the bot or the author? The answer depends on whether the conflict is mechanical or semantic, and treating the two as the same job is what makes every flat policy, “the bot rebases everything” or “the author handles it,” wrong.

The clean rebase is settled and out of scope; automate it on every push to main. The live question is the conflict, and the conflict is not one thing. Resolving a conflict means deciding what the code should now say, and that decision is a claim about what the author meant. Someone has to make it, and the interesting question is who can support it with intent they actually hold. The choice depends entirely on whether the conflict has a unique correct answer an oracle can check, or whether its correctness lives only in the author’s head.

Why the Question Got Loud
#

For most of git’s history the conflict question was quiet. A contributor resolved their own conflicts, or the maintainer did, and the round trip was short because the people involved shared a mental model of the code. Two things broke that equilibrium.

The first is auto-merge. Once low-risk changes land on green without a human clicking merge, a PR that cannot merge cleanly becomes the thing that stalls the whole lane. A human can absorb a conflicted PR by glancing and clicking; an automated merge pipeline cannot. Auto-merge makes conflict resolution a prerequisite of the lane, and that turns “who resolves” from a courtesy into a structural question.

The second is the model-authored pull request. When the author is a model, the conflict round trip behaves differently at both ends. The bot never sleeps, so a resolve request adds no latency the way it does for a human. But the bot also has no private knowledge of what the code was meant to do, so its resolution is a guess about intent presented as a fix. This is the plausibility problem, and it is worst exactly where the conflict is hardest.

So the modern repository has a lane that wants every PR mergeable on green, and a growing share of PRs whose intent lives only in the code a model produced, with no description, linked issue, or spec that states it independently. When the intent is encoded somewhere, a conflict has an oracle to check a resolution against. When it is not, there is nothing to verify the resolution by, and the absence of a way to verify the resolution, not the absence of an author to interrogate, is the actual problem. The old default, the contributor resolves, is too slow for the merge lane, and the bot is too confident to stand in as the oracle, so the live question is not who authored the PR but whether the intent exists outside someone’s head.

The Conflict Has a Gradient
#

Conflicts are not all the same, and the gradient is what lets you route them accurately.

At one end is the mechanical conflict: an import added in both branches, a trailing comma, two functions inserted at adjacent positions, a formatting drift the formatter can settle. Resolve it and there is exactly one answer any reasonable developer would accept, and a test suite that passes before will pass after. At the other end is the semantic conflict: two people changed the same logic for different reasons, and the correct resolution depends on which reason was right. The mechanical conflict has a unique correct answer that a test can falsify; the semantic conflict has no correct answer without the intent that motivated the change.

The mechanical conflict is therefore still logistics, just logistics that needs an oracle. Resolve it, run the suite, and if the suite passes the resolution is, by construction, the one the project already trusted. Here the test acts as the independent oracle the model cannot be, the same role it plays in closing the bug gap when a fix claims to be done. A conflict whose wrongness a test can catch is a conflict the bot may resolve, because the test, not the model, approves it.

The semantic conflict is a different kind of problem. Nothing external is left to disagree with the model’s plausible resolution, because the definition of “correct” is the intent, and the intent is precisely what the model does not have. A bot that resolves a semantic conflict is making an intent claim it cannot source, and the harder the disagreement, the more likely the claim is a confident fabrication.

Where the Intent Lives
#

The semantic conflict splits once more, and it splits on a cleaner axis than who wrote the PR: whether the intent is encoded anywhere a resolver can read it.

Intent can live in two places. It can be encoded, written down in the PR description, a linked issue, an acceptance criterion, a failing test, anything a human authored to state what the change was supposed to do. Or it can be tacit, held in someone’s head and never written down, surfacing only if you ask. A conflict is safe for the bot to resolve when the intent is encoded and the encoded source covers the disputed region, because then the resolution has an oracle. It is not safe when the intent is tacit, because then nothing external can falsify the bot’s resolution, and the resolution will look right whether or not it is.

Author type matters, but only as a proxy for where the intent lives, and the proxy is worth stating because it inverts the naive assumption.

When the PR is human-authored, the human’s own description is an independent statement of intent. If the description covers the conflict, the bot may resolve against it, and the description, not the model, is the oracle. If the description is thin or silent on the disputed region, the intent is effectively tacit, and the only defensible move is to ask the human who holds it.

When the PR is bot-authored, the trap is that the bot’s description is not an independent oracle. It was written by the same model that wrote the code, or a sibling of it, so checking the resolution against the description is checking the model against its own narration, which provides no independent verification. The only trustworthy intent source for a bot-authored PR is something a human wrote upstream: the issue, the spec, the prompt, an acceptance test. When that upstream source exists and covers the conflict, the bot may re-derive against it, and the re-derivation is checkable. When it does not exist, the intent was never encoded by anyone, and the bot’s resolution is a guess about intent no one ever wrote down.

The distinction above is the refinement the flat “maintainer picks a default” framing misses. A single repository-wide default is too coarse, because the right default is a function of whether the PR carries an encoded, independent intent source, and that varies PR by PR, not repository by repository. The PR that links a human-authored issue with acceptance criteria is safe for bot resolution regardless of who wrote the code; the PR that arrives as code and a self-description is not safe regardless of who wrote the code.

The Label Is the Right Mechanism, the Default Is the Question
#

Given the split, the mechanism worth reaching for is the one the question already points at: a label on the PR that decides who resolves a conflict. Labels are already how a modern triage layer expresses every other routing decision, and conflict resolution is no different in kind. The label should express the thing that actually varies, which is not “is this PR rebased” but “who is allowed to resolve a semantic conflict on it.”

For a mechanical conflict there is no decision worth encoding: the bot resolves, the test gates, and a conflict: auto-resolved note is enough audit trail. The label is worth having at the semantic tier. Something like conflict: author-resolves versus conflict: bot-may-resolve, defaulted per repository by the maintainers and overridable per PR, is the right form, and it is the form the question proposes.

The interesting work is in the default, and the sound default routes on intent availability, not on author type. A PR that links a human-authored issue or spec, with acceptance criteria that cover the change, defaults to bot-may-resolve, because the resolution has an oracle. A PR that arrives as code alone, with no independent statement of intent, defaults to author-resolves, because nothing can falsify the bot’s resolution. The maintainer’s actual choice is not “auto or manual.” It is “what do we assume about a PR whose intent source we cannot verify.” The conservative answer, treat the PR as unencoded and ask the author, costs a little latency where the bot could have handled it. The permissive answer, resolve and trust the model, costs intent claims that are silently wrong. Between losing a little speed and shipping a confidently wrong merge, the speed is the cheaper loss, so the default for a PR with no verifiable intent source should be author-resolves.

This makes the highest-leverage change a documentation change, not a tooling one. Require, in the pull request template, a linked issue or a short intent statement that covers the change, and ask whether the PR was model-assisted. The linked issue turns an unencoded PR into an encoded one; the model-assisted flag tells you whether the PR’s own description can serve as an oracle or whether you need the upstream source. Both shrink the “no verifiable intent” bucket, and shrinking that bucket is what lets the auto-merge lane actually run.

What to Do Next
#

First, surface the conflict instead of hiding it. Add eps1lon/actions-label-merge-conflict so any PR that falls behind main gets a merge-conflict label the moment it conflicts, and loses it the moment it merges again. Your auto-merge lane can now filter -label:merge-conflict and only act on work that is actually ready. The author gets an automated nudge that a genuine decision is needed, without you being the one to say it.

Second, classify the conflict before you route it. Mechanical conflicts (imports, formatting, adjacent non-overlapping edits) go to the bot behind the test gate, and the test is the oracle that makes the resolution trustworthy. If the suite fails, the resolution is wrong by definition and the bot escalates rather than ships.

Third, route the semantic conflict by whether an independent intent source exists. If the PR links a human-authored issue or spec that covers the disputed region, let the bot resolve and check the result against that source, with an encoded test that must pass on the resolved tree. If the PR carries no such source, post the conflict as a comment and drop the PR to author-resolves until someone provides the intent or resolves it by hand.

Fourth, choose the conservative default for the case where you cannot tell. When a PR arrives with no linked issue and no intent statement, default to author-resolves, because the cost of a confidently wrong merge is higher than the cost of a round trip, and the round trip at least asks the question the silent resolution skips.

Fifth, make the routing signal cheap to produce. Require a linked issue or a short intent statement in the pull request template, and ask whether the PR was model-assisted. The link is what turns an unencoded PR into an encoded one, and the model-assisted flag is what tells you whether the PR’s own description can count as the oracle.

Do not write a policy that says “the bot resolves everything” or “the author resolves everything.” Both are the original mistake in policy form. Gate the mechanical conflict behind a test, defer the semantic conflict to an independent intent source when one exists, and defer to the author when one does not.

See also
#

  • Triaging Open Source Pull Requests - the upstream layer this refines: labeling merge conflicts and routing by risk is the triage move, and conflict-tier routing is what makes the auto-merge lane inside that layer safe
  • The Merge Gate - the case for gating on the properties of a change rather than on the existence of a PR, which is the same principle applied here to “is this a mechanical conflict or a semantic one”
  • The Acceptance Gap: Why an LLM Solution Is Not a Shipped Solution - why a conflict resolution is a mini-acceptance gap: the mechanical tier wants a check (verification), and the semantic tier wants the author’s reaction (validation)
  • Rethinking Code Review in the Age of LLMs - the plausibility problem in full, the reason a bot resolving a semantic conflict is making a claim about intent it cannot source

Feature Parity Is Not a Moat: Compete on What Does Not Clone

For most of software’s history, shipping a feature gave you a window of time. A competitor had to understand the need, design the thing, build it, test it, and ship it. That took weeks or months, during which you could compound users, data, and trust. That window has collapsed to days, sometimes hours, and the uncomfortable implication is that the feature was never the moat. The slowness of copying it was. Once the copying is nearly free, every hour spent perfecting a feature is an hour spent on the one layer the market just commoditized, and the advantage has moved somewhere developers were trained to undervalue.

The Moat Was Always Somewhere Else
#

An economic moat is the structural thing that lets a business keep its profits against competitors. The standard list of what counts as a moat is worth noting: switching costs, network effects, cost advantages, intangible assets like brand and patents, and efficient scale. A feature is not on that list. A feature is something you build in the hope that a moat forms around it, through usage, data, or habit, before anyone else arrives.

For a long time the list and the feature felt like the same thing, because copying was slow. You shipped the recommendation engine, and by the time a rival reverse-engineered it, your users had generated enough behavior to make yours better. You shipped the dashboard, and by the time it was cloned, your customers had wired it into their weekly review and would not move. The feature was the seed, and the copy-time was the growing season, and the moat was the harvest. The copy-time is gone, which means the growing season is gone, which means a feature shipped with no other plan now arrives at the market with no moat attached.

What Actually Clones, and What Does Not
#

It is worth being clear about the boundary, because the claim is not that everything is copyable.

What clones easily is the surface. The visible feature, the UI flow, the API surface, the integration that calls a public endpoint, the report that joins two tables. A capable engineer with an LLM can reproduce any of these from a screenshot or a description in an afternoon, and a competitor can ship a credible copy in a week. The commoditization of the application layer is under way, and it is fast.

What does not clone is everything that had to be true for the feature to be valuable in the first place. The two years of usage data that makes the recommendation correct. The thousand integrations already in your customers’ workflows. The track record of reliability that makes a procurement officer sign. The deep, specific knowledge of a narrow industry that lets you pick the right feature to build next. The competitor can copy the feature and still be left holding an empty shell, because the value was never in the shell. It was in the data, the relationships, and the context the shell was built to carry.

Where the Moat Moves
#

When the feature stops being the moat, the advantage moves to the layers that surround it, and almost all of them are things that compound through time rather than through code. Five of them matter for a developer deciding where to spend next quarter.

The learning loop, not the feature
#

The competitor copies your feature, but they do not copy what you learned by shipping it. Shipping a feature to actual users tells you which half of it was wrong, which edge case matters, and which adjacent problem is now the actual problem. That knowledge is yours alone until competitors run the same experiment themselves, and they will now run it by copying your result rather than discovering their own. The moat is the build-measure-learn cycle compressed to days, run against actual users, faster than anyone who is copying can run it. A feature is no longer a deliverable; it is a probe, and the team that runs more probes per month learns more about the market than the team that ships fewer, more perfect features.

Proprietary data and feedback
#

A clone of your feature without your data is a weaker feature. The recommendation is bland, the search results are generic, the anomaly detection fires on the wrong things, because the model behind the clone has nothing the field has not already seen. Every interaction with an actual user adds signal to your data that the clone cannot synthesize, and that data advantage compounds quietly until the gap is not closeable by better code. This compounding is the mechanism behind network effects, and it is one of the few things that genuinely does not clone.

Workflow depth and switching costs
#

A feature you can use in a minute is a feature you can leave in a minute. A feature that is wired into six other systems, that holds three years of configuration, that your team’s runbook depends on, is a feature that survives a cheaper clone because the cost of moving exceeds the cost of staying. These switching costs are a legitimate moat when they are a side effect of genuine usefulness, and they are built by going deep into the customer’s workflow rather than wide across a feature checklist. Shallow integrations clone in a day; deep ones do not, and depth is a function of how much of the customer’s actual job you have absorbed.

Distribution, brand, and trust
#

Engineers historically treated these as someone else’s problem, which was always a mistake and is now an expensive one. The feature that arrives through a channel the user already trusts, from a name they recognize, beats the marginally better clone that arrives cold. Trust is built slowly, through reliability and through presence, and it is one of the slowest things in the product to reproduce. A developer who invests zero attention here has chosen to compete purely on the layer that clones fastest.

Judgment about what to build
#

Judgment is the layer the rest of this blog keeps arriving at, because the bottleneck keeps moving toward it. When everyone can build anything, building things stops being the differentiator, and deciding which thing to build, for whom, and in what form is what matters. That judgment is a personal and team-level moat, it compounds with practice, and it does not clone because the competitor cannot copy the years of being close to the problem that produced it.

The Feature Race Is a Trap
#

The failure mode to avoid is the feature-parity race, and it is the default response if you do not see the shift.

You ship a feature, the competitor copies it, you ship another to pull ahead, they copy that, and the cycle repeats. Both teams are running as fast as they can and neither is building a moat, because every feature is neutralized within a week of landing. You are exhausting yourselves producing advantages that confer no advantage, and the only thing accumulating is technical debt from features nobody had time to integrate properly.

The feature-parity race is the same trap as first-mover advantage misunderstood. Being first was valuable only because first movers got the growing season, the time for data and habit to form before the copiers arrived. When the growing season disappears, being first stops being the point. Being the one who learns fastest from being there is the point.

The deeper error is treating the feature list as the measure of progress. It feels like progress, because the features keep shipping and the release notes keep growing. But a competitor watching your changelog now has your roadmap for free, and a model lets them execute it at your speed. A changelog is no longer a strategic asset; in a clonable world it is a plan you give your competitors every week.

What to Do Next
#

A few concrete moves follow from treating features as probes and moats as the target.

Audit where your engineering effort actually lands. If the large majority sits on building and polishing features, you are over-invested in the layer that clones and under-invested in the layers that do not. Reallocate deliberately, not by abandoning features, which you still have to ship, but by capping the time each feature gets and routing the surplus into data, integration depth, and the learning loop.

Instrument every feature as an experiment from day one. If you ship a feature and cannot tell within a week whether it moved the metric you built it for, you have shipped a deliverable, not a probe. You have also given up the one advantage the copy does not get. The measurement is the moat, because it is what turns the feature into learning the competitor does not inherit.

Pick a narrow domain and go deep rather than spreading wide. The competitor who clones your broad, shallow surface gets a credible copy. The competitor who tries to clone your deep, specific understanding of one industry’s workflow has to do the years of work you already did, and most will not bother. A moat that survives cloning is built from depth.

Invest in the layers engineers usually skip. Distribution, documentation that lives where users find it, reliability that earns trust, onboarding that keeps users through genuine usefulness rather than lock-in. These compound while features are being copied, and they are the reason the copy reaches a market with no unclaimed customers.

And protect the judgment layer on your own team. The skill of deciding what to build is now the highest-leverage skill a developer can hold, and it is built by staying close to users, not by staying close to the IDE. Delegate the building aggressively so that the time you reclaim lands on the judgment, the data, and the relationships, because those are the three things the clone will never contain.

The Moat Moved
#

The feature stopped being the moat the day it became cheap to copy. That shift is not a threat to the developers who notice, because the moat did not disappear; it moved to a set of layers that reward exactly the attention most engineers have been trained to underinvest in. Data, distribution, workflow depth, trust, and judgment all compound, and none of them clone.

The teams that win this era will still ship features, because they have to. They will just stop treating the features as the point. The feature is the seed; the moat is the growing season, and the growing season is now made of everything except the code.

See also
#

References
#

  • Economic moat - the structural sources of sustained advantage, the canonical list against which a feature is revealed to be a seed rather than a moat
  • Commoditization - what happens when a previously differentiated layer becomes cheap and standard, which is what happened to the application feature layer
  • Switching barriers - the workflow-depth moat that does not clone, built by absorbing more of the customer’s actual job
  • Network effect - the data-and-usage moat that makes a clone strictly weaker than the original it copied
  • Lean startup - the build-measure-learn framing that turns a feature from a deliverable into a probe, the learning loop that outpaces copying
  • First-mover advantage - why being first was only valuable during the growing season, and why that advantage compresses as copy-time collapses

The Acceptance Gap: Why an LLM Solution Is Not a Shipped Solution

Generation is solved. You describe, the model produces, and the candidate looks plausible in seconds. The bottleneck is no longer producing a solution; it is deciding that a solution is acceptable, and that gap is the one the model cannot close on its own.

I keep noticing the gap in two forms, and they fail for different reasons, which is why a single strategy like “review the output” addresses neither one well. One is a bug fix that the model swears it fixed and did not. The other is a feature that meets the description and still is not what I wanted. Both feel like the model failed, but the failure is in a different place each time, and the difference is the whole point.

The Gap Is Not One Gap
#

The distance between “a description of what I want” and “an acceptable solution I would ship” is not a single gap. It is two, stacked, and they respond to completely different interventions.

The first gap is about correctness. Did the change do the thing it was supposed to do, in the world, against ground truth the model cannot see? The second gap is about fit. Does the result match the thing I actually wanted, most of which I never wrote down?

Correctness is, in principle, objective. You can check it. Fit is, in practice, subjective. You can only feel it.

Most of the frustration people report with LLM coding comes from treating these two gaps as the same gap, and applying the tool that closes one to the other. The bug gap wants a check; the feature gap wants a reaction, and reaching for one when you need the other is why the work keeps coming back unfinished.

Bug Fixes: The Silent-Failure Gap
#

When I ask an LLM to fix a bug, it returns a patch that looks correct. It will explain the root cause in a confident voice, write a plausible diff, and tell me the issue is resolved. Often it is. Sometimes the patch addresses a symptom and the root cause resurfaces next week. Sometimes it fixes the exact reproduction I pasted but not the general case. Sometimes it fixes nothing at all, and merely rearranges the code into something that looks like a fix.

The model cannot tell me which of those happened, because the model has no access to the ground truth that would let it verify its own claim. It optimized for “a plausible fix,” not for “a verified fix.” Its confidence is not evidence, and nothing in the tone of the output distinguishes a genuine fix from a convincing one.

This silent failure is the dangerous kind, which testers have always called the oracle problem: without an independent way to decide what the correct output is, you cannot tell that a plausible output is wrong. The model has the problem in its most acute form, because it is both the thing producing the answer and the thing narrating why the answer is right. The narrator is not independent of the guess.

So for bug fixes I end up verifying manually. I reproduce the original failure, apply the patch, and check whether the failure is gone. Not because I enjoy the step, but because the model’s report of “fixed” is a hypothesis, not a result. A fix is not fixed until something independent of the model says so.

Features: The Taste Gap
#

When I ask an LLM to build a feature, the gap is different, and it fails for a different reason.

The feature works, in the narrow sense that it does what the description said. The gap is that “meets the description” is not “matches what I wanted,” because most of what I wanted was never in the description. It lived in taste. The feel of the interaction. The expectation about a default. The sense that this control belongs here and not there, that the empty state should say this and not that, that the feature is finished when it feels light rather than heavy.

I did not write any of that down, because I did not know I wanted it until I saw the result that lacked it. Subjective requirements work this way: they are discovered by contact with the artifact, not enumerated in advance. A specification can carry the objective part of what you want; it cannot carry the taste, because the taste is a reaction you have not had yet.

So the loop for features is not “verify,” it is “try.” Generate, run it, notice the distance between what I got and what I expected, describe that distance, regenerate. When the distance closes to zero, I stop. That stopping point, the “just ship it” moment, is a taste judgment, not a verification step. No test defines it. I define it, by being satisfied.

Why the Model Cannot Close Either Gap Alone
#

The two gaps look different, but they share one cause. Both are acceptance gaps, and acceptance requires information the model does not possess.

For bugs, the missing information is the ground truth of correct behavior. That truth lives outside the model, in an executable check or in a human observation. The model can guess at it, but it cannot consult it. For features, the missing information is my taste. That lives outside the model too, in the reaction I will have when I try the result. The model can guess at it from my description, but it cannot feel it.

In both cases the model can generate candidates freely, and in both cases it cannot sign off on them. Signing off requires exactly the external information that was never in the prompt, which is why no amount of rephrasing the prompt closes the gap.

This situation follows the same pattern I described in The Shifting Bottleneck: automating a layer does not remove the layer, it moves the constraint to the layer above it. Here the automated layer is generation, and the layer above it is acceptance. The work did not go away. It turned into deciding whether the generated thing is good enough.

What Closes Each Gap
#

The two gaps respond to different interventions, and the strategy is to match the intervention to the gap.

The bug gap is closeable in principle, and the mechanism is encoding. Write the acceptance criterion as a check the model cannot fake. A failing test that must pass. A reproduction script that must go green. A property the output must satisfy, stated before the fix is written. Once the check exists, the model can run it, and its claim of “fixed” becomes trustworthy only when the check agrees. The bug gap shrinks to exactly the set of bugs for which I have not yet written a check.

This practice is test-driven development, rediscovered as the answer to “why don’t I trust the model’s fix.” The discipline is to write the check before, or alongside, the fix, not after. A bug fixed without a check is a bug I will have to verify by hand, forever, because nothing is keeping it fixed. A bug fixed with a check stays fixed, because the check fails loudly if the bug regresses, whether or not I am looking.

The feature gap is not closeable the same way, and this is the uncomfortable part. I cannot write a test for “this feels right,” because I do not know the specification of my own taste until I see the result. Exploratory testing and specification by example help me identify more of what I want, but they cannot identify all of it, because the residual is a reaction, not a requirement. The feature gap closes only through iteration, and the terminal condition is a human saying “good enough, ship it.”

There is no encoding that removes the human from the feature loop, because the human’s reaction is the signal the loop is measuring. The best I can do is make the loop fast, so that each try is cheap and I can afford many of them. A model that generates in thirty seconds and an environment where I can try the result in ten more is a loop I can run ten times in a morning, and the tenth version is usually the one I ship.

The Mistake Is Treating Them as the Same Gap
#

Most of the waste in LLM-assisted work comes from applying the wrong tool to the wrong gap.

Treat the taste gap as a verification problem, and you write tests that pass and ship features nobody likes. The tests confirm the feature does what the spec said, which was never the question. The question was whether the spec was the right spec, and no test answers that.

Treat the verification gap as a taste problem, and you manually re-check, by hand, things the model could have checked for itself. You read a diff looking for whether the bug is really fixed, applying your eyes as a slow and unreliable oracle, when a failing test would have answered in a second and answered the same way every time. You waste on work the model could have done for you attention you should have spent on taste.

The split is sharp once you see it. If the gap is “did it do the thing,” that is a check, and the move is to encode the check so the model runs it without you. If the gap is “is this the thing I wanted,” that is taste, and the move is to iterate fast and trust the stopping judgment. Verification and validation are the old words for this distinction, and they have never been more useful than they are now: verify against the spec, validate against the want. The model can help with the first. It cannot do the second for you.

What to Do Next
#

Sort the next ten things you hand to the model into two piles.

The bug pile gets a check for each item, written first. If you cannot write a check that would fail before the fix and pass after it, you do not yet understand the bug well enough to delegate it, and the model’s fix will be a guess you have to verify by hand anyway. The check is the thing that makes delegation safe.

The feature pile gets a fast iteration loop. Cut anything that slows down the try-it-and-react cycle, because the cost of a feature is no longer the cost of building it, it is the cost of the number of tries it takes to match your taste. A model that builds cheaply makes many tries affordable, and that is what matters.

Leave the “is this good enough to ship” judgment to yourself. Generation is no longer the bottleneck, and neither, soon, is verification, once you encode it. The bottleneck is acceptance, and the half of acceptance that is taste is the last compounding thing you do. Nobody can write that prompt for you, which is exactly why it is the part worth keeping.

See also
#


github-board - A kanban board for any GitHub search

GitHub’s issue and PR lists are flat. When you’re tracking a dozen repositories, or triaging hundreds of items, a flat list stops being a useful view of your work. GitHub Projects exists, but it demands manual triage and won’t adapt to an ad-hoc layout you want for the next ten minutes.

I built github-board to fix this. It turns any GitHub search into a customizable kanban board, defined entirely by small filter expressions. There is no backend, no build step, and no framework. You open index.html in a browser, paste a token, and build a board in seconds against live data. Try it live at tomzxcode.github.io/github-board.

github-board overview

The problem
#

A GitHub issue list answers one question well: what is in this repo, in this state. It answers almost every other question poorly. Which of my open PRs are drafts, which are waiting on review, and which have gone stale? How are issues spread across the area:* labels? What does the backlog look like across an entire organization?

You can build a GitHub Project board to answer these questions, but each card has to be placed and maintained by hand. The board reflects triage you did, not the state of your data right now. For a recurring view that’s worth the effort. For a one-off question you want to answer in 30 seconds, it isn’t.

How it works
#

You give github-board a search query, the same syntax the GitHub search bar already understands.

repo:owner/name

Then you define columns with boolean expressions over the fetched items. The board fetches up to 2000 issues and PRs through the GraphQL API with pagination, then groups them into columns entirely in your browser.

A default board ships with Draft PRs, Open PRs, Open Issues, and Closed columns. Each column is just an expression, and the board re-renders live as you type.

Columns and swimlanes as expressions
#

Every column is a filter expression against fields like type, state, labels, assignees, and dates.

type:pr and state:open and draft
type:issue and state:open and label:bug
state:closed and updated > -7d

The expression language supports the operators you’d expect: ==, =~, <, contains, in, exists, empty, plus regex matching and relative date math like -7d, -2w, -1y.

You can add a second dimension with swimlanes (rows), so a board can group issues by assignee across the same set of status columns, all from one query.

Auto-split with a $1 capture
#

Auto-split is the feature I use most. If you put a $1 capture in a column or lane rule, github-board expands it into one bucket per distinct matched value.

Name a lane area:$1 matching labels =~ /area:(.*)/i, and you instantly get one row per area label that exists in your data. Name a column the same way and you get one column per label. No manual setup per bucket, and buckets with no items simply don’t appear or stay empty depending on your preference. The $1 capture is the fastest way to see how work distributes across a category you didn’t know you cared about until just now.

Shareable links and presets #

The entire view (query, filter, columns, swimlanes, and sort) is encoded in the URL hash. Click Share, send the link to a teammate, and they see your exact board using their own token. The token is never included in the link, so you can paste it anywhere.

For boards you return to, save the full configuration as a named preset. Presets persist in localStorage alongside your token, so everything stays in your browser.

Privacy and scope
#

github-board is read-only and has no backend. Requests go directly from your browser to api.github.com. You bring your own personal access token, which is stored only in your browser’s localStorage. There is no OAuth flow, no server logging, and no way for the tool to modify your issues or pull requests.

github-board is a view, not a project-management tool. There is no drag-and-drop across columns, because the board reflects your data, not where a card was dropped.

Getting started
#

No install is required.

  1. Open tomzxcode.github.io/github-board (or open index.html from the repo).
  2. Paste a GitHub personal access token with read access to what you want to view.
  3. Open Settings and enter a query like repo:owner/name or org:your-org.
  4. Click Refresh and adjust the columns to match your workflow.

If you already use gh-cached to browse issues without burning your rate limit, or ghx for agentic code reviews, github-board is the visual counterpart: the same GitHub data, arranged the way you want to see it right now.

See also
#

  • gh-cached - browse GitHub issues and PRs from a local cache to avoid rate limits
  • ghx - a CLI for the inline comment and review operations the gh CLI doesn’t expose
  • Triaging open source pull requests - the kind of high-volume review work github-board is built to visualize
  • Backlog management best practices - principles for keeping a backlog scannable that expression-driven boards make concrete