Your AI Coding Pilot Worked. So Why Isn't Your Team Shipping Faster?

I’ve spent thirty years shipping production software, and the last stretch of it working with AI-generated code every day. Here’s the pattern I see over and over.

The pilot that “worked”

The typical pilot goes like this. A few developers get an assistant (Copilot, Cursor, Claude Code, Codex, take your pick). Within a week they’re enthusiastic: boilerplate disappears, tests write themselves, that annoying migration took an afternoon. Leadership hears “we’re 30% faster” and rolls it out to everyone.

Three months later, the release calendar looks the same. Pull requests are bigger. Reviews take longer. A few production bugs trace back to code nobody fully read. Some developers quietly stop using the tool.

Nothing went wrong with the AI. The process around it never changed.

Bottleneck 1: specification

An assistant does exactly what you ask, quickly and confidently, including the parts you didn’t think through. A vague ticket used to be fine because a developer would spend two days on it and ask questions along the way. Now the vague ticket becomes 600 lines of plausible code in four minutes, built on assumptions nobody agreed to.

What to do:

• Write the acceptance criteria before anyone opens the assistant. Inputs, outputs, edge cases, what must not change.
• Put the rules of your codebase in a project rules file (AGENTS.md, CLAUDE.md or .github/copilot-instructions.md, depending on the tool): how to run the tests, which libraries are allowed, which folders are off-limits. Every session then starts with your conventions instead of the model’s guesses. I wrote a guide to project rules files with a template.
• Treat “ask me questions first” as a standard instruction. A good assistant will surface the ambiguity your ticket hid.

Bottleneck 2: decomposition

Assistants are excellent at well-bounded tasks and unreliable at sprawling ones. “Add a status filter to the orders API, with tests” goes well. “Modernize the billing module” goes badly, because the assistant has to make dozens of design decisions it has no business making.

What to do:

• Break work into steps that each end in something you can run or test: a migration, an endpoint, a component.
• Keep each step’s diff small enough to review in one sitting. If a change touches fifteen files, it was two or three tasks.
• Ask the assistant for a plan first, approve or edit the plan, then let it execute one step at a time. This one habit removes most “it went off in the wrong direction” failures.

Decomposition is a senior skill. It’s also where an experienced architect pays for themselves: the quality of the cut decides the quality of everything generated afterwards.

Bottleneck 3: review

If code generation gets five times faster and review capacity stays the same, review becomes the constraint, and it’s usually your most experienced people who do it. Worse, generated code looks right. It compiles, it’s well named, and it’s commented. The bugs are in what it assumed.

What to do:

• Require tests with every change, and read the tests first. If the tests don’t describe the behavior you wanted, the code doesn’t either.
• Review for intent, not style: does this do what the ticket said, and nothing else? Did it add a dependency? Did it touch security, money or personal data?
• Use a checklist. Mine is in How to review AI-written code: intent, tests, invented APIs and packages, security, error handling and diff size.
• Let the assistant help with review too: “list every assumption this change makes” is a surprisingly useful prompt.

Measure the right thing

“Lines of code” and “suggestions accepted” will tell you the rollout is a triumph. Measure what the business feels instead:

• Cycle time: from ticket started to running in production.
• Change failure rate: how often a release needs a fix or rollback.
• Review time per pull request, and how big those pull requests are.

If cycle time isn’t dropping, find which of the three bottlenecks is eating the gain.

A 30/60/90-day rollout that works

First 30 days: pick one team and one well-tested service. Write the project rules file together. Agree on the review checklist. Measure cycle time before you start.

Days 31–60: make “plan first, small steps, tests included” the default way of working. Track pull request size and review time. Fix the rules file every time the assistant repeats a mistake.

Days 61–90: expand to the next team, with the first team’s rules file and checklist as the starting point. Compare cycle time and failure rate against the baseline.

The bottom line

AI coding tools are the biggest change in how software gets built that I’ve seen in thirty years, and the gains are real. But they belong to teams that change how they work, not just which tools they have. The model isn’t the bottleneck; specification, decomposition and judgment are.

If your team is in the “pilot worked, nothing shipped faster” phase, that’s exactly the problem I help with: setting up the workflow, the rules and the review practice so the speed shows up in your release dates. Let’s talk.