The adoption curve is no longer the story. GitHub, Google, Anthropic and a dozen smaller vendors all report enterprise seat counts that would have seemed implausible in 2024, and internal developer surveys at large companies consistently show daily usage above half of engineering staff. The tool is on the desk; the question is what it did to the work.

That question turns out to be slippery, because software productivity has never had a clean unit of measurement, and AI assistance makes the existing problems worse.

Why it matters

Enterprise contracts for coding assistants now run to eight figures annually at large organizations. CFOs signing those renewals want a number, and the number they get depends entirely on methodology: accept-rate telemetry, self-reported surveys, controlled experiments and delivery-metrics analysis each produce different answers, in roughly that order of optimism.

The stakes extend beyond procurement. If assistants genuinely multiply output, staffing plans, outsourcing economics and the entry-level hiring pipeline all change. If they mostly shift effort around — faster typing, slower reviewing — the planning implications are opposite.

How it works

Modern assistants operate at three levels: autocomplete in the editor, chat-based generation of larger blocks, and agentic modes that plan and execute multi-file changes with tool access. Each level has a different measurement profile. Autocomplete gains are easy to see in keystroke telemetry but small in absolute terms. Agentic gains are potentially large but come with review overhead that telemetry rarely captures.

The controlled-experiment literature has matured: randomized trials generally find speedups on well-scoped, greenfield tasks, and much smaller or even negative effects on large, mature codebases where context is the bottleneck. Field studies of real delivery pipelines find cycle-time improvements concentrated in code review and testing rather than in writing itself.

Evidence

Vendor-published studies report task-completion speedups in the 20-to-55-percent range, typically on benchmark-style tasks. Independent academic replications tend to land lower, and one widely discussed 2025 randomized trial of experienced open-source developers found a slowdown on familiar codebases — a result vendors dispute on methodology grounds but which matches many practitioners' intuitions about context-heavy work.

The most reliable signal may be behavioral: renewal rates are high, and organizations that ran structured rollouts report the strongest gains in test coverage, documentation and migration work — exactly the tedious categories where generation quality matters less than volume.

The competing read

Optimists argue the measurement lag is temporary: agentic workflows are improving faster than studies can track them, and the compounding effect on junior-developer ramp time alone justifies the spend. Skeptics counter that the industry is repeating the pattern of every previous productivity tool — real gains, overstated by an order of magnitude in the sales deck — and that code-volume metrics actively mislead by rewarding output rather than outcomes.

Both sides agree on one thing: the organizations getting the most value treat assistants as a process change requiring training and review-policy updates, not as a license purchase.

What happens next

Watch the shift from seat-based to outcome-based pricing, which several vendors are piloting, and watch whether agentic coding's review bottleneck produces a new tooling category around verification of AI-generated changes. The next rigorous large-sample field studies, due from several research groups, will matter more than any benchmark.