What AI Assistants Actually Change: Compressing the Build-Test-Refactor Loop

Watch a capable engineer work and you will not see a steady stream of finished code. You will see a cycle repeated hundreds of times a day: make a small change, run it, look at what happened, adjust. This loop, tight and mostly invisible, is where the hours actually go and where quality is actually won. The interesting question about an AI coding assistant is therefore not how fast it types but what it does to this loop. Framed that way, the answer is specific and measurable: it shortens the time between making a change and learning whether the change was right, and because that loop runs so many times, shortening it compounds. This article takes the inner loop apart, shows which legs an assistant compresses, and explains why the leg it cannot compress is the one that determines whether the speed is worth having.

The inner loop is where the time goes

Software is not written in a single pass; it is converged upon. An engineer forms an intention, expresses a small piece of it in code, runs it, compares what happened to what was wanted, and adjusts. This build-run-observe-refactor cycle is the inner development loop, and its defining feature is repetition: it runs not once per feature but dozens or hundreds of times, each pass nudging the work closer to correct. The total time to build something is far less about the length of any single edit than about how many loops it takes and how long each loop lasts.

That decomposition points at the real lever. If the number of iterations to reach a working result is roughly set by the difficulty of the problem, then the way to finish faster is to make each iteration shorter. Anything that reduces the latency between making a change and learning its consequence multiplies across every pass of the loop, which is why fast test suites, hot reload, and good error messages have always felt so disproportionately valuable. They do not speed up one step; they speed up the step you take a thousand times.

This is the frame in which an AI coding assistant should be evaluated. Not as a faster typist, which barely matters, but as a device that reduces the latency of the inner loop. Everything useful it does, drafting a first implementation, proposing a test, explaining a failure, suggesting a refactor, is an intervention on one of the loop's legs, and its value is the loop-time it removes multiplied by the number of times the loop runs.

Anatomy of one iteration

A single pass of the loop has four legs, and each carries its own latency. First is implement: turning a fragment of intent into a concrete change. Second is run: executing that change so it can be observed. Third is observe: reading the result, whether a passing test, a failing assertion, or an error message, and comparing it to the goal. Fourth is refactor: reshaping what now works so the next iteration starts from cleaner ground. The loop then repeats, carrying forward what was learned.

The cost of the loop is the sum of these legs, and the slowest leg dominates. A ten-second test run tucked inside a loop that runs two hundred times is over half an hour of pure waiting, most of it spent watching rather than thinking. The observe leg is subtler but just as expensive: time spent deciphering an opaque stack trace or reconstructing why a result is wrong is latency exactly as real as a slow build, and it is often the larger share.

Seeing the iteration as a loop rather than a line matters because it explains where speedups come from. You do not accelerate development by making the finished code appear faster; you accelerate it by shrinking one or more legs of a cycle that repeats. The figure shows the loop as it actually runs, a ring rather than an assembly line, with learning carried from each pass into the next.

What the assistant actually compresses

Mapped onto the loop, an assistant's contributions stop looking miscellaneous and start looking targeted. On the implement leg it produces a plausible first draft of a change from a stated intent, collapsing the blank-editor latency of getting from idea to something runnable. On the observe leg it does something less obvious but often more valuable: it reads an error or a failing test and explains it in context, compressing the interpretation time that is usually the loop's hidden bottleneck. On the refactor leg it proposes structural improvements the engineer can accept or reject at a glance.

Crucially, the assistant also shortens the loop by generating the very tests that make the observe leg fast and trustworthy. A loop is only as good as its feedback, and feedback is only as good as the checks behind it; an assistant that drafts a test alongside a change tightens the cycle at both the writing and the checking ends at once. The engineer's role shifts from producing every artifact to stating intent, steering, and judging, which is a different and more leveraged use of the same hour.

The net effect is a loop whose stages remain the same but whose latencies shrink. The figure lays out the compressed cycle as a pipeline of stages from intent to a committed change, with the human supplying intent and judgment at the ends and the assistant collapsing the slow middle.

What the evidence actually says

The clearest measurement to date comes from a controlled trial reported by Sida Peng and colleagues in 2023, in which developers given a defined programming task and an AI assistant finished markedly faster than a control group without one, with the study reporting a reduction on the order of fifty-five percent in completion time. The headline number is striking, but the mechanism is what matters for this argument. A single task completed far faster is consistent with exactly the story above: a loop whose per-iteration latency fell, run many times, yielding a large aggregate saving.

It is worth being precise about what such a result does and does not establish, because honesty is the price of using it. It measures time to complete a bounded, well-specified task, not the correctness of large systems, the maintainability of the output, or the effect on a whole team over months. Treating a task-level speedup as a blanket productivity multiplier would be exactly the kind of unearned extrapolation this series avoids. The defensible reading is narrower and still important: on well-scoped work, the assistant shortens the inner loop enough to move the finish line substantially closer.

The direction also fits a long-standing finding from delivery research. The DORA program summarized by Nicole Forsgren, Jez Humble, and Gene Kim ties strong performance to fast feedback and small batches, which is the same claim as this article's from an organizational altitude: shorten the loop, reduce the amount in flight, and throughput and quality both improve. An assistant that compresses the loop is, in these terms, a fast-feedback mechanism operating at the scale of a single keystroke rather than a deployment.

• The trial measured a bounded task, so read it as loop-latency reduction, not a blanket multiplier.
• A large task-level saving is what many shortened iterations look like in aggregate.
• Fast feedback and small batches are the same lever seen from the delivery-metrics altitude.
• Correctness, maintainability, and team effects are separate questions the number does not settle.

Who does what in the compressed loop

Compressing the loop does not remove the human; it reassigns them. The assistant is strongest at the mechanical, generative legs: producing a draft, drafting a test, explaining an error, proposing a refactor. The human is irreplaceable at the legs that require holding intent and exercising judgment: deciding what the change should achieve, recognizing when a plausible-looking result is subtly wrong, and choosing which of several suggestions actually fits the design. The loop runs fastest when each party does the leg it is suited to.

This division is not a nicety; it is the source of both the speed and the risk. The assistant's drafts are fluent and frequently correct, which is precisely why an inattentive human can wave through a change that is confidently wrong. The value of the arrangement depends on the human genuinely occupying the judgment legs rather than ceding them to the fluency of the output. The figure assigns the legs across the two roles, making explicit which steps belong to the person and which to the machine.

Read the assignment as a contract. The assistant accelerates production and interpretation; the human owns intent and verification. When that contract holds, the loop is both fast and sound. When the human quietly hands the verification leg to the assistant as well, the loop is merely fast, and the next section is about why that trade rarely ends well.

Faster loops help only if the checking leg keeps pace

There is one leg of the loop that resists compression, and it is the most important one. Verification, actually establishing that a change is correct and not merely plausible, cannot be shortcut without changing what the loop produces. An assistant can draft a test, but deciding whether the test checks the right thing, and whether the passing result means what it appears to mean, is judgment that does not compress. This is a familiar shape: just as the inherently serial part of a computation bounds how much parallelism can speed it up, the irreducible checking work bounds how fast the loop can safely go.

The danger is specific. Because an assistant makes the writing legs so cheap, it becomes tempting to spend the savings on more speed rather than on more checking, and confidently-worded but incorrect output is exactly the failure mode that a rushed verification leg lets through. A loop that has been sped up everywhere except verification does not produce correct software faster; it produces unverified software faster, which is a different and more dangerous thing.

The way to bank the speed rather than squander it is to reinvest part of it in the checking leg: more tests, better assertions, and genuine human review of the diffs that matter. The figure frames the trade as a two-by-two of loop speed against verification rigor, and the only durable quadrant is the one where both are high. Fast and sound is the goal; fast and unchecked is a liability wearing the costume of productivity.

What this changes economically

Step back from the loop and the aggregate effect is a change in how much finished, verified work one person can produce in a given stretch of focused time. If each iteration is cheaper and the checking leg is kept honest, the same engineer converges on a correct result in fewer wall-clock hours, which means the production work embodied in a task, the effort a plan implicitly assumes must be divided among people, has shrunk. Work that once seemed to justify several contributors can now sit comfortably with one.

This does not repeal the reasons to involve more than one mind, and it is not a claim that verification, judgment, or design can be automated away; those remain stubbornly human and are exactly where the compressed loop concentrates the human's time. It is a narrower and sturdier point: the mechanical middle of the development loop has become cheap, and cheap middles change the arithmetic of how work is best organized around a single, well-supported person.

The practical stance that follows is to optimize for loop latency deliberately. Measure and attack the slowest leg, whether that is a sluggish test suite, an opaque failure, or a blank-editor stall, and use the assistant precisely where it removes that latency, while guarding the verification leg as the thing that must not be compressed. Do that, and the loop becomes the quiet engine that lets one engineer carry a whole task at a pace that used to require a crowd.

Practitioner Toolkit

Copy-paste, strictly defensive artifacts you can use today. Nothing here attacks a real system.

loop until done:
    intent   = state_next_small_goal()
    change   = assistant_draft(intent)          # implement leg
    test     = assistant_draft_test(intent)      # checking leg, written up front
    result   = run(change, test)                 # run + observe
    if result.failed:
        explanation = assistant_explain(result)  # compress the observe leg
        continue                                 # iterate, model updated
    review(change, test)                         # human judgment, not optional
    commit(change, test)

Glossary

Inner development loop

The tight, repeated cycle of implementing a change, running it, observing the result, and refactoring.

Loop latency

The time for one pass of the inner loop; because the loop repeats, reducing it multiplies across every iteration.

Observe leg

The step of reading a result or error and comparing it to intent, often the loop's hidden bottleneck.

Verification

Establishing that a change is actually correct rather than merely plausible; the leg that resists compression.

Batch size

The amount of work in flight at once; smaller batches shorten feedback and lower the context that must be held.

Fast feedback

The property of learning quickly whether a change worked, associated with higher delivery performance.

References

1. Peng, S., Kalliamvakou, E., Cihon, P. & Demirer, M. The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. arXiv:2302.06590 (2023)
2. Amdahl, G. M. Validity of the Single Processor Approach to Achieving Large Scale Computing Capabilities. AFIPS Conf. Proc. 30 (1967)
3. Sweller, J. Cognitive Load During Problem Solving: Effects on Learning. Cognitive Science 12(2), 257-285 (1988)
4. Forsgren, N., Humble, J. & Kim, G. Accelerate: The Science of Lean Software and DevOps (2018), ISBN 1-942788-33-1
5. Brooks, F. P. The Mythical Man-Month: Essays on Software Engineering (Anniversary ed., 1995), ISBN 0-201-83595-9