This is from March 2026, but it does match what I've encountered professionally. Code review was always a significant bottleneck and is even more so with agentic coding. We've had discussions about leaning more on automated review, but it hasn't translated into any practical improvements so far. I think senior engineers find it very difficult to give up on line-by-line review, no matter how much better the agents have gotten.

I think if there is a major change in code review culture, it's not going to happen first in established engineering teams. It'll happen in startups that adopt a no-human-review policy from jump and then maintain it as they mature. I'm sure that's happening, but it's likely invisible to most studies and surveys at the moment. If it proves successful, and such companies start shipping significantly faster, the rest of the industry will be forced to adopt.

> firms finds that human code review forms a significant “bottleneck” for the overall efficiency of AI coding tools

true, but what if you just don't

use skills & multipasses to keep code clean and compact (which does work if you do it right, but drives up the cost)

use a different AI to review the first AI

give it a short not in depth human review, _except_ for anything security critical

rely on lintable language specific limitations (e.g. rust no unsafe, typing, white/banned lists of imports, no eval, language sandbox etc.) to reduce what can go wrong outside of the small part of properly reviewed code (and yes, this makes languages like C & C++ close to a no-go as many of the previous points are just too brittle)

I can not say I like this idea, I really really don't.

But I have seen a small startup having rather large success with it (at least during initial bootstrap time, rest will have to be seen), so as much as this idea repels me I fear this will become somewhat of an standard. But as mentioned they also did a _lot_ more then "just dropping in" AI. (Also maybe worth a shoot out is them doing decent requirements engineering, sure using AI, but I have seen way to many places skipping it fully at a hidden but extremely high long term cost)

I also have seen the opposites, large blobs of not well written AI generated code which produces a slew of subtle easy to overlook bugs & inconsistencies (partially due to giving the AI wrong requirements due to not having done requirements enginering) which passes tests but will come back to bite you later. This did drain teams hard. Not only did it eat a lot of time for reviewing, but also those kind of "not how humans would have messed it up" bugs drained even more time. Maybe worst all the new time drains where not the kind of work devs like to do, so that drained motivation to a point where it can affect productivity.

That and they do not necessarily generate better code. They tend to generate code that is repetitive. It misses opportunities for extraction and refactoring, the result is bloated and hard to read code (because those blocks are there literally instead of with a usable name) and they tend to over-comment where it doesn't add but is 'safe' because it is obvious (effectively re-writing code as comments) and fail to document where documentation would be essential. There also typically isn't any real plan to what is written, as long as 'the tests pass' it is all good.

You can fix a lot of this stuff by shepherding the whole process but it can get quite tedious. I'm impressed by the number of people that are letting Jesus (one of these days an AI provider is going to name their AI that way :) ) take the wheel and I wonder how long they will manage to support the output. So far the score isn't very good. I've gotten good mileage out of these tools by letting them loose on very narrowly scoped pieces of code that are written in isolation. As soon as the codebase gets larger than 100K LOC or so managing the whole thing becomes an exercise in putting up barriers to avoid spillover from a tiny issue into a much larger one.

Well, imagine racing car manufacturers competing on making better, faster engines. At some point engines are so strong that other parts become the limiting factors, such as tires for traction. Now someone invents superfuel, but cars are still only marginally faster because the additional engine power didn't change the bottom line much.

That's what's happening in software. We knew that coding is only a fraction of the overall work, and even completely automating it away won't increase productivity by an order of magnitude.

Yes, the job is changing, but only parts of it. High-level engineering best practices still apply even without the coding part. And there is no free lunch.

As is standard for academic papers, this is an attempt to describe the state of the world as of when they did the research. But then it takes time to write it up and get it out, after which newspapers report it as current events.

In this case, we're looking at a picture from 2025. The models involved in the main agent results were primarily Claude 3.5 through Sonnet 4.5, GPT-4.1 through GPT-5, and Gemini 2.5. And the paper's design means you're mostly looking at the very initial effects of deployment, before companies developed good review processes.

A dimension not much discussed here is that code review is not only about quality but accountability and security. An automated review is going to help to ensure the code behaves as intended, but it is not going to ensure the intent was correct. Employees inserting back doors, bypassing tests, "cheating" on design decisions to get the work done faster etc are not going to be identified if the person driving the change tells the AI it is the intended outcome.

For this reason alone, I don't expect code review to go away entirely in most contexts where the code "matters" - in the sense that there is substantial liability to it being mis-specified at the implementation level, not just technically wrong in implementation. I do expect it to evolve a lot and become dramatically AI-assisted.

The main protection ultimately is a human being willing to put their reputation on the line by saying "I understand this change and I approve it". That, inherently, means review of some kind.

Man, I'm probably going to expose my own inexperience here, but like, did anybody actually expect code reviews to catch and prevent bugs on a line by line basis? Like ever?

I never have operated this way. Code reviews were a basic sanity check, a chance to implement best practices, and a chance to teach these things to juniors.

If I was concerned about something breaking, I'd duscuss test coverage.

Was I doing it wrong this whole time??

It doesn't match my experience. We are producing more software, and being a lot more ambitious about the things we develop. The trick is mostly to do with getting good guidance in place and then you can spend a lot more time iterating the product itself. I spend a lot more time play testing the product now and trying different things / refining. However, there is a lot of previous work that's been done to create a huge amount of technical guidance on how to do things. Lots of things on Specification, Design/Architecture/Coding, Performance, Testing, QA testing, etc. I think getting all this guidance/tooling/process in place is critical to producing software.

Note that they define “more software” by “more tickets closed.” Make of that what you will. They argue that this is a good measure because the scope of tickets did not increase. Worth considering whether you believe this captures what they claim it does. Are the extra tests you write thanks to coding agents represented in the tickets for your work?

“But all that extra code doesn’t translate directly into improved software output on the firm level. On the contrary, the resolution rate for Issues and Epics (i.e. wholesale software features) tracked by tools like Jira did not change in a statistically significant way after AI tools were introduced (the researchers also found no “compositional shift” in the size or complexity of those Jira-tracked issues across the AI introduction).”

I’d strongly disagree with one aspect. Homemade personal use software has Definately skyrocketed. Projects I never thought I’d even have the time for are now possible.

I read this paper several days ago, when my director shared it. Several things jumped out at me:

- the data is mostly from 2025 and earlier

- the baseline mean lines of code per month was 5017, the estimated increase was 1494 lines per worker-month

- the baseline is 5.34 PRs per month, and increased by 1.22

- the baseline mean for code review is 7.03 days

The second and third bullets are confirmation that this is old data. In late 2026, people aren't delivering 30% more lines of code. If people were just producing 30% more lines, they'd still be reading every line. My point isn't that "more lines is good". I'm worried about a few too many lines of code that no one reads. My point is that this shows the data is old.

For the latter, I'm a bit annoyed by using mean, median would be more valuable, but I think it's safe to say that the engineering culture was already pretty bad. If your average pull request sits for 7 days, you evidently don't care about shipping anything, and agents aren't going to change that.

The bottleneck is, and has been, the whole team evolving the product. Having a clear, actionable, vision and being good at biting off the right next step and adjusting for the step after that is always the issue. Before AI teams took the wrong step size in the wrong direction or were so wrapped up in process that they couldn't take a step or so distracted by 'things' that they never even took steps. After AI the same is still true. Our process sucks in this industry. We love to make process because we love to make machines but the processes we love to make rarely help us and that is a fundamental problem that AI isn't changing.

I'm noticing AI catching many bugs in code that previously went unnoticed. Some of them are important, and it's great to catch them early.

The other 11 items in the list of 12 would probably never be hit in practice, but now that you know, you have to sort out how to handle each (even if that just means documenting, filing, or ignoring: you have to understand each of the disparate 12 issues first.)

From the study:

> We study the impacts of AI coding assistants and agents on software engineering work... across 718 firms.

> Our primary source of data is Jellyfish (“JF”), an analytics platform used by firms to understand the activity of their software engineering teams.

What about all the software being built by everyone else not using Jellyfish? What about the small companies that don't have the same operational bottlenecks and truly are building more than their larger competitors? What about the solo founders starting companies and building software which they never would have built without AI?

I don't think my team has delivered any more usable and needed features than before. If anything, we're running faster in the wrong direction, or things just gets never-ending scope creep or whatever.

I'm completely uninterested in the opinion of a video game journalist on what does and doesn't work well in writing software. The author knows about as much about writing code as he knows about performing heart surgery

Agree. There's more code, but what's the trade-off? Fewer people on the team understand what's actually in there, so there are more and more bugs. Of course the agent will happily fix each one, and break three other things on the way, because nobody stopped to ask why the code looked the way it did.

Feels like we're building legacy code at a record speed.

Any study that wants to say anything about the impact of coding agents on whether more software is being created or not can't limit itself to what firms are doing that create a footprint of "work events — including GitHub coding activity, Jira issues, and Google Calendar events".

This is software that would not existed without coding agents: https://fluximp.org

Sure its a coding agent itself, but I would never have tried to create it manually as I would have expected it to be too much work to see it through as solo dev. There were no issues created anywhere, no calendar events, and there is very little github activity (I primarily used Mercurial on a private server; git is just for distribution).

I started it with Kilo Code and after a couple of months it was purely self-modified and has been since then. I was a bit worried that it wouldn't work on Linux and I'd either have to use another agent or use it running on my Windows box to modify the code on my Linux box to get it working, but it just worked so the streak of being self-modified has continue.

I want to make an indie game and I'm working towards that with the project I'm using to benchmark FluxImp. My first job was as game dev and I took a shot at indie game dev 20 years ago (failed; not fun). Its something I had long written off as too time consuming to even start, but with coding agents I no longer feel that's true.

All of the stories about non-software engineers making small software that is useful to them also seems unlikely to generate the kind of footprint that lends itself to being easily studied.

"There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult." - T. Hoare (1981)

I don't see how AI coding agents are going to create 'more software' until the PO's and PM's of the world step back and let developer's communicate directly with the stakeholders. Or the PO's and PM's could cross-train on AI coding agents.

We are still being stressed to create all stories with such fidelity that 'any' developer can pick it up and implement it in the timeframe allowed for the size the story was given. When the story is written in that detail, we may as well copy it into an AI agent and have it implement it. But we can't do that, so we have people who have SME write up story's that sit in the backlog for 2 weeks until a developer can bring it in, so that they can spend a day implementing it (really 2 hours with 2 hours of meetings and 4 hours of video games) because the smallest pointing size we can give is 1 point and people aren't allowed to take on more than 8 points a sprint.

How can we expect things to get done faster when the team that reports work up to senior management put's such strict limitations on our throughput?

Yes, instead of reviewing 1,000 lines a day I now "review" 10,000 lines a day. My brain is not 10x faster, so either I spend 10x more time reviewing code, or I just opt to accept that AI is probably right based on the complexity of the problem. However I realize that if we just let the agents loose on the codebase, everything would speed up like 100x.

Invariably once certain AI models in certain domains reach "full automation" quality this will happen, until now the human eyeball bottleneck is holding a leash on millions of tireless superhuman coders who have just a few narrow but egregious contextual awareness issues.

Given what I've been able to accomplish with even just local models at home, and my experience at my day job, I imagine this has almost everything to do with the organizations themselves and less to do with using AI to build software.

Wrong thing to index on. Always bet on the slope, not on points in the slope. Things will only get better and better and factor in that downward slope thought while taking actions and pick ones that are favorable. The folks who focus on a point in time will be left behind, period.

The PR->merge->deploy SDLC is probably not what we want anymore.

Something like continuously delivery and validation through the whole stack so you can develop an incremental change, validate it as much as you can locally, then validate it as much as you can in the cloud using integration tests with synthetic, then with replayed production, and then finally allow it become production. If stuff starts breaking, you have your AI swam go in, identify the culprit, bounce it, and the train never stops.

My conclusion from all this is that Douglas Adams joined the writing team some time ago and has been responsible for the whole LLM arc.

There are many ways to say the same thing, that code throughput wasn't the only bottleneck, communication is by far the hardest, most brittle and time consuming part of software production.

We need to start becoming better writers, capable of expressing complex thoughts concisely and with technical precision. Software production has become a sibling field to linguistics, more than it already was.

Aside from other comments mentioning flaws and cutoff of study, anyone in the industry or even working on hobby products can say this conclusion is wrong.

I have a pet theory. LLMs are trained on many successful open source projects. Due to their success, the projects are used very widely and they cover extreme edge cases which have accumulated over many years and which are usually not covered, and don't need to be covered, in majority of regular software. So, when LLMs write code, they try to mimic the code they have been trained on and incorporate these edge cases into each fragment they write. It results in more code which is also harder to read and comprehend.

LLMs generate more code yes.

I generate more beautiful code, yes.

But the longer the codebase survives the more hair and cruft it accumulates. My once beautiful logical paths get carved up by compounding requirements and edge cases.

That's what the AI was trained on. The end result - the hairy ball of wax that noone can look at sideways.

I often think of that whenever AI comes up with what I consider to be an irrational but plausible edge case.

My experience is that agents are decent in coding a feature, solving a bug - doing specific well defined task. They are very bad at judging UI/UX and user flows through app. Like really, really bad. They can quickly put together vibe coded app but make it usable takes a lot of extra manual work.

Did anyone solve this problem? Any good approaches?

I've spent most of my time working on tools to keep my agents on track, and keep code legible. Where I'm probably most strict in the tooling is on documentation, which makes the code review much more manageable.

They generate more software too. It's just organizations haven't adapted to remove the bottlenecks yet. I replaced so much software for my self and its better than all the alternatives I used to use. And there are tons of other people doing the same thing.

Maybe for some orgs, I know theres orgs out there where there is no such review only QA and unit testing.

one thing i noticed, there are a lot of new things, but the software has not changed at all, we are building the same things again and again, but no new things.

The paper is from January, so the data is probably from 2025. Very unlikely anything in the paper is relevant now (if it ever was).

The title should include “IF you’re still tied to a traditional human-reviewed dev process” at the end.

The reality is that new apps and new companies are being built without the bottleneck, constantly now.

Look at the shipping speed of the frontier labs themselves. It’s all the evidence you need to know that this isn’t true.

Along these lines: agents will write mountains of super repetitive test code, seemingly uninterested in refactoring it (without explicit instruction) to reduce redundancies.

In any non-trivial integration test scenario, it's easy to end up several times the LoC in tests compared to actual shipped code.

Second problem: different models, agents, and harnesses seem to have different tendencies and they tend to avoid touching code that is not their own unless explicitly instructed. What I mean by this: it will happily duplicate some helper method from another test file rather than pulling it out and reusing it, seemingly avoiding coupling (good kind in this case since model level changes should be propagated atomically).

Tests are where I see agents just spew absolute mountains of code.

How come? For example, "Brandon Thomas released Artcraft, a free, open-source suite of apps ... alternatives to the entire Adobe Creative Suite".

I checked and it looks and works great!

This headline being in the present tense, while reporting on a data from 2025, makes it basically dishonest. 2026 AI coding agents bear little resemblance to 2025 AI coding agents, and early-2026 coding agents are not like like-2026 coding agents.

I take both approaches. just /yolo or /goal or measure twice and cut once.

yolo works when I need to focus on higher priority, and make sure things just work (though not "great").

For the latter, I use it for more like spec-driven fashion with HITL. Basically everytime it ffinishes a task, I review and then move on.

Both appraoches work. just make sure your yolo slop isn't bad and passes stuff.

LoC don't mean jack shit.

Creating code is free now. Maintaining it is not. At least not at this point. The amount of slop we need to maintain is becoming very big. There is no other way than to use ai to deal with this. But that creates even more slop. Hopefully slop will be dealt with at some point.

Maybe but I’d argue the frontier corps will self destruct via green annd product management enough to make this impractical. So far here’s how I see that going 1) can’t easily autopilot due to security. Ganna keep on prompting you into running weird shell commands 2) it’ll launch your code on weird Anthropic or ChatGPT sites behind a paywall 3) between the two of them it’ll Misalign the solution so it’s unusable

[flagged]

[dead]

Counterpoint: https://getartcraft.com

I'd say that's "more software."

The paradigm shift is trusting the agents' code, and not reviewing it, the same way that an engineering manager does not read every line of every PR.

Way back in the day, everyone wrote assembly. Then things like C came around, which had to be compiled to assembly, and most people objected: "I don't want to be that far from what the CPU is doing!".

Each time we move up an abstraction layer, people object, saying that nothing will work right if we move up that high in the stack. Give it time. I'll bet in 5-10 years, almost no one will look at code anymore, except in special cases.

We still have people who write in assembly, we still have people who write in C, mostly for performance reasons. Because you still need to be close to the CPU to get that last bit of performance. But in most cases, it doesn't matter. It doesn't matter if the LLM makes bad code if it works and passes all the tests.

Granted, this is the first time we're going from deterministic to non-deterministic, but is it really? Large scale distributed systems are already non-deterministic. They create inputs and situations that programs aren't designed to handle all the time. What's a little more chaos? :)