And the exact same things you would need to safely give up on PRs for human developers (auto-formatters, linters, comprehensive end-to-end tests, continuous deployment pipelines, etc), are also things that place meaningful guardrails on LLMs, and help them maintain a reasonable quality bar.
If that's genuinely your attitude then your org has a problem.
Code review is slow and less fun, for the average sw eng. But for high quality work it's indispensable. So treat code reviews as a scarce resource. Optimize for code reviewer time and attention. Have your PRs the right size? Are they well described? Do you give context? Do they fit in the bigger story? Do you mix in unrelated drive-by fixes? How easy is it to deal with you once you have received comments? Do you address them promptly? Do you give your reviewers credit (if not praise) for their help? Do you give back by doing code reviews yourself with high quality feedback? There are lot of things you can do to streamline things and give code reviews the place in a teams workflow that it deserves.
And that's not rare in teams. Lots of teams and developers do code review wrong.
I even hear other people complain that I "block" their code review. I mean, if there are issues in your code, of course I am going to flag them, what do you think the purpose of code review is?
The argument here is that all code reviews are done with attention and care, but quality of a code review is highly dependent on the reviewer and the team’s review process, and in the real world the quality of reviews pretty much follow the same distribution curve as, say, agile project management: For the time invested in reviewing, a handful of teams get excellent utility from them, most teams get little benefit, and a sad few actually cause harm.
If most code reviews provide only a little benefit at base for most teams, recommending that most teams should also delay shipping quality work is going to sound a lot like bad advice.
I’ve noticed that large PRs aren’t just a problem for human reviewers: they’re a problem for AI reviewers too.
If I submit a 100 line PR I’m likely to get some useful comments back from both humans and LLMs. In fact the LLM is likely to come back with so much feedback it gets down to the nitpicky/annoying level.
If I submit 1000+ lines in my PR, the humans either don’t have time and/or get scrolling blindness, and the AI reviewer is likely to give me a response that amounts to, “<<slaps roof>> Looks good to me bro: ship it!”
I guess they have a limited token budget for reviews so you can bamboozle them simply by blowing most or all of that budget.
It is definitely very grunt-like for an LLM.
Code reviews, documentation, static analysis, only retrieving deps from internal repos, unit tests, integration tests, ....
Especially in domains where shipping software is not the main product, and a plain cost center to the main business of physical goods.
That said, sometimes low-trust environments are the issue, not PRs. In a higher trust environment, PR review is a helpful thing you usually desire, not dread.
Respectfully, in a high-trust environment, feedback should be delivered well before the PR stage. If you've let someone write a whole bunch of code without having a shared understanding of how the solution should work, you may have earlier process issues that PRs are papering over
If you have these parties review each other's code, I agree that rarely brings much value.
I think the best way to understand our experience with reviews is to stop and say: in a few sentences, what do you expect out of a quality code review? (sounds like nothing in your case, but I am curious)
For bigger changes, of course you need feedback on designs. But that could easily be in the form of draft PRs.
I definitely would push back on anything that required feedback before PRs. That's way too much process. Just going to slow you down for no benefit.
I've worked with people who consider themselves 'prolific humans'. Someone always has to tidy upp later, and its never them
I run both infrastructure and security - that means a lot of relatively self-contained PRs to infrastructure-as-code and dependency management systems. I'm also the team lead, which makes me responsible for a lot of throwaway prototyping, as well as cleaning up anyone else's mess...
Yes, the prolific-but-damaging engineers are all too common in corporate. But particularly in startup land, you tend to find your high-performers wearing a lot of hats at once.
So in essence you have one guy working at 4x and e.g. four other getting just 0.7x - net effect is still positive, but everyone save for that one person is miserable.
Mind you, the 4x dev doesn't necessarily have to be particularly talented - they only need to get their foot in the door before anyone else.
Back during the ZIRP days you could immediately tell that this is the case in a team by staff rotation alone. Nowadays people understandably cling to their jobs, so you might now know until it's too late.
IME, it’s because they lack the experience to have the Taste one develops as a senior engineer. “This works, and is somewhat understandable” is as far as they get. Little to no understanding of how this solution could fit better in the codebase.
I've managed some incredibly prolific developers and some very slow ones, and the prolific ones are pretty much always the ones more available, more willing to fix things, more willing to take feedback.
And also: they make less mistakes because their skills are sharp. This anecdote comes to mind: https://austinkleon.com/2020/12/10/quantity-leads-to-quality...
If you have to constantly rationalize performance differences by demeaning others, this says more about you than the prolific people.
1. Your skills are >2 standard deviations above everyone else's.
2. You're fast at producing a lot of half-baked garbage, and your coworkers are too shy to confront you, so they just try to ignore it.
(one of these scenarios is much more likely)
We had better management for a few months, and many on the team were actually quickly closing the skill gap with me, but we had another shuffle and things are stupid once more.
So I'd offer that's option 3. (There's always a third option to any suggested either-or fallacy.)
But of course HN has to with the most uncharitable interpretation.
I am not sure there is a good analogue for reviews in the AI world. The human operating the AI should obviously review everything produced but that is clearly not as good as a second pair of human MK1 eye balls from pair programming.
Code review is not for feedback, it’s for ensuring quality (many eyes on the output) and have a shared involvement in the evolution of the code. The time for feedback is earlier, once you have an idea of the solution.
1. Solution under-engineered/over-engineered. 2. Code is hard to read or comprehend. 3. Design/Archtecture lacking. 4. Principles decided upon by team not adhered to.
These are just some of the reasons I've rejected functionally correct code before.
To summarize, in any software engineering course you learn that there are other metrics used to evaluate code other than correctness (maintainability, readability, scalability, portability, efficiency etc.)
While PRs may have been used to correct style, that shouldn't have been their only or even main purpose. That's on whoever was using it that way, not on the concept of reviews.
Then don't review the code. Ask Agents to review and merge it, also shift the responsibilities to the AI agents as well.
If you think human is a bottleneck, then either optimize for humans, or remove humans. What's the problem?
Sadly, in my case, it is the auditor. Our SOC2 documents have this lovely "every change has been reviewed by at least one other human", and it's going to be a fun battle to get that reworded
If a coworker is creating a ton of AI-made PRs, I think the first step should always be to run an AI against them with the "assume this is low quality code and find all problems, big and small" text that was suggested in a comment here, and let that be the first line of defense.
To keep the dev on their toes, each dev should come up with their own prompt for AI PR review and they can switch off who reviews it each time, until there are no problems remaining.
Then a human can start to review it.
It will quickly show the low quality code being produced and the massive waste of time it is for everyone, not to mention all the money spent on tokens for the whole process.
Or it'll work, and everyone will have their way, and only have to review code that's pretty decent.
Change your auditor and compliance, SOC2 is created for a trust between organizations employing humans, if you think agents can own the things, lead the way, introduce a new compliance, if companies sign up for it, then you will be the first who is removing the human bottleneck.
Prolific humans should scale to the review/test/QA/staging backpressure - not just push to have whatever they produce accepted.
Prolific is not a badge of honor, and "lines of code" is not a quality metric.