How long does it take? Does it find the issue? In more or less time than the engineers took?
Ask it to review your code for design and cohesiveness issues. How does it do?
A bug that is not immediately obvious, no syntax error, logic is sound, works as expected on local and QA envs. But let it run in prod for 2 months and you have a massive problem.
I'm not dismissing LLM's here btw, just pointing out that there is probably a lot of things that go into the category "you don't know what you don't know". These kinds of issues may or may not be a problem depending on the business you are in.
Most of the time the latest models perform better than I expect.
> Making software is no longer very hard. It's becoming a few steps up from burger flipping. Maybe somewhere around line chef.
You did acknowledge that "some" skill was required:
> There's still some skill involved, but the skill is mostly in manual testing, and accurately phrasing what went wrong. The AI is better at debugging than people are, and the code isn't great, but perfectly adequate for pretty much everything. And it's even fine at system design these days.
All I asked was for you to back this up, since the burden of proof is on you to prove your claim, not on me to prove it for you.