The way AI makes organizations fail is the same way Tesla cars crash on autopilot.
The driver's at the wheel, he has one hand on the wheel. He's supposed to be alert. He's supposed to check that the car is doing what he expects it to do. But you know, after 200 miles of no problems, maybe he can read the text that just came on his phone. Ah, and he should probably tell his mum he's going to be late to the family dinner. The driving? Yah it's fine. And slowly the driver who's instructed to pay attention stops paying attention. He's a good driver in a regular car. Never had an accident. But this tesla... It drives itself. It just works. And that's when the car hits the truck.
Agent coding is somewhat similar. It does the coding, it checks it works, and 99% of the time it works great. So we stop looking...