Hacker News new | past | comments | ask | show | jobs | submit
Soon HN will realize, and eventually the world will realize, that the problem isn't AI or accelerationists, it's that the IT 'engineering' industry has been building crap and now AI might expose it at an automated scale. [0]

We already knew about the crap - it's been discussed endlessly - but I don't see HN commenters connecting the dots to these outcomes. For decades, with few exceptions we've built rushed, fragile structures and depended mostly on security - and safety - through obscurity. Probably nobody would discover the hardcoded backdoor for vendor access or the plaintext ftp transfers or what would happen if that one router broadcast everything; nobody would look and the damage would be localized enough that only those immediately impacted would care.

Now attackers can use AI systems to look at much more, and the damage may affect society - a society ever more dependent on IT. Our BS is exposed. There's an argument that things were engineered for the environment in which they were deployed, and now that environment has changed dramatically (i.e., AI systems are in it) - like taking your sailboat designed for a placid lake into the open ocean. But I think that's a big stretch: Vulnerabilities were already widespread and widely exploited; you just hoped no attackers turned attention to your systems.

It will be valuable and interesting to see whose work product stands up to this brutal new environment. Whose engineering was legitimate the whole time? Which practices worked? 'When the tide goes out, you find out who's naked.'

[0] This is one way to look at it. Also I don't want to overreact or contribute to social media panic over recent IT issues, many of which seem unrelated to AI.

There's a scene in the movie Ocean's 11 where Don Cheadle's character gets really upset because the electric company accidentally discovers, and fixes, the exploit he was going to use to knock out power for the heist.

I think about that a lot with AI now, accelerating the rate at which these old bugs are discovered. Hopefully most of the time the people discovering things are whitehats.

I don't think that a pre-AI world in which complex systems were ever built in a way resistant to post-AI exploits could have ever existed.
loading story #50536095