Hacker News new | past | comments | ask | show | jobs | submit
bottlepalm | on: A thousand fewer paths to root
All software is Swiss cheese to smart enough AI, and unless you can provably create secure software - you never really can. Breaking it just requires more intelligence than the thing securing it.

It's a good reason why we may need to borderline shutdown the internet in the future because to a super intelligent AI, there are no barriers. Everything is essentially open. It creates its own zero day exploits as it goes.

semi-extrinsic | parent | next
I don't think this is true in general. If you deploy a good ole web application with sufficiently locked down inputs and thorough input validation, the AI will not succeed. The security engineering of 10 years ago still works in this regard. But it requires discipline to build small, minimalist, well engineered software that you as a human actually understand.
reassess_blind | root | parent | next
And then your language has a gaping hole (PHP or NodeJS), or the box you're running on has a gaping hole (Linux), or the PaaS you use has a gaping hole (Railway). There are just too many layers underneath the good ol' web application to claim that AI won't succeed.

Honestly, at this point the best we have is to hope the whitehat AI scanning for issues is stronger than the blackhat AI.

locknitpicker | root | parent
> There are just too many layers underneath the good ol' web application to claim that AI won't succeed.

I think you got it backwards. For your AI to succeed, it needs to poke a hole on all layers, not one in many layers. If an attacker pokes a hole in n-1 layers but the 1 layer remains untouched, the system's integrity is preserved.

Why do you think defense in depth is a basic concept?

tsimionescu | root | parent
Not really, no. If you're running an application server, then your entire app can get compromised if there's a security exploit in any of your app code, your web server, your kernel, your hypervisor.

Defense in depth says that simply compromising your application code shouldn't provide access to the physical hardware you're running on, and that's hopefully the case, but it doesn't mean your app is not hacked. It just hopefully means that a hack on your app code doesn't automatically give the attacker access to your entire company.

throwawayffffas | root | parent | next
As the TCP bug that surfaced a couple of weeks ago, that is not true, you depend on the kernel, your application runtime, and many layers of dependencies.

Many bugs live in all those layers that may undermine your ability to lock down inputs and outputs.

imafraidnot | root | parent
And then PHP (name your lang/OS/platform of choice) comes along and pulls your sufficiently locked down app into the gutter.

This is more fundamental than better input checking. Your compute substrate itself is flawed. Intelligence and speed is indeed the only thing between “safe” and “completely open”.

I fully agree it’s a good idea to be minimal etc but I don’t see it being sufficient in the long run.

throwawayffffas | parent | next
Nah we just all have to switch to a handfull of heavily audited memory safe unikernels.

Your http proxy and your web server don't need a shell, drivers for wifi, mail daemons, systemd, python, PHP, perl, docker, multiple users, hell it might not even need a file system.

You can ship a stripped down docket image running on a stripped down Linux distro, but that still ships the entire kernel with a huge amount of functionality you don't need that may be exploited.

__d | root | parent
I am kinda expecting the second coming of unikernels. The reduction in both attack surface and depth seems likely to be useful.
mike_hearn | root | parent
Why would unikernels fix this? They don't even attempt privilege separation at all.
bee_rider | parent
I don’t think that is right. Defense and offense are basically different games. Currently my desktop is 100% safe (it is powered off).

That’s a dumb edge case, but it makes the point, you can always be more secure by cutting features. Maybe intelligence moves up the amount of complexity you can securely handle? But it is never much.

bottlepalm | root | parent
That's why I said there's a good chance we shut down the internet. An off/unconnected computer is the only way to be secure. Our stacks are so unimaginably complex that just connecting it to the network opens it up to every undiscovered vulnerability that it contains at every layer of the stack.

Even disconnected, if the computer is just passively listening to bluetooth or wifi, it is vulnerable to exploits in those stacks as well. AI has already demonstrated it can zero day with ease. An AI another order of magnitude stronger than what we have today could potentially zero day worm lockout every electronic device tangentially connected to the internet in an afternoon.