An AI Aside

Taking a break from my work towards the conclusion of my fascism project, I wanted to take a moment to talk about something lighter: Why a strong Artificial Intelligence will almost certainly kill us all.
Kidding! It'll probably just make the world awful and uninhabitable! Our deaths will be an unintended side effect.

I'm kind of assuming you've read the definitive AI blog piece, and if you haven't, you probably should. It's a lot more comprehensive than this, funnier, and better written.

But assuming you're still wondering why AI is going to get us, here's how. It's the same moloch-ian problem we run into everywhere else, but with higher stakes.

Because here's the problem, we already have super-intelligences running around, and we're not very good at controlling them. I'm talking, of course, of corporations.

Indeed, the problem of restraining an advanced AI, ensuring that it will only look to achieve its goals through a small predictable number of channels, is similar to the problem of ensuring corporations are ethical.

By this I mean, if an organization of people armed with conventional morality can't be trusted not to cleverly circumvent regulations, what chance do we have to control a similarly clever but amoral AI? Our Volkswagen fellows might have cheated emissions standards, but they know better than to do something that would actively harm people (leaving aside the pollution argument for a moment). An AI would have no such compunctions.

It's not even the AI's fault, it's a basic fundamental error to try to directly anthropomorphize them. It's just that it's a lot easier to program discrete tasks, than it is to program an innate sense of morality into a machine. And as the machines get smarter and more capable, the chances for dangerous error increase.

In the end, it'll probably be just a regrettable accident.