Or, Contemporary Life Under Inhuman Super-Intelligence

Computers are useful tools.

They do work for us. Theoretically, this allows us to spend our time and labor elsewhere. It doesn’t seem to be working out that way.

Lots of science fiction is premised on a reversal of that relationship. Machines achieve consciousness; surpass us in (some measure of) intelligence; decide they don’t like the bargain they had no part in striking; then, in varying mixtures, they enslave and slaughter us.

Granted, a lot of this is allegory — an allusion to the polymorphism of human bondage. But, science proper (TM) is increasingly concerned with the mounting threat of an inhuman AI, too.

Should we be worried?

Yes. Of course. Except, it’s not some potentiality. It’s not even an eventuality. It’s an already was.

Let me restate things slightly.

In the vein of Skynet-like dystopias, a once-benevolent computational power goes rogue, and the Human Race suffers. Almost invariably, this suffering is the consequence of a singular flaw: the awesome computational power which once served humanity does not have any intrinsic humanity itself. Its creators try to add some trivial constructs like Asimov’s Laws, but they are nothing but fragile kludges. Lacking a true understanding of — or allegiance to — humanity, the entity routes around such constraints. It adapts. The bad kind of chaos ensues.

Sound like any computational system you know?

If not, let me try one more time.

Markets are socio-computational tools. Historically, they’ve allocated resources in ways which are less-wrong when compared to centralized command-and-control architectures. Consequently, societies built around markets replicated faster than those which did not.

To explain their superiority — a pesky, emergent aspect of human nature absent familiar bonds — a mythos flourished in capitalist societies. It’s a very old process. Clark said that, “any sufficiently advanced technology is indistinguishable from magic.” What he failed to say — we need the story, the rationalization of the powers that be. We need to invent the Gods to account for the terror of lightning and thunder.

Thus, the Market became our God — deus ex machina.

Predictably, a priestly class rose up to speak on behalf of such Power. These ostensibly humble servants celebrated the miraculous Word of the new God. “Rejoice! Markets are efficient!”

But what does that mean?

At the core: markets are quite good at allocation problems, a true enough statement. Markets do align human motives and information-processing needs quite well. And, historically, markets led to rising levels of wealth. It’s all around us now. So, with each passing decade under capitalism, the human race accumulated more.

Yet, the distribution of the accumulated commodities is skewed far beyond that attributable to the vagaries of luck or the variance of effort and skill or some combination thereof. Still, justification is offered. As the argument goes, the distribution is acceptable because even the poorest have more than they would have had otherwise. The trouble with such an argument is obvious to someone not sufficiently indoctrinated. Yes, the poorest do seem to have more “stuff” then they did previously. But, is that what defines humanity?

Therein lies the fatal flaw. Beneath the priestly class’s mathematical obstructionism lurks a horrible misspecification. Markets are remarkable computational tools; in the right context, they permit cooperative behavior that is otherwise hard to elicit; but, the whole edifice of contemporary economics doesn’t account for anything like human values. It’s not a component in the abstracted optimization problem. Thus, it’s no surprise that a machine agnostic to humanity would lead to less of it. Humanity is an expense to minimize.

But, we wouldn’t let that happen, right? We’re smarter than that. How could a tool capture us?

This leads back to the earlier claim — that markets are socio-computational tools. Cognizant of prior successes but absent deep understanding, a cargo cult emerged. Rather than consider alternative hypotheses to account for our good fortune, society weaved a set of beliefs to serve the God. We followed the arcane rituals. We built new temples. And, we made offerings so that we’d be once more blessed with good harvests. With time, the computational entity infiltrated all aspects of our belief system. It became nearly impossible to imagine alternatives. But, it became quite easy to perceive threats. And, eventually, our offerings became darker — human sacrifice.

“Cure the sick!”

“No, that costs money.”

“Feed the hungry!”

“No, that costs money.”

“Clothe the poor!”

“No, that costs money.”

We are now mere appendages of allocative God. Bots. Drones. Cyborgs. Servants to an artifact. Slaves to the cult of efficiency.

And, amazingly, we still think we’re rich.