What Should We Be Worried About?

AUTHORITARIAN SUBMISSION

MICHAEL VASSAR

Cofounder & chief science officer, MetaMed Research; former executive director, Singularity Institute

No one really knows what we should be worried about; the smartest people around seem to generally think the answer is machine intelligence. They may be right, but other types of inhuman intelligence have a track record worth worrying about. I’m worried that so few of us are doing anything to fight back against it.

In a pre-verbal tribe, most of the monkeys don’t need to decide where they’re going—because if they go off on their own, either the leader will kill them for rebelling or a jaguar will kill them once they’re separated from their pack. Only pack leaders need to activate the messy symbolic intuitions that tell them what’s out there and what needs to be done. As Abraham Maslow might say, only people who see themselves as chiefs, who have satisfied their need for esteem, are in a position to self-actualize.

Maslow suggests that at one level of description a human being consists of five programs. One escapes immediate danger, one seeks comfort and physical security, one finds a social context in which to be embedded, one builds esteem within that context, and one directs big-picture intentionality. When a given program reports satisfaction, the next program is turned on and starts competing in its more subtle game.

Some of those programs attend to things that can be understood fairly rigorously, like a cart, a plow, or a sword. Others attend to more complicated things, such as the long-term alliances and reproductive opportunities within a tribe. The former programs might involve situational awareness and detailed planning, while the latter might operate via subtle and tacit pattern-detection and automatic obedience to crude heuristics. If so, the latter programs might be fairly easily hacked. They might also offer a less fertile ground for the emergence of a Universal Turing Machine. Mechanical metaphors of solidity and shape might constitute a good substrate for digital, and thus potentially abstract, cognition, while social metaphors for vague properties like weirdness, gravitas, and sexiness might constitute a poor foundation for universal cognition. These programs seem to have been disfavored by history’s great scientific innovators, who tend to make statements like “I do not know what I may appear to the world, but to myself I seem to have been only like a boy playing on the seashore, and diverting myself in now and then finding a smoother pebble….” or “What do you care what other people think?”—which sound like endorsements of physical over social cognition.

So I worry about the consequences of assisting a population in satisfying its physiological needs but not its love, belonging, and esteem needs. Such a population would be at risk for underdevelopment of the programs for satisfying physiological and safety needs. That would be harmful if these functions are the basis for precise thought in general. One might argue that the solution to this problem is to hurry people through their love, belonging, and esteem needs, but that approach incurs further hazards.

Dacher Keltner and other psychologists have showed that higher socioeconomic class—essentially, higher satisfaction of esteem needs—leads to increased unethical behavior; other research suggests that external efforts to boost self-esteem tend to produce antisocial narcissism. Research of this type fits in well with Jonathan Haidt’s theories on the authoritarian dimension of moral cognition. It may not be a coincidence that one of the world’s most popular religions is literally named “submission.” In general, authoritarian impulses have always prompted rulers to keep their subjects’ bellies full and pressed firmly against the ground. Steven Pinker has compellingly established the success of modern society in bringing us unprecedented peace. People who don’t fear for their safety but who despair of ever achieving love or belonging are the most submissive.

When we say someone is “smart,” we largely mean that they learn quickly. In circumstances where most students reach a seventh-grade reading level, the “gifted” students absorb the full content of their schooling. According to New York City’s three-time Teacher of the Year John Taylor Gatto, that curriculum consists primarily of six lessons, but it seems to me that these lessons can be summarized in one word: submission.

Robert Altmeyer’s research shows that for a population of authoritarian submissives, authoritarian dominators are a survival necessity. Since those who learn their school lessons are too submissive to guide their own lives, our society is forced to throw huge wads of money at the rare intelligent authoritarian dominants it can find, from derivative startup founders to sociopathic Fortune 500 CEOs. However, with their attention placed on esteem, their concrete reasoning underdeveloped, and their school curriculum poorly absorbed, such leaders aren’t well positioned to create value. They can create some, by imperfectly imitating established models, but can’t build the abstract models needed to innovate seriously. For such innovations, we depend on the few self-actualizers we still get—people who aren’t starving for esteem. And that does not include the wealthy, the powerful, and the “smart”; they learned their lessons well in school.