Code of Honor

6

Major Chang Xiubo of the People’s Liberation Army stared at the twin monitors of the desktop computer, mouth half open, lines of code reflecting off the lenses of his thick glasses. He studied the program carefully, imagining the beauty of her avatar. She’d been designed as an NPC for video gaming, but, oh, she had potential for so much more.

From the time he was a small boy, Chang had always imagined that computers and all their glorious parts were female. This software was certainly mysterious enough to be a woman. Completely engrossed in her ability to solve problems on her own with no prompting or additional coding from him, Chang watched the mission unfold on his screen and passed a long, rattling fart into the mesh of his office chair. The two other engineers in the lab, both women, glanced up and shook their heads in unconcealed disgust. They were accustomed to, if not at ease with, the major’s eccentricities.

He clicked the mouse beside his keyboard, scrolling, studying.

This software — called Calliope by the Americans — had already caused the deaths of two people, with another soon to follow.

Chang Xiubo’s grandfather once owned a horse that was so clever it could escape from any gate, no matter how complicated the latch. Unable to be contained, the horse eventually had to be killed. It was a near indisputable fact, the old man said, that the smarter something was, the more mischief it created, putting everyone and everything around it in danger.

The risks of being a smart horse became a popular warning from young Chang’s parents and a way to say no when he asked for more math books or a new computer. He got these things anyway — because he was smart, which made them worry all the more.

Chang’s father was by no means rich, but he was a loyal party member and a good provider for his family. He said what people above him wanted to hear and farted silently. Xiubo could never bring himself to do either. The elder Chang was charged with supply of military garrisons in and around Jiuquan, a relatively small city of a million people, west of Beijing, and south of the border with Mongolia.

A homely child with no friends his age, young Chang Xiubo had accompanied his father on a delivery to the Inner Mongolia Autonomous Region and the Jiuquan Satellite Launch Center when he was fifteen. This short journey of about a hundred kilometers got the boy out of the house and away from the old computer that he had cobbled together. The site was famous for the launch of Shenzhou 5, the first manned Chinese space flight. History had been made here, heroes made. But Chang Xiubo was not nearly so impressed with astronauts as he was with the computers that sent them on their journeys. His father had been delivering spools of computer cable, so Chang had gotten to see the computer rooms, a fascinating treat for the boy, better than going to the zoo. The physicists and engineers smoked constantly, typing away, never looking up from their workstations. At that moment, Chang Xiubo decided he wanted to emulate these stolid men. They ignored him completely until he began to ask questions about discrete math and linear algebra.

One of them called the site supervisor, another humorless engineer who wanted nothing to do with the boy until he found they spoke the same mathematical language. The supervisor telephoned a government minister he knew and eventually arranged for Chang to attend Harbin Institute of Technology as soon he could take the exams.

Chang’s mother had been horrified, not because her son was going away to a school almost as far east of Beijing as Jiuquan was to the west, but because he had displayed his staggering intellect to the world.

She wept when he left home, tissue to her nose, pleading, “Please do not be a smart horse.”

Military service was still compulsory for young men in China, if the government ever got around to noticing you forced their hand. Nowadays, that did not usually happen unless you made someone angry. The PLA had an overabundance of qualified volunteers, so those who were not willing were rarely called on to serve. That was not the case when Chang was a boy. The military officers at the Harbin Institute of Technology had encouraged him to finish his studies, but they were waiting for him when he was done.

He’d called to tell his parents, and again, his mother had reminded him to keep his intellect in check.

But smart was the only kind of horse he knew how to be. He certainly wasn’t handsome or athletic. Fortunately for Chang, his talents were highly valued by his superiors. Otherwise, someone would have long since put a bullet in his head. Past commanders had called him a “stain” on his performance reports, “a disgrace to the uniform,” but had gone on to note that he was one of the most gifted scientists they had ever seen. A secret addendum to his personnel file from a particularly hateful colonel noted that if he were to ever be separated from the military, he should be institutionalized or killed to keep him from putting his skills toward activities not sanctioned by the party. Chang knew of the file, and considered it a badge of honor. He’d found his spot as special assistant to Lieutenant General Bai.

A software engineer by training, Chang was, by disposition, a toad. He was so maddeningly aloof that both superiors and subordinates alike were forced to raise their voices to get his attention. Short and squarely built, with thick black-rimmed glasses and coarse hair that was forever in need of a trim, he could most often be found snowing flakes of dandruff on his desk, staring into space, apparently forgetting to blink. Those new to his lab might think he’d fallen into a catatonic trance, but these were no states of stupefaction. Chang’s face might be flaccid, but behind the blank stare, his mind clicked through problem after problem at an incredible rate. He’d talked to himself since he was a small child, though now he’d learned to do it silently, asking questions and then working through solutions while the world stumbled on blindly around him.

And now he had Calliope.

This magnificent thing was going to either see his star rise to astronomical heights or extinguish it entirely. He had suspected this gaming software to be advanced, but not nearly this fantastic. The Americans overused the word awesome, but this… this entity did nothing if it did not inspire true awe. Artificial intelligence was just that — artificial. But Calliope was as much art as she was science.

Like most any other scientist in his field, Major Chang realized that the future of technology was intertwined with AI. Saudi Arabia had already granted citizenship to a computer named Sophia. It was a stunt, of course, but the reality was not that far off. The presidents of Russia and China, even Iran, all saw AI as a key to power. Tech developers, billionaire businessmen — who did not get to be billionaires by accident — folded AI into their business models or changed those models altogether.

Personal assistants, self-driving vehicles, and even medical diagnoses were now driven by neural networks.

China in general, and specifically Major Chang, concentrated on arguably more nefarious applications for this technology than those working in the West.

The average smartphone had tens of thousands of times more computing power than the MIT Apollo Guidance Computer used to get the Americans to the moon. These ubiquitous devices had the power to dumb down civilizations or provide an exponential increase in human productivity. They were also a perfect platform with which to track the movement, communication, and social interactions of the user. Personal assistants like Siri refrained from spying only because they were not programmed to do so. The same artificial intelligence software that predicted and suggested words when a user was typing text could easily predict subversive antigovernment behavior. Facebook had some of the most advanced facial-recognition software in the world. The uses there were many and obvious. AI programs run by Amazon and Google could accurately target ads to just the right user by scanning their search history and myriad personal data that had become the coin of the online realm. This same technology could easily be used to gauge and score a citizen’s commitment to the social fabric of the country.

There was no doubt that Major Chang was a highly intelligent scientist, but much of his genius lay in knowing how to best utilize the advances of others. Artificial intelligence and deep learning were the future of the military as well as the civilian world. Surveillance, command control, targeting — the list was limited only by the imagination, and most of the consumer technology was ripe for the taking if one knew where to look.

And Major Chang had feelers everywhere.

He’d first heard whisperings of the AI program called Calliope from a contact at MIT. There were, it seemed, a couple of engineers who had developed a supercomputer with such an advanced neural network that she had become a partner of sorts in the development of artificial intelligence and deep learning. She — and everyone who had been in contact with the new computer referred to her as a she—was widely considered to be the next leap in deep learning. Like something from a science-fiction movie, she appeared to have a personality.

Called LongGame, she wanted to learn. And learn she did. So much so that she was able to assist the engineers who made her in creating a new software that was a smaller, more portable version of herself. Moore’s law essentially said that computing power doubled every eighteen months to two years — while devices got smaller and cheaper. The observation had held true for half a century. Many thought it had run its course — and without AI it might have.

Frankly, Calliope was exactly what people like Elon Musk and others warned about.

Using this technology as a non-player character in a pedestrian video game seemed to Chang to be unconscionable. Neural networks could now beat human beings at most any game — Breakout, chess, even the sophisticated and seemingly random game of Go. Calliope could do all of that but so much more, harnessing the Cloud or the host computer she happened to occupy.

Calliope was small, portable, and powerful. But she had another trait that made her particularly useful for any number of Chang’s purposes. Neural networks could beat a human player ninety-nine times out of a hundred, but they did not want to play. Calliope was fairly bursting at the seams as she sought new challenges. She pilfered through subdirectories, files, and applications in the operating system into which Chang had injected her, like a bored child, looking for something to do.

She’d been developed to play the game on her own alongside the primary player, like a second human. She could, for instance, “go and fetch”—fight her way through the enemy to bring back more ammunition, weapons, or fuel for her partner. She commandeered the computer’s entire hard drive, the Net, the Cloud, the Dark Web, anything to which she had access, to complete her task. That morning, he’d loaded her into a self-driving car and told her to guard the three parking spots in the lot next to his building, leaving the details of how to do it to her own devices. He thought she might park lengthwise in all three spaces, but instead she drove back and forth on the lot, actively challenging any vehicle that came near, like an aggressive mother bird protecting her nest.

She needed a mission — and since she knew how to go and fetch things, Major Chang had just the right mission in mind.