The Kraken Project

63

Ford smelled the fresh coffee as soon as he entered Lockwood’s office. The fall sunlight filtered through the gauzy curtains in the window, casting a warm glow over the antique desk and Persian rugs. Lockwood sat behind his desk, in his blue suit, white shirt, and pink power tie, looking relaxed, confident, and full of self-assurance. No wonder: the president had just been reelected, even if it was by the skin of his teeth, and that meant Lockwood would keep his job as science adviser for another four years.

“Wyman, glad you could come. And Dr. Shepherd, welcome. Coffee, tea?”

Ford sat down opposite Melissa. He hadn’t seen her in a week, and she looked very different, dressed in a gray suit, her hair pulled back, her face scrubbed. Funny, he’d never seen her with a clean face before.

They both opted for coffee. The stiff waiter pushed the creaking antique cart in, poured them coffee all around, and wheeled it out.

“I read your report with great interest,” said Lockwood, tapping a file on his desk. He was in an expansive mood. “While I’m not happy with your going rogue there for a few days, at least the outcome was good. Excellent, in fact. As long as you’re sure the AI was destroyed.”

“Absolutely,” said Melissa. “Incinerated. There were no stray Wi-Fi fields in the area, no way Dorothy could have escaped. And there were no copies. As you know, the Dorothy software would not allow copies of herself to exist, for reasons we don’t quite understand. She’s … gone for good.” Ford noticed that Melissa gave a little swallow, cleared her throat, crossed her arms.

“That’s a huge relief.”

Melissa leaned forward. “The one that survived, Moro — is he cooperating?”

“Oh yes. As they say in the movies, he’s singing like a canary. It appears that he and this fellow, G. Parker Lansing, of Lansing Partners, wanted the program for some algorithmic trading scheme.”

“How did they find out about Dorothy?”

“Moro was one of the founders of a hacking group called Johndoe. Through a fellow hacker he got access to one of your programmers, Patty Melancourt.”

“I feared as much,” said Melissa.

“Melancourt told them all about the Dorothy program, gave them the classified coding manual — and then for her trouble they murdered her. Made it look like suicide. They also killed a man who owned an ISP in Half Moon Bay and stole his customer data. They had a couple of Kyrgyz hit men working for them. These were some bad people, and, as you know, they died in the fire.”

Lockwood’s phone buzzed. He picked it up, listened, put it down. “I have a little surprise for you.”

A moment later two Secret Service men came in, took up their usual positions, and the president’s chief of staff came in — followed by the president and a four-star general.

Though the president had just won reelection, he still looked awful. He was dressed in an impeccable gray suit, every hair in place, but his face was still sunken and his skin gray. It had been a nasty election, and a great deal had been made of the president’s alleged poor health and bad heart. He looked like all the life had been sucked out of him.

“Dr. Shepherd, what a pleasure.” The president came over and enveloped her hand in his, giving it a clammy press. He did the same with Ford and Lockwood before sitting down himself. Even without being summoned, the waiter was there with the coffee cart, pouring him a cup.

The chief executive swept a hand through his close-cropped grizzled hair. “I want to tell you both how grateful I am for all you did. The outcome was exactly what was required, and you managed to keep this unfortunate incident under wraps. Our national security was protected.”

And there was no inconvenient scandal right before the election, Ford thought.

“But I’m not here just to give you my thanks. I’d like to introduce you to General Donnelly. General?”

The general removed a file from his briefcase. “Dr. Shepherd, I’m the chief of the Defense Intelligence Agency, which, as you may know, is a branch of the Department of Defense. The DIA manages all military intelligence involving foreign powers. We exist to prevent strategic surprise and to deliver a decisive advantage to our military establishment.”

He paused.

“Dr. Shepherd, I’ll get to the point: we’d like to offer you a job.”

“What kind of job?” she asked evenly.

“We’ve been briefed on the so-called Dorothy program you created for NASA. Now, we fully understand that this program malfunctioned and was defective, and it eventually destroyed itself. But we also know that it represented a major programming breakthrough in the field of artificial intelligence. You were the one responsible for that. We want you to lead a team to develop autonomous AI software for the Defense Intelligence Agency — software that will give us a strategic advantage.”

He laid the sky-blue file on the table in front of her. “The offer is right here. It’s a classified position — indeed, even this job offer is classified. It is a highly compensated, high-prestige position, with a lot of responsibility, support, and unlimited financing. And it also involves a commission.”

“A commission?”

“That’s right. You will be commissioned as a lieutenant colonel in the U.S. Army.”

The president placed his hands on his knees and leaned forward. “Now that the election is behind us, I’ve got a mandate to upgrade and expand our military capabilities. Especially in the area of cyberwarfare. This is the great military challenge of the twenty-first century. AI systems are the future. AI is going to revolutionize warfare. It will enable us to develop smart cruise missiles that can recognize individual targets. For example — and this is classified — we’re developing a line of drones the size of insects that can search out and destroy targets, drones that can spend days prowling cities and bunkers, looking for a programmed target. The big stumbling block has been a lack of strong AI. These dronesects, as we call them, have to be autonomous. And that’s just one of a hundred of the exciting military projects you’ll be working on, every one of which depends on autonomous software. To give other examples: AI will allow us to deploy small, rat-sized unmanned all-terrain vehicles that can penetrate enemy lines, play hide-and-seek with unfriendlies, search houses, eavesdrop, and seek out deeply fortified underground targets. It will allow us to develop small underwater vehicles disguised like fish that can journey thousands of miles across seas and up rivers to gather intelligence, sink enemy ships, and attack harbors. AI will enable us to break through enemy firewalls, destroy their homeland infrastructure, disable their weapons, and crash their planes. AI is going to restore the United States as a number-one military superpower, not through brute stockpiles of nukes, which can never be used, but through intelligence warfare capabilities. This is something the Chinese have been working on now for several years. Already there’s a growing AI gap between them and us. With your help, we’ll close that AI gap.”

“AI gap?” asked Melissa. “Like the missile gap of old?”

“Same idea.”

Ford glanced at Melissa. Her face was pale.

“The details of the offer are in the file. Please take it home and think about it. We just ask that you not discuss it with anyone.”

She pushed the file away. “The answer is no.”

“You’re declining?” said the president. “But you haven’t even looked at the offer.”

Melissa stood up. “I don’t need to. You don’t have any idea what you’re getting into with AI. Just like I didn’t when I designed Dorothy at NASA.”

“What do you mean?” said the president.

She looked around at the small group. “True AI, strong AI, is like creating a human mind. There’s something immoral about doing it at all. But to do it for the purpose of warfare, for killing … No. It is extremely dangerous to create a weapon that can make its own killing decisions — loaded with software that is taught to kill, that wants to kill. You’ll never control it. Just like we couldn’t control Dorothy. It’ll be opening a Pandora’s box. With nukes, at least a human finger is on the button.”

“That’s an absurd notion,” said General Donnelly. “Any AI system we deploy will be absolutely under human control.”

“Isn’t it nice to think so. You never met Dorothy.”

“Dr. Shepherd, this is a once-in-a-lifetime offer,” said the president, his voice rising in irritation. “You can spare us the lecture. A simple yes or no will suffice. There are plenty of others, including some on your own NASA team, who’ll be glad to work for us.”

“Then consider this my simple no.” She picked up her briefcase. “Good day, Mr. President. General Donnelly.”

“I’ll remind you that the offer you just rejected was strictly classified.”

She paused, and then abruptly turned around. Her voice was suddenly pleading. “Mr. President, I beg you, do not go down that road. It’ll be the beginning of the end for the human race. Please think this through.”

“Thank you, Dr. Shepherd, but I do not need you to tell me how to conduct myself as commander in chief.”

Melissa Shepherd turned and left. Ford watched her go.

The president turned to Lockwood, scowling. “You didn’t tell me she was some kind of anti-military nutcase.”

“I didn’t know it, Mr. President. I sincerely apologize.”

The president turned to Ford. “And you?”

Ford rose. “Having seen AI in action, I’m afraid I have to agree with Dr. Shepherd on this. The last thing the human race might do before our extinction is to weaponize AI. It’s that dangerous.”

“The Chinese are already doing it,” said the president.

“Then God help us all.” And Ford walked out of the room.