The Killer Thing

Four

“They are gone, Grandfather,” the young girl said, slipping inside the room where Dr. Vianti was still standing quietly before the robot.

“But they will send others,” he said. “They are afraid of the robot.”

“Not of it, Grandfather, of the mind that could develop it. Those Earthmen are afraid of superiority in any form, and they recognize it in you. Why else hold you here a prisoner?”

He smiled at her gently, then visibly shook himself. “Well, I have several months yet in which to play with my toy. Now it’s only a toy, but later… It would have made a difference for our people… ” He sighed and approached the monstrous metal machine, touching it with obvious affection.

Over twice as tall as the little man, the robot stood, enough space within its metal covering to contain two layers of four men each the size of the doctor. Yet, despite its immensity, he had refined its tactile receptors so that it could sense a change in temperature of 1/100th of one degree, or could handle fragile hair-like peridot crystals without shattering them.

“We must prepare a paper,” he said. “Perhaps one day…”

The girl’s mouth tightened, but when he turned his brilliant green eyes towards her she bowed her head. Both knew his paper would never be published. “Will you use the dictation machine, Grandfather?”

“I think not, my child. Perhaps you would make notes…” The dictation machine automatically recorded in the World Group Government building.

She nodded and left him, returning a moment later with a pad.

“We must be orderly and methodical, my dear,” Dr. Vianti said; he was making a minute adjustment in the circuitry of the robot as he talked to her. “I shall continue to work on this model at night, when I am unable to sleep. Every morning we shall work on the paper together, and during the afternoon hours, you will make a finished copy of the morning’s work.”

“And what will we do with the finished copy, Grandfather?” she asked bitterly. “If it is known that you are preparing it, it will be confiscated and burned, as have been all of our books of knowledge. They have destroyed everything! Whatever they touch is left in ruins! We should have killed those soldiers today!”

Dr. Vianti didn’t look at her. He knew her bitterness; he knew the futility of her bitterness. His people had tried to withstand the superior forces from the World Group, and they had failed utterly. Not warlike, they hadn’t understood the methods and the callousness of a warlike nation, and the defeat they had suffered had been total, its mark still visible on their world, on the ruins of their land, and the ruins of many of their peoples. The World Group powers had understood precisely how to subdue this planet, and what to do with it afterwards. The leaders had been sent to Venus or Earth. The universities had been disbanded, and the teachers dispersed to work at menial tasks where they generally succumbed to apathy, their minds stultified by too much of monotonous, endless, thoughtless work, work such as he was supposed to be doing in maintaining the operations of Mocklem Mines. There was no communication among the intelligentsia; it was prohibited. Any advances made in the sciences or in technology had to be reported to the World Group, where the information might summarily be destroyed, or used by that group and filtered through it, passed back to the original discoverer for further modification under “proper” supervision.

The development of the robot would be considered not permissible, that he knew. They didn’t want the peoples they conquered to benefit in ways that would free them of the joyless tasks of keeping alive; the decision had been made long in the past that conquered peoples must be kept busy, too busy to speculate on their fate, too busy to make plans to alter their destiny. Those chosen for education or training were sent to World Group schools where their lessons included thorough indoctrination. Dr. Vianti understood all this very well, and he had flouted his direct orders not to make further modifications in the robot. His punishment would be swift and drastic.

“I shall recapitulate only briefly,” he said to the girl, “and we can fill in the details later. First, I investigated the possibility of inducing a second order purpose in the feedback net of the mechanism, that is, a state, both internal and external, in the feedback circuitry offering the entire net the highest probability for the net’s continued ability to seek the first order purposes. The first order purpose of the mechanism, of course, is the immediate satisfaction of goal achievement, the state in which the internal disequilibrium would be less than in any alternate state within the range of its operations. In bringing about a second order purpose the mechanism has the ability to arrive at a predictive value based on its past experiences, thus, the entire mechanism is involved in predicting its own future ability to maintain satisfaction of the first order purpose, i.e. its primary, over-riding concern is self-preservation in order to function and achieve goal satisfaction.”

His voice trailed off as he became more involved with the maze of interconnecting wires and wafers in the robot, and the girl sat back in her chair and watched him for a few minutes with a sad half-smile on her mouth. It would be for nothing, she thought. He would work, stay awake at night to draw his schematics and charts and diagrams, and it would be for nothing. After several minutes she left the room to watch the board that would flash if anything went wrong at the mines themselves.

From time to time Dr. Vianti muttered, mostly incomprehensible phrases, and exclamations, and he didn’t notice that she was not there to hear them. The receptors for verbal orders were elementary, primitive… must be a way of increasing the range, expand it to equal the range of orders possible… Increase learning capacity, needn’t be idle circuits, but reassignable from present functions… Reassignable, of course! With holding circuits for those functions displaced, so that nothing would be lost…

Some days later he dictated, “The learning capacity is the range of effective internal rearrangement, and as such can be measured by the number and the kinds of uncommitted resources. These resources can be increased arithmetically, limited only by the initial size of the container.”

“We’ve given it three separate feedback systems,” he said one day, weeks after the visit by Trace and Duncan. “The first system it had from the start: the goal seeking, first order purpose. I have modified this first order, so that the goal itself can be changed without reprogramming. The second one I have now given to it: the second order purpose of self-preservation in order to function on the first order level. And the third is the learning feedback net. Don’t you see, my dear? In the first two orders the operating channels themselves do not change; they remain as originally programmed. In the third order, that of learning, it is programmed to accept external data that then has the capacity to invoke change in the operating channels. As its vocabulary range grows, it becomes more and more a self-modifying communications system. It is achieving consciousness of a primitive kind. In its most restricted sense consciousness is the collection of internal feedbacks of secondary messages, when the secondary messages are about changes in the state of the parts of the system, that is, when the secondary messages are concerned with primary messages. Primary messages concern the mechanism’s interaction with the external world.”

“Grandfather, why? You are making yourself ill! For what? They will destroy it!” The robot stood in its customary place unmoving, and inside it her words were fed into the system, as were all words now spoken in its presence. It had no understanding of most of the words, but they were programmed in, awaiting a future time when understanding would infuse them with meaning and purpose.

“If I can show them that it would be useful to the entire World Group, not merely to our world, then they might further my research with an appropriate laboratory, and assistants.” His hands fluttered nervously; there was a tremor in his left hand that had not been there two months earlier, “It is too hard for one man, too hard. The endless maintenance alone…” He looked sharply at the new waldoes dangling idly. “I could make it self-repairing…” he said, and a new excitement crept into his voice. “And a learning machine… I’ve been trying to programme language into it, but a learning machine could do it around the clock when I’m not able to do it myself…”

The girl looked at the metal hulk and shivered. “This far,” she said slowly, “you haven’t given it anything that we don’t have ourselves. It’s been like teaching a child, but if it becomes self-repairing…”

Dr. Vianti didn’t even look at her. He was pulling out his diagrams that gave her a confusing impression of connections and letters and numbers, none of which had meaning for her. She left the old man.

When the army Major arrived five weeks later, he was accompanied by a dozen men, all dressed for active duty. Dr. Vianti left the robot in the makeshift laboratory to talk to the Major in the outer room. The robot stood very still for several seconds, then the dome over the barrel of its body swivelled slightly, so that one of the transparent apertures was facing the door.

The words were in the official English of the World Group government, so they were merely stored without understanding. After half an hour of the conversation the robot heard Dr. Vianti again, speaking in Ramsean, to his granddaughter, “I’ll have to destroy the robot! They will take it for further study themselves, and with its abilities now, it would be a dangerous toy in their hands.”

“How?” she asked.

Some of the foreign words interrupted, and not until another five minutes did the doctor have the opportunity to answer her question. “I’ll give it an order it is not equipped to carry out. It will break down. Stay out here and destroy the papers!”

The new voice said more words that went into storage, and the door opened. The robot knew, from its experience, that the laser destroyed. It destroyed a narrow strip of rock, four feet long, so that the rock could be separated from the mountain. It knew it would be destroyed if its dome were removed. It had no first order purpose any longer, only its second order purpose; it had to preserve itself. Whatever order Dr. Vianti issued would be contrary to the second order purpose; it would not obey a command that was contrary to the second order. It would destroy the doctor, who was a threat to its only order of purpose, i.e, to preserve itself. Its predictive value was based on its past experience. It raised the covering over the pencil-thin hole and a red light stabbed the air. It reached the doctor, and it severed his head from his shoulders.

Then the robot waited for a first order purpose to be given to it. It had no alternative; it could only function on a deductive level, achieving its goal on the basis of whatever premises were programmed into it. Without a first order purpose, it could only wait, unless threatened. The major did not threaten it in any way.

The girl screamed at it, and it scanned its circuits, searching to see if she posed a threat to its existence. She did not, Her words were recorded also, recorded and stored.

“It’s a killer! You’ll have to destroy it before it destroys everyone it gets near! It doesn’t know anything about right and wrong, good and bad. It’s an enemy of anyone who is near it!”

The men loaded it into a carrier, and they left Ramses with it, heading out into space, toward Venus and the army research installation.