How to Create a Mind: The Secret of Human Thought Revealed

CHAPTER 2 THOUGHT EXPERIMENTS ON THINKING

I started thinking about thinking around 1960, the same year that I discovered the computer. You would be hard pressed today to find a twelve-year-old who does not use a computer, but back then there were only a handful of them in my hometown of New York City. Of course these early devices did not fit in your hand, and the first one I got access to took up a large room. In the early 1960s I did some programming on an IBM 1620 to do analyses of variance (a statistical test) on data that had been collected by studying a program for early childhood education, a forerunner to Head Start. Hence there was considerable drama involved in the effort, as the fate of this national educational initiative rode on our work. The algorithms and data being analyzed were sufficiently complex that we were not able to anticipate what answers the computer would come up with. The answers were, of course, determined by the data, but they were not predictable. It turns out that the distinction between being determined and being predictable is an important one, to which I will return.

I remember how exciting it was when the front-panel lights dimmed right before the algorithm finished its deliberations, as if the computer were deep in thought. When people came by, eager to get the next set of results, I would point to the gently flashing lights and say, “It’s thinking.” This both was and wasn’t a joke—it really did seem to be contemplating the answers—and staff members started to ascribe a personality to the machine. It was an anthropomorphization, perhaps, but it did get me to begin to consider in earnest the relationship between thinking and computing.

In order to assess the extent to which my own brain is similar to the computer programs I was familiar with, I began to think about what my brain must be doing as it processed information. I have continued this investigation for fifty years. What I will describe below about our current understanding of how the brain works will sound very different from the standard concept of a computer. Fundamentally, however, the brain does store and process information, and because of the universality of computation—a concept to which I will also return—there is more of a parallel between brains and computers than may be apparent.

Each time I do something—or think of something—whether it is brushing my teeth, walking across the kitchen, contemplating a business problem, practicing on a music keyboard, or coming up with a new idea, I reflect on how I was able to accomplish it. I think even more about all of the things that I am not able to do, as the limitations of human thought provide an equally important set of clues. Thinking so much about thinking might very well be slowing me down, but I have been hopeful that such exercises in self-reflection will enable me to refine my mental methods.

To raise our own awareness of how our brains work, let’s consider a series of mind experiments.

Try this: Recite the alphabet.

You probably remember this from childhood and can do it easily.

Okay, now try this: Recite the alphabet backward.

Unless you have studied the alphabet in this order, you are likely to find it impossible to do. On occasion someone who has spent a significant amount of time in an elementary school classroom where the alphabet is displayed will be able to call up his visual memory and then read it backward from that. Even this is difficult, though, because we do not actually remember whole images. Reciting the alphabet backward should be a simple task, as it involves exactly the same information as reciting it forward, yet we are generally unable to do it.

Do you remember your social security number? If you do, can you recite it backward without first writing it down? How about the nursery rhyme “Mary Had a Little Lamb”? Computers can do this trivially. Yet we fail at it unless we specifically learn the backward sequence as a new series. This tells us something important about how human memory is organized.

Of course, we are able to perform this task easily if we write down the sequence and then read it backward. In doing so we are using a technology—written language—to compensate for one of the limitations of our unaided thinking, albeit a very early tool. (It was our second invention, with spoken language as the first.) This is why we invent tools—to compensate for our shortcomings.

This suggests that our memories are sequential and in order. They can be accessed in the order that they are remembered. We are unable to directly reverse the sequence of a memory.

We also have some difficulty starting a memory in the middle of a sequence. If I learn to play a piece of music on the piano, I generally can’t just begin it at an arbitrary point in its middle. There are a few points at which I can jump in, because my sequential memory of the piece is organized in segments. If I try to start in the middle of a segment, though, I need to revert to sight-reading until my sequential memory kicks in.

Next, try this: Recall a walk that you took in the last day or so. What do you remember about it?

This mind experiment works best if you took a walk very recently, such as earlier today or yesterday. (You can also substitute a drive, or basically any activity during which you moved across some terrain.)

It is likely that you don’t remember much about the experience. Who was the fifth person you encountered (not just including people you know)? Did you see an oak tree? A mailbox? What did you see when you turned the first corner? If you passed some stores, what was in the second window? Perhaps you can reconstruct the answers to some of these questions from the few clues that you do remember, but it is likely that you remember relatively few details, even though this is a very recent experience.

If you take walks regularly, think back to the first walk you took last month (or to the first trip to the office last month, if you commute). You probably cannot recall the specific walk or commute at all, and if you do, you doubtless recall even fewer details about it than about your walk today.

I will later discuss the issue of consciousness and make the point that we tend to equate consciousness with our memory of events. The primary reason we believe that we are not conscious when under anesthesia is that we don’t remember anything from that period (albeit there are intriguing—and disturbing—exceptions to this). So with regard to the walk I took this morning, was I not conscious during most of it? It’s a reasonable question, given that I remember almost nothing about what I saw or even what I was thinking about.

There happen to be a few things I do remember from my walk this morning. I recall thinking about this book, but I couldn’t tell you exactly what those thoughts were. I also recall passing a woman pushing a baby carriage. I remember that the woman was attractive, and that the baby was cute as well. I recall two thoughts I had in connection with this experience: This baby is adorable, like my new grandson, and What is this baby perceiving in her visual surroundings? I cannot recall what either of them was wearing or the color of their hair. (My wife will tell you that that is typical.) Although I am unable to describe anything specific about their appearance, I do have some ineffable sense of what the mom looked like and believe I could pick out her picture from among those of several different women. So while there must be something about her appearance that I have retained in my memory, if I think about the woman, baby carriage, and baby, I am unable to visualize them. There is no photograph or video of this event in my mind. It is hard to describe exactly what is in my mind about this experience.

I also recall having passed a different woman with a baby carriage on a walk a few weeks earlier. In that case I don’t believe I could even recognize that woman’s picture. That memory is now much dimmer than it must have been shortly after that walk.

Next, think about people whom you have encountered only once or twice. Can you visualize them clearly? If you are a visual artist, then you may have learned this observational skill, but typically we are unable to visualize people we’ve only casually come across to draw or describe them sufficiently but would have little difficulty in recognizing a picture of them.

This suggests that there are no images, videos, or sound recordings stored in the brain. Our memories are stored as sequences of patterns. Memories that are not accessed dim over time. When police sketch artists interview a crime victim, they do not ask, “What did the perpetrator’s eyebrows look like?” Rather, they will show a series of images of eyebrows and ask the victim to select one. The correct set of eyebrows will trigger the recognition of the same pattern that is stored in the victim’s memory.

Let’s now consider faces that you know well. Can you recognize any of these people?

You are undoubtedly able to recognize these familiar personalities, even though they are partially covered or distorted. This represents a key strength of human perception: We can recognize a pattern even if only part of it is perceived (seen, heard, felt) and even if it contains alterations. Our recognition ability is apparently able to detect invariant features of a pattern—characteristics that survive real-world variations. The apparent distortions in a caricature or in certain forms of art such as impressionism emphasize the patterns of an image (person, object) that we recognize while changing other details. The world of art is actually ahead of the world of science in appreciating the power of the human perceptual system. We use the same approach when we recognize a melody from only a few notes.

Now consider this image:

The image is ambiguous—the corner indicated by the black region may be an inside corner or an outside corner. At first you are likely to perceive it one way or the other, though with some effort you can change your perception to the alternate interpretation. Once your mind has fixed on an understanding, however, it may be difficult to see the other perspective. (This turns out to be true of intellectual perspectives as well.) Your brain’s interpretation of the image actually influences your experience of it. When the corner appears to be an inside one, your brain will interpret the grey region as a shadow, so it does not seem to be as dark as when you interpret the corner as being an outside one.

Thus our conscious experience of our perceptions is actually changed by our interpretations.

Consider that we see what we expect to ___

I’m confident that you were able to complete the above sentence.

Had I written out the last word, you would have needed only to glance at it momentarily to confirm that it was what you had expected.

This implies that we are constantly predicting the future and hypothesizing what we will experience. This expectation influences what we actually perceive. Predicting the future is actually the primary reason that we have a brain.

Consider an experience that we all have on a regular basis: A memory from years ago inexplicably pops into your head.

Often this will be a memory of a person or an event that you haven’t thought about for a long time. It is evident that something has triggered the memory. The train of thought that did so may be apparent and something you are able to articulate. At other times you may be aware of the sequence of thoughts that led to the memory but would have a hard time expressing it. Often the trigger is quickly lost, so the memory appears to have come from nowhere. I often experience these random memories while doing routine procedures such as brushing my teeth. Sometimes I may be aware of the connection—the toothpaste falling off the toothbrush might remind me of the paint falling off a brush in a painting class I took in college. Sometimes I have only a vague sense of the connection, or none at all.

A related phenomenon that everyone experiences frequently is trying to think of a name or a word. The procedure we use in this circumstance is to try to remind ourselves of triggers that may unlock the memory. (For example: Who played Queen Padmé in Revenge of the Sith? Let’s see, it’s that same actress who was the star in a recent dark movie about dancing, that was Black Swan, oh yes, Natalie Portman.) Sometimes we adopt idiosyncratic mnemonics to help us remember. (For example: She’s always slim, not portly, oh yes, Portman, Natalie Portman.) Some of our memories are sufficiently robust that we can go directly from a question (such as who played Queen Padmé) to the answer; often we need to go through a series of triggers until we find one that works. It’s very much like having the right Web link. Memories can indeed become lost like a Web page to which no other page links to (at least no page that we can find).

While executing routine procedures—such as putting on a shirt—watch yourself performing them, and consider the extent to which you follow the same sequence of steps each time. From my own observation (and as I mentioned, I am constantly trying to observe myself), it is likely that you follow very much the same steps each time you perform a particular routine task, though there may be additional modules added. For example, most of my shirts do not require cuff links, but when one does, that involves a further series of tasks.

The lists of steps in my mind are organized in hierarchies. I follow a routine procedure before going to sleep. The first step is to brush my teeth. But this action is in turn broken into a smaller series of steps, the first of which is to put toothpaste on the toothbrush. That step in turn is made up of yet smaller steps, such as finding the toothpaste, removing the cap, and so on. The step of finding the toothpaste also has steps, the first of which is to open the bathroom cabinet. That step in turn requires steps, the first of which is to grab the outside of the cabinet door. This nesting actually continues down to a very fine grain of movements, so that there are literally thousands of little actions constituting my nighttime routine. Although I may have difficulty remembering details of a walk I took just a few hours ago, I have no difficulty recalling all of these many steps in preparing for bed—so much so that I am able to think about other things while I go through these procedures. It is important to point out that this list is not stored as one long list of thousands of steps—rather, each of our routine procedures is remembered as an elaborate hierarchy of nested activities.

The same type of hierarchy is involved in our ability to recognize objects and situations. We recognize the faces of people we know well and also recognize that these faces contain eyes, a nose, a mouth, and so on—a hierarchy of patterns that we use in both our perceptions and our actions. The use of hierarchies allows us to reuse patterns. For example, we do not need to relearn the concept of a nose and a mouth each time we are introduced to a new face.

In the next chapter, we’ll put the results of these thought experiments together into a theory of how the neocortex must work. I will argue that they reveal essential attributes of our thinking that are uniform, from finding the toothpaste to writing a poem.