The Memory Prediction Theater

A synthesis of Bernard Baars’ Global Workspace Theory, Jeff Hawkins’ Memory Prediction Framework, and Dale Antanitus’ Theory of Cortical Neuron-Astrocyte Interaction yields a new explanation for how consciousness manifests in the brain.

The Million Euro Question

My purpose with this article is to lay before you a new hypothesis of human cognition, one that anchors consciousness more solidly in neurophysiology—albeit a theoretical neurophysiology. In doing so I address a question Thomas Ramsøy, Science and Consciousness Review’s Managing Editor, asked Baars in a recent interview on the subject of Global Workspace (SCR 2005, July).

RAMSØY: So it seems that the coupling of your cognitive theory of consciousness with neuroscience and artificial intelligence research has led to a fruitful elaboration of the theory. In what way would you say that the Global Workspace manifests itself in the brain?

BAARS: That’s the million Euro question!

I’m a software engineer with an interest in artificial intelligence, not a member of the consciousness or neuroscience communities, but I’ve noticed a powerful synergy I believe deserves your attention. Should my hypothesis prove compelling, I hope you will join with others of like mind in moving it forward into the light of true theoryhood and scientific legitimacy.

I begin with a brief summary of the ingredients of the new hypothesis starting with I find myself wondering where the neurophysiological roots are Global Workspace, but focus mainly on the recent work of Jeff Hawkins, of Silicon Valley fame, a new brain theory called the Memory-Prediction Framework. Next I delve into the Astrocyte Hypothesis of Dale Antanitus, Harvard Medical School, which appears to provide the physiological underpinnings of cortical regionalization so critical to Hawkins’ MPF. Antanitus’ theory also seems to help solve the binding problem, suggesting a cellular basis for gamma synchrony. This of course brings us back full circle to Global Workspace. What is offered is a high-level, conceptual progression from the more abstract domain of consciousness to the concrete, physiological domain of neurons and astrocytes.

Global Workspace Theory

First proposed to a wide audience by Baars in his 1987 book, In the Theater of Consciousness, the theory describes consciousness as the product of a perpetual reentrant process involving the aggregation of sensory input and the broadcast of neural output within the metaphor of a theater.

“Global Workspace theory suggests a fleeting memory capacity that enables access between brain functions that are otherwise separate. This makes sense in a brain that is a brainweb, viewed as a massive parallel distributed system of highly specialized processors. In such a system coordination and control may take place by way of a central information exchange (the global workspace), allowing some specialized processors-such as sensory systems in the brain-to distribute information to the system as a whole. (The) theory may be thought of as a theater of mental functioning. Consciousness in the metaphor resembles a bright spot on the stage of immediate memory, directed there by a spotlight of attention, under executive guidance. The rest of the theater is dark and unconscious. For sensory consciousness the bright spot on stage is likely to require the corresponding sensory projection areas of the cortex. Once a conscious sensory content is established, it is distributed widely to a decentralized “audience” of expert networks sitting in the darkened theater, presumably using corticocortical and corticothalamic fibers. This is the primary functional role of consciousness: to allow a theater architecture to operate in the brain, in order to integrate, provide access, and coordinate the functioning of very large numbers of specialized networks that otherwise operate autonomously.” (Baars 2003)

A more elaborated version of the theory is summarized graphically in the figure below (Baars 1987). Here we see sensory stimuli as actors competing for the attentional spotlight on the stage of working memory, mediated by prevailing contextual elements. Consciousness is seen as a central hub facilitating data exchange between distributed cortical processors.

The resolution of the spotlight competition produces a kind of snapshot video frame which is immediately broadcast to all unconscious processes in the audience and backstage. Each unconscious process reacts to the broadcast in its own way, producing feedback for return to the stage where it competes for inclusion into the next spotlight frame.

Global Workspace is an elegant model, but like many others, including Dr Baars himself, I find myself wondering where the neurophysiological roots are. I believe the answer lies in a better understanding of how cortical neurons actually interact—specifically, how they’re organized and reorganized at the cellular level. A compelling original theory introduced late last year by Jeff Hawkins, inventor of the PalmPilot, addresses this very topic, and does so decisively in my opinion. Hawkins calls his new theory the Memory Prediction Framework.

The Memory Prediction Framework

Given the fact that I’m principally addressing the consciousness community, I’ll begin by summarizing Hawkins’ stated views on the subject. As evidenced in the title of his book, On Intelligence, he’s chosen intelligence rather than consciousness to label the neural gestalt; nonetheless, he doesn’t dodge the issue:

“Some scientists, such as Christof Koch, are willing to tackle the issue of consciousness, but most consider it a question of philosophy bordering on pseudoscience. I think it deserves consideration if for no other reason than many people are curious about it… (It’s) not a big problem. I think consciousness is simply what it feels like to have a cortex.” (pp. 193-194)

Despite this somewhat glib dismissal, Hawkins goes on to separate consciousness into two categories, self-awareness and qualia (p. 196). He reduces self awareness to declarative memory, providing the following thought experiment as proof.

“Let’s assume you go through today and wake up tomorrow. But just as you’re waking up, I flip (a) switch and erase the last twenty-four hours (from your memory). You would have zero memory of the previous day. From your brain’s perspective, yesterday never happened. I would tell you it’s Wednesday and you’d protest, “No, it’s Tuesday. I’m certain of it.” Everyone you met with on Tuesday would say you had been conscious throughout the day… Finally, shown a video of you having lunch, you gradually become convinced that the day did happen, even though you have no memory of it… Your belief that you were conscious disappeared only when your declarative memory was erased.” (p. 197)

Hawkins discusses qualia in terms of the qualitative differences between the senses, offering two possible explanations for the phenomenon, (1) That sensory input is processed uniquely prior to reaching cortex, and that this processing is connected to the emotional centers; and (2) That the structure of perceptual input itself determines qualitative differences in experience. In support of the latter he cites the fact that the optic nerve has a million fibers and carries primarily spatial information, while the auditory nerve has only thirty thousand fibers and carries more temporal information.

Moving on to the subject at hand, I would say Hawkins’ basic position is that the brain does not compute anything, there are no functional processors in cortex. In his view, the brain simply retrieves everything it needs to produce intelligence from There are no functional processors in the cortex memory. He bases this contention on neuroscientist Vernon Mountcastle’s 1978 paper, An Organizing Principle for Cerebral Function, wherein Mountcastle proposes that because all areas of the neocortex are essentially the same in appearance and structure, they’re actually performing the same basic operation. Hawkins calls this operation the cortical algorithm, and proceeds to define it as a memory system based on a hierarchy of cortical regions. To Hawkins, the processors of Global Workspace are in actuality regions of cortex arranged in a hierarchy, with the same algorithm governing memory storage and recall running in each. He identifies four attributes essential to neocortical memory (p. 70):

1. It stores sequences of patterns
2. It recalls patterns auto-associatively
3. It Stores patterns in an invariant form
4. It stores patterns in a hierarchy

Hawkins refers to these attributes collectively as, “the necessary ingredients to predict the future based on memories of the past,” (p. 84) having earlier proposed the idea that making predictions is itself virtually synonymous with intelligence. “It is the ability to make predictions about the future that is the crux of intelligence (p. 6)… Prediction is not just one of the things your brain does, it’s the primary thing… The cortex is an organ of prediction.” (p. 89)

All of Hawkins’ neocortical attributes are important, and he fully elaborates upon them in his book, but the essence of the MPF lies in the hierarchical nature of cortex, the arrangement of the myriad self-similar regions that comprise it.

“Pick any region and you will find many lower regions providing converging sensory input. The receiving region sends projections back to its input regions telling them what patterns they should expect to see next.” (p. 123)

The world is organized hierarchically—objects are composed of subobjects, which are themselves subobjects of larger objects. Auto-association and invariance are the mechanisms that support cortical hierarchy, especially the flow of information up and down its structure. Memories are stored invariantly and recalled auto-associatively. This means that the exact same set of neurons need not simultaneously fire to produce memory recall; just being close to the invariant is enough. And of course, cortex doesn’t just record static patterns, but patterns that evolve over time (i.e., sequences of patterns), like frames of video.

“(Your) cortex has a clever learning algorithm that naturally finds whatever structure exists and captures it (p. 127)… Brains are pattern machines. It’s not incorrect to express the brain’s functions in terms of hearing and vision, but at the most fundamental level, patterns are the name of the game. No matter how different the activities of various cortical areas may seem from each other, the same basic cortical algorithm is at work.” (p. 62)

What is this cortical algorithm? At the conscious level, Hawkins sums it up this way:

“(Your) neocortex stores sensory information in its memory. At a future time when (you) encounter the same or a similar situation, the memory recognizes the input as similar and recalls what happened in the past. The recalled memory is compared with the sensory input stream. It both “fills in” the current input and predicts what will be seen next. By comparing the actual sensory input with the recalled memory, (you) not only understand where (you are) but can see into the future.”(p. 99)

Each memory is like a video clip with its own unique name, its invariant representation. When similar input occurs subsequent to memorization, if the input name is similar enough, the video clip is recalled and played; it is remembered.

The scientific core of Hawkins’ theory is laid out in Chapter 6 of his book. He goes into considerable detail explaining the cortical algorithm at the level of neurons and cortical regions. Unfortunately, a summary is beyond the scope of this article. However, its basic mechanics might be gleaned from a brief discussion of how information flows up and down the hierarchy.

“As you move about the world, changing inputs stream into lower regions of the (cortical hierarchy). Each region tries to interpret its stream of inputs as part of a known sequence of patterns. The columns try to anticipate their activity. If they can, they will pass on a stable pattern, the ‘name’ of the sequence, to the next higher region… If an unexpected pattern arrives, it is automatically passed to the next higher region (i.e., without a name). The higher region may be able to understand this new pattern as the next part of its own sequence. But if such recognition does not occur, an unexpected pattern will keep propagating up the hierarchy until some higher region can interpret it as part of its normal sequence of events. The higher the unexpected pattern needs to go, the more regions of the cortex get involved… Finally, when a region somewhere up the hierarchy thinks it can understand the unexpected event, it generates a new prediction. The new prediction propagates down the hierarchy as far as it can go. If the new prediction is not right, an error will be detected, and again it will climb up the hierarchy until some region can interpret it as part of its currently active sequence. Thus we can see that observed patterns flow up the hierarchy and predictions flow down.”(p. 159)

In short, Hawkins explains how neurons in the abstract are arranged to record perceptions as memories and then recall those memories to produce intelligence. But once again, I find myself wondering what’s actually going on at the cellular level. How is cortex regionalized and sub-regionalized, from the broad scope sensory areas (e.g., visual, auditory, etc.) comprising hundreds of millions of neurons, all the way down to the one-hundred-neuron microcolumns? In other words, What is the cellular basis of neuronal hierarchy? How do these groups form and change over time?

The Astrocyte Hypothesis

In a nutshell, the Astrocyte Hypothesis proposes “that motile astrocytes control synaptic activity and organize synapses into synchronously firing groups.“ They accomplish this through the mechanisms of infotropism and neurotransmitter-induced calcium waves. (Antanitus 2002)

After reading Antanitus’ theory a couple of times, I started to realize that his concept of infotropism, if true, is almost certainly the key to cortical regionalization. Further, the astrocytic calcium waves he describes could also be the key to another pervasive neural mystery, gamma synchrony. To me, this kind of theoretical synergy implies that astrocytes, ten times as plentiful as neurons in the brain, might well be the missing link in anchoring both consciousness (Global Workspace) and intelligence (MPF) firmly in the neurophysiological domain.

So, how does infotropism work? Antanitus:

“This theory proposes that the neurotransmitter-tropic filopodial response of astrocytes and/or activity-related astrocytic swelling in vitro also occur in vivo where they provide a mechanism that mediates the astrocytic envelopment and segregation of synapses. Just as the extent of filopodial growth in vitro is dependent on neurotransmitter concentration and exposure time and the degree of swelling is dependent on extracellular ionic concentrations, the extent of astrocytic envelopment of synapses in vivo depends on the synapse’s firing history. An astrocyte would thus be expected to change its envelopment of individual synapses based on the relative firing frequencies of each synapse. I refer to this in vivo astrocytic movement guided by synaptic activity as infotropism.” (Antanitus 2002)

Here we have the neurophysiological basis for Hawkins’ regions of cortex, a seedling explanation for how they form and change over time. Astrocytes appear to be the What is the cellular basis of neuronal hierarchy? literal glue (glial) holding these synaptic hierarchies together, as well as the medium facilitating their synchronized firing. But how do we make the leap from regions of cortex to synchronized firing across whole hemispheres? Astrocytic syncytia, masses of astrocytes connected via unidirectional gap junctions, might also help solve the binding problem at the core of Global Workspace. This help would presumably manifest as a role for astrocytic calcium waves in the activation and/or modulation of gamma synchrony.

“Astrocytic calcium waves can be evoked in vitro by exposure to both excitatory and inhibitory neurotransmitters applied in suprathreshold concentrations. Although most of the evidence for calcium waves has come from observations in vitro, there is some evidence for calcium wave production in situ as well. This theory proposes that similar astrocytic calcium waves also occur in vivo. It is unlikely that sufficient neurotransmitter would be released at one astrocytically enveloped synapse to provoke a calcium wave in the enveloping astrocyte. One astrocyte, however, can envelop many synapses in vivo. Thus, if calcium waves occur in vivo within the cortex, coordinated exposure to neurotransmitters at many of the synapses one astrocyte controls could be sufficient to trigger calcium wave production. A reciprocal relationship would then be established between an astrocyte and the synapses it envelops. The astrocyte controls neurotransmitter release through calcium wave modulation of synaptic (Ca2+) and an astrocytic calcium wave is precipitated in turn by coordinated neurotransmitter release at the many synapses the astrocyte envelops.” (Antanitus 2002)

The current best explanation for gamma synchrony is offered by Stuart Hameroff of the University of Arizona.

“The neural correlate of consciousness is in dendrites of cortical neurons interconnected by gap junctions, forming Hebbian ‘hyper-neurons’. Chemical synapses and axonal spikes convey inputs to, and outputs from, conscious processes in hyper-neuron dendrites, consistent with gamma EEG/coherent 40 Hz and the post-synaptic mechanism of general anesthesia.” (Hameroff 2005)

Could it be that astrocytic calcium waves somehow trigger and/or modulate these fleeting hyper-neuronal manifestations? Given the ubiquitous presence of astrocytes wherever neurons are in the brain, their larger population, and the fact that the syncytia of both are mediated by gap junctions, it seems likely they play a key role. Pursuing this hypothetical connection would therefore seem promising, perhaps even yielding an alternative explanation for hyper-neuronal gamma synchrony. It’s interesting that Antanitus offers an anesthesia connection in his theory as well.

Intelligence and consciousness, both rendered more comprehensible by a memory-prediction framework extended into the cellular domain. Hawkins has shown us how cortex produces intelligence, and we’ve seen neurophysiological support for his theory in the work of Antanitus and Hameroff, with their gap junction concepts of cellular aggregation. Now the problem is, How do we tie it all back to consciousness? What about Global Workspace?

The Memory Prediction Theater

Recall Baars’ theater diagram above. In an effort to render it more visually expository, I produced the image below. Essentially the same information is offered, but this depiction is configured like a real theater, with the audience facing the stage and the backstage area behind. The arrows of varying thickness and density was an early attempt by me to approximate data volume emanating from each of the modules; it’s the only modification I made to Baars’ concept, and not critical to understanding what follows.

Shortly after producing this image I discovered and quickly devoured Hawkins’ On Intelligence. Deeply moved, I wondered, How might Hawkins’ insights play out in Baars’ theater model? Answering this question led me to produce this image.

In this new theater model, the audience consists not of a set of functional processors but the invariant representations (IRs) of discrete memory sequences. Conscious broadcast from the spotlight into the audience manifests as an auto-associative memory search for best-match of the IR of the current spotlight frame—either a perception or a memory. The best-matched memory makes it back into the spotlight.

Baars’ stage of working memory actually functions more like a movie screen in the MPT. On this stage-screen appears a perpetual sequence of spotlight frames representing the stream of consciousness. Each spotlight frame is either collapsed sensory input sampled over a few milliseconds (i.e., a perceptual frame) or a memory auto-associatively recalled from the cortical audience. In other words, memory recall can be stimulated either by the broadcast of a perceptual spotlight frame or another memory frame.

Baars’ backstage context area now houses the Memory Prediction Loop. The MPL process appears to be event-driven, but it helps to think of it as a loop. The event in question is the saccade, the slight refocusing of the eye that ordinarily occurs about three times a second, but whose actual frequency is probably directly proportional to perceptual novelty. I hypothesize that each saccade produces a new perceptual spotlight frame. The MPL essentially replaces Baars’ context modules. There is no executive functionality, only the comparison of perceptual spotlight frames to recalled memory sequences in order to inform the current prediction of what’s about to occur. Here’s the basic MPL algorithm.

Memory frames recalled from the cortical audience are multiplexed into the perceptual flow. Given the rate of the saccade (~3 Hz) as compared to the overall rate of gamma synchrony (~40 Hz), it seems reasonable to suspect that many memory frames follow each perceptual frame.

The little windsock-like structure backstage is the Prediction Array. This is the reality of working memory, its physical correlate perhaps being the hippocampus. The Prediction Array is that list of memory-based prediction sequences every perceptual spotlight frame is compared against after it scales the cortical hierarchy. The MPL involves a comparison of pre-broadcast spotlight frames with the contents of working memory, whose store of memory spotlight sequences collectively produce the predictive gestalt we call context.

Context implies prediction, so the essential function of the backstage area has not changed in the transition from Baars to Hawkins, we just understand better now how context is produced. The stage of working memory accepts the IRs of memory sequences from the audience instead of feedback from supposed functional modules. Massive feedback does manifest in cortex, but in support of the singular cortical algorithm, not as discrete inputs and outputs exchanged by disparate processors.

Summary

Hawkins and Baars are not in disagreement. Hawkins’ memory-prediction framework, extended by Antanitus’ neuron-astrocyte hypothesis, is simply the underlying physical architecture of Baars’ distributed processors. The theater model of Global Workspace still functions beautifully to explain consciousness after being reinterpreted in memory-prediction terms.

Baars’ theater is an attractive building, but one admittedly lacking a strong neurophysiological foundation. Hawkins and Antanitus appear, at least to me, to supply that foundation, assuming their theories prove correct. It is my hope that the promise of such a synergy will stimulate investigation into the correctness of this source material, and therefore the correctness of the Memory Prediction Theater as a model of human consciousness.

© 2005 Carl Carpenter

References

1. Antanitus D, A Theory of Cortical Neuron-Astrocyte Interaction, 2002
2. Baars B, The Global Brainweb, Science and Consciousness Review, 2003
3. Baars B, In the Theater of Consciousness, Oxford University Press, 1997
4. Hameroff S, Consciousness, Neurobiology and Quantum Mechanics: The Case for a Connection, The Emerging Physics of Consciousness, Springer-Verlag, 2005
5. Hawkins J, On Intelligence, Times Books, 2004