Introduction and Mission of Numenta:

Jeff Hawkins, the speaker, introduces his talk titled “A Thousand Brains Theory of Intelligence.” He outlines the mission of his company, Numenta, which is twofold: to understand the workings of the brain, particularly the neocortex, and to apply this understanding to AI and machine intelligence. The talk aims to share progress in these areas.

The Need for a New Framework:

Hawkins cites a 1979 article by Francis Crick, “Thinking About the Brain,” which noted that despite an abundance of data on the brain, a coherent framework to understand its functioning is missing. Hawkins aims to present such a framework that aligns with known facts about the brain.

The Neocortex: The Center of Intelligence:

Hawkins describes the neocortex as the organ primarily responsible for human intelligence. This “wrinkly sheet of cells” manages sensory perception, motor behavior, and abstract thought. It is roughly the size of a large dinner napkin and about 2.5 millimeters thick.

Attributes of the Neocortex:

The neocortex is characterized by continuous and rapid learning, low power consumption, and extreme flexibility. Unlike artificial neural networks, it can learn from single exposures, uses only around 20 watts of power, and can perform a multitude of tasks, showcasing its ability to generalize well.

Comparing Brain and Current AI:

Hawkins emphasizes that contemporary AI systems are not nearly as capable as the human brain. Existing AI does not possess the attributes of continuous learning, rapid assimilation, energy efficiency, and flexibility. In essence, the brain operates fundamentally differently from today’s AI.

Overall, the talk establishes the need for a novel understanding of the brain to advance AI, explores the significance of the neocortex in human intelligence, and highlights the shortcomings of current AI systems in comparison to human cognitive capabilities.

Primary Role of Neocortex:

Jeff Hawkins starts by clarifying the primary function of the neocortex. Contrary to popular belief, the neocortex is not like a computer that merely processes inputs and outputs. Instead, its main purpose is to learn a model of the external world and create an internal representation of it. This model serves as a basis for recognizing objects, knowing locations, planning, and other cognitive functions.

Complexity of the Internal Model:

Hawkins elaborates on the complexity of this internal model. It not only knows about thousands of physical objects but understands them in hierarchical terms. For example, a bicycle isn’t just a single object; it consists of wheels, pedals, a frame, and other components. This internal model also captures the relative positions of objects to each other. For instance, the wheels are positioned relative to the frame in a specific way.

Objects and Behaviors:

The neocortex also learns the behavior of objects. For example, when using a smartphone, one expects certain actions upon touching or swiping. This behavioral knowledge forms a crucial part of our internal world model and helps in interacting with the environment.

Abstract Concepts:

The internal model is not restricted to physical entities; it also incorporates abstract concepts like math and the definition of democracy. These abstract elements are vital for our understanding and functioning in a complex world.

Goal-Oriented Behavior:

Lastly, Hawkins explains that the neocortex allows for goal-oriented behavior. Once an internal model of the world is established, it can be used to plan and decide the best course of action to achieve desired outcomes. Thus, the neocortex is pivotal in enabling intelligent, purposeful actions.

In summary, Jeff Hawkins articulates that the primary function of the neocortex is to construct a rich, multi-layered internal model of the world. This model is indispensable for object recognition, spatial understanding, behavioral prediction, and ultimately, goal-oriented actions.

Predictive Nature of the Neocortex:

Jeff Hawkins emphasizes that the neocortex operates in a predictive mode. It is always forecasting what it will encounter next. Whether you’re touching an object or entering a familiar room, your brain has expectations based on previous experiences. This predictive ability is not something we’re necessarily conscious of, but we become aware of it when something unexpected happens. Our attention is immediately drawn to anomalies, suggesting the brain had specific predictions.

Role of Predictions in Learning:

Predictions serve as a training signal for the brain. If a prediction is incorrect, the brain directs attention towards that inconsistency. It’s a cue to learn and refine the internal model of the world. The brain doesn’t require repetition of millions of images or actions to learn; it builds the internal model incrementally through correct and incorrect predictions.

Neocortical Structure: Cortical Columns:

Hawkins delves into the architecture of the neocortex, which is organized into ‘cortical columns.’ These are not visibly distinct but have been inferred to exist. In humans, there are approximately 150,000 such columns, each about a millimeter in diameter. These columns process inputs from specific sensory patches, and this columnar organization is present across sensory modalities.

Evidence for Columns:

The existence of columns is backed by research. Vernon Mouncastle’s work, for instance, found that cells in a particular column react to stimulation from the same sensory patch, whether it be skin, retina, or cochlea. After a column of about a millimeter, the responsive cells switch to a different sensory patch, indicating a new column. This serves as one piece of evidence for the columnar organization of the neocortex.

Uniformity Across Columns:

Despite their involvement in different sensory modalities, cortical columns are remarkably uniform in structure. This repetitive element is what the neocortex is essentially built with.

Through predictions and columnar organization, the neocortex constructs a versatile and adaptive model of the world, forming the basis for learning and goal-oriented behavior.

Early Studies on Neocortex Architecture:

Jeff Hawkins discusses the first scientific investigations into the architecture of the neocortex, dating back to 1899. Scientists found that the cortex is organized into columns, with varying densities and sizes of cells. These columns are about 2.5 millimeters thick.

Vertical Information Flow:

Most connections between the cells in these columns are vertical, indicating that information primarily flows up and down within a column. This vertical flow happens between different layers of cells within each column, and it is where most of the brain’s processing occurs.

Neocortical Complexity:

Columns in the neocortex are complex and well-organized structures. They contain about 100,000 neurons and 500 million synapses, with dozens of different cell types connected in specific circuits. Contrary to some beliefs in AI research, these columns do not merely perform simple tasks like feature extraction. Their circuitry performs complex computations.

Motor Outputs:

An interesting feature is that every column has a ‘motor out,’ meaning a layer of cells that project to other parts of the brain that control movement. For example, the parts of the cortex that receive input from the eyes also have cells that direct how the eyes should move.

Uniform Architecture:

Surprisingly, the architecture of columns processing different types of information—such as vision, touch, or language—is remarkably similar. Hawkins mentions Vernon Mountcastle’s 1979 paper, which speculated that this similarity means all columns perform the same intrinsic function but specialize based on what they are connected to.

Quest for Understanding:

Understanding the function of these columns could offer generalizing significance in neuroscience, according to Mountcastle. Hawkins suggests that his theory will provide insights into what columns do and how they work, contributing to this major scientific quest.

Thought Experiment Introduction:

Jeff Hawkins begins by describing a thought experiment involving a Numenta coffee cup. He holds the cup and notes that when his finger moves around it, his brain anticipates the sensation he will feel at different parts of the cup, such as its rounded side or rim.

The Role of Prediction:

Hawkins finds it intriguing that the brain can make these predictions even without visual cues, like when in the dark. This suggests that the brain has a model of the cup, and it can predict the sensory input based on the model.

Brain’s Tracking Mechanism:

The brain tracks the location of each patch of skin that comes in contact with the cup. It also checks its model to verify that the sensations felt are as expected. If something unexpected is felt (e.g., a crack in the cup), the brain recognizes it immediately.

Reference Frame Concept:

To make these predictions, the brain must have a ‘reference frame’ for each patch of skin touching the object. This is surprising because the reference frame is not relative to the body but relative to the object being touched (e.g., the cup). The brain must keep track of these locations for its sensory predictions.

Column Functionality in Neocortex:

All these observations led to a theory about how columns in the neocortex work. Each column captures input from a particular patch of skin. It pairs this sensory input with a location relative to the object, allowing it to learn a model of the object. By sensing and moving repeatedly, the column learns the object’s shape or structure.

Theoretical Contribution:

Hawkins mentions that these insights have led to a published theory, encapsulated in a paper titled, “A Theory of How Columns in Your Cortex Enable Learning of the Structure of the World.” The theory posits that columns can learn the structure of any object through a combination of sensation and location.

Reference Frames and Object Models: Jeff Hawkins speaks about the role of ‘reference frames’ in the cortical columns of the brain. These frames help to understand different features at different locations, like sensing the texture of a cup. An object is defined as a layer of cells that represents a collection of all sensed features and locations on the object.

Multiple Inputs and Column Voting: When multiple fingers touch an object, each finger connects to a different cortical column. Each column has its own location relative to the object and its own model of it. The columns communicate with each other through “voting,” which allows for a unified understanding of what object is being touched.

Application to Vision: Hawkins extends this mechanism to visual perception. The brain does not see one unified image but rather interprets various patches of visual information received by different columns. These columns similarly “vote” to construct a cohesive visual image, akin to fingers collectively identifying an object by touch.

Neural Reference Frames: The concept raises a question: How do neurons represent reference frames? Hawkins points to existing knowledge about ‘grid cells’ in the hippocampus and entorhinal cortex, which act as reference frames in spatial navigation for animals.

Confirmation through Studies: Hawkins discusses growing evidence for the presence of grid cells in the human neocortex, which supports his theory. These cells form the basis for organizing knowledge about objects or even abstract concepts, as shown in fMRI experiments involving human subjects and birds.

Hierarchical Systems: Finally, Hawkins talks about the hierarchical structure of both the brain and most AI systems. He compares traditional hierarchical models with the complex connectivity in an actual brain, highlighting the depth and complexity that real neural networks exhibit, as compared to their AI counterparts.

Concluding Remarks: Hawkins asserts that these insights into cortical columns, reference frames, and grid cells have significant implications for understanding brain functions and could potentially guide the development of more advanced AI systems.

Multiple Models for One Object:

Jeff Hawkins presents the “Thousand Brains Theory of Intelligence,” emphasizing that the brain does not contain a single model for any given object like a coffee cup. Instead, there are thousands of complementary models distributed across different cortical columns.

Sensory and Motor Integration:

These models don’t just represent how an object looks, but also how it feels, implying the crucial role of sensory-motor integration in understanding the world. Movement isn’t just optional; it’s necessary for learning and model-building.

Model Voting Mechanism:

In this paradigm, most of the brain’s connections are employed for “voting” between different models in the cortical columns. These votes contribute to a holistic understanding of objects or situations.

Hierarchy of Models:

While there is still a hierarchy in the brain, it is not merely a hierarchy of features. The hierarchy comprises “models of models,” allowing for complex, structured understanding of the world.

Implications for Artificial Intelligence:

Hawkins argues that true machine intelligence must operate on the same principles as the human brain, which includes a distributed model, sensory-motor integration, and a voting mechanism among models. According to him, most AI today lacks these critical components and thus fails to possess a structured understanding of the world.

Future of AI:

Hawkins is confident that his principles will be the foundation of AI technology in the 21st century. His company, Numenta, is working to implement these ideas into machine intelligence.