Mead’s Start in Electron Tunneling:
Carver Mead began his research in electron tunneling in solids, exploring the ability of electrons to flow through insulators when made thin enough.

Moore’s Law and Tunneling Limits:
Consulting for Gordon Moore, Mead learned about Moore’s law and its implications for transistor miniaturization. He realized that electron tunneling would eventually limit the size of transistors.

Automated Design and Large-Scale Circuits:
To address the challenges of designing smaller and more complex circuits, Mead developed automated design methods and taught courses on large-scale circuit design.

Considering Costs in Computing:
Mead recognized that computing costs involve silicon area, operation time, and energy consumption, requiring trade-offs and optimization.

The Medium of Silicon:
Mead viewed silicon as a medium with unlimited potential, allowing for efficient computation through data movement optimization.

Pipeline Designs for Efficiency:
Mead and his colleagues developed pipeline designs to improve computational efficiency, reducing energy and silicon area requirements.

Seeking a New Paradigm:
The limitations of standard computer designs led Mead to search for a new way of thinking, inspired by the brain’s capabilities.

Brains as Examples of a New Medium:
Mead recognized the brain’s ability to perform complex tasks despite its slow speed and low density, suggesting a new paradigm for computing.

Borrowing from Biology:
Mark Anderson proposes using biological evolution as a model for creating efficient solutions to problems. By studying evolution, we can learn from the efficient solutions that nature has developed over time.

Intelligence Distribution in Mammals:
Carver Mead emphasizes the distributed nature of intelligence in mammals. Intelligence is not concentrated in a central data center but rather spread out across the sensory input and motor output systems. This distributed intelligence allows for efficient processing and response to stimuli.

Applying Biological Lessons to Self-Driving Cars:
Mead draws a parallel between the distributed intelligence of biological systems and the challenges of creating self-driving cars. He argues that self-driving cars need to have intelligence embedded in their sensors and motor systems to effectively navigate their environment.

Mead’s Interest in the Retina:
Mead’s interest in the retina stems from its role as a piece of the brain located behind the lens. The retina processes sensory information and is an example of distributed intelligence in the biological system.

Carver Mead on the Brain’s Image Processing:
There is a significant amount of brain activity in the image plane, suggesting that this is a key location for visual processing. Traditional computer inputs, such as large files in memory, do not resemble the continuous stream of photons entering the eye.

The Retina and Dynamic Vision Systems:
Carver Mead’s student made significant advancements in understanding how the retina processes visual information in the 1980s. Dynamic vision systems, or silicon retinas, are inspired by the retina’s structure and function.

Mark Anderson on the Retina’s Complexity:
Mark Anderson highlights the intricate connections and layers within the retina, making it a complex system for processing visual information. The retina’s ability to process a barrage of photons per second and reduce it to a manageable amount of data is remarkable.

The Difficulty of Replicating the Retina’s Function:
Replicating the retina’s functionality with artificial systems remains a challenge due to its complexity and the vast amount of data it processes. The retina’s efficiency in processing visual information is still a source of inspiration for researchers in the field of computer vision.

Neural Coding:
Neural signals are transmitted as nerve pulses, or spikes, with information encoded in the relative timing of arrival on different channels. This coding mechanism allows for efficient data compression and event-driven processing.

Event-Driven Processing:
Real-world interactions are often event-driven, meaning that relevant information is contained in changes or events rather than in static data. Traditional computing systems struggle with event-driven processing due to the need for large memories and complex algorithms.

Neuromorphic Sensors:
Recent advancements have led to the development of neuromorphic sensors that output nerve spikes, providing a native source of information in a format that is compatible with the brain’s processing mechanisms.

Pattern Recognition:
Pattern recognition is closely related to neural coding and is essential for understanding the world around us. The brain’s ability to recognize patterns allows it to make decisions and take appropriate actions based on limited and noisy information.

Robustness and Adaptability:
The nervous system exhibits remarkable robustness and adaptability, allowing it to function effectively over a wide range of conditions and sensory inputs. This adaptability is particularly evident in the ability of neurons to adjust their response properties based on experience.

Deep Secrets of the Nervous System:
One of the key secrets of the nervous system is its ability to amplify and de-amplify signals over a wide range, allowing it to process information effectively in both bright and dark conditions. This capability is challenging to replicate in traditional computing systems due to the risk of exponential explosion of data.

Background:
Carver Mead is a prominent scientist known for his deep understanding of science and recognizing the potential of scientific discoveries to benefit chip design and humanity. He started with scientific curiosity, like tunneling, and then explored how it could benefit chip technology and society.

Biological Inspiration in Chip Design:
The future of chip design may be influenced by biologically inspired ideas and pattern recognition. Neuromorphic systems can learn from biological nerves, particularly how they process patterns in real-time. Real-time input sources are crucial for developing pattern recognition systems.

Patterns in Nature and Computing:
Patterns inherent in nature are ideally suited for pattern recognition and computation systems. Spatial-temporal patterns, including time-based patterns, offer rich information for pattern recognition.

Sensory Inspiration and Bio-Design:
Carver Mead’s work includes sensory inspiration, such as hearing and vision. He explored cochlear mimicry and built devices for hearing assistance. He also investigated the retina and visual assists, leading to the development of the Foveon chip.

Motivations for Bio-Design:
Carver Mead’s motivations for bio-design were inseparable. He was inspired by the beauty of nature’s designs and believed that understanding and mimicking them could lead to groundbreaking technologies.

Importance of Practical Implementation:
Carver Mead emphasizes the importance of building and testing ideas to truly understand them. He believes that theoretical understanding is insufficient without practical implementation.

Event-Driven Computation:
Event-driven computers, also known as spiking computation, are gaining attention. Even Intel has ventured into this field with their Loihi chip. Event-driven computation requires a different mindset, programming techniques, and learning algorithms.

Current Developments and Optimism:
Carver Mead is excited about the current support for event-driven computation and biologically inspired chip design. He believes that this is a great time for researchers and innovators to explore these concepts.

Shifting Focus:
Carver Mead had been away from the field due to a lack of progress, but the True North chip and the subsequent recognition it received brought him back.

Air Force Research Lab Involvement:
The Air Force Research Lab took over the True North project, limiting its availability for purchase.

PRP (Pattern Recognition Process) Evolution:
Mark Anderson and Dharmendra Modha coined the term “PRP” to describe the True North chip’s operation. Mead believes that similar approaches will continue to emerge.

Kwabena Bon’s Contributions:
Kwabena Bon at Stanford has made significant progress in pattern recognition computation, with his latest work being particularly impressive.

Other Promising Projects:
A group in Atlanta is developing chips capable of learning. A group in Zurich, led by Chi-Chi Liu and Toby Delbruck, is driving the development of spike-oriented retinas. Christoph Posch’s company produces advanced neuromorphic chips.

Neuromorphic Advancements:
Neuromorphic chips, inspired by the brain’s neural structure, are gaining traction in pattern recognition.

Overall Outlook:
Mead is optimistic about the future of pattern recognition technology, highlighting several promising projects and advancements.

History of Chip Design and the Influence of Moore’s Law:
The focus of chip design in the past few decades has been on scaling transistors to pack more functionality onto a single chip, following Moore’s Law. This approach allowed companies to beat the competition by simply making their designs smaller and adding more features, without necessarily focusing on overall system design.

The Shift Towards Pattern Recognition Processors and Software-Inspired Chip Design:
The future of chip design is expected to see a shift towards pattern recognition processors, inspired by biological neural networks. Software excellence is likely to play a significant role in driving new chip designs, with important software algorithms being implemented directly into hardware accelerators.

The Gradual Evolution of Technology and the Importance of Small Steps:
Technological advancements often occur through a series of small, incremental steps, rather than sudden breakthroughs. This is because taking small steps allows for parallel exploration of different approaches, reducing the risk of failure compared to attempting a single large leap.

The Scale of Bio-Relevant Computation and the Lessons from Software:
There is a scale at which nerve spikes and recognition of ordered nerve spike subsets become relevant in bio-inspired hardware. The lessons learned from scaling in the software domain may also apply to bio-inspired hardware, suggesting that there is a sweet spot for the scale of computation in neuromorphic systems.

The Magic of Learning and Built-In Patterns in Biological Systems:
Biological systems exhibit a combination of learning and built-in patterns, which is essential for their intelligence. This combination allows biological systems to adapt to their environment and perform complex tasks, such as pattern recognition and decision-making.

The Rapid Intellectual Growth of Babies and the Importance of Early Learning:
Babies experience rapid intellectual growth in their first year of life, acquiring fundamental knowledge and skills. This highlights the importance of early learning and the development of basic cognitive abilities in the first few months of life.

The Difficulty of Teaching Machines Basic Concepts and the Importance of Analog Computation:
Teaching machines basic concepts, such as arithmetic, has proven to be more challenging than expected. This suggests that analog computation, which is more akin to the way biological systems process information, may play a role in solving these challenges.

Analog and Digital Worlds:
Analog and digital technologies are like two worlds, similar to musicians like Neil Young embracing vinyl records amidst a digital age.

Analog and Digital in Bio and Chip Design:
In bio, the nervous system utilizes both analog and digital aspects. Analog values are represented by digital numbers in backpropagation, and computation in memory involves storing analog values in memory technologies.

Analog Advantages in Computation:
Analog computation offers significant advantages in energy efficiency. By using analog values and computation in memory, energy consumption can be reduced by a factor of 10,000 compared to digital implementations.

Analog and Digital in the Nervous System:
The nervous system combines digital spikes in amplitude with analog time dimension. Analog processing occurs locally within neurons and their surroundings, demonstrating the harmonious integration of both analog and digital aspects.

Nature’s Analog Dominance:
Nature is predominantly analog, even though quantum mechanics may offer differing perspectives. This analog nature presents opportunities for mirroring and implementing low-power computation.

No Digital Circuits in Nature:
Interestingly, there are no digital circuits in nature, further emphasizing the significance of analog computation in the design of future technologies.

Digital Circuits:
Digital circuits are analog circuits that are used in a particular way to restore the signal every stage, preventing errors and noise from overwhelming the signal. The digital nature of these circuits lies in their ability to keep the errors and noise from overwhelming the signal.

Neural Networks and Co-evolution:
Neural networks are currently in a similar stage to early chip design, where advancements in hardware (smaller chips) outpaced smarter designs. Neural nets have co-evolved with various bells and whistles (batch norm layer, optimizers, etc.) over the past 10 years, making it challenging to introduce smarter designs. New, smarter methods may not perform as well initially due to the lack of co-evolution with existing bells and whistles.

Overcoming Resistance to Innovation:
Entrenched ecosystems tend to resist innovations that challenge the status quo and require a change in mindset. Successful innovations often require a combination of technological potential, a commercial foothold, and the development of new bells and whistles to support the new approach. The availability of silicon foundries that can fabricate chips with unique designs encourages innovation and experimentation.

The Revolution of Software Algorithms on Chips:
In the 1980s, Carver Mead and Lynn Conway pioneered the concept of using software algorithms to create chips, empowering mere mortals to design their own chips. This revolution was driven by the desire to find a more efficient way to design chips than the manual methods used at the time. The success of this approach led to the development of courses and programs that taught students how to design chips using software algorithms.

AI and Deep Learning:
Carver Mead compares deep learning to a calculator for a much bigger space, emphasizing its role in making humans smarter. Mark Anderson highlights the limitations of neural networks, including the black box problem, declining return on investment, and their inability to engage in creative thinking or make big discoveries.

The Role of AI in the Future:
Mead suggests that deep learning and AI should be seen as tools that augment human intelligence, rather than as replacements for it. Anderson emphasizes the need for explainable AI systems that can provide insights into their decision-making processes.

Upcoming Events at the Conference:
The conference will feature an AI Advanced Day on Wednesday, focusing on advanced AI and related topics. Larry Smart will present on very large scale networks and exascale computing, including their applications in Elon Musk’s Tesla car network. A top-secret project will be revealed on Wednesday at 5 pm, showcasing a real xAI system that works on neural networks and other AI systems.