Carver Mead (CalTech Professor) – Semiconductors (Apr 2014)

Chapters

Abstract

“Revolutionizing Computation: The Journey Beyond Digital Systems to Exponential Problem Solving”

In the rapidly evolving landscape of computation, a striking shift is emerging from traditional digital systems to pioneering approaches inspired by quantum mechanics and the intricacies of the nervous system. Key to this transformation is the quest to solve intractable problems, those that elude the grasp of current computing capabilities due to their exponential complexity. Renowned scholar Carver Mead envisions a future where computation transcends conventional boundaries, harnessing the potential of ultra-parallel structures, neural computing, and quantum computing. This article delves into the core ideas shaping this future, exploring the limitations of digital systems, the promise of biological systems as models, and the groundbreaking possibilities of quantum computation, all converging towards a new paradigm of real-world interaction and complex problem solving.

Turing Machines and Computable Functions:

The foundation of computation lies in the concept of the Turing machine, a theoretical model that encapsulates the belief that any computation can be executed within a reasonable timeframe. This idea extends into computable functions, focusing on the time and space efficiency of algorithms, an area scrutinized by complexity theory. The assumption of similar computational capabilities across computers forms the basis for this analysis.

Intractable Problems and the Exponential Challenge:

However, the field of computation faces a critical challenge with intractable problems, particularly in optimization and perception. These problems are characterized by an overwhelming number of alternatives, rendering their efficient resolution elusive. This exponential complexity remains a formidable hurdle, despite significant technological advancements. The difficulty in replicating the perception abilities of even simple animals using computers is a stark example of this challenge, and it has become an embarrassment for computer scientists.

Envisioning a Different Kind of Machine:

To address these challenges, Carver Mead proposes a radical shift towards machines whose computational capabilities increase exponentially with size, a stark contrast to the linear or sub-linear growth observed in current computers. Current predictions in speech recognition, artificial vision, and AI continue to fail due to exponential problem complexity.

Candidate Structures for Exponential Computation:

In pursuit of this exponential computation, various structures are being explored. Ultra-parallel VLSI structures, neural computing architectures, and quantum computing models stand out as promising candidates. Progress in designing and fabricating ultra-parallel VLSI structures has been made, with success in achieving working silicon implementations. However, speedup is limited to polynomial or linear growth with increased resources.

Challenges in Ultra-parallel VLSI Structures:

Ultra-parallel VLSI structures, while promising, face limitations in achieving exponential computation due to their polynomial or linear speedup in relation to hardware growth. The complexity of design and implementation further constrains their potential.

The Quest for Exponential Computation:

Mead’s work in this field underscores the ongoing efforts to overcome the limitations of current computing systems, driving the exploration of alternative computing paradigms to efficiently solve intractable problems.

Digital Systems:

Digital systems, which represent information using discrete symbols, have been a cornerstone since the invention of the alphabet. They offer advantages like noise immunity, error correction, high-speed processing, and efficient long-distance transmission. Digital systems consist of logic gates, combinatorial logic, and sequential logic, which allow for state storage and sequential behavior. A trade-off exists between speed and power consumption in digital systems.

Limitations of Digital Systems:

However, digital systems are not without limitations. They struggle with finite precision, quantization error, and aliasing. These constraints hinder their ability to represent continuous variables and complicate the processing of exponential problems. Digital information must be stored and transmitted physically, subject to the laws of physics. Repeated processing can lead to information degradation and loss due to noise and interference.

Alternative Hypotheses and Biological Systems:

To address these limitations, alternative approaches are required. Biological systems, particularly the nervous system, demonstrate an elegant handling of exponential problems through analog representations and continuous processing. The nervous system’s ability to track a moving target efficiently illustrates this capability.

The Marvelous Computation of the Nervous System:

The nervous system offers a starkly different computational model from traditional digital computers. With neurons capable of receiving inputs from myriad sources and encoding information in the timing and distribution of nerve pulses, it exemplifies a system maintaining numerous possibilities simultaneously, enabling exponential computational capability. Ramon Quijal’s book on “The Neuron” provides a detailed exploration of this fascinating subject.

Challenges in Emulating the Nervous System:

Replicating the nervous system’s computational prowess remains challenging. While progress has been made in mimicking specific functions, achieving self-tuning and continuous learning in artificial systems is a significant obstacle.

A New Computing Paradigm Inspired by the Nervous System:

Inspired by the nervous system, a new computing paradigm is emerging, focusing on predicting future inputs and extracting relevant information from real-time data through layers of artificial neurons.

Quantum Computing as an Alternative:

Quantum computing emerges as another frontier, with qubits enabling multiple states simultaneously and offering exponential computational capabilities. It presents a novel approach to tackling complex computation challenges. Quantum computing involves manipulating electron wave functions, but practical implementation remains elusive. Coupling of wave functions with external elements poses a major challenge, leading to information loss.

Future of Computing:

The future of computing, as envisioned by Mead, transcends digital systems. It anticipates a diverse array of computing paradigms emerging over the next 50 years, promising richer interaction with the real world and a deeper understanding of complex problems. This vision includes the emergence of natural systems like the nervous system and the uncharted territory of quantum mechanics, marking a significant leap in the journey of computational science.



Carver Mead’s vision for the future of computing extends beyond the digital field, embracing natural systems like the nervous system and the uncharted territory of quantum mechanics. This evolution holds immense promise for solving complex problems and revolutionizing our interaction with the world, marking a significant leap in the journey of computational science.

Supplemental Update:

Neurons: A Structure Unlike Standard Digital Computers

– Neurons, the building blocks of the nervous system, have an enormous number of inputs compared to traditional logic gates and encode information in the relative timing of nerve pulses, a continuous variable.

– They perform distributed amplification, ensuring signals arrive at junctions with the right velocity, allowing them to keep alive an exponential number of possibilities simultaneously, leading to exponential computational capability.

Challenges in Building Artificial Neurons

– Tuning the gain and maintaining the right balance between signal amplification and quantization is crucial.

Progress in Building Artificial Neural Systems

– Researchers have made progress in building artificial neural systems, including retinas, motion detectors, cochleas, and learning systems, which operate in continuous time and can learn and adapt as they receive input.

A Possible Computing Paradigm Inspired by Neurons

– A potential computing paradigm involves an input layer receiving real-time data, a network of artificial neurons predicting the next input, a comparison between the prediction and the actual input, correction of the model, and sending the output to the next level.

Quantum Computation Concepts

– Quantum computation utilizes a quantum system where the phase of the electrons’ wave function is preserved.

– Information is encoded in the wave function of electrons, and encoding in the wave function of multiple electrons creates a collective system with an exponentially larger space for evolution.

– Observing the time evolution of this collective system allows for computations with exponential character, such as factoring large numbers.

Challenges in Building Quantum Systems

– Maintaining the coherence of the quantum system and overcoming technical difficulties pose significant challenges.

Carver Mead’s Insights on the Future of Computing

– Mead believes physical principles, such as those in quantum mechanics, will eventually be embodied in physical reality, leading to exciting advancements in computation.

– He anticipates the recognition of other forms of computation, such as those in the nervous system and quantum systems, and envisions a future where computing paradigms extend beyond digital systems, addressing real-world problems and integrating inputs from the natural world into cyberspace.

Notes by: MatrixKarma