Questions for the Panel:
Should more research effort be dedicated to comparing large language models (LLMs) and other deep learning models to human intelligence? Can neuroscience help AI, and can AI help neuroscience?

Points to Consider:
The curse of dimensionality: Neural networks seem to avoid the exponential growth in parameters required for representing complex functions. Why is this the case? Common principles of intelligence: Are there fundamental principles that underlie different forms of intelligence, including LLMs and human intelligence? The difference between classical and quantum computers in relation to intelligence.

Neuroscience’s Impact on AI:
Neuroscience’s central influence on AI stems from the idea that extensive networks of simple processes, capable of learning through connection strength adjustments, can perform intricate tasks. This concept is not immediately plausible, but neuroscience motivated the exploration of neural networks.

Specific Influences from Neuroscience:
Smaller details, such as dropout and ReLU activations, emerged from considering how neurons occasionally fail to fire and that sigmoid functions are inadequate neuron models, respectively.

Future Hardware for Fast Weights:
Hinton anticipates the integration of fast weights into AI, where weights are implemented as conductances. This approach requires specialized hardware.

Digital Intelligences May Be Superior:
Hinton’s recent epiphany suggests that digital intelligences, utilizing backpropagation and allowing identical model copies, might be superior to existing AI. They are more compact, energy-efficient, and capable of storing more knowledge with fewer weights.

AI’s Limited Insights for Neuroscience:
Hinton posits that recent AI developments may not significantly contribute to our understanding of neuroscience. We may have already extracted most of the valuable inspiration from the brain, and new AI developments may not directly translate to brain insights.

Language Models and Statistical Efficiency:
Large language models, especially multimodal ones, challenge the notion that the brain is more statistically efficient than artificial neural networks. Previous comparisons between MIT undergraduates and tabular neural networks may have been misleading. Large language models demonstrate similar statistical efficiency to humans in few-shot learning tasks.

Bayes’ Influence on Statistical Efficiency:
Hinton suggests that Bayes’ concept of a rich prior applies to digital intelligences as well, implying that statistical efficiency is not unique to biological brains.

Pietro Perone’s Perspective on AI:
Embodiment is crucial for intelligence, and purely text-based AI is limited. Current AI over-indexes on data analysis but lacks a theory of mind and struggles to interact with the real world. To understand causation, AI systems need to conduct and design experiments. Machines excel in recognition tasks but lack the ability to design and interpret experiments like biologists. AI has advantages like vast information access and parameter sharing, leading to potential superiority in some domains.

Theory in Deep Networks:
The most successful theory for deep networks is dropout, which helps prevent overfitting and improves regularization. Theory development is challenging due to the rapidly changing landscape of AI, with 50,000 researchers experimenting globally. Theoreticians should focus on the most promising targets and collaborate with experimentalists to advance the field.

David Siegel’s Perspective on AI in Finance:
Finance is a challenging domain for machine learning due to its complexity and lack of labeled data. AI can assist in financial decision-making, but human expertise remains essential. The future of AI in finance involves developing explainable and interpretable models, incorporating domain knowledge, and addressing ethical considerations.

Understanding Intelligence:
Neuroscience and AI are crucial for understanding intelligence. The goal is to develop a theory of intelligence that explains how the brain computes intelligence. This theory should go beyond mere prediction and provide a deep understanding of intelligence.

Inspiration from Neuroscience:
Neuroscience has significantly contributed to AI, especially in deep learning and reinforcement learning. Ideas from neuroscience, such as memory replay and episodic memory, have been implemented in AI systems.

The Role of the Internet:
The vast amount of human-curated data on the internet serves as a valuable resource for AI systems. These systems can draw on this data to learn and improve their performance.

Missing Elements in Current AI Systems:
Current AI systems lack proficiency in planning, handling factuality, and episodic memory. There is potential for further inspiration from neuroscience to address these limitations.

The Shift towards Analysis Techniques:
The focus is shifting from inspiration to analysis techniques in understanding AI systems. Neuroscientists can contribute their analysis skills to gain a deeper understanding of AI systems.

The Need for Collaboration:
Collaboration between leading labs and independent academic institutes is essential to advance research in this field. Sharing of resources and data can facilitate progress and ensure safety and performance implications are addressed.

The Urgency of Understanding:
As AI systems become increasingly powerful, understanding them becomes more urgent. Researchers should explore additional possibilities, such as AI systems explaining themselves, to enhance our understanding.

Theory in AI:
Theory can mean different things to different people. Due to the complexity of neural networks, having very precise theory at the level of physics is unlikely. Theory is still useful, as seen in the scaling of parameters, activations, and normalization, which have all been beneficial to AI. More such theories will continue to be useful.

Neuroscience’s Contribution to AI:
Important ideas from neuroscience, such as the concept of neurons and distributed representations, have already been incorporated into AI. There might be a few more significant ideas that can be borrowed from neuroscience, but it requires skill and taste to do so successfully. The brain’s complexity makes it challenging to determine which ideas are incidental and which have potential for AI inspiration. Copying the brain directly is not a viable approach.

AI’s Potential to Aid Neuroscience:
There is growing evidence that the representations learned by artificial neural networks and the brain show similarities in vision and language processing. Studying these neural networks may provide insights into how the human brain works. This line of research holds promise and is worth pursuing.

Practical Contributions from Academia:
Academia can contribute by evaluating and understanding the capabilities of AI models. Assessing their strengths and weaknesses, including their impressive abilities and strange failure modes, is crucial. Gaining insight into future developments in AI, even a year or two in advance, would be valuable. These contributions can be made outside of large companies and would be significant.

Theory: Despite the importance of theory, it is not necessary to have the same level of precision as in physics. Fundamental principles and systematic studies of different forms of intelligence can lead to useful insights. Neuroscience: AI has contributed to a better understanding of language in the brain, challenging Chomsky’s innate language theory. Demis Hassabis suggests applying AI techniques to analyze brain states and asks for better questions in neuroscience research. Better theories can lead to better benchmarks and a better understanding of emerging capabilities. Benchmarks: The creation of the right benchmarks is crucial, requiring theory on emerging capabilities and emergent properties. Jim, a neuroscientist in the audience, emphasizes using AI as predictors to drive experiments and create new phenomena. A shift in mindset is needed to focus on building deeper understanding through platforms and treating AI generators as hypothesis builders.

The Brain vs. Transformers:
The brain is far more complex than transformer architectures, making it harder to study. Studying transformer architectures can benefit neuroscience, providing insights into simpler models of intelligence.

Over-Indexing on Language:
While language is a significant field of interest, it should not overshadow other forms of intelligence across various species. Studying simpler embodiments of intelligence, like C. elegans or fruit flies, can help us understand fundamental principles more quickly.

Rat vs. Human Intelligence:
Different perspectives exist on the relationship between understanding rat and human intelligence. Some believe understanding rats would be close to halfway towards understanding humans, while others believe it would be much less.

Psychophysics in the Discussion:
Incorporating psychophysics into the discussion can provide benchmarks and insights into animal behavior. Psychophysics reflects the behavior of humans and animals in response to physical stimuli.

Psychophysics as a Guiding Principle:
Psychophysics, cognitive science, and behavioral measures are crucial for assessing AI performance. Demis Hassabis emphasizes the importance of rigorous behavioral testing under controlled conditions. Open-source models and accessible resources facilitate rapid experimentation and validation.

Balancing Commercial Goals with Research Needs:
Industry tends to focus on incremental improvements (hill climbing), while academia encourages exploration and innovation. There is an inherent tension between product needs and research goals, but commercial incentives drive continuous AI improvement. Long-term AI research, including safety and alignment, requires a balance of freedom for researchers and top-down guidance.

Rethinking Performance Measurement:
Pietro Perone highlights the challenges in defining tasks and benchmarks for large vision and language models. Simple-minded benchmarks may not capture the ecological value of intelligence. Understanding the ecological fitness of AI systems is crucial for evaluating their performance. Ilya Sutskevar emphasizes the difficulty of measuring performance due to factors like training set size and context-dependent behavior. Geoff Hinton notes the challenges of conducting systematic experiments with modern AI models due to their access to real-time information.

Current Issues in AI Benchmarking:
Benchmarking is challenging due to the lack of well-defined problems with right answers. Many problems depend on context or philosophical frameworks, leading to fuzzy outputs. Incorrect benchmarking functions can lead to models that are optimized for the wrong things.

The Need for Theory and Data:
Theory is crucial to avoid over-reliance on data-driven approaches, especially when data is imperfect. Incorrect data can lead to undesirable outputs, highlighting the importance of careful data selection.

Bringing Large-Scale Experimentation to the Ground:
Access to large-scale models from leading labs is essential for advancing alignment research. Collaboration between academia and industry can facilitate robust experimentation at scale.

Open Science and Model Access:
Open science and open access to models are vital for progress, but must be balanced with concerns about bad actor use cases. Striking the right balance between open science and responsible AI development is a complex challenge.

Future Research Directions:
Developing new theories and meta-theories of learning to address the limitations of current AI systems. Conducting psychophysical experiments at scale to better understand and align AI systems with human values. Exploring embodied AI and the integration of language and motion for more complete AI systems. Addressing the challenges of open science and responsible AI development in collaboration with government and other stakeholders.

Evolution and Efficiency of the Brain’s Design:
The brain’s complexity, with its diverse neuronal and glial cell types, may be a result of evolutionary optimization. This intricacy might not be essential for intelligence; a simpler system with fewer neuronal types could achieve similar results.

Inspiration from Neural Diversity:
AI models have incorporated elements of neural diversity, like layer normalization, inspired by inhibition in the brain. However, the extent to which this diversity is necessary for intelligence in AI systems is still uncertain.

Exploration of Existing AI Models:
Trained neural networks, particularly large open-source models, may already exhibit interesting neural types. Investigating these models could provide insights into the relationship between neuronal diversity and intelligence.

Current Applications of AI in Science:
AI has been successfully applied to fields such as biology, chemistry, and fusion, demonstrating its versatility and potential for scientific discovery. AlphaFold, developed by Demis Hassabis, stands as a prime example of AI’s impact in biology, enabling the prediction of protein structures with remarkable accuracy.

Challenges for AI in Science:
Despite significant progress, AI still faces limitations in terms of creativity and out-of-the-box thinking. While AI systems can interpolate and extrapolate data, they struggle with invention and creating truly novel concepts.

Levels of Creativity in AI:
Demis Hassabis proposes three levels of creativity in AI: interpolation, extrapolation, and invention. Current AI systems exhibit interpolation and extrapolation capabilities, but lack the ability to invent or come up with groundbreaking ideas.

Benchmarking Creativity:
The challenge of benchmarking creativity in AI systems remains a significant obstacle in evaluating their progress. Without proper benchmarks, it becomes difficult to measure and compare the creative capabilities of different AI models.

The Future of AI in Science:
While AI may not currently possess the full range of human creativity, Demis Hassabis believes that future systems will achieve this milestone. Continued research and development in AI algorithms hold the promise of unlocking even greater creative potential, leading to groundbreaking scientific discoveries.

Backpropagation in the Brain:
The brain likely does not use backpropagation through time, as it requires multiple cortical areas. Big language models store more information per connection than the brain, suggesting a different learning mechanism. Hinton suspects the brain may be using a simpler learning algorithm than backpropagation.

Future Architectures Beyond Transformers:
The next architecture after transformers is unknown, and researchers are secretive about their ideas.

Breakthroughs in Neuroscience:
Understanding how the brain learns could have a significant impact on machine learning. Backpropagation and gradient descent have been critical to the success of neural networks, but their biological plausibility is questionable. Identifying how the brain performs learning without backpropagation could lead to new AI algorithms.