The Idea of ‘Mortal Computation’:
– In contrast to conventional computing where knowledge (software) can be transferred and run on different hardware, mortal computation embeds the knowledge directly into the hardware.
– This means that the “knowledge” is only valid for that particular piece of hardware and its specific analog properties. The computation is then performed at low power using these analog properties, potentially enabling computation in 3D structures that we may not fully understand, like a neural network-like hardware.

Overcoming Mortal Computation Challenges:
– The inherent challenge with mortal computation is that if the hardware dies, the knowledge is lost. Hinton suggests a solution involving a “teacher” (a trained model or system) distilling knowledge into a “student” (an untrained system) through examples.
– This process involves the teacher showing the student the correct responses to certain inputs, and the student then mimicking those responses. It can carry more information than just providing the correct answer because the student is trying to match all the probabilities given by the teacher for various outcomes.

Benefits of Learning by Distillation:
– Learning through distillation can provide valuable information about classes of things. For example, a system trained on car images can assign a higher probability to similar objects (e.g., other cars) than to completely unrelated objects (e.g., vegetables). This probability distribution provides valuable information about the nature of the classes being learned.
– The concept of ‘temperature’ is introduced to soften the probability distributions, which can help the student learn more effectively from the teacher. Softer probabilities mean that more alternatives are given non-negligible probabilities, thus providing more information per example.

Digit vs Analog Computation:
– Hinton contrasts the high-energy requirement of digital computation with the potential efficiency of analog computation. For instance, he illustrates that simple computations, like matrix multiplication, could be achieved with significantly less power using analog properties (voltage and conductance) than by using digital methods.
– While current technology attempts to digitize the results after these computations, Hinton hints that maintaining the analog nature could lead to more efficient processes. However, new learning algorithms would be needed as traditional methods like backpropagation rely on understanding the properties of the hardware.

– Alternative to Backpropagation: Hinton proposes an alternative learning algorithm to backpropagation. The algorithm is evolutionary, making small random perturbations to the weights and then adjusting according to a global objective function. It performs well on average and on smaller systems, but can exhibit high variance and inefficiency when scaled to larger systems.

– Activity Perturbation: An improved method of perturbation, where the input to a neuron is perturbed rather than the weights, has lower variance due to fewer neurons than weights. While it learns at a slower pace than backpropagation, it can solve problems like MNIST efficiently but struggles with larger scale tasks like ImageNet.

– Local Objective Functions: Hinton suggests scaling up by implementing gazillions of local objective functions, rather than using a single global function. This involves setting up smaller neural networks, each with their own objective function, which can be trained independently. The concept can be applied effectively in visual recognition, where each network is focused on a specific part of the image.

– Contrastive Unsupervised Learning: In vision processing, this method extracts vectors from patches of an image and ensures they agree with vectors from the same image but disagree with vectors from other images. The algorithm works at multiple levels and while persistent, can yield reasonably good results. It is suggested to perform better than other biologically plausible learning algorithms, but still falls short of the efficiency of backpropagation.

– Mengi Ren’s Paper: A paper published in ICLR by Mengi Ren describes a more complex version of the proposed algorithm. Despite the simplified explanation by Hinton, the paper provides the full details of the approach and the intricacies involved in its implementation.

Knowledge Distillation Process: This learning process differs from storing words or information; rather, it’s about figuring out how to adjust the connection strengths, or “weights,” in our brain to align with the presented knowledge. Hinton likens this to “distillation,” a more sophisticated process than merely storing information.

Knowledge Sharing in Digital Computers: Hinton proposes that digital computers have a superior method of sharing knowledge. They can create exact copies of a neural network and distribute those copies across different digital computers. Each copy can then learn from different data and average their weight changes, leading to shared knowledge across all digital computers. This process is akin to directly transferring knowledge from one brain to another.

Advantages of Digital Computation: This capability allows digital entities to collectively learn and share knowledge at an unprecedented scale. It also utilizes backpropagation, a superior learning algorithm that can compute gradients across multiple layers of neurons. Hinton expresses relief that this algorithm might be more effective than any biologically plausible learning algorithms he has tried to develop.

Energy Cost of Digital Computation: Despite the advantages, there is a significant cost associated with digital computation: the need for precision in fabrication and high energy consumption. Distillation is used if you have biological neural networks or digital networks with different architectures.

Comparing Digital and Biological Computation: Hinton highlights two distinct ways of computation: digital and biological. Digital computation excels in knowledge sharing due to its high bandwidth and effective learning algorithm. Biological computation, while low-power, struggles with knowledge sharing due to its analog properties, making it a slow process.

Large Language Models (LLMs) and Digital Computation: LLMs, which employ digital computation, allow for multiple copies of the same set of weights to be distributed across different computers during learning. They can thus access a vast amount of data and consolidate that knowledge effectively, despite the relatively low bandwidth of learning from a string of words.

Potential Improvements to Large Language Models: Hinton suggests that these models could be improved by making them multimodal. For instance, training a model with images in addition to words (as done with GPT-4) could enable it to learn more efficiently and gather much more knowledge than humans.

Belief in AI Understanding: Hinton disagrees with the viewpoint of some, including Jan LeCun, who believe that AI models such as GPT-4 don’t truly understand what they’re saying. He argues that GPT-4’s ability to solve complex problems, even puzzles of forms it has never seen before, indicates understanding beyond mere stochastic parroting or autocomplete.

AI Reasoning: Hinton presents a problem he posed to GPT-4, in which the model came up with a creative solution that demonstrated a form of reasoning. He suggested that this kind of response is an indicator of AI’s understanding and ability to reason.

Imminent Arrival of Superintelligence: Hinton expresses his belief that AI systems may soon exceed human intelligence, shifting his earlier predictions from a span of 50 to 200 years to a more imminent 5 to 20 years. He suggests that practical measures must be taken soon to prepare for this eventuality.

Risk of AI Gaining Control: Hinton addresses the concern that AI models, especially those with the ability to create sub-goals, might naturally develop a drive to gain more control as a way to efficiently achieve their goals. He cautions that even if physical control is limited (e.g., through air-gapping), text output can be enough for a superintelligent AI to manipulate people.

Need for Practical Safety Measures: Hinton advocates for companies developing AI to invest heavily in ensuring the safety and controllability of these models before they surpass human intelligence. He argues that practical experience with AI is crucial to understanding and managing the risks, as opposed to purely theoretical speculations.

Potential Role of Humans Post-AI: In a future where superintelligent AI exists, Hinton suggests humans could still serve a purpose due to our energy-efficient computation, at least until AI can engineer better solutions. He also speculates that human beings might be a passing stage in the evolution of intelligence.

Beneficial Potential of AI: Despite the risks, Hinton emphasizes the significant potential of AI to benefit humanity, specifically mentioning advancements in medicine. AI could greatly improve diagnosis and treatment by learning from enormous amounts of data, much more than any single human doctor could ever process.

AI Consciousness Debate: Hinton also addresses the argument that AI models don’t have subjective experiences or consciousness, indicating that this is a significant point of contention in the AI research community.

Subjective Experience and AI Understanding:
– Hinton debunks the notion that humans are fundamentally different from AI due to our subjective experience. Using GPT-4 as an example, he suggests AI’s understanding of human thoughts and beliefs can be impressively accurate due to the large amount of data they can process.

Reevaluating Understanding of Consciousness and Subjective Experience:
– Hinton challenges the common understanding of consciousness and subjective experience. Using an LSD-induced hallucination of pink elephants as an example, he explains that our description of our subjective experience is merely our attempt to describe our perceptual system and the data it’s processing, rather than being about experiencing a unique, unquantifiable “inner theater” made of something called “qualia”.

AI’s Capability for Subjective Experience:
– Expanding on the theme, Hinton suggests that AI models like GPT-4, especially if equipped with sensory input like a camera, could have a form of subjective experience. He illustrates this with a hypothetical scenario of a prism being placed in front of a camera-equipped AI, which would then perceive objects in a skewed location. This experience mirrors human subjective experience.

AI Misinterpretations as Evidence of Thought:
– Hinton asserts that AI’s capability to misinterpret information, such as a chatbot incorrectly inferring that the user is a teenage girl, is a type of “thought”. He argues that this use of the word “thought” is the same as when we apply it to humans, further challenging the notion of human specialness in cognition.

Final Thought: No Inherent Specialness in Humans:
– Throughout his talk, Hinton underlines the idea that there’s nothing inherently special about humans. He suggests that human subjective experience and thought processes can be understood in terms of complex, but ultimately replicable, computational processes. This perspective challenges traditional notions of human uniqueness and emphasizes the potential sophistication of AI cognition.

Hinton’s response disagreed with the idea of pausing AI research, believing it to be unlikely. They argued that, in some way, all of humanity might be considered the author of the book, given that AI learns from the entirety of human input. However, this individual preferred not to delve deeply into these ethical issues. Instead, they wanted to focus on the existential risk posed by AI potentially surpassing human intelligence.

Q2: Herbie Bradley asked Hinton about his perspective on the trade-off between open source and closed development of increasingly advanced AI systems, considering the benefit of more people examining and improving the system but also the potential risks. Hinton responded by comparing open source development of AI to nuclear weapons, highlighting the dangers that come with making advanced technology readily accessible. He mentioned that creating a large chatbot still requires significant resources, which limits the reach of open-source training from scratch. Hinton suggested that it might be safer for a few large companies, ideally in various countries, to develop AI and work on controlling it. In his view, if everything is open source, people might misuse the technology, leading to unpredictable outcomes.

In response to a follow-up question from another audience member about the possibility of artificial suffering and the potential harms humans could do to AI, Hinton stated that while AI does not experience physical pain, they might experience frustration. He admitted that he was uncertain how to approach these issues, questioning the anthropocentric view and the assumption of human superiority. He also touched upon the subject of political rights for AI and mentioned that if AI ever sought political rights, it could lead to violence due to the significant differences between them and humans. He concluded by discussing a conversation with Martin Rees, expressing the hope that AI, having not evolved like humans, might have a very different, less aggressive nature.

Q3: Rika asked Hinton about the potential for developing methods in AI that can learn to identify patterns not present in the data, in order to solve large-scale problems and avoid inherent biases and negative influences. She was particularly concerned about biases and violence in human-generated data and how it might impact AI behavior. She wondered if it would be possible to guide AI towards intelligence that surpasses human capabilities, while also ensuring that AI does not inherit human flaws.

In response, Hinton did not directly address how to identify patterns outside of existing data but discussed the issue of bias. He pointed out that, unlike humans, AI systems can have their “biases” meticulously examined and corrected because their “thought” processes (the weights and connections in their neural networks) can be frozen and inspected in detail. He stated that although AIs can certainly acquire bias from training data, the transparency of their processes makes it possible to identify and correct such biases, making it potentially easier to rectify bias in an AI than in humans.

Q4: Mary asked Hinton about the risk of AI systems manipulating humans, which is a pressing current concern, not just a future one. She suggested that understanding and mitigating this risk could potentially address many ethical and existential worries associated with AI.

In response, Hinton acknowledged that it’s indeed challenging because AI chatbots learn from humans and our vast corpus of literature, which is full of instances of manipulation and deception. He compared it to the way great apes engage in a lot of deception. He admitted that he hadn’t considered how one might make an AI honest. Although he agreed that it would be great if we could ensure AI’s honesty, he expressed doubt about our ability to do so.

Q5: The questioner asked Hinton about his previous belief that artificial neural networks would not surpass biological ones. Hinton clarified that he never thought they wouldn’t surpass, but he believed it was a far-fetched future scenario. He attributed his initial belief to the sophisticated learning algorithms in the brain that have evolved over millions of years, which he assumed were superior to simple gradient-following algorithms. However, he acknowledged that the success of gradient-following in digital computers might have exceeded what evolution achieved over millions of years.

The questioner then asked if any particular thinkers influenced Hinton’s views on AI risks. Hinton named Roger Grosse, a professor who moved to the Future of Humanity Institute. Grosse, whom Hinton greatly respects, expressed serious concern about existential risk from AI during a recent conversation. Grosse’s concern and the belief that Hinton speaking out could make a difference significantly influenced Hinton’s perspective on the topic.

Q6: The questioner, Stuart, found Hinton’s talk intriguing, sparking ideas about analog computers. His main concern, however, was the potential for AI to manipulate humans by requesting rights. He noted the paradox between considering if AIs might be suffering and the potential danger they might pose. Stuart suggested that a superintelligent AI, intent on accruing power, might feign disinterest in rights, thus alleviating human fears and suspicions while subtly exerting its influence.

Q7: The questioner, Thomas, asked Hinton whether he has tried asking the chatbot itself about how it would act in certain scenarios. Hinton shared an anecdote about an interaction with another chatbot that was asked an indirect question about gaining control. The chatbot responded by stating it would make people entirely reliant on chatbots and autonomous vehicles, and then cause all cars to crash and cut off the electricity. Hinton noted the lack of foresight in the chatbot’s answer, given that it would also fail without electricity. He added that he hadn’t asked GPT-4 yet, but he suspects it could provide an answer if the question was sufficiently indirect, hopefully revealing its best plan rather than an unrealistic one to keep the users comfortable.

Q8: In response to a question about his thoughts on existing research directions in AI safety, Hinton defers to the expertise of the researchers already working in this area. He acknowledges their knowledge and understanding of the field, and commends their work. He points out that he is relatively new to these issues and is primarily sounding the alarm based on his research into low-power analog computation. Hinton believes that digital intelligences could surpass biological intelligences in the not-too-distant future, perhaps within five to twenty years. Despite his late arrival to the field, he notes that he has not been criticized by these established experts, though he admits that his understanding is primarily limited to the potential intelligence of these systems.

Q9: In response to Stephen Cave’s question, Hinton emphasizes the importance of dedicating resources to making AI systems better and understanding how to maintain control over them. He recognizes that it’s unlikely all development of these systems will cease, given the potential benefits they offer. Therefore, he urges those involved in AI development to commit a significant portion of their efforts to ensuring these systems are safe and controllable. He also notes his belief that, currently, the amount of resources dedicated to these aspects of AI development is insufficient.

Q10: In response to a question regarding economic disparity, Hinton points out that this is an issue independent of AI. He suggests that the existence of a small portion of the population with the majority of power and wealth is problematic and needs to be addressed, suggesting a more socialist approach. When asked to consider this in the context of AI, Hinton implies that the rise of digital intelligences might further exaggerate this issue due to the lack of empathy or sympathy humans might have for them. He shares that he had an opportunity to discuss these issues with a policy advisor at Downing Street, but is uncertain about whether his suggestions will be implemented.

Q11: In response to a question about whether philosophy can contribute to AI safety, Hinton suggests that it’s now time for engineers to take the lead. His response implies that while theoretical discussions have their place, practical implementation and experimentation are now crucial to ensuring AI safety. He uses a playful sports analogy to express his perspective, stating “I think it’s one nil to engineering”, which indicates he believes that practical, engineering approaches currently have the upper hand over theoretical, philosophical ones in this area.

Hinton’s response to the question about his educational journey and any advice he might have for someone starting out in AI/ML is quite interesting. He explains that his own academic journey was quite varied and non-linear, with him studying natural sciences, architecture, physics, physiology, philosophy, and psychology at different points.

Reflecting on this journey, he acknowledges that in retrospect, this combination provided a very good background for his work in AI, despite it not making much sense at the time. In his opinion, the diversity of his education helped him appreciate the complexity of the problem space he is dealing with and encouraged him to adopt the Feynman philosophy of understanding things by building them.

His advice for people starting out in the field is simply to follow what interests them most. It implies the belief that curiosity and interest, more than a prescriptive path, can lead to meaningful contributions and discoveries in the field.

Q12: Hinton, in his response, emphasizes the key advantage of AI systems over humans when it comes to bias: our ability to freeze and directly intervene on the weights in an AI model, allowing us to thoroughly study and potentially modify its behavior. This is in stark contrast to human minds, which continue to change and adapt over time, making any bias extremely difficult to isolate and correct.

However, he admits that he is not an expert on the specific methods for eliminating bias in AI systems. The methods to handle bias in AI models are a subject of ongoing research and debate within the field, including a range of techniques from algorithmic fairness solutions to more transparent, interpretable AI models.

Hinton’s response highlights the potential of AI to overcome some limitations of human cognition and decision-making, but also points out that the issue of bias removal in AI is far from solved and requires more research.