The Royal Society’s Role:
The Royal Society, with a history spanning over 350 years, has been instrumental in scientific discoveries and practical applications. In April 2017, the Society released a series of reports on various aspects of digital technology, including cybersecurity, machine learning, and teaching computer science.

Machine Learning Report:
The Society’s report on machine learning called for action in several areas to ensure that the benefits of this technology are widely felt. The report emphasized the importance of careful stewardship of machine learning data, encouraging public debate on the technology’s benefits and potential risks.

Lecture Series Objectives:
The series of events and lectures aims to foster a public conversation about machine learning and AI. It provides a platform for discussing the impact of these technologies on society, both positive and negative.

Demis Hassabis’ Background:
Demis Hassabis, the speaker for the first lecture, is an accomplished figure in the field of machine learning and AI. He holds degrees in computer technology and neuroscience, and co-founded DeepMind in 2010. DeepMind was acquired by Google in 2014 and has grown significantly since then.

Hassabis’ Recognition and Achievements:
Hassabis has received numerous accolades, including being named one of the 100 most influential people in the world by Time magazine. He was awarded a CBE (Commander of the Order of the British Empire) for his contributions to science and technology.

AI and Public Engagement:
Hassabis believes it is crucial to facilitate public debate and engagement between AI researchers and the broader public. He emphasizes the importance of open and robust conversations about AI’s potential and pitfalls as it increasingly impacts our lives.

AI’s Potential in Scientific Discovery:
Hassabis is passionate about harnessing AI’s capabilities to advance science itself. He believes that AI can help drive scientific progress by providing powerful tools for discovery.

Defining AI and DeepMind’s Vision:
AI is described as the science of making machines smart. DeepMind, Hassabis’ company, aims to push the frontiers of AI by combining top researchers, engineers, and computational power.

Fusing Academic and Startup Culture:
DeepMind’s unique culture blends the best aspects of academic rigor and startup energy. The company seeks to fuse the strengths of both worlds to accelerate AI research.

DeepMind’s Mission:
DeepMind’s mission is articulated as a two-step process. Step one is to fundamentally solve intelligence, and step two is to use it to solve everything else.

Understanding and Recreating Intelligence:
Solving intelligence involves understanding the phenomenon, identifying its key processes, and artificially recreating them. The goal is to make intelligence universally abundant and available.

Building a General-Purpose Learning Machine:
DeepMind aims to create the world’s first general-purpose learning machine. Such a machine would be capable of learning and mastering tasks automatically from raw experience.

Generality and the Example of the Human Brain:
The idea of generality implies that a single set of algorithms can operate across diverse tasks, even those it has never encountered before. The human brain serves as a powerful example of a general-purpose learning algorithm.

Artificial General Intelligence (AGI):
DeepMind uses the term Artificial General Intelligence (AGI) to distinguish it from traditional handcrafted AI. AGI aims to replicate the human mind’s ability to learn, adapt, and solve problems flexibly.

Deep Blue, an Expert System:
Deep Blue, IBM’s famous chess computer, epitomizes rule-based AI. It was pre-programmed with rules and heuristics, making it an expert in chess. However, it lacked adaptability and couldn’t perform tasks beyond chess.

Limitations of Rule-based AI:
Rule-based AI systems like Deep Blue are inflexible and lack general learning capabilities. They are designed for specific tasks and cannot handle unexpected situations or variations.

Hallmarks of AGI Systems:
AGI (Artificial General Intelligence) systems possess flexibility, adaptability, and robustness. They have general learning capabilities that allow them to cope with new tasks and problems.

Garry Kasparov’s Intelligence:
Garry Kasparov’s cognitive abilities extend beyond chess, encompassing politics, languages, and writing. His intelligence is versatile and applicable across diverse domains, unlike Deep Blue’s narrow expertise.

Missing Elements in Rule-based AI:
Rule-based AI lacks adaptability, learning capabilities, and the ability to operate across various tasks. These missing elements hinder the development of true intelligence and AGI.

Reinforcement Learning:
Reinforcement learning is a framework for AI systems to learn through interaction with their environment. Agents observe their environment through perceptual inputs, build a model of the environment, and make plans to achieve their goals. Actions are taken, and the agent receives feedback through new observations. This process continues in a feedback loop, with the agent constantly trying to improve its actions to reach its goal.

Deep Learning:
Deep learning involves hierarchical neural networks, loosely approximating how real neural networks work in the brain. Neural networks are trained using supervised learning, where they are shown thousands of examples and adjust their weights to improve accuracy. Back propagation is used to adjust the weights, making the neural network more likely to give the correct answer in the future.

Deep Reinforcement Learning:
DeepMind combines reinforcement learning and deep learning to create deep reinforcement learning. Deep learning processes perceptual inputs to make sense of the world, while reinforcement learning makes decisions and takes actions towards the agent’s goal. This combination allows for end-to-end learning, where the system can process raw inputs and make decisions without intermediate steps.

DQN: The First End-to-End Learning System:
DQN was the first end-to-end learning system, taking raw pixel inputs and making decisions in Atari games. DQN learned from scratch, without any knowledge of the game or its controls. The DQN architecture includes a neural network that processes the screen input and outputs possible actions. The agent selects the best action based on the current screen input, aiming to maximize its score.

Introduction of Deep Q-Networks (DQN):
DQN is a reinforcement learning algorithm used in Atari games. DQN learns by playing the game and receiving rewards or punishments. Over time, DQN develops strategies to maximize rewards and minimize punishments.

Breakthrough with Breakout Game:
DQN’s initial performance in Breakout was poor. After 100 games, DQN started to grasp the concept of moving the bat toward the ball. After 300 games, DQN became proficient at returning the ball. DQN surprisingly discovered an optimal strategy of digging a tunnel to trap the ball.

Significance of the Breakout Breakthrough:
This was a watershed moment for DeepMind, demonstrating the potential of reinforcement learning. It showcased the ability of AI to discover new strategies beyond human knowledge.

Introduction of AlphaGo:
AlphaGo was a program developed by DeepMind to play the ancient Chinese board game Go. Go is a simple game with two rules but challenging to master. The goal is to wall off areas of the board with stones, and the player with the most walled-off territory wins.

Challenges of Go for Computers:
Go has a vast search space of possible board positions. Explicitly defining an evaluation function for Go is difficult due to its esoteric nature.

AlphaGo’s Breakthrough:
AlphaGo overcame these challenges using two neural networks. One neural network handled the combinatorial explosion and search space. The other neural network approximated the evaluation function.

Conclusion:
Demis Hassabis’s breakthroughs with DQN in Atari games and AlphaGo in Go highlighted the potential of reinforcement learning and neural networks in AI. These achievements opened new avenues for AI research and applications.

Policy Network:
AlphaGo used a policy network to predict the next move a human player would make in a given board position. The policy network was trained on data from strong amateur games downloaded from the internet. The output of the policy network was a probability distribution over possible moves, with higher probabilities assigned to more likely moves. The policy network significantly reduced the breadth of the search tree by focusing on the most probable moves.

Value Network:
AlphaGo also utilized a value network to evaluate the current board position. The value network was trained by playing AlphaGo against itself millions of times. The value network learned to predict the winner of the game and the level of certainty in the prediction. The value network provided a single real number between 0 and 1, representing the probability of white winning.

Combining Policy and Value Networks:
By combining the policy and value networks, AlphaGo overcame the challenges inherent in Computer Go without relying on explicit evaluation functions. AlphaGo learned to play Go through experience, playing against itself millions of times, rather than relying on pre-programmed rules.

Challenge Match Against Lee Sedol:
AlphaGo challenged Lee Sedol, one of the greatest Go players in the world, to a $1 million match in South Korea in 2016. The match attracted immense attention and viewership across Asia. AlphaGo unexpectedly won the match 4-1, defying predictions from experts and Go players. The victory was considered a decade ahead of its time, revolutionizing the field of AI and Computer Go.

AlphaGo’s Creative Gameplay:
AlphaGo surprised the Go world with its unique and creative moves, revolutionizing the game. It played unconventional moves, like move 37 in game two, which was initially considered a mistake but later proved decisive. This move demonstrated AlphaGo’s ability to plan long-term strategies and impact the game’s outcome.

The Importance of Creativity in AI:
AlphaGo’s creativity highlights the importance of AI systems generating their own ideas rather than relying solely on human knowledge. Original moves can significantly impact the game’s result, making them beautiful and impactful.

Lisa Doll’s Reaction and Subsequent Success:
Lisa Doll, the professional Go player who competed against AlphaGo, was gracious in defeat and inspired by the match. He won a game against AlphaGo and went on a three-month unbeaten winning streak in human championship matches. He experimented with new ideas and techniques, showcasing the positive impact of the match on his gameplay.

AlphaZero: Generalizing AI to Multiple Games:
AlphaZero, the successor to AlphaGo, can play any two-player game, including chess and shogi, demonstrating its generalization capabilities. It starts learning from scratch through self-play, eliminating the need for human data to bootstrap its learning.

AlphaZero’s Performance in Chess:
AlphaZero defeated the top chess program, Stockfish, after only four hours of training, starting from random knowledge. This achievement shocked the chess community, as Stockfish was considered unbeatable by humans. AlphaZero’s victory highlights the potential of AI to excel at complex strategic games.

The Future of AI:
DeepMind’s work on AlphaGo and AlphaZero showcases the rapid progress in AI, particularly in the area of strategic gameplay. These advancements have implications for the future of AI, raising questions about the limits of AI’s capabilities and its potential impact on society.

AlphaZero’s Astonishing Performance in Chess:
AlphaZero achieved a remarkable feat, defeating the world’s leading chess engine, Stockfish 28-0 with 72 draws in a 100-game match. It not only surpassed existing AI programs but also demonstrated an astounding level of skill that challenged traditional chess strategies.

AlphaZero’s Unconventional Style of Play:
AlphaZero introduced a new style of chess that emphasizes mobility over materiality, departing from conventional valuation systems. It readily sacrifices material to gain mobility, leading to long-term positional advantages. Chess grandmasters have praised AlphaZero’s human-like style, in contrast to the mechanical and unappealing play of previous computer chess programs.

AI’s Rapid Progress and Ongoing Challenges:
AI has experienced tremendous progress in recent years, but there are still significant challenges to overcome before achieving true artificial intelligence. The journey towards understanding intelligence is just beginning, with many exciting avenues for research and exploration.

Unsupervised Learning:
Unsupervised learning is a significant challenge in AI, as it involves learning without direct feedback or correct answers.

Memory and One-Shot Learning:
True intelligence requires the ability to remember past experiences and use them to inform future decisions. One-shot learning involves learning new concepts from just a single example, which is a challenging but essential skill for AI.

Imagination-Based Planning:
Imagination allows humans to plan for the future by visualizing different scenarios and their outcomes. AI systems need to develop this ability to make effective plans in complex and uncertain environments.

Learning Abstract Concepts:
True intelligence requires the ability to learn and understand abstract concepts, such as language and mathematics. Current AI systems struggle with this task, but it is crucial for achieving higher levels of intelligence.

Transfer Learning:
Transfer learning involves applying knowledge learned in one domain to a new and different domain. Humans are exceptionally good at this, but AI systems need to improve their capabilities in this area.

The Importance of Language:
Language is a key component of intelligence, allowing humans to communicate, reason, and express complex ideas. AI systems need to develop the ability to understand and use language effectively.

AI in Current Scientific Research:
AI systems are being used to analyze data from telescopes to discover new exoplanets. AI is used to control plasma in nuclear fusion reactors. AI assists in solving quantum chemistry problems. AI is helping radiographers quickly triage retinopathy scans to look for macular degeneration.

AI in Protein Folding:
Predicting the 3D structure of proteins from their amino acid sequence is a major challenge in biology. DeepMind is working on a project to apply AI to protein folding. Accurately predicting protein structures could aid in drug discovery and disease research.

General Properties of Problems Suitable for AI:
Massive combinatorial search base. Clear objective function or metric to optimize against. Lots of real data or an accurate and efficient simulator to generate data.

AI Applications in the Real World:
AI is being used in healthcare, education, and personalized education. AI is being integrated into Google Assistant to make it more intelligent.

AI as a Meta-Solution:
AI can help us make sense of the vast amount of data we are confronted with. AI can help us understand complex systems like climate and nuclear fusion. AI can help us extract insights from unstructured data.

Intelligence as Converting Information to Knowledge:
AI can automate the process of converting unstructured information into useful, actionable knowledge.

AI-Assisted Science:
The goal is to make AI-assisted science possible, where AI works in tandem with human scientists. AI scientists could help accelerate scientific discovery and solve complex problems.

AI’s Ethical Considerations and Responsible Deployment:
Demis Hassabis emphasizes the need for responsible AI development and deployment, considering its impact on society and ensuring benefits for everyone. DeepMind has established an ethics and society team to work with stakeholders on ethical AI deployment. The Partnership on AI is an industry-wide collaboration aimed at establishing best practices and protocols for AI research and public engagement.

The UK’s AI Ecosystem and Leadership Role:
Hassabis highlights the UK’s strengths in AI research and development, with world-renowned universities and institutions. DeepMind actively supports the UK’s AI ecosystem through sponsorships, scholarships, internships, and lectures. The UK has a rich heritage in computing and AI, dating back to Charles Babbage and Alan Turing. Hassabis expresses his aspiration for the UK to become a global leader in AI.

Challenges in AI Deployment and Testing:
Hassabis acknowledges the remaining challenges in AI development, including overcoming difficulties with intelligence tasks and achieving breakthroughs. He emphasizes the need for careful testing of AI systems, as they are adaptive and learn continuously, making traditional software testing methods insufficient.

Addressing Irrationality in AI Systems:
Hassabis recognizes the challenge of dealing with human irrationality in AI systems, particularly in areas like economics. He suggests that AI systems may need to model certain aspects of irrationality to understand and interact effectively with human experts and systems.

Public Engagement and Shaping AI’s Future:
Hassabis encourages public involvement in understanding AI technologies and their societal implications. He emphasizes the importance of engaging in discussions about how AI should be deployed for the benefit of society. AI has the potential to be transformative and beneficial if used responsibly, and it could become one of the most significant inventions in human history.

Introduction:
Demis Hassabis offers a balanced perspective on the risks and benefits of AI technology.

Mid-Term Concerns:
Hassabis believes the near-term focus should be on issues such as autonomous weapon systems and the safety of self-driving cars. He emphasizes the need for responsible testing and development of these technologies.

Technological Challenges:
Hassabis acknowledges the challenge of ensuring the safety and reliability of AI systems, especially as they become more adaptable and operate in the real world.

Interpretability and Accountability:
Hassabis highlights the importance of interpretability and accountability in AI systems, especially for safety-critical applications. He mentions ongoing research to develop visualization tools and other methods to improve the understanding and explainability of AI decisions.

Balancing Progress and Risks:
Hassabis emphasizes the potential transformative benefits of AI technology, but stresses the need for careful consideration of its ethical and societal implications.

Governance and Trust:
Hassabis recognizes the importance of governance and trust in AI development and deployment. He acknowledges the tendency for technologies to resist governance, and expresses interest in exploring the lessons learned from past technologies in this regard.

AI Regulation:
Demis Hassabis believes it’s crucial for governments to stay informed about AI advancements and consider the ethical and regulatory implications. Immediate knee-jerk regulations are not advisable, as researchers are still determining the appropriate protocols and control mechanisms for AGI. Regulations should focus on upgrading existing regulations in areas like transport and healthcare to accommodate new AI technologies.

DeepMind’s Role in the Future of AI:
DeepMind aims to play a major role in advancing AI research, particularly in areas like concepts and memory. The company strives to be a beacon for the ethical use of AI, serving as a role model for other organizations in addressing ethical and philosophical questions surrounding AI.

Ethics of AGI and Consciousness:
Hassabis acknowledges the potential ethical considerations related to AGI and consciousness. He raises the question of whether an AGI of sufficient power might require rights if it approaches the border of consciousness.

Practical AI Applications:
Hassabis highlights the need to upgrade regulations for specific deployed AI systems like self-driving cars. Existing regulations in areas like transport and healthcare should be adapted to handle new AI technologies.

Definition of Consciousness:
We don’t have a clear definition of consciousness yet, and there is ongoing debate among neuroscientists and philosophers. However, we all intuitively feel that we have consciousness.

Intelligence and Consciousness as Double Dissociable:
Intelligence and consciousness are not necessarily linked. Animals like dogs and cats may possess some aspects of consciousness, while AI systems like AlphaGo lack it.

Implications of Creating an AGI System Without Consciousness:
If we can create a highly intelligent AGI system that lacks consciousness, it would provide insights into the nature of intelligence. It would help us identify the missing ingredient that distinguishes human consciousness from artificial intelligence.

AI as a Tool to Explore Consciousness:
By collaborating with neuroscientists and other experts, AI can be used as an experimental tool to study consciousness. AI systems can be used to test theories about consciousness and explore related issues like qualia.

DeepMind’s Paradigm for AI Deployment:
DeepMind’s paradigm for AI deployment involves transcending the limitations of existing systems. The company uses a system to determine the paradigm, which helps allocate resources and attention to different aspects of AI, including the AI itself and its relationship with humans.

Organizational Process:
Demis Hassabis compares DeepMind’s approach to agile software project management methods used in computer games and engineering projects. He emphasizes the importance of rapid iteration and collaboration while preserving bottom-up creativity.

Factors for Deployment:
DeepMind considers a variety of factors before deciding where to deploy AI, including social good, fit with current technology, commercial opportunity, and alignment with research goals. Healthcare is prioritized due to its societal significance and the personal motivations of many team members.

Examples of Deployment:
AlphaGo, a variant of the program used for playing the game Go, was successfully applied to control cooling systems in Google data centers, resulting in significant energy savings.

Neuroscience and Deep Neural Networks:
A question from the audience highlights the differences between neurons and synapses in the brain and the elements of deep neural networks. The speaker acknowledges these differences but does not provide a direct answer about the incorporation of these elements into deep nets.

Demis Hassabis’s Perspective on Neuroscience and AI:
Current neural networks are simplified versions of the brain’s complexity. Real neurons are probabilistic and use spike trains for information passing. DeepMind has a neuroscience team exploring inspiration from neuroscience.

Implementation Details and Systems Neuroscience:
Hassabis believes copying all biological implementation details is unnecessary. He focuses on the algorithmic and computational level (systems neuroscience). The line between inspiration and copying is movable, depending on new discoveries.

Addressing Unforeseen Events in Deep Reinforcement Learning:
Current systems cannot handle unexpected, incomplete, or probabilistic information. Research on games like poker and Starcraft offers some insights. Transfer learning may help adapt to new domains with similar underlying structures.

Additional Challenges and Transfer Learning:
Hassabis emphasizes the importance of learning from fewer examples. Transfer learning can accelerate learning in new domains with similar structures.

Royal Society’s Role in the Debate:
The Royal Society is independent of government, industry, and universities. It aims to fuel debate and be trustworthy in discussions about AI and neuroscience.