Demis Hassabis (DeepMind Co-founder) – Learning from first Principles | NIPS (Dec 2017)

Chapters

Abstract

Revolutionizing Intelligence: DeepMind’s Groundbreaking AI Journey

In the rapidly evolving field of artificial intelligence, DeepMind stands as a beacon of innovation and progress. Their mission is to solve fundamental intelligence and use it to solve other problems, potentially affecting every aspect of our lives. Their unique approach emphasizes learning versus hand-crafting solutions, generality, groundedness in sensory modes, and active participation in learning.

From mastering complex games like Go and chess to exploring the boundaries of machine learning, DeepMind’s journey offers profound insights into the nature of intelligence, both artificial and human. This article delves into the key milestones and methodologies of DeepMind, revealing how their cutting-edge approaches are reshaping our understanding of AI and its potential applications.

DeepMind’s Approach to Building Artificial General Intelligence

DeepMind’s mission is to solve fundamental intelligence and use it to solve other problems, potentially affecting every aspect of our lives. Their philosophy emphasizes learning versus hand-crafting solutions, generality, groundedness in sensory modes, and active participation in learning.

Their approach is guided by four key axes:

1. Learning vs. Hand-Crafting: Systems learn for themselves from first principles and raw data, rather than relying on spoon-fed or handcrafted solutions.

2. Generality: Systems are designed to work across a wide range of environments and tasks, potentially including unseen tasks.

3. Groundedness vs. Logic-Based Systems: Systems are fully grounded in sensory modes of reality, tracing the origins of knowledge to real-world inputs.

4. Active vs. Passive Learning: Systems are active participants in their own learning, similar to how children and animals learn.

DeepMind’s Breakthroughs: DQN and AlphaGo

DeepMind’s first system, DQN, combined deep learning and reinforcement learning to create deep reinforcement learning. DQN was an end-to-end learning agent that learned from rewards and achieved impressive results in Atari games, mastering dozens of different games and outperforming humans.

AlphaGo was DeepMind’s system for playing the game of Go, known for its vast complexity and difficulty for computers. Traditional techniques like brute force search and evaluation functions were ineffective for Go due to the enormous search space and lack of materiality. AlphaGo used two neural networks: a policy network trained on human games to predict human moves and a value network trained on game data to predict the winner. These networks narrowed down the search space and provided valuable information for decision-making. AlphaGo was combined with Monte Carlo Tree Search, which called the neural networks to evaluate positions and guide the search process. This combination allowed AlphaGo to challenge and defeat top human Go players.

Overcoming Go’s Challenges

Developing AlphaGo involved addressing three core challenges:

1. Search Space: Go’s vast search space rendered traditional brute force methods ineffective.

2. Evaluation Function: Crafting a function to assess Go positions was complex due to the game’s subtle tactics and lack of clear material advantage.

3. Intuition and Feel: Replicating the intuitive decision-making of professional Go players was a formidable task.

AlphaGo’s solution combined two neural networks, the Policy Network and the Value Network, with Monte Carlo Tree Search, achieving superhuman performance and showcasing the power of AI in mastering complex tasks.

AlphaGo’s Impact on Game Strategy

AlphaGo’s victory featured unconventional strategies, most notably the 5th line move in game 37, initially deemed illogical by human standards but later recognized as a stroke of genius. This move dramatically altered the course of the game, highlighting AlphaGo’s innovative approach and advanced understanding of Go.

AlphaGo’s victory over Lee Seung-ho, the Go player, in a $1 million challenge match marked a historic moment. AlphaGo’s triumph shocked the world as it surpassed experts’ predictions by a decade. Through self-play, AlphaGo developed novel ideas and motifs not seen before in human play. One striking example is move 37 in game two, where AlphaGo played an unconventional move on the 5th line. This move showcased AlphaGo’s ability to think presciently and plan moves several dozen moves ahead. Inspired by AlphaGo, Nita Dole executed his genius move in game 4. A documentary directed by Greg Coase provides insights into AlphaGo’s journey. The documentary is available on various streaming platforms for those interested in learning more.

Human Response and Adaptation

The interaction between AI and human players has been a crucial aspect of DeepMind’s journey. Professional Go player Nita Dole’s ingenious response in game 4, move 78, demonstrated the dynamic and evolving relationship between human and machine intelligence. The documentary “AlphaGo,” directed by Greg Coase, further explores these interactions, offering insights into the challenges and opportunities presented by AI in strategic thinking.

Intuition and Creativity in AI

DeepMind’s work also sheds light on the concepts of intuition and creativity in AI. Intuition, an implicit knowledge gained through experience but not consciously accessible, plays a crucial role in decision-making. Creativity, the ability to synthesize existing knowledge into novel ideas, is another aspect that AI systems like AlphaGo Zero and AlphaZero are beginning to explore.

Evolution of AlphaGo: Zero to AlphaZero

AlphaGo Zero marked a significant evolution, learning solely from self-play without human data, while AlphaZero further extended this approach to other games like chess and shogi. These developments illustrate the generality and adaptability of DeepMind’s AI, capable of mastering complex games rapidly and evolving beyond established human strategies.

AlphaZero’s Chess Breakthrough

AlphaZero’s performance in chess, particularly its victory against the world champion engine Stockfish, demonstrated its ability to apply its learning algorithm across different games. It showcased a human-like approach to move selection and a propensity for long-term positional sacrifices, challenging conventional chess strategies and tactics.

The Limitations and Potential of AlphaZero

Despite its remarkable achievements, AlphaZero faces limitations, such as the need for a clear objective function and substantial data or efficient simulators. However, its potential applications extend beyond games, with principles being applied in healthcare, energy optimization, data science, and scientific research.

The Future of Human-Machine Collaboration

DeepMind’s ultimate goal is to foster collaboration between humans and machines to tackle complex scientific questions. The journey of DeepMind, from mastering games to exploring general intelligence, signifies a transformative era in AI, where the fusion of human creativity and machine precision can unlock new frontiers in knowledge and innovation.

In conclusion, DeepMind’s journey through the fields of AI is not just a story of technological triumph but a profound exploration of the nature of intelligence itself. The implications of their work extend far beyond the fields of gaming, offering a glimpse into a future where AI can greatly impact every aspect of our lives.

Notes by: Hephaestus