Mustafa’s Background:
Co-founder and CEO of Inflection AI, Venture partner at Greylock Co-founder of DeepMind, now part of Google Vice President of AI product management and AI policy at Google Author of “The Coming Wave: Technology, Power, and the 21st Century’s Greatest Dilemma”

Topics Discussed:
The new book, “The Coming Wave” Progress in AI achieved by DeepMind, including landmarks like Atari DQN, AlphaGo, AlphaZero, and AlphaFold The ability of technology to invent new knowledge Risks associated with advancements in AI Superintelligence as a distraction from pressing issues Inevitable spread of general purpose technology Nature of intelligence, productivity growth, and labor disruption The containment problem, scale’s importance, and open source LLMs Changing norms of work, leisure, and the redistribution of value Introducing friction into AI deployment and regulatory capture Misinformation apocalypse, digital watermarks, and asymmetric threats Conflict and cooperation with China, supply chain monopolies, and other topics

Early Entrepreneurial Endeavors:
Mustafa Suleiman’s entrepreneurial journey began at the age of 18 when he started a point-of-sale system and sales company, installing early PDAs and networking equipment. Despite not achieving success in this venture, it marked his initial foray into entrepreneurship.

Shifting to Social Impact:
After dropping out of Oxford, Suleiman pursued a more impactful path by co-founding a secular telephone counseling service for young British Muslims. His motivation stemmed from his newfound atheist beliefs and a desire to promote human rights principles and universal justice.

Practical Action and Local Government Experience:
Suleiman’s passion for practical action led him to work in local government as a human rights policy officer for the Mayor of London, Ken Livingston. While the role was significant and exciting, it ultimately proved unsatisfying due to its limitations.

Seeking Greater Impact:
Driven by a desire to scale up his impact, Suleiman co-founded a conflict resolution firm at the age of 22. This consultancy allowed him to work with international organizations and governments to address complex conflicts around the world.

DeepMind’s Founding:
Mustafa Suleiman’s background in negotiation and peace-building led him to realize the importance of technology, particularly in advancing AI. He recognized the transformative potential of AI and its ability to enhance productivity and efficiency across various domains. Suleiman reconnected with his childhood friend Demis Hassabis, whose brother was involved in AI research. During a poker game, they discussed the impact of AI and the need to pursue positive and responsible development.

Shane Legg’s Research on Intelligence:
Shane Legg, a co-founder of DeepMind, had conducted extensive research on intelligence definitions. He aimed to synthesize various definitions and create an algorithmic construct to measure progress towards a defined goal. Legg defined intelligence as the ability to perform well across diverse environments, emphasizing its generality. This concept of generality became a core concept in the field of AI research.

Founding DeepMind:
Suleiman, Hassabis, and Legg founded DeepMind with the mission to create safe and ethical artificial general intelligence (AGI). DeepMind became the first company explicitly focused on inventing AGI. The company’s goal was to push the boundaries of AI capabilities and explore the potential of AGI.

AI Safety and Risk:
Suleiman and Sam Harris met at conferences focused on AI safety and risk. The Puerto Rico conference in 2015 sparked Harris’ realization of the significance of AI development. The subsequent conference in Asilomar in 2017 provided a platform for further discussions on AI safety and risk.

DeepMind’s Legacy:
DeepMind’s groundbreaking work in AGI and AI safety has significantly influenced the field. The company’s contributions have paved the way for advancements in AI capabilities and responsible AI development. While DeepMind has been overshadowed by OpenAI recently, its impact on the field of AI remains profound.

Early Contributions and the Bet on Deep Learning:
DeepMind’s early focus on deep learning, a then-obscure field, allowed it to attract top talent and make significant breakthroughs. The combination of deep learning and reinforcement learning led to the Atari DQN, an AI system that achieved human-level performance in multiple Atari games.

Acquisition by Google:
Google’s acquisition of DeepMind in 2014 provided the company with ample resources to scale its operations and pursue ambitious AGI goals.

Merger with Google Brain:
Google’s AI division, Google Brain, merged with DeepMind in 2023, consolidating open-ended AI research under Google DeepMind and focusing applied research on Google Research.

Breakthroughs with AlphaGo, AlphaZero, and AlphaFold:
AlphaGo’s victory over the world’s top Go players and AlphaZero’s subsequent dominance across multiple board games demonstrated the power of AI in complex strategic environments. AlphaFold’s ability to accurately predict protein structures revolutionized the field of protein folding and opened up new avenues for drug discovery and medical research.

Deep Learning Methods in Games:
Traditional game-playing models relied on hand-crafted rule-based features, as seen in IBM’s Deep Blue model for chess. Go, with its vast possible configurations, required a new approach.

AlphaGo’s Success:
AlphaGo progressively improved by learning from hundreds of thousands of prior games, akin to acquiring expert knowledge from centuries of human play. This success highlighted the generality of deep learning methods, which can scale with more compute.

AlphaZero’s Breakthrough:
AlphaZero achieved superhuman performance without learning from prior games. It learned entirely through self-play, simulating billions of games against itself. This approach proved more valuable than bootstrapping from human knowledge.

Uncoupling from Human Knowledge:
AlphaZero’s self-play learning led to remarkable insights. It surpassed AlphaGo and other systems after just a day of self-play.

Generalizability and Scalability:
The generality of deep learning methods allows them to be paralyzed and scaled up. This enables leveraging traditional computing infrastructure and advancements like Moore’s law.

Impact on Algorithm Quality:
The barrier to applying high-quality algorithms is lowered by the turbocharging effect of underlying drivers.

AlphaZero’s Novel Move:
During a match against the world champion, AlphaZero made a move that initially appeared to be a mistake but later proved to be a brilliant novel strategy. This showcased AlphaZero’s ability to discover new knowledge through self-play.

Suleiman’s Inspiration from AlphaGo’s Creative Discovery:
Suleiman’s inspiration to invent new knowledge through AI algorithms was sparked by AlphaGo’s surprising creative discovery of new strategies.

AlphaFold’s Breakthrough in Protein Folding:
AlphaFold’s ability to generate examples of protein folding structures opened up possibilities for understanding the value and properties of molecules. The open-sourcing of 200 million protein structures in AlphaFold 2 represented a massive breakthrough and a testament to the potential of AI methods.

Comparison of AlphaFold’s Accomplishments to Academic Research:
AlphaFold’s achievements were comparable to millions of PhD theses worth of research in solving protein folding problems.

Technology’s Compressive Effect:
Suleiman highlights technology’s massive compressive effect, where a small amount of labor or resources can now produce immense results, such as 50 years of light from the same amount of energy that once produced 50 minutes of light.

Scale and Complexity of Large Language Models:
The sheer scale and complexity of large language models are astounding, with calculations comparable to the entire Pacific Ocean if each operation were a drop of water.

Suleiman’s Growing Concerns about AI Risks:
Suleiman acknowledges his increasing concern about the potential risks of AI, despite his initial optimism. He recognizes the attendant risks of synthetic biology and the lack of a “brake” to halt AI development, emphasizing the need to address these issues as AI advances.

Geoff Hinton’s Recent Worries:
Suleiman mentions Geoff Hinton’s recent resignation from Google and public expression of concerns about AI, particularly large language models. He finds it inscrutable that Hinton’s concerns emerged suddenly, given the self-evident basis for these concerns.

Ethics and Safety in AI Development:
Mustafa Suleiman has long prioritized ethical and safe AI development, emphasizing the need for oversight and independent scrutiny. During the founding of his company, he advocated for an “ethics and safety board” to ensure technologies align with the public interest and prevent military or surveillance applications.

Divergence from Bostrom’s Superintelligence:
Suleiman disagrees with the focus on “superintelligence” and the “intelligence explosion” as major concerns. He views these concepts as distractions, consuming excessive time and attention.

Near-term Concerns and Practical Risks:
Suleiman emphasizes practical, near-term issues such as misinformation and reduced barriers to power. He highlights the potential for AI to amplify misinformation and grant greater power to individuals or entities.

Modern Turing Test and Artificial Capable Intelligence:
Suleiman proposes a modern Turing test that evaluates capabilities rather than conversational skills. He introduces the term “artificial capable intelligence” (ACI) to describe AI with the ability to learn APIs, initiate actions, and interact with environments.

Proliferation of Power and Existential Risk:
Suleiman foresees a proliferation of conventional power rather than an intelligence explosion. While he acknowledges the importance of existential risk and AGI safety, he prioritizes addressing near-term, practical risks.

Defining the Coming Wave:
General purpose technologies have historically enabled other technologies and proliferated widely. The next wave of technology is intelligence, which is the ability to take actions, synthesize information, make predictions, and affect the world. Intelligence has produced all the value, products, and affected the landscape around us.

Potential Benefits of Artificial Intelligence:
AI can distill the essence of human capabilities into an algorithmic construct that can benefit from scale, perfect memory, compute, and vast data consumption. This can add knowledge, science, and technology to the human ecosystem. In the future, everyone could have access to the best doctor, educator, personal assistant, and chief of staff. AI has the potential to create a more meritocratic society where access to these benefits is widely available.

Meritocratic Access to Technology:
Everyone in the Western world, regardless of wealth, has access to the same smartphones and laptops. The cost of chips has exponentially declined, leading to mass proliferation of technology.

Intelligence and Productivity Proliferation:
If intelligence and life are subject to exponentials, over the next two to three decades, Intelligence, the ability to get stuff done, is about to proliferate, leading to a Cambrian explosion of productivity. Everyone will have access to tools that enable them to pursue their agendas and become smarter, more productive, and more capable.

Labor Disruption:
AI is a true replacement for human intelligence, not just an augmentation. It is replacing higher cognitive, higher status, white-collar jobs first. The argument that new lanes of human creativity will open up due to AI’s productivity gains is not compelling.

AI’s Steady Climb Up the Ladder of Human Cognitive Abilities:
AI models deliberately aim to replace human cognitive abilities. They have progressed from image and audio recognition to generation, text understanding, and text generation. Creativity, judgment, and empathy, once considered unique to humans, are now being challenged by AI.

Long-Term Implications:
If exponential trajectories continue, AI will eventually turbocharge humans temporarily. We need to consider the consequences of widely proliferated, powerful, and cheap AI systems.

Containment:
The concept of containment becomes relevant to manage the massive consequences of AI’s spread. It involves thinking about what actions to take in response to the overwhelming incentives for using AI in statecraft and serving citizens.

AI’s Driving Forces:
AI’s development is driven by various forces: Individual scientific curiosity and the desire to explore, invent, and discover. Commercial incentives for profit and competitive advantage. State-level competition for commercial competitiveness and labor market dominance.

Pessimism Aversion:
Some individuals are constitutionally unable to engage in discussions about potential negative outcomes. The framing of pessimism and optimism as good or bad is misleading and unhelpful. It is important to assess facts objectively and make predictions based on evidence, even if those predictions may be wrong.

Allergic to Predictions:
Post-financial crisis, there has been a tendency among intellectuals and the public to avoid making predictions due to fear of being wrong. Overcoming this aversion to predictions is essential for addressing potential problems and taking steps toward mitigation and adaptation.

The Containment Problem:
The containment problem refers to the challenge of managing and mitigating the potential negative consequences of AI development. This includes addressing issues such as job displacement, inequality, and the potential for AI to be used for malicious purposes.

Containment of AI:
Containment refers to our ability to control and limit new technologies throughout their development and deployment. Its goal is to prevent technologies from running out of control and having catastrophic consequences. Containment can be challenging as technologies often proliferate quickly and become widely accessible.

The Current State of AI:
The digital genie is out of the bottle, and many powerful AI models are already in the wild. Open source development and easy access to AI models have accelerated their proliferation. This raises concerns about the safety and accountability of these technologies.

Open Source Development:
Mustafa Suleiman believes that open source development can be beneficial for AI safety. Openness allows for scrutiny, pressure testing, and accountability of AI models. However, the exponential growth of compute and model size may lead to challenges in controlling the proliferation of powerful AI technologies.

Holding Developers Accountable:
It is important to hold developers of mega AI models accountable, whether they are centralized or open-source. Developers need to take responsibility for the potential risks and impacts of their creations. Mechanisms for accountability and regulation need to be developed.

Rapid Advancement of AI:
The compute used in AI models has increased by 10 orders of magnitude in a decade, far exceeding Moore’s law. This rapid advancement poses challenges for containment and safety measures.