Demis Hassabis’ Introduction:
Demis Hassabis, a renowned expert in AI and founder of DeepMind, shares his journey and achievements, including his exceptional chess skills, academic pursuits, and groundbreaking work at DeepMind.

AlphaFold’s Impact on Protein Folding:
AlphaFold, a pioneering AI system developed by DeepMind, revolutionized protein folding prediction, enabling scientists to determine the complex structures of proteins from their amino acid sequences with remarkable accuracy. This breakthrough has profound implications for drug discovery, vaccine development, and understanding biological processes at the molecular level.

The Importance of Artificial General Intelligence (AGI):
DeepMind’s mission is to solve intelligence and advance science and humanity by understanding natural intelligence and replicating it in an artificial construct. AGI, or Artificial General Intelligence, aims to create a system capable of performing at human-level or above across various cognitive tasks and domains.

Two Main Approaches to AI:
Expert systems rely on hard-coded knowledge programmed by engineers and scientists, limiting their ability to handle unexpected situations and solve novel problems. Learning systems, in contrast, learn solutions from first principles, data, and experience, enabling them to generalize to new tasks and situations.

The Promise of Learning Systems:
Learning systems hold the potential to solve problems that humans do not yet know how to solve, opening up new frontiers in scientific discovery and technological advancement. DeepMind’s work on AlphaFold exemplifies the transformative power of AI in tackling complex scientific challenges.

AI Learning Systems:
Learning systems are inspired by neuroscience, emphasizing generality and learning. Reinforcement learning is a type of learning system that learns from first principles through trial and error.

Reinforcement Learning:
In reinforcement learning, an agent interacts with an environment, observes its surroundings, and takes actions to achieve a goal. The agent builds a model of the environment and predicts the consequences of its actions. The agent selects the best action to take based on its model and its goal.

AlphaGo:
AlphaGo is a reinforcement learning system that plays the game of Go. Go is a complex game with a vast search space and no straightforward evaluation function. AlphaGo learned to play Go by playing against itself and improving its evaluation function and move selection through self-play.

AlphaGo’s Training Process:
AlphaGo played 100,000 games against itself, generating a dataset of positions and outcomes. A new version of AlphaGo was trained on this dataset to predict moves and evaluate positions. The new version played a match against the old version, and if it won by a significant margin, it replaced the old version. This process was repeated until AlphaGo reached world championship level.

AlphaGo’s Implications:
AlphaGo’s success demonstrates the power of reinforcement learning and self-play. AlphaGo’s approach can be applied to other complex games and domains. AlphaGo’s ability to learn from its mistakes and improve over time has broader implications for AI and machine learning.

AlphaGo’s Success in Go:
AlphaGo’s victory over Lisa Dole, a 18-time world champion, showcased the algorithm’s capability to defeat the best human players in the game. The neural network used in AlphaGo significantly reduced the search space and guided the Monte Carlo tree search towards the most promising moves. AlphaGo’s remarkable move on the fifth line in game two of the match, which went against conventional wisdom, ultimately proved to be strategically advantageous.

AI Techniques and Suitable Problem Criteria:
The types of problems suitable for AI techniques developed by DeepMind have specific characteristics: Massive combinatorial search space or state space. Clear objective function or metric for optimization. Availability of data for learning and an accurate simulator for generating more data. Additional considerations include the existence of external benchmarks or tests and the potential for significant downstream impact.

Protein Folding as an Ideal Problem for AI:
Protein folding fits all the criteria for a suitable problem for AI techniques: Enormous state space due to numerous possible protein conformations. Clear objective of predicting the 3D structure from an amino acid sequence. Abundant data available from experimental studies and accurate simulators. Protein folding has remained a challenging problem for decades, with attempts to solve it dating back to the 1970s.

Personal Journey and Timing in Scientific Discovery:
Demis Hassabis emphasizes the importance of timing in scientific discovery and innovation. Being ahead of one’s time can be beneficial, but being too far ahead can lead to challenges and frustration. The story of Charles Babbage, who was perhaps a century ahead of his time, is used as an example of the potential difficulties faced by pioneers.

Background:
Demis Hassabis first encountered the problem of protein folding as a student at Cambridge in the 1990s. Later, he became fascinated by the idea of citizen science games and the potential of Foldit to harness gamers’ motivation and time for useful science. The success of AlphaGo in matching the intuition of Go masters gave Hassabis hope that a system could be built to match the intuition of Foldit players.

AlphaFold Project:
In March 2016, DeepMind launched the AlphaFold project to develop a system for protein structure prediction. The project was driven by the existence of CASP, a blind prediction assessment that provides a gold standard benchmark for protein structure prediction.

CASP Competitions and Progress:
AlphaFold’s entry in CASP 13 in 2018 resulted in a significant jump in the winning score by nearly 50%, demonstrating the effectiveness of machine learning in protein structure prediction. In CASP 14 in 2020, AlphaFold2 achieved atomic accuracy, with a median error of less than one angstrom, competitive with experimental methods.

AlphaFold2 Architecture and Key Advances:
AlphaFold2 is the most complex system built at DeepMind, consisting of 32 component algorithms and requiring 60 pages of supplementary information. It employs an end-to-end system, iteratively refining the structure through a recycling stage. AlphaFold2 utilizes an attention-based neural network to infer the graph structure between residues, addressing the limitations of convolutional neural networks. Evolutionary and physical constraints are integrated into the architecture without impacting the learning process, blending prior knowledge with machine learning.

Conclusion:
AlphaFold2 represents a groundbreaking achievement in protein structure prediction, reaching atomic accuracy and demonstrating the power of machine learning in solving complex biological problems.

AlphaFold2: A Multidisciplinary Effort:
AlphaFold2’s development involved a large, diverse team of researchers, including machine learning experts, biologists, chemists, and physicists, highlighting the interdisciplinary nature of the project.

Domain-Specific Knowledge vs. Generality:
Unlike AlphaZero, which aimed for generality across games, AlphaFold2 prioritized performance for the specific problem of protein structure prediction. The team employed a “kitchen sink” approach, incorporating both general techniques and domain-specific knowledge to achieve optimal results.

Speed and Efficiency:
AlphaFold2’s training process is surprisingly fast, taking only two weeks on a modest amount of compute resources. Inference, or obtaining a protein structure prediction, is even faster, taking minutes or seconds on a single GPU machine.

Broad Access and the Human Proteome:
To provide broad access to AlphaFold2, the team decided to fold entire proteomes, starting with the human proteome over Christmas 2020. This ambitious effort resulted in predictions for over 36% of the human proteome with high accuracy, significantly expanding the experimentally determined structures.

Confidence Measures and Experimental Coverage:
The team developed confidence measures to assess the accuracy of AlphaFold2’s predictions. Despite the high coverage, experimental coverage of the human proteome was still limited to 17% after decades of research. AlphaFold2 more than doubled this coverage to 36% with high-accuracy predictions, demonstrating its potential impact on structural biology.

Future Directions: Striving for AGI:
AlphaFold2 represents a significant step towards DeepMind’s ultimate goal of building AGI, a general system capable of tackling a wide range of tasks. The team aims to extract domain-specific aspects from AlphaFold2, similar to the transition from AlphaGo to AlphaZero, to create a more general system applicable to various problems.

AlphaFold’s Accuracy and Confidence Measures:
AlphaFold provides accurate protein structure predictions, with different levels of confidence assigned to each residue. PLDDT is a novel confidence metric developed specifically for AlphaFold, ranging from 0 to 100. Predictions with PLDDT above 90 are considered highly accurate, with both the backbone and side chains being reliable. Predictions with PLDDT between 70 and 90 have high accuracy for the backbone but may have less reliable side chains. Predictions with PLDDT below 50 may indicate protein disorder, but further research is needed for confirmation.

Expansion of AlphaFold Predictions:
AlphaFold has been expanded to include 20 proteomes of model organisms and pathogens. The AlphaFold database now contains millions of predictions per day, including curated proteins of interest. AlphaFold has been particularly useful for neglected tropical diseases, where structural information is often lacking.

Applications and Impact of AlphaFold:
AlphaFold has been used in various applications, including: Modeling complex biological structures like the nuclear pore complex. Predicting protein disorder. Assisting experimentalists in resolving protein structures. Identifying structures of top pathogens that could cause future pandemics. AlphaFold has received positive feedback and accolades from the research community.

Future Plans for AlphaFold:
Plans to fold every protein in UniProt, covering over 100 million known proteins. Ongoing collaboration with the EBI to improve the AlphaFold database and add new features. Welcoming feedback and suggestions from users to further enhance AlphaFold’s capabilities.

Future-Looking Work:
AlphaFold Multima: Handles Multimas, disordered proteins, point mutations, link and docking, protein interactions, and protein design. Digital Biology: Biology as an information processing system; AI as the right description language for biology; goal of creating a full virtual cell simulation. Scientific Discovery: AlphaFold as proof of concept; breakthroughs in quantum chemistry, pure mathematics, fusion, and genomics. AI as a Tool: Hubble telescope analogy; aiding scientific discovery.

Addressing Biases in AI:
Data sources: analyzing biases and generating synthetic data to rebalance. Diverse Workforce: diverse set of people designing and engineering systems. Broadening the Ecosystem: sponsoring diversity scholarships and initiatives.

Intuition and Consciousness:
Intuition: Knowledge gained through experience, not consciously accessible. Pattern Recognition: Key part of intuition; AI systems can find patterns and signals. Consciousness: Not necessary for intelligence; conscious expression is slower than intuition. Animal Consciousness: Dogs and dolphins have consciousness but not as intelligent as humans. Studying AI: Can help understand human consciousness by comparing to the human brain.

Evolution and AI:
Evolution as a Tinkerer: Nature’s solutions are often complex and not optimal. Evolutionary Constraint: AI predictions consider evolutionary constraints.

Evolutionary Constraints and Efficiency:
Evolutionary constraints are essential for the multi-sequence alignment in AlphaFold2’s system, but not vital for its high performance. Evolution is a powerful yet inefficient algorithm. Brute force algorithms like evolution may not be the most efficient approach for AlphaFold’s goals.

Moving Up the Hierarchy of Complexity:
AlphaFold’s journey continues beyond static protein structure prediction. The next steps involve understanding protein-protein interactions, protein ligand binding, and the dynamic behavior of these systems.

Exploring Fundamental Physics and Translation:
AlphaFold’s team aims to model all the physics and energies involved in protein folding. This would unlock insights into activation energies of enzymes and aid protein design. AlphaFold is seen as a translation model, converting amino acid sequences into 3D structures.

The Dream of a Virtual Cell:
AlphaFold is one component in the larger goal of creating a virtual cell. Interim stages and promising ideas are being explored to reach this long-term objective.

Continuous Improvement and Addressing Weaknesses:
AlphaFold’s team is committed to improving the accuracy of the system. They analyze cases where the system struggles and incorporate special problem positions to enhance performance.

AlphaGo Zero’s Influence on AlphaFold:
AlphaFold’s development draws inspiration from AlphaGo Zero’s approach. AlphaGo Zero was trained to beat the old version and excel in specific problem positions where the previous system struggled.

AlphaFold2’s Performance and Future Improvements:
AlphaFold2 demonstrated remarkable performance in predicting protein structures, achieving 950 out of 1,000 correct predictions in challenging positions. Ongoing efforts focus on creating a database of problematic proteins to further refine AlphaFold2’s accuracy.

AlphaFold2’s Ability to Predict Synthetic Sequences:
AlphaFold2 exhibits proficiency in predicting structures of de novo proteins, including those presented in the CAS competition. Caution is advised when relying on mutated predictions, as AlphaFold2 may struggle with point mutations that drastically alter protein conformation.

AlphaFold2’s Handling of Point Mutations and Multiple Conformations:
AlphaFold2 is less effective in predicting the effects of point mutations that significantly change protein conformation, a common occurrence in disease. Efforts are underway to enhance AlphaFold2’s ability to detect and prioritize these mutations. Exploring the possibility of obtaining multiple conformations from AlphaFold2, including misfolded proteins, is an ongoing area of research.

The Role of Chaperones in Protein Folding:
Chaperones contribute to protein folding in vivo, influencing intermediate conformations. Modeling the contribution of chaperones could refine AlphaFold2’s predictions for problematic structures, potentially leading to virtual cell simulations. The negative view suggests that solely relying on amino acid sequences without considering dynamic factors may limit AlphaFold2’s accuracy.

Research Priorities:
Prioritize research and collaborations that will have the most impact on advancing AI and biology. Continue to explore the potential of AI to solve complex biological problems, including protein design and structure prediction. Develop methods for AI to understand and generate the relationship between protein structure and function.

Ethical Considerations:
Educate the public, politicians, and governments about AI and its potential risks and benefits. Establish principles and regulations for the ethical and responsible use of AI, especially in real-world applications. Prevent malicious use and bad training of AI systems through careful oversight and regulation.

Applications of AI in Biology:
Use AI to design proteins with specific functions and properties, such as stickman proteins or molecular motors. Collaborate with biologists to validate AI predictions and apply them to practical problems in biology. Establish a new company, Isomorphic Labs, to focus on advancing the biology aspects of AI research.

Future of AI and Biology:
AI has the potential to revolutionize biology, leading to breakthroughs in protein design, drug discovery, and personalized medicine. There will be a continued focus on developing AI methods that can understand and model dynamic biological systems. AI will play a crucial role in addressing complex biological challenges and improving human health.

AI’s Black Box Nature:
AI systems, like deep learning, are often seen as black boxes due to their complexity and lack of transparency. Understanding how these systems make decisions and the principles behind their operations is essential, especially in critical domains like healthcare and protein folding.

Engineering Science vs. Natural Science:
Demis Hassabis distinguishes AI as an engineering science rather than a natural science. In engineering science, the artifact (AI system) must be built first before it can be analyzed, unlike natural science, where phenomena already exist and are studied.

Effort in Understanding Black Boxes:
Until recently, AI systems were not advanced enough to warrant the effort required to understand their inner workings. Now, with systems like AlphaGo and AlphaFold demonstrating remarkable capabilities, there is more incentive to analyze and comprehend these black boxes.

Reverse Engineering AlphaZero:
A collaboration between Demis Hassabis, Vladimir Kramnik, and AlphaZero programmers resulted in a paper analyzing the game system’s decision-making. This analysis aimed to extract rules and strategies from AlphaZero’s network, comparing them to classical chess systems.

Future of AI Analysis Tools:
Demis Hassabis believes that the next 10 years will see a surge in the development of tools for visualizing and analyzing complex AI systems. These tools will be crucial for understanding the ‘how’ of AI systems, particularly those with significant impact, like AlphaFold.

Neuroscience and Artificial Networks:
Demis Hassabis emphasizes the importance of understanding artificial networks at least as well as we understand brains. Brains are also black boxes, but advancements in neuroscience techniques like fMRI and single-cell recording are providing insights into their workings. In contrast, artificial brains offer more freedom for experimentation and analysis.

Ablation Studies on AlphaFold2:
AlphaFold2, a breakthrough AI system for protein folding, was subjected to ablation studies to analyze its components. The results revealed the necessity of all components, prompting further investigation into why each part is essential.

Understanding AI as a Goal:
Demis Hassabis sets a lower bound for understanding AI systems: they should be understood as well as natural brains. Currently, our understanding of artificial networks falls short of this goal, presenting a significant area for research. A team at DeepMind is dedicated to developing virtual brain analytics tools to facilitate this understanding.