Demis’s Return to MIT:
Demis Hassabis, co-founder of DeepMind, has maintained a close relationship with MIT since his time as a visiting postdoc in 2009. His advisory role in MIT’s Center for Brains, Minds, and Machines (CBMM) has been instrumental in shaping the center’s research directions. Hassabis’s regular visits to MIT every three years provide an opportunity for him to share DeepMind’s progress and engage with the academic community.

AlphaFold’s Impact on Biology:
Hassabis emphasizes the significance of AlphaFold, a DeepMind project that predicts protein structures from amino acid sequences. He invites biologists in the audience to share their feedback on AlphaFold and explores potential collaborations.

Recapitulating Past Work Leading to AI in Science:
Hassabis highlights DeepMind’s journey over the past 10-12 years, leading to the current stage where AI is meaningfully applied to scientific discovery. He aims to connect the dots between DeepMind’s earlier work on games and its current focus on using AI for scientific advancements.

DeepMind’s Founding and Mission:
DeepMind was established in 2010 with an ambitious vision to develop artificial general intelligence (AGI) or human-level AI. The company’s mission is to solve intelligence and use it to advance science and benefit humanity.

Solving Intelligence:
Hassabis defines solving intelligence as fundamentally understanding the phenomenon and recreating it artificially to achieve AGI. He believes that a general approach to solving intelligence can lead to algorithms applicable to a wide range of challenging tasks, mirroring the capabilities of the human brain.

Learning Systems Approach:
DeepMind’s approach to AI involves learning systems, particularly deep reinforcement learning (RL). RL systems observe data streams, build models of their environment, and learn to achieve specific goals or maximize rewards within those environments. These systems can potentially discover new knowledge from first principles through trial and error.

AlphaGo’s Success and Significance:
AlphaGo, DeepMind’s program for playing the game of Go, was a landmark achievement in deep RL. Hassabis highlights the challenge of Go for computers due to its vast search space and esoteric nature.

Background:
Go is a challenging game due to its complexity and reliance on intuition and pattern matching. AlphaGo and its successors use a self-play training approach to improve their evaluation and move prediction skills.

Training Process:
Start with a random neural network. Train the network through hundreds of thousands or millions of self-play games. Use the generated data to train a new version of the network (V2). Conduct a mini-tournament between V2 and V1. If V2 wins by a certain threshold (e.g., 55%), replace V1 with V2. Repeat the process until V2 consistently beats V1.

Results:
AlphaGo achieved superhuman performance in various two-player games, including Go, within 17 to 20 iterations.

Generalization to Other Games:
AlphaGo’s underlying approach can be applied to any two-player perfect information game.

Searching a Constrained Space:
AlphaGo’s neural network learns a model of the environment, constraining the search space for possible moves.

Original Go Strategies:
AlphaGo demonstrated creativity by introducing original strategies, such as move 37 in game two. This move, initially considered unconventional, later became a decisive factor in winning the game.

AI Systems and Interpolation:
AI systems excel at interpolation tasks, such as distinguishing between images of cats and dogs.

polation to Invention

AI Creativity:
AI systems can exhibit creativity in Go by developing original strategies that are not simply interpolations of known strategies. However, AI systems currently lack the ability for true invention, such as inventing new games like Go or Chess.

Generalization:
DeepMind’s approach to engineering involves focusing on performance first and then generalizing the system to make it less specific to a particular task. AlphaZero achieved remarkable results by beating the best programs in various games, including AlphaGo, within a few hours of training from nothing. Generalization can also provide performance benefits, as seen with AlphaZero’s improved strength compared to AlphaGo.

Analytics and Understanding:
AlphaZero’s success prompted efforts to analyze and understand the system’s inner workings, aiming to open the “black box” and gain insights into its knowledge representation and limitations.

Chess and Stockfish:
AlphaZero’s capabilities were also demonstrated in chess, where it surpassed Stockfish 8, the reigning world champion chess computer.

AlphaZero’s Unique Style of Chess:
AlphaZero invented a new style of chess, favoring mobility over materiality, and its games were considered beautiful and aesthetically pleasing. AlphaZero sacrificed pieces and pawns to imprison black pieces, leading to the “immortal Zagzwang game,” where no move could improve the black position.

Advantages of AlphaZero over Traditional Chess Engines:
AlphaZero doesn’t have inbuilt rules and valuation tables like traditional chess engines. It is freer to evaluate positions based on contextual factors and make sacrifices when advantageous. AlphaZero balances different factors more effectively and can dynamically adjust its evaluation based on the position. It requires less search for strong play, making it more efficient than traditional chess engines.

AlphaZero’s Influence on Human Chess Players:
Magnus Carlsen and Garry Kasparov praised AlphaZero’s style and approach to chess. Carlsen acknowledged AlphaZero’s influence on his own play. Kasparov noted that AlphaZero’s learning process allows it to reflect the “truth” of chess.

DeepMind’s Approach to Scientific Discovery:
DeepMind seeks out massive combinatorial search spaces where exhaustive brute force approaches are not tractable. It requires a clear objective function or measure to optimize against. Sufficient data or an accurate and efficient simulator is necessary for learning and progress. DeepMind has successfully used synthetic data generated from simulators to augment real data.

DeepMind Technologies in Commercial Products:
DeepMind technology is used in various Google products, including YouTube recommendations, Android battery life optimization, and text-to-speech systems.

Protein Folding and Its Importance:
Protein folding is the process of determining a protein’s 3D structure from its amino acid sequence. Proteins are essential to life and their 3D structure determines their function and drug targets. DeepMind’s AlphaFold tackles the protein folding problem and has achieved significant breakthroughs.

Understanding the Protein Folding Problem:
Determining protein structure experimentally is a time-consuming and laborious process. Christian Anfinsen’s Nobel lecture sparked a 50-year quest to predict protein structure directly from its amino acid sequence.

Leventhal’s Paradox:
Leventhal’s paradox highlights the computational challenges of protein structure prediction due to the vast number of possible shapes a protein can take. Despite this, proteins spontaneously fold efficiently in nature, indicating the existence of an efficient solution.

Personal Journey with Protein Folding:
Demis Hassabis’s early fascination with the protein folding problem as a student in Cambridge. Belief in AI’s potential to solve the problem in the future, despite limitations of AI techniques in the 1990s.

Background:
Demis Hassabis’ first encounter with the protein folding problem was through the Foldit game, a citizen science game that allowed gamers to contribute to science by solving protein folding puzzles. Hassabis was intrigued by the idea of using gamers’ intuition and pattern matching skills to find protein folds, and he noticed that some players were able to find real structures that were published in scientific journals.

Mimicking Intuition:
After AlphaGo’s success in mimicking the intuition of Go masters, Hassabis realized that it might be possible to apply similar techniques to protein folding. Protein folding is a highly complex problem, and greedy energy minimization methods often get stuck in local maxima, resulting in incorrect folds. Hassabis believed that AlphaFold could use its intuition, gained through training on large datasets, to find better solutions than traditional methods.

CASP Competition:
CASP is a biennial competition that assesses the accuracy of protein structure prediction methods. Experimental departments produce new protein structures and submit them to CASP organizers, who withhold the structures from the competing teams. Teams then have one week to predict the structures using their methods, and the organizers compare the predicted structures to the experimentally determined ground truth. CASP has been running since 1994 and is considered the Olympics of protein folding.

AlphaFold’s Breakthrough:
AlphaFold was the first method to use cutting-edge machine learning as the core component of its solution to the protein folding problem. In CASP 2018, AlphaFold won the competition by a significant margin, achieving an accuracy level that was almost 50% higher than the next best team. This breakthrough revolutionized the field of protein folding and demonstrated the power of machine learning in addressing complex scientific problems.

Stagnant Progress:
Prior to AlphaFold’s entry in CASP 2018, there had been minimal progress in protein structure prediction for over a decade. Traditional methods had reached a plateau, and the field was in need of new approaches. AlphaFold’s success highlighted the potential of machine learning to accelerate scientific progress and address long-standing challenges.

Key Advances of AlphaFold2 over AlphaFold1:
End-to-end system with iterative recycling stage. Direct prediction from amino acid sequence to 3D structure. Attention-based neural network for inferring implicit graph structure. Incorporation of evolutionary and physical constraints into the model architecture.

Accuracy and Performance:
Atomic accuracy achieved at Cas14 competition. Less than 1 Angstrom error, surpassing experimental methods. Significantly more accurate than other systems.

Broad Access to the Biology Community:
Folded entire human proteome within two weeks. Doubled the experimentally determined human proteome to 36% with sub-angstrom accuracy. Determined another 58% at high accuracy, useful for fundamental research.

Confidence Metric for Biologists:
Developed a confidence metric to assess the reliability of predictions. Greater than 90 indicates experimental level accuracy. Greater than 70 indicates high accuracy regime. Lower than 50 may indicate disorder.

Folded 20 Model Organisms and Major Diseases:
Expanded the scope beyond human proteome to include important research organisms and diseases. Aimed to cover more of the biology space.

Database Creation:
AlphaFold Protein Structure Database was created in collaboration with EMBL-EBI to store 3D structures and other tools for biologists.

Free and Open Access:
The database was released for free with unrestricted access for any use, maximizing scientific impact and benefit to humanity.

Database Content:
It initially contained 330,000 structures for 20 model organisms and the human proteome. Now expanded to over a million predictions, including 440,000 annotated ones from Swiss prot.

Focus on Neglected Tropical Diseases:
Collaborated with the WHO to prioritize understudied neglected tropical diseases like Chagas disease and Leishmaniasis. This enables non-profits to initiate drug design efforts with the available protein structures.

Applications and Impact:
AlphaFold has been used to model complex biological structures previously not possible, such as the nuclear pore complex. It has aided in predicting protein disorder and identifying protein structures for pathogens with pandemic potential. Over 350,000 researchers have utilized the database, reaching nearly all biologists worldwide.

Future Directions:
Aiming to expand the database to include nearly 200 million known proteins, covering a vast range of organisms.

What’s Next for AlphaFold?:
Researchers are working on applying AlphaFold to study protein complexes, disordered proteins, point mutations, and protein dynamics. They are also exploring the use of AlphaFold in protein design.

AlphaFold as a Proof of Concept for Digital Biology:
Demis Hassabis believes that AlphaFold is a proof of concept for a new era of digital biology. He envisions creating a virtual cell that can be used to make useful predictions in silico.

AI as the Perfect Description Language for Biology:
Hassabis believes that AI is the perfect description language for biology because it is an emergent process that is too complex to be described mathematically in a clean way.

AI as the Ultimate General Purpose Tool for Scientists:
Hassabis believes that AI used properly could be the ultimate general purpose tool to help scientists see further, much like the Hubble telescope was for cosmologists.