Background:
Traditional approaches to training machine learning algorithms faced skepticism from academia and industry. DeepMind’s mission was to distill human intelligence into algorithmic constructs to scale up and enhance human capabilities.

Motivation for Using AI to Address Global Issues:
DeepMind’s focus on using AI tools and methods to create positive impact in the world. Belief that businesses should consider global challenges and their role in addressing them.

Global Social Problems:
800 million people lack access to clean water, projected to double in the next decade. 800 million people suffer from malnutrition daily, despite a third of produced food going to waste. Consumption patterns and resource requirements are unsustainable. Rising consumption in developing nations further exacerbates the problem.

Income Inequality:
Top 1% incomes have increased by 300% in the last three decades. Real incomes for the bottom two-thirds of the global population have stagnated or declined.

Opportunity for AI and Businesses:
AI companies and organizations have the potential to address these challenging global issues.

Motivation for Enhancing Problem-Solving Capacity:
Mustafa Suleyman emphasizes the need for enhanced problem-solving capacity to address complex global challenges. Traditional methods often struggle to predict phenomena like stock market trends or climate change impacts.

DeepMind’s Hybrid Approach:
DeepMind combines elements from academia, industry, and the public sector to create a unique research organization. The company focuses on long-term, complex problems while maintaining agility and commercial motivation. DeepMind’s values are grounded in social mission and public service.

Training Algorithms in Controlled Environments:
Before partnering with Google, DeepMind faced difficulties in accessing data for training algorithms. The company turned to the old school Atari test bed, consisting of 100 Atari games from the 70s and 80s. This controlled environment allowed DeepMind to adjust complexity based on the algorithm’s capabilities. The algorithm received raw pixel data from the games, similar to a robot observing an arcade.

Learning from Scratch:
DeepMind’s approach involves learning everything from scratch, without pre-programmed rules or heuristics. The goal is to maximize score by learning rewarding actions in the environment. The algorithm learns its own representation of rewards, which is often more complex and non-linear than human-crafted rules.

Training Success and Game Examples:
DeepMind successfully trained its algorithm to play a variety of Atari games. Examples include Enduro, River Raid, a 3D tank shooting game, Pong against the inbuilt AI, Sea Quest, and an adversarial game with clever strategies. The algorithm’s ability to learn new knowledge, such as cornering an opponent, demonstrated the effectiveness of the training approach.

Go and Its Complexity:
Go is an ancient Chinese board game with simple rules, involving black and white stones placed on a board to surround territory. The complexity of Go lies in the vast number of possible board configurations, estimated to be 10 to the power of 170, far exceeding the number of atoms in the known universe.

Traditional Methods and Neural Networks:
Traditional methods of defining rules based on human expertise don’t scale well for Go’s complexity. The approach used involved training two neural networks: a policy network and a value network. The policy network estimates the likelihood of rewarding moves, while the value network calculates their usefulness.

Success in Go and Nature Paper Publications:
The neural network approach led to success in playing against the human world champion, Lee Sedol, winning four games to one in a live match. The achievement resulted in two nature paper publications, including a front page feature.

Applying the Approach to Global Climate Change:
The research team aims to apply the same methods to address intractable problems like global climate change. Energy consumption, particularly data center energy consumption, is a significant contributor to carbon emissions.

Reducing Data Center Cooling Costs:
The neural network model trained by the team reduced the overall cost of cooling Google data centers by approximately 40%. This finding surprised data center engineers, who have years of expertise in cooling systems.

Data Center Cooling Insights from the Neural Network:
The neural network identified opportunities for cooling optimization that were not apparent to human experts. The network leveraged data from sensors and historical records to understand cooling patterns and adjust settings accordingly.

Data Center Cooling System:
Data centers use a complex system of cooling to remove heat generated by compute load. The goal is to maximize power usage efficiency while maintaining safe temperature and pressure.

Data Collection:
Two types of data are used for training: state data (sensor readings) and action data (cooling system adjustments). State data includes IT load, pressure, temperature, water flow, pump and fan speeds, and clock speeds. Action data includes the number of cooling towers, chillers, and pumps active, as well as set points for pressure, temperature, and flow rates.

State and Action Representations:
1,200 state variables and 20 actions are aggregated into 120 state representations. Actions can be continuous or discrete, allowing for fine-grained control. Time step is set to one hour.

Optimization Goal:
Given a state, the goal is to determine the actions that optimize power usage efficiency at the next time step, while staying within safe constraints.

Neural Network Predictions:
Neural networks are used to predict pressure, temperature, and power usage efficiency at the next time step. The model is fine-tuned to be more or less risk-averse in its recommendations.

Operational Constraints:
Heuristic-driven boundary constraints are defined to ensure safe operation. The model is designed to maximize long-term reward, not short-term performance. Confidence estimates are provided for each prediction.

Automatic Failover Mechanisms:
Backup systems are in place to handle recommendations that violate safe boundary constraints. Human oversight remains a central part of the system.

Results:
The system achieved a 40% reduction in energy required for cooling, resulting in a 15% improvement in power usage effectiveness.

Future Research:
DeepMind is interested in discovering new knowledge and making 10x contributions to complex social problems. Insights gained from this project can be applied to other domains.

Learnings from Google’s Data Center Optimization:
More Cooling Equipment Leads to More Efficiency: Contrary to intuition, spreading load thinly over multiple pieces of equipment and running them at lower capacity results in more efficient power consumption. This discovery reveals interesting linear versus exponential power efficiency curves. Optimal Flow Rate for Chillers: Engineers often assume that concentrating flow through chillers is best, but lower flow can lead to improved efficiency. While local energy consumption for the chiller may increase, the overall system efficiency can be significantly better. Adaptive Response to Incoming Compute Demand and Temperature: The ability to shift loads to different parts of the system based on incoming compute demand and temperature conditions allows for a flexible and fast response to data center conditions. This adaptive approach leads to significant efficiency gains.

Potential for Wider Application:
Generalizable Optimization Systems: The developed optimization systems are inherently general and can work effectively in various environments, given appropriate data and sufficient data volume. DeepMind is exploring the possibility of offering these methods as a service outside of Google.

DeepMind’s Mission and Partnerships:
Mission-Driven Organization: DeepMind is building a unique organization driven by the goal of using its methods to improve the world. The company believes that partnerships are key to delivering and scaling the benefits of AI. Open to Collaboration: DeepMind welcomes partnerships and discussions with individuals and organizations interested in collaborating on AI for positive impact. The company is actively hiring individuals who share its mission.