Cool Things of the Week:
Google has released Data Set Search in beta, a search tool specifically for indexing metadata on open data sets. Google has expanded its public data sets for geospatial and ML-based analytics, adding 5 petabytes of BigQuery storage available for public data sets for the next five years. Kaggle and Google AI have launched a new challenge to encourage the development of models that are more robust to blind spots in data sets, addressing issues of machine learning bias and fairness. The Google Cloud Platform Podcast is now among the Top 100 Podcasts in Tech on iTunes. A blog post explaining what makes TPUs fine-tuned for deep learning is available, providing information on neural nets, TPUs, and pricing.

Interview with Jeff Dean, Head of Google AI:
Jeff Dean is a computer scientist and engineer leading Google AI. He shares his excitement about the upcoming Deep Learning Indaba conference in Stellenbosch, South Africa. Dean discusses the importance of fostering a diverse and inclusive community in AI research and development. He emphasizes the value of open-source software and collaboration in advancing AI. Dean highlights the significance of responsible AI, considering ethical and societal implications. He explains Google’s efforts in promoting responsible AI through initiatives like the AI Principles and the Advanced Technology External Advisory Council.

Research Focus of Google AI:
Google AI, also known as Google Research and Machine Intelligence, conducts a wide range of computer science research. The research includes fundamental algorithm and technique development, product-oriented research, and systems infrastructure development. Google AI strives to create tools that benefit internal and external developers, such as the popular machine learning toolkit TensorFlow.

Jeff Dean’s Role and Goals as Leader of Google AI:
Jeff Dean’s goal as the leader of Google AI is to inspire researchers to tackle ambitious problems and achieve significant impact. He spends Mondays coding and collaborates with colleagues on multiple technical projects. The rest of his time is dedicated to guiding the large organization, focusing on the right problems, and ensuring impact.

Jeff Dean’s Definition of Artificial Intelligence (AI):
AI means creating computer systems capable of solving problems that require human intelligence. He emphasizes the concise and straightforward nature of this definition.

Challenges in the Field of Computer Science Research:
The field is experiencing an explosion of interest and progress, particularly in the past seven to eight years. Computer vision has witnessed remarkable advancements, with deep learning models enabling computers to see effectively. The impact of these advancements is already being felt and will continue to have profound effects on various aspects of the world.

Implications for Computer Vision:
Machine learning has enabled automated machines to perceive and understand their surroundings. This has significant implications for general computer vision, such as image recognition and classification. Google Photos utilizes these capabilities to provide users with enhanced experiences, such as categorizing images by content.

Transformative Impact in Healthcare:
Machine learning algorithms are tackling medical imaging-related problems. Automated systems assist medical professionals in analyzing images, leading to improved diagnosis and treatment. Robotics also benefits from machine vision, enabling robots to navigate and interact with their environment effectively.

Advancements in Natural Language Understanding and Speech Recognition:
Machine learning has made significant strides in natural language understanding and speech recognition. These advancements have led to improved user experiences and enhanced interactions with devices and applications.

Opportunities and Excitement in Machine Learning:
New capabilities in machine learning present opportunities for positive impact on the world. There is a surge of interest in studying and researching machine learning to address various challenges.

Collaboration between Brain and Google AI:
Brain and Google AI collaborate closely with different groups across Google and Alphabet. This collaboration includes bringing research projects from Google AI Research Division to market through Cloud AI. Cloud AI provides products like Cloud AutoML, which enables customers to train models without advanced machine learning expertise.

Inspiration behind Tensor Processing Units (TPUs):
Jeff Dean’s interest in solving interesting problems with computers led to the development of TPUs. The need for more computation for sophisticated algorithms and larger data sets drove the pursuit of TPUs.

Development of TPUs for Speech Recognition:
Collaboration with the speech recognition team to improve accuracy using deep learning models. Back-of-the-envelope calculations revealed the massive computational requirements for improved speech recognition. This prompted the exploration of computationally efficient solutions for deep learning models.

Properties of Deep Learning Models:
Deep learning models are tolerant of reduced precision, allowing for hardware optimization. The algorithms are composed of a handful of building blocks, such as matrix multiplies and vector dot products.

Advantages of TPUs:
TPUs are designed specifically for low-precision linear algebra operations, making them highly efficient for deep learning models. This focused design enables complete hardware rethinking, leading to specialized and powerful TPUs.

Applications Suited for TPUs:
TPUs excel in applications requiring intensive low-precision linear algebra operations. They are particularly suitable for deep learning tasks such as image recognition, natural language processing, and speech recognition.

TPU Design Balance:
TPUs strike a balance between specialization and general applicability. They are designed to accelerate operations commonly found in deep learning models, enabling them to run various models like vision, speech, and language.

TPU Versions:
TPUv1: Initially focused on inference, targeting reduced precision requirements. TPUv2: Expanded to support both training and inference, addressing the complex hardware design challenges of training.

TPUv3 Cooling System:
TPUv3 introduced liquid cooling for improved heat dissipation. Each board contains four TPU chips, with water cooling directly contacting the chips to remove excess heat.

TPUv3 Pod Scale:
TPUv3 pods have a significantly larger footprint compared to TPUv2 pods. TPUv3 pods house eight times the computational capacity of TPUv2 pods.

Jeff Dean’s Interest in Parallel Processing:
Dean has been fascinated by parallel processing since his undergraduate years at the University of Minnesota. He viewed more computation as the key to solving many problems.

Early Work on Neural Nets:
Dean’s undergrad thesis focused on parallel training of neural nets. He aimed to harness the power of 64-processor machines to accelerate neural net training.

Moore’s Law and the Rise of GPUs:
Dean highlights the significance of Moore’s law in driving computational advancements. The emergence of GPUs as powerful tools for training deep learning models.

Rekindled Interest in Machine Learning:
Dean’s encounter with Andrew Ng at Google X in 2011 sparked his renewed interest in machine learning. Ng shared promising results from his Stanford students using neural nets.

The Genesis of Google Brain Project:
Dean, along with Greg Corrado, formed the Google Brain Project. Their goal was to develop a computer system capable of training large neural nets using CPUs.

The Hype and Potential of Deep Learning:
Dean acknowledges the hype surrounding deep learning but emphasizes its justified aspects. He highlights significant advancements in computer vision, speech recognition, translation, and other applications.

Tempering Expectations and Recognizing Progress:
Dean cautions against unrealistic expectations of super intelligent machines in the near future. He emphasizes the importance of recognizing the remarkable progress in computer capabilities.

Underappreciated Applications of Machine Learning:
Dean points to applications in scientific simulations, chemical interactions, earthquake fault simulations, and medical imaging. These applications have the potential to transform major problems in the world.

Balancing Hype and Progress:
Dean believes it is essential to temper excessive hype but also acknowledge the significant advances in machine learning.

Neural Networks as Scientific Simulators:
Neural networks can be trained to approximate expensive scientific simulators, resulting in significant speedups. In the quantum chemistry domain, a neural network achieved similar accuracy to a traditional simulator, but was 300,000 times faster. In earthquake fault simulation, a simple neural network was 10 to 100,000 times faster than the original simulator, with no noticeable difference in accuracy.

Games and Neural Networks:
Neural networks can potentially enhance game performance by simulating complex game elements more efficiently.

Neural Networks in General Computer Systems:
Many aspects of general computer systems, such as operating systems, compilers, and storage systems, rely on handwritten heuristics. These heuristics can be replaced with neural networks, which can learn to adapt to the specific usage patterns of the system.

Example: Neural Networks for Loop Ordering in Compilers:
Compilers use heuristics to determine the order of loops in generated code. Neural networks can learn to optimize loop ordering based on the specific program being compiled.

Bigtable Server Cache:
Bigtable servers use an in-memory cache to store frequently accessed data, improving performance by avoiding disk reads.

Data Access Patterns:
Information about user groups, job names, and job types is available when data is requested.

Heuristic Approach:
A simple heuristic might be to avoid caching data from MapReduce jobs since they typically read data once and don’t reuse it immediately.

Machine Learning Approach:
A learning heuristic could analyze usage patterns and automatically learn to identify data that is not likely to be reused and should not be cached.

Potential Benefits:
Machine learning can adapt to changing usage patterns, leading to more efficient caching decisions.

Deep Learning Indaba Conference:
Jeff Dean will speak at the Deep Learning Indaba conference in South Africa.

Conference Focus:
The conference addresses important engineering problems identified by the National Academy of Engineering in 2008.

Machine Learning’s Role:
Machine learning can contribute to solving these problems in various ways.

Jeff Dean’s Talk:
Dean’s talk will highlight Google’s work in applying machine learning to solve some of these grand challenges.

Deep Learning Indaba Attendees:
The conference brings together researchers and practitioners from Africa and around the world.

Jeff Dean’s Personal Connection:
Dean lived in Uganda and Somalia as a child and is passionate about using machine learning to address issues in Africa.

Expanding Brain to Ghana:
Brain has set up an office in Ghana led by Mustafa Sis. The office is being renovated, and a team of researchers has been hired. The office will focus on both basic machine learning research and applied problems specific to Ghana and Africa.

Deep Learning Adapa:
Deep Learning Adapa is a great resource for those interested in deep learning. Resources will be provided in the show notes for those who want to learn more.

Best Advice for a Career:
Partner with people who have different expertise than you. Working on projects with diverse expertise allows you to learn new things and continuously stimulate your brain. This approach helps you develop a broader understanding of various techniques and perspectives.

Keeping Up with Research and Resources:
Surround yourself with people who know things that you don’t. Network with others to stay updated on interesting developments and research in their areas of focus. Be open to learning new things and exploring different fields.

Staying Current in a Rapidly Changing Field:
Jeff Dean emphasizes the challenge of keeping up with advancements in machine learning due to its rapid evolution. He suggests focusing on a few areas of interest and gaining a cursory understanding of others. Reading a hundred abstracts provides a high-level grasp of various concepts, allowing for further exploration as needed.

The Power of Automated Experimentation:
Dean highlights the significance of computers conducting experiments, measuring results, and generating new experiments autonomously. This approach enables exploration of diverse problem-solving methods, surpassing human capabilities in terms of the number of experiments conducted.

Jeff Dean’s Unique Educational Journey:
Dean shares his experience of attending 11 schools in 12 years, gaining exposure to diverse environments and perspectives. He believes this adaptability has been instrumental in his success, allowing him to appreciate the positive aspects of different places.

Resources for Learning Deep Learning and Machine Learning:
Dean recommends various resources for individuals interested in deep learning, TPUs, Google Brain, and ML research. He suggests blogs, courses, and textbooks tailored to different expertise levels, ranging from high school students to advanced researchers.

Jeff Dean’s Coding Routine:
Despite his senior position, Dean dedicates Mondays to coding, emphasizing the importance of hands-on involvement in development.

IoT Core Data Display on a Web Front End:
Gabe, an IoT expert, shares his experience building an end-to-end IoT application during Cloud Next 2018. The setup involved Raspberry Pi devices communicating with IoT Core via MQTT, telemetry data being bridged to Cloud PubSub, and a Cloud Function pulling data into Cloud Firestore. A front-end Angular app subscribed to Firestore callbacks to display real-time heart rate data.

Upcoming Conferences and Events:
Mark Mandel discusses upcoming conferences he will be attending, including Tokyo Next and Strange Loop. Gabe shares his plans for a European trip, attending conferences in London, Paris, and Barcelona.

Mark Mandel:
Barcelona hosts the IoT Solutions World Congress, a major European IoT conference.

Melanie Warricks Lukescuk Higgins:
They plan an enjoyable trip to Barcelona.

Mark Mirchandani:
He expresses gratitude to Melanie and Gabe for their participation in the podcast.

Melanie Warricks Lukescuk Higgins:
She acknowledges Mark’s appreciation.

Mark Mirchandani:
He thanks the audience for listening and promises a new episode next week.