Jeff Dean (Google Senior Fellow) – Data Science Career AMA (Mar 2019)


Chapters

00:00:00 From Academia to Industry: Jeff Dean's Journey
00:10:14 Jeff Dean's Transition from Academia to Industrial Research
00:13:32 Evolution of a Computer Scientist's Career from DEC to Google
00:21:08 Journey from Researcher to Manager: Balancing Hands-On Work and Leadership
00:23:49 Influences on a Career in Machine Learning
00:29:33 Machine Learning Skills and Techniques
00:32:27 Automating Machine Learning Experimentation
00:36:44 Advancing Machine Learning through Multitask Learning and Talent Development
00:43:54 Machine Learning: Opportunities, Challenges, and the Importance of Diversity

Abstract

Exploring the Journey of Jeff Dean: From Childhood to AI Leadership

Abstract:

Jeff Dean’s remarkable journey, marked by a global childhood, a passion for computer science, and a transformative career in AI and machine learning (ML), serves as a beacon for aspiring professionals. His experiences, from the early days of programming in BASIC to leading Google’s AI initiatives, highlight the significance of embracing challenges, valuing diversity, and maintaining a balance between personal growth and professional responsibilities. This article delves into Dean’s life, exploring his influences, career decisions, and contributions to the field of AI, while also reflecting on the evolving landscape of machine learning, the impact of AutoML tools, and the importance of inclusivity in technology.

Childhood and Influences:

Jeff Dean’s childhood, characterized by frequent relocations due to his parents’ careers in tropical disease research and anthropology, instilled in him an adaptability and curiosity that would later define his professional life. Immersed in diverse cultures and exposed to a range of experiences, Dean’s early introduction to computers sparked a lifelong passion. This nomadic upbringing laid the foundation for his innovative approach to problem-solving and his interest in technology.

Growing up, Dean moved around the world due to his parents’ work in public health research and anthropology. He attended 11 schools in 12 years, living in various locations, including Hawaii, Uganda, Minnesota, and Somalia. At the age of nine, his father introduced him to a kit computer, the IMSI 8080, sparking his interest in computers. He started programming in BASIC, writing computer games, and modifying existing games.

University and Career Decisions:

Dean’s academic journey, balancing computer science and economics, was pivotal in shaping his career path. His work with the World Health Organization and the choice to focus on computer science for his PhD reflect a commitment to practical impact and a keen understanding of his strengths. This period was crucial in solidifying his interest in research and setting the stage for his future contributions to AI and ML.

In college, Dean pursued a double major in computer science and economics. During his undergraduate studies, he took a parallel and distributed computing class that sparked his interest in research and neural networks. After graduating, Dean worked for a year at the World Health Organization in Geneva before pursuing a PhD in computer science.

Academia vs. Industry:

The decision to pursue a career in industrial research over academia was a defining moment for Dean. His tenure at Digital Equipment Corporation’s Western Research Lab, where he engaged in diverse projects and honed his technical skills, was a critical stepping stone. This choice underscored his preference for practical application over theoretical exploration.

After completing his PhD, Dean initially considered a career in academia but only received one academic job offer. Influenced by factors such as his wife’s career aspirations and the desire to work in the Bay Area, he decided to pursue industrial research. Dean interviewed at various labs before accepting a position at Google.

Jeff Dean’s Journey at Google:

Dean’s transition to Google marked a significant shift in his career. His roles evolved from hands-on problem-solving to leading large-scale projects like Google Brain. This transition from individual contributor to manager highlights the duality of his expertise: a deep technical understanding coupled with the ability to guide and inspire large teams.

At Google, Dean has made significant contributions to the company’s engineering developments, including MapReduce, BigTable, and TensorFlow. His work has had a profound impact on the field of AI and ML, driving advancements in various applications and industries. He co-founded the Google Brain project with Andrew Ng and Greg Corrado, which investigates large-scale neural networks for computer vision, speech recognition, and language understanding. Unlike other Google X projects, Google Brain focused on software relevant to core Google teams.

Jeff Dean’s Career Evolution: During his tenure at Google, Dean’s career took an unexpected managerial turn. Initially tasked with managing a team of seven people at Google X, he eventually assumed the role of leading all of Google’s research division. Throughout this journey, Dean has struck a balance between managing his large organization and engaging in hands-on coding projects, allocating two days a week to pursue his own research interests while steering the organization, ensuring impactful research, and fostering collaboration.

Mentorship and Influences:

Throughout his career, Dean has been influenced by a range of mentors, from teachers to colleagues like Sanjay Gemmawat. These relationships not only shaped his technical skills but also his approach to collaboration and problem-solving.

In addition to his formal education, Dean credits his mentors and colleagues for shaping his career. One of his most influential mentors was Sanjay Gemmawat, a Google engineer who worked with Dean on the design and implementation of MapReduce. Dean also acknowledges the contributions of his other colleagues at Google, who have helped him develop his skills and broaden his perspective.

Collaboration with Sanjay Gemmawat: Jeff Dean and Sanjay Gemmawat had a long working relationship, complementing each other’s strengths. They often pair-programmed together, combining their logical and frenetic approaches. Their collaboration led to significant contributions to the field, including the development of MapReduce, a framework for distributed processing of large data sets.

Skills for Aspiring Data Scientists and ML Practitioners:

Dean’s advice for aspiring data scientists and ML practitioners emphasizes a strong foundation in mathematics, programming, and an understanding of machine learning algorithms. He advocates for adaptability and interdisciplinary knowledge, highlighting the dynamic nature of the field.

Dean emphasizes the importance of a strong foundation in mathematics, programming, and machine learning algorithms for aspiring data scientists and ML practitioners. He believes that adaptability and interdisciplinary knowledge are essential in this rapidly changing field.

Tools, Knowledge, and Skills: Programming, machine learning algorithms, and computer architecture are essential for understanding computation at a high level of abstraction. Knowledge of cache and branch predictors, as well as the strengths and weaknesses of different accelerators, is beneficial. Interdisciplinary skills, such as healthcare, neuroscience, or quantum chemistry, can be valuable when applying machine learning to various problems.

Impact of AutoML Tools:

Dean recognizes the transformative potential of AutoML tools in automating repetitive tasks, allowing experts to focus on strategic decision-making and innovation. He foresees these tools expanding into new domains and potentially tackling global challenges.

Dean believes that AutoML tools have the potential to revolutionize the field of AI by automating repetitive tasks and allowing experts to focus on more strategic decision-making and innovation. He anticipates that these tools will continue to expand into new domains and potentially help solve global challenges.

AutoML Tools and the Human Role:

– AutoML techniques can automate the iterative process of running experiments, analyzing results, and designing new experiments in machine learning.

– AutoML tools can enhance human capabilities by automating repetitive tasks and enabling higher-level insights.

– Humans should focus on guiding AutoML tools, interpreting results, and making strategic decisions rather than performing manual, repetitive tasks.

Limitations of AutoML Tools:

– Current AutoML tools are limited in their ability to generate completely new techniques or explore vast search spaces.

– AutoML tools excel in searching within defined architectural spaces, but struggle with excessively large search spaces.

Multitask Learning and Sparse Activation:

His vision for the future of AI includes the development of models capable of multitask learning with sparse activation. This approach, focusing on efficiency and adaptability, is a testament to his forward-thinking philosophy.

Dean’s vision for the future of AI includes the development of models capable of multitask learning with sparse activation. He believes that this approach, which focuses on efficiency and adaptability, will be essential for solving complex real-world problems.

A Single Model for All Machine Learning Problems:

– Jeff Dean envisions training a single model with large capacity and sparse activation to solve all machine learning problems.

– Different parts of the model would be specialized for different tasks, and the routing would be learned to activate the relevant parts for a given task.

– This approach would allow for sharing of knowledge across related tasks and examples.

Promoting Representation and Inclusiveness:

A significant aspect of Dean’s ethos is the promotion of inclusivity and representation in the field of machine learning. He underscores the importance of inspiring a diverse range of individuals to explore computing and AI, advocating for a culture of respect and support in workplaces.

Dean is a strong advocate for diversity and inclusion in the field of machine learning. He believes that it is essential to inspire a diverse range of individuals to explore computing and AI and to create a culture of respect and support in workplaces.

Fostering Inclusivity and Diversity in Machine Learning:

– Dean expresses his concern about the lack of gender and racial diversity in the machine learning field, particularly among his daughter’s peers.

– He emphasizes the need to inspire and expose all individuals, regardless of background, to the possibilities and career paths available in computing.

– Dean stresses the importance of creating inclusive work environments that promote respect and inclusion for everyone, recognizing the value of diverse perspectives in shaping machine learning products and their impact on society.

Addressing Socioeconomic Disparities:

– Dean acknowledges the inherent advantages enjoyed by individuals with access to technology and resources at home, which can create disadvantages for those who lack such privileges.

– He advocates for initiatives and efforts aimed at reducing these disparities and ensuring equal opportunities for all individuals interested in pursuing a career in machine learning.



Jeff Dean’s story is not just a tale of personal achievement; it’s a blueprint for future generations in AI and ML. His journey underlines the importance of lifelong learning, the willingness to embrace new challenges, and the need for diversity and inclusivity in technology. Dean’s contributions extend beyond his technical accomplishments, influencing the way we approach, teach, and think about AI and ML. As the field continues to evolve, his insights and experiences provide valuable lessons for those looking to make their mark in this dynamic and impactful domain.

The Future of Machine Learning and the Importance of Inclusion and Diversity in the Field:

– Jeff Dean believes there is great potential for machine learning to solve complex tasks and make a positive impact in areas such as healthcare and education.

– He sees opportunities for advancements in machine learning systems that can address real-world problems and drive meaningful outcomes.

– The conversation concludes with a note of gratitude to the audience and a preview of the final day of CareerCon 2019, focusing on “Getting the Job” with a lineup of engaging sessions and activities.


Notes by: WisdomWave