Jeff Dean (Google Senior Fellow) – Keynote (Jan 2022)

Chapters

Abstract

Updated Article: Revolutionizing Chip Design with Machine Learning

Introduction

The field of chip design has witnessed a transformation driven by the integration of machine learning (ML) techniques. This article delves into the profound impact of ML on various facets of chip design, from reducing design cycles to enhancing performance and efficiency, drawing insights from Harry Foster and Jeff Dean, a Senior Vice President at Google.

Custom chip design involves significant time, effort, and expertise. Researchers aim to reduce the design time to a few people and a few weeks.

Harry Foster’s Acknowledgment of Resilience

Harry Foster’s admiration for the DAC executive committee’s resilience sets the stage. He pays homage to the long-standing team and talented individuals who have contributed significantly to the field. Foster acknowledges sponsors like CETA, CIGDA, SSCS, Cadence, Siemens, Synopsys, Perforce, and CleoSoft for supporting I Love DAC. Attendees are encouraged to visit their booths to show appreciation.

Jeff Dean: A Visionary in Machine Learning

Jeff Dean, who has been with Google since 1991, has significantly contributed to the field of machine learning. His involvement in developing core systems like MapReduce and TensorFlow, along with his academic achievements and prestigious recognitions, highlight his expertise in this domain.

Machine Learning as a Transformative Force

In his presentation at DAC, Jeff Dean emphasized the transformative role of machine learning in hardware design. He envisions a future where chip design time is reduced from years to days, indicating a paradigm shift in the industry. Advances in neural networks have notably reshaped machine learning and inference methods.

Real-World Impact of Machine Learning

The practical applications of machine learning are vast and varied. Innovations in speech recognition, computer vision, and natural language processing have led to significant improvements in computational capabilities, facilitating more intuitive human-computer interactions.

Democratizing Machine Learning Through Open Frameworks

Open-source frameworks such as TensorFlow, JAX, and PyTorch have made advanced ML techniques more accessible, spurring innovation across various sectors.

The Evolution of Computer Design for ML

The impact of machine learning is evident in the development of specialized computers for deep learning tasks. Google’s Tensor Processing Units (TPUs), which have evolved from TPUv1 to TPUv4, show substantial improvements in connectivity, cooling, and compute power, greatly enhancing performance in complex ML tasks.

Chip Design: From Traditional to ML-Driven Methods

The traditional chip design process is being transformed through the integration of machine learning. Reinforcement learning, in particular, has proven effective in automating complex tasks such as chip placement and routing. The work on automated placement and routing, published in 2020 and 2022, has shown promising results.

Reinforcement learning in chip design involves making placement decisions based on a system of rewards. This process is more complex than games like Go, as it must consider chip area, timing, congestion, and design rules. The evaluation of the true reward function is time-intensive, requiring hours of iterations with EDA tools.

The process begins with a blank canvas, using a floor planning environment and a distributed PPO RL algorithm. Each node placement is evaluated, utilizing a proxy reward function for faster assessment. Multiple objective functions are combined with appropriate weighting. A hybrid approach is employed, combining RL agent placement of macros with force-directed placement of standard cells.

The ML Placer has demonstrated its efficiency in handling TPU design blocks, surpassing human experts in certain aspects. It has been able to place a design in 24 hours, achieving a shorter wire length and a comparable number of design rule violations to human expert designs. The ML Placer explores more rounded areas for optimization, as opposed to the human inclination towards straight lines. Researchers are now training RL agents on multiple designs to enable quick and effective placement of new designs.

Automating Verification and Architectural Exploration

Machine learning is also making strides in verification, with graph-based neural networks leading to more automated test case generation. This approach, combined with deep representation learning, significantly improves verification efficiency and quality.

AI’s role in optimizing placement in ASIC designs is evident, as it can achieve better results with fewer iterations compared to traditional methods. A pre-trained policy can quickly achieve better accuracy. Zero-shot placement can provide good results in less than a second of computation.

RL-based tools have shown to produce better quality routing results than human experts in real-world production settings.

A graph neural network learns from the design to predict coverage points and test cases. This learned representation enables efficient test case generation, reducing computational cost.

Machine

Learning in Accelerator Design

The challenge of keeping up with rapidly evolving ML models is being addressed through ML-driven design space exploration. This approach, which includes the co-design of hardware and compilers, is redefining accelerator design, making it more efficient and performant.

TPU Efficiency and Customization

Advancements in TPUs, especially with compiler optimizations, have led to significant performance gains. Tailoring chips for specific models like EfficientNet and BERT has resulted in marked improvements in performance-per-watt, showcasing the advantages of workload-specific customization.

The Role of Machine Learning in Reducing Design Cycles

Machine learning is playing a crucial role not only in enhancing chip performance but also in automating design cycles. Techniques like reinforcement learning are speeding up experimentation and decision-making processes in chip design, pointing towards a future where chips can be designed in days or weeks.

Conclusion

The integration of machine learning in chip design represents a significant paradigm shift, leading to more efficient, powerful, and customized computing solutions. As machine learning continues to evolve, its role in revolutionizing chip design will only deepen, bringing us closer to designing sophisticated chips in a remarkably short time frame.

Machine learning’s capacity to automate time-consuming tasks in chip design, such as architectural exploration and synthesis, is pivotal. It is particularly beneficial in designing custom chips for specific machine learning models. An archived version of this work is publicly available, and an expanded version has been accepted for publication in S++. Deep representation learning enhances verification efficiency and quality, and its ability to generalize across designs benefits slightly modified designs or new designs trained on multiple earlier versions.

Automating Design Cycle for Machine Learning Accelerators

Developing ML accelerators involves anticipating future ML models, a challenging task due to the rapid evolution of these models. Shorter design cycles could enable the creation of single-workload accelerators and leverage machine learning for efficient design space exploration.

Co-Designing Hardware and Compiler Optimizations

The performance of accelerators involves both hardware data paths and the mapping of workloads by compilers. Co-designing compiler optimizations with data paths can significantly enhance design space exploration.

Exploring Data Path, Schedule, and Compiler Decisions

Automated search techniques are optimizing compute and memory aspects of ML accelerators. The search space includes decisions on data path mapping, the fusion of operations, and compiler choices.

Exploring Systolic Array Size, Cache Sizes, and More

The meta search space allows for the exploration of various design decisions, such as systolic array size, cache sizes, and compiler optimizations.

Considering Compiler Co-Design for Comprehensive Analysis

Co-designing compiler optimizations with hardware design ensures a thorough evaluation of design choices.

Benchmarking Against a Hypothetical TPUv3 Chip

The results of these explorations will be benchmarked against a baseline hypothetical TPUv3 chip.

Customizing TPU Designs for Specific Workloads

The TPUv3 architecture is optimized for compute-intensive AI workloads. Integrating hardware and software improvements has led to enhanced performance and efficiency.

Customized Designs for Computer Vision Models

EfficientNet models B0-B7 have been explored for tailored chip designs, resulting in performance improvements ranging from 3 to 6 times.

Expanding to Natural Language Processing Models

The customization has been extended to BERT-128 and BERT-1024 models, with consistent performance improvements observed.

Optimization for Multiple Models Simultaneously

A single chip has been customized for five different machine learning models, showing a moderate performance drop compared to model-specific optimizations.

Performance vs. Power Considerations

The absolute performance results are even better for multi-model optimization, with variations depending on whether the priority is on performance or performance per watt.

Benefits of Customization

Significant improvements have been achieved by optimizing for specific workloads, potentially leading to shorter design cycles and faster customization.

AI in Chip Design

AI is being used at multiple stages of the chip design process, including floorplanning, architecture exploration, and RTL synthesis. AI-driven floorplanning tools optimize chip layouts, reducing design time and enhancing performance. AI-powered architecture exploration tools assist engineers in selecting the most suitable architecture by evaluating various design alternatives. AI-enabled RTL synthesis tools automate the conversion of high-level design descriptions into efficient hardware implementations.

Potential Benefits of AI in Chip Design

AI has the potential to streamline the chip design process, enabling a small team to design a chip in a matter of days or weeks. The automation of design tasks using reinforcement learning can expedite the design cycle by reducing the need for human intervention and experimentation.

Conclusion

Machine learning is poised to revolutionize the way computer chips are designed, facilitating faster and more efficient design processes. AI’s role extends to the design of customized chips, allowing a small team to complete the process in a remarkably short time frame. The integration of machine learning in chip design is a significant advancement, leading to more efficient, powerful, and customized computing solutions. As machine learning continues to evolve, its impact on revolutionizing chip design is expected to grow, bringing us closer to the era of designing sophisticated chips quickly and efficiently.

Notes by: Flaneur