Andy Bechtolsheim (Arista Co-Founder) – A discussion with Arista’s Andy Bechtolsheim (Apr 2012)
Chapters
00:00:00 Software-Enabled Functions and Extensions in OpenFlow
OSS Multi-Node Applications: OpenSwitch’s EOS model of stateless agents and centralized state management in a single database seamlessly extends to multi-node systems. This simplifies and enhances features like hitlist upgrades, which have traditionally been challenging to implement. The model also offers opportunities for expansion into wider multi-chassis or multi-node settings.
Centralized OpenFlow Management: Bechtolsheim acknowledges the customer use case of centralized management for a collection of switches using OpenFlow. OpenSwitch is committed to implementing this feature in a predictable manner.
Replacing Traditional Routing with Centralized Agents: Bechtolsheim expresses skepticism about replacing traditional routing with centralized routing agents for Internet-scale applications. The original purpose of distributed routing protocols was to ensure survivability in case of node failure. A centralized approach would introduce a single point of failure and compromise fault tolerance.
Benefits of Software-Enabled Functions: Bechtolsheim emphasizes the advantage of OpenSwitch’s software-defined architecture in implementing software-enabled functions. The nature of the software makes it easier for OpenSwitch to introduce and manage these functions compared to other vendors.
Open Questions and Discussion Topics: Bechtolsheim invites questions related to the discussed topics, including chip roadmaps, system roadmaps, and the implications of software-defined networking on controlling small groups of tightly-defined systems.
00:02:52 Challenges and Obstacles in Network Architecture Innovation
OpenFlow Reality Check: OpenFlow, announced in 2008, has not resulted in any significant production deployments to date.
Flow-Based Network Management Challenges: Centralized flow-based network management is impractical due to scalability issues and complexity. The throughput of flow-based networks is limited by flow setup time in controllers. Greenfield environments are rare, making it difficult to introduce new network architectures into existing customer networks.
Real-World Networking Challenges Networks evolve slowly due to protocol maturity and investment in existing infrastructure. People are reluctant to change what works, leading to long transition periods for new technologies.
Success Stories: ECMP (Equal-Cost Multi-Path) has proven to be a successful and scalable deployment model. Leaf-spine architecture with ECMP and layer two MLEC in the spine has become the standard deployment model for large cloud data centers.
Conclusion: OpenFlow has not met initial expectations, and flow-based network management faces scalability and practicality challenges. Networks evolve slowly due to factors such as financial investment, reluctance to change, and protocol maturity. Successful network technologies like ECMP and leaf-spine architectures have emerged based on proven and reliable approaches.
Service Provider Focus: Arista focuses on data center networks within service providers, known as service provider customers. They do not offer the routing tables or features of full-fledged routers.
Critique of SDN and Next-Gen Data Center Models: The speaker points out that Arista’s existence challenges the concepts of SDN and next-gen data center models. Their success indicates agreement among many in the industry.
Centralized Management: Centralized management is a key goal in cloud environments to reduce OPEX and the number of network engineers required. Arista’s approach involves robust and trusted implementations to ensure reliability in the absence of maintenance windows.
00:10:51 Networking Technology Scaling and Evolution
Arista’s Proven Success in High-Availability Environments: Arista has achieved substantial success in deploying scalable networking solutions in demanding environments. They have demonstrated the effectiveness of their scalable architecture in large-scale deployments.
Continuous Technology Evolution Through Silicon Advancements: Arista relies on various chip vendors’ roadmaps to drive technology improvements. Regular upgrades in silicon every 18 months to two years enhance performance, add ports, and increase memory and table sizes.
Seamless Integration of Different Chip Architectures: Arista’s software supports diverse chip architectures with a single release. This approach simplifies system administration by eliminating the need for separate release trains or version tracking.
Cost-effective Optics for Enhanced Performance: Arista anticipates reduced costs for high-speed optics, such as 40-gigabit and high-gauge optics, in the coming years. This development opens up opportunities for performance upgrades without extensive network changes.
Solving Scaling Challenges through Incremental Upgrades: Arista’s approach to scaling focuses on incremental upgrades rather than a complete system overhaul. This enables customers to address scaling needs by leveraging existing networks.
Centralized Management for Efficient Configuration: Arista provides centralized management capabilities to streamline the configuration process. This feature automates provisioning, reducing the need for manual CLI commands and eliminating the requirement for additional network engineers.
Customer-Driven Software Prioritization: Arista’s software roadmap is heavily influenced by the needs and feedback of its largest customers. This customer-centric approach ensures that the most pressing requirements are prioritized and addressed.
00:13:59 Network Complexity Solutions: VXLAN and NVGRE
Complexity in Networking: Networking involves multiple layers, introducing unnecessary complexity for layer two and layer three.
VXLAN and NVGRE: VxLAN and NVGRE are standards that solve the problem of tunneling layer two over layer three. Ken Duda, Arista’s CTO, played a key role in developing these standards.
Market Dominance of Standard Space Protocols: Standard space protocols, like VXLAN and NVGRE, will succeed in the market due to interoperability between multiple vendors.
Hardware Termination in the Switch: Current silicon lacks hardware termination forVXLAN in the switch. Arista is considering implementing hardware termination in future silicon or FPGAs.
Arista’s Approach to Chip Selection: Arista uses a variety of chips from different vendors, focusing on the best performing options available. The company avoids endorsements to maintain vendor neutrality.
FPGA Programming and Silicon Chip Design: Arista currently does not engage in silicon chip design but may explore FPGA programming in the future.
00:17:45 Custom Hardware vs. Programmable Network Gear
FPGAs (Field Programmable Gate Arrays) Use Models: Customers can design their own functions in Verilog or C transitive model, directly in proprietary applications. Partners can provide turnkey solutions like market, feed select, or FX protocol permissions. Arista can do its own standard networking functions in FPGA.
Benefits of Arista’s FPGA Switches: The same box can handle all three use cases. Arista supports the platform worldwide, helping developers with low code and no hardware support capabilities. Developers can license their code in any way, maintaining their own business model.
Challenges in Developing Silicon Chips: High design verification cost (around $20-30 million for a good switch chip in 20nm technology). Multiple vendors invest over $100 million each year on next-generation chip development. Arista would need a chip development team as large as its current software team to compete.
Differences in Chip Architectures: Racktop switches require integrated switches with onboard memory buffers. Modular chassis benefit from chips with a visual queuing architecture. Different architectures have different IP investment and validation verification requirements.
Network Edge Resolution: Some chips can perform network edge resolution in hardware, while others cannot. Choosing the right chip is crucial for wire-speed network edge resolution.
SDN (Software-Defined Networking) and Arista’s Approach: Arista’s application switch (7124FX) is not a load balancer but runs applications and the switch together. This approach is suitable for functions that can be done at FPGA speed (hundreds of millions of packets per second). It solves a different problem than traditional load balancers or other application switches.
Faster Order Execution: FPGAs offer sub-microsecond execution by implementing logic close to the hardware level in the network switch, eliminating the latency associated with software implementations. Traditional Linux systems typically have latencies of around two microseconds, while FPGAs can perform the same operations in 200 nanoseconds.
Competitive Advantage: In the competitive world of high-speed trading, low latency is essential for success. FPGAs provide an edge by enabling faster decision-making and execution compared to software solutions.
Efficient Hardware Functions: FPGAs are used to perform functions that are traditionally handled by software, such as risk analysis and monitoring. These functions are more efficient when implemented in hardware, resulting in instantaneous profitability gains.
Suitable Applications: FPGAs are ideal for applications that can be implemented with a small number of gates or clocks. Complex applications, such as Oracle databases, are not well-suited due to the increased coding complexity and inability to keep up with the switch’s packet rate.
Conclusion: FPGAs provide significant advantages in high-speed financial applications by enabling sub-microsecond order execution and decision-making. Their suitability for specific tasks and their ability to optimize profitability make them a valuable tool in this competitive trading environment.
00:24:15 Software-Defined Networking: Challenges and Opportunities
Theory vs Reality: The evolution of SDN is still unclear. Few customers have implemented unique or weird things on Arista switches. Stability is paramount in production networks.
Problem-Solving Focus: Arista focuses on understanding the true use case behind customer requirements. The company aims to solve problems rather than blindly implementing SDN standards.
Complexity of SDN Standards: Another writing protocol for SDN adds complexity and requires support. Arista highlights the challenge of making SDN standards useful and scalable.
Alignment with EOS and Cloud Vision: Arista will map SDN use cases to its EOS platform and Cloud Vision. Cloud Vision enables multiple switches on a network to perform tasks beyond a single switch.
Conclusion: Arista emphasizes stability and problem-solving in its approach to SDN. The company will integrate SDN use cases into its existing EOS and Cloud Vision offerings.
Abstract
Revolutionizing Network Infrastructure: Arista’s Pioneering Approach in Multi-Node Systems, OpenFlow, and SDN
A New Era in Network Management: Arista’s Innovative Multi-Node System
In the field of network infrastructure, Arista Networks stands as a vanguard, particularly with its groundbreaking extension of the EOS (Extensible Operating System) model to multi-node systems. OpenSwitch’s EOS model of stateless agents and centralized state management in a single database seamlessly extends to multi-node systems. This simplifies and enhances features like hitlist upgrades, which have traditionally been challenging to implement. The model also offers opportunities for expansion into wider multi-chassis or multi-node settings.
The switch architecture is highly scalable, with some switches supporting thousands of nodes and managing multiple petabytes of traffic. Arista’s success in deploying scalable networking solutions in demanding environments is a testament to its effectiveness.
The OpenFlow Dilemma: Balancing Demand and Practicality
While OpenFlow has garnered significant interest for centralized switch management, Bechtolsheim notes its challenges, particularly in replacing traditional routing with centralized agents. The resilience required in node failure scenarios remains a critical consideration, tempering the enthusiasm for a complete shift towards OpenFlow. Nonetheless, Arista’s commitment to software-defined networks (SDNs) shines through, with a focus on leveraging software-enabled functions to enhance network capabilities.
However, OpenFlow has not met initial expectations, and flow-based network management faces scalability and practicality challenges. Networks evolve slowly due to factors such as financial investment, reluctance to change, and protocol maturity. Successful network technologies like ECMP and leaf-spine architectures have emerged based on proven and reliable approaches.
Challenges and Alternatives to OpenFlow
OpenFlow, despite its potential, faces hurdles in widespread production deployment. Issues like scalability and integration with existing network infrastructures, which aren’t typically managed on a flow basis, hinder its adoption. Bechtolsheim introduces ECMP (Equal-Cost Multi-Pathing) as a practical, scalable alternative. This approach, eschewing the need for centralized controllers, optimizes traffic distribution and reliability.
Complexity in networking involves multiple layers, introducing unnecessary complexity for layer two and layer three. VXLAN and NVGRE are standards that solve the problem of tunneling layer two over layer three. Ken Duda, Arista’s CTO, played a key role in developing these standards. Standard space protocols, like VXLAN and NVGRE, will succeed in the market due to interoperability between multiple vendors. Current silicon lacks hardware termination for VXLAN in the switch. Arista is considering implementing hardware termination in future silicon or FPGAs.
The Slow Evolution of Networking Protocols and the Rise of Virtualization
Bechtolsheim highlights the sluggish pace of change in networking protocols, exemplified by the protracted transition to IPv6. This inertia is attributed to substantial investments in existing infrastructure and a general reluctance to disturb stable systems. However, virtualization emerges as a transformative force, offering enhanced flexibility and resource utilization, thereby propelling new technological adoptions and architectures.
Advancements in Data Center Networking: Leaf-Spine Architecture
The shift from traditional three-layer architecture to leaf-spine design signifies a major advancement in data center networking. This topology, coupled with ECMP and layer two MLEC in the spine, optimizes network resource utilization, ensuring efficient load balancing and failover mechanisms.
Leaf-spine architecture with ECMP and layer two MLEC in the spine has become the standard deployment model for large cloud data centers. Arista focuses on data center networks within service providers, known as service provider customers. They do not offer the routing tables or features of full-fledged routers. Centralized management is a key goal in cloud environments to reduce OPEX and the number of network engineers required. Arista’s approach involves robust and trusted implementations to ensure reliability in the absence of maintenance windows.
Service Providers and Cloud Business Integration
Service providers are increasingly delving into cloud services, building expansive data centers within their networks. Arista, focusing on these data center networks, champions simpler and more robust solutions over traditional models. The key lies in centralized management, essential for cloud networks, aiming to slash operational expenses and reduce network engineer requirements. This approach, alongside the uniform deployment and standardization of cloud networks, offers pronounced efficiency and scalability benefits.
Arista’s Technological Edge: Scalability, Performance, and Compatibility
Arista’s switches are a testament to scalability, supporting extensive ECMP with large switches managing thousands of nodes. Regular silicon upgrades enhance performance, with features like increased bandwidth and larger tables. Importantly, Arista’s software supports various chip architectures while maintaining a single binary release, simplifying system administration.
Arista’s success in deploying scalable networking solutions in demanding environments is a testament to its effectiveness. Regular technology improvements are driven by Arista’s reliance on various chip vendors’ roadmaps. Silicon upgrades every 18 months to two years enhance performance, add ports, and increase memory and table sizes. Arista’s software supports diverse chip architectures with a single release, simplifying system administration by eliminating the need for separate release trains or version tracking. Arista anticipates reduced costs for high-speed optics, such as 40-gigabit and high-gauge optics, in the coming years. This development opens up opportunities for performance upgrades without extensive network changes.
The Complexities and Innovations in Networking
Bechtolsheim brings attention to the complexities of traditional networking, such as the layered VLAN approach. Innovations like VXLAN and NVGRE, tunneling protocols that extend layer 2 networks across data centers, offer solutions to these complexities. The potential of FPGA implementation in future chips further underscores Arista’s innovative approach to networking, providing flexibility and reducing vendor dependency.
FPGA: A Game-Changer in High-Speed Trading
In the high-speed trading arena, FPGAs (Field-Programmable Gate Arrays) offer remarkable advantages. Their ability to execute critical functions faster than software implementations, with sub-microsecond processing speeds, significantly reduces latency. This capability is especially beneficial in environments like Wall Street, where every microsecond counts. However, their suitability is confined to simpler applications, as complex functions like database management remain beyond their scope.
Faster Order Execution and Competitive Advantage:
– FPGAs provide sub-microsecond execution speeds by implementing logic close to the hardware level in the network switch.
– They offer a competitive edge by enabling faster decision-making and execution compared to software solutions.
Efficient Hardware Functions and Application Suitability:
– FPGAs handle functions traditionally handled by software, such as risk analysis and monitoring, efficiently.
– They are ideal for applications with a small number of gates or clocks.
– However, complex applications like Oracle databases are not well-suited due to increased coding complexity and inability to match the switch’s packet rate.
The Uncertain Path of Software-Defined Networking
The evolution of SDN is still shrouded in uncertainty, with limited adoption due to unique network requirements. Discussions around SDN are now centered on understanding its true use cases and exploring alternative solutions. Arista’s approach to SDN involves integrating it with their EOS and Cloud Vision platforms, enhancing network functionality without compromising on stability and operability.
Arista’s Practical Approach to SDN and Complexity Avoidance:
– Arista focuses on solving customer problems rather than blindly implementing SDN standards.
– The company highlights the challenge of making SDN standards useful and scalable.
Mapping SDN to Existing Platforms:
– Arista will map SDN use cases to its EOS platform and Cloud Vision.
– Cloud Vision allows multiple switches on a network to perform tasks beyond a single switch’s capabilities.
Arista’s Balanced Approach in Network Evolution
In conclusion, Arista Networks stands at the forefront of network technology evolution, adeptly balancing innovation with practicality. From extending EOS to multi-node systems to exploring the potential of FPGAs in high-speed environments, Arista demonstrates a commitment to advancing network infrastructure while maintaining the stability and reliability essential in production networks. Their approach, focusing on incremental upgrades and compatibility, paves the way for a more efficient, flexible, and robust networking future.
Vinod Khosla advocates for disruptive innovation, re-engineering engineering practices, and adapting to rapid change in the computing landscape. Khosla emphasizes the importance of evolvability, specialization, and experimentation in engineering methodologies to navigate the complexities of modern computing systems....
VMware's NSX and Arista's collaboration transform network infrastructure with virtual cloud networks and software-defined security, enhancing scalability, flexibility, and security. NSX provides a unified fabric for policy management across heterogeneous environments, integrating software-defined networking, intrinsic security, and automation to meet modern networking challenges....
Moore's Law, networking innovations, and silicon photonics have revolutionized data center technologies, enabling exponential growth in computing power and connectivity speeds. Data centers are evolving rapidly to meet modern computing demands, emphasizing speed, efficiency, and cost-effectiveness....
Moore's Law and merchant silicon fuel networking chip advancements, leading to higher density, speed, and power efficiency. Optics technology evolution enables higher speeds and cost-effectiveness, driving innovation in data center interconnects and long-haul networks....
Pioneering disruption, adaptation, and customer-centric approaches in cybersecurity are crucial for success in the evolving landscape of security, according to industry leaders at the RSA conference. A strong company culture, diversity, inclusivity, and a focus on long-term customer satisfaction are essential for sustained growth and success in the cybersecurity industry....
Andy Bechtolsheim's journey from a modest upbringing in Germany to becoming a Silicon Valley icon highlights his ability to foresee and adapt to changing technological landscapes. His contributions to workstations, networking, and cloud computing have shaped the modern computing and networking world....
Ethereum roadmap focuses on scalability, security, and decentralization, aiming to make transactions affordable for everyday users. Cryptocurrencies can potentially address global challenges like financial inclusion and climate change, but a truly decentralized, neutral, and scalable blockchain ecosystem is crucial....