Introduction of Andy Bechtolsheim:
Andy Bechtolsheim, Chairman, Chief Development Officer, and Co-Founder of Arista Networks, delivered a keynote address. He is a renowned figure in the tech industry, having co-founded Sun Microsystems and receiving numerous accolades.

Multi-Chip Packaging:
Multi-chip packaging is gaining significance in the context of Moore’s law. Integrating different devices on a single package substrate is essential for future advancements. Challenges lie in interconnecting high-density chips with high IO bandwidth.

2.5D and 3D Technologies:
2.5D technology, represented by Intel’s embedded multi-die interconnect bridge, is an example of chip-to-chip interconnects. Vertical stacking, or 3D technologies, offer new possibilities for dense chip packaging. These technologies enable “system on a single package” designs, evolving from traditional “system on a chip.”

Standardization and Interfaces:
Standardization of chip-to-chip interfaces is crucial for multi-vendor compatibility. Intel’s advanced interface bus is one example of a proposed standard. Memory interfaces, HPM memories, full-size SRAM dies, and various IO chiplets are also part of the interface landscape.

Thermal Power Dissipation:
Thermal power dissipation is a limiting factor in multi-chip packaging. Conventional methods can dissipate up to 1000 watts per package. New developments like diamond substrates could significantly increase this limit.

I/O Limits and Speed:
The current limit for service speed on multi-chip packages is 100 gigabits per second (Gbps), with developments pushing for 200 Gbps per lane. The limiting factor for IO is the electrical power required to drive the IO, which can exceed the available power on the chip.

Power Consumption and Scaling:
The perception that electrical power requirements would exceed the total power budget as low as 20 terabits per chip is outdated. Current estimates suggest that the threshold is at least 100 terabits and closer to 200 terabits in terms of total electric power. A petabit per chip IO solution is not available today, but conventional electrical IO from the chip package can scale to much higher power levels.

Co-Packaged Optics:
Multi-chip modules can integrate optical components, but current co-packaged optic solutions do not offer higher shoreline density or lower power consumption than electrical IO. Attaching fiber to co-packaged optics is challenging, requiring high-density fiber connectors or direct interconnection with polymer waveguides, which are not yet in production. Reliable optics are crucial for co-packaged optics, as the failure of a single laser can necessitate replacing the entire system.

Rack-Level Interconnectivity:
Existing electrical IO services, such as VSR, MR, and LR, work well for rack-level interconnectivity, covering distances of up to several meters. Copper-based solutions are cost-effective, reliable, and low-power, making them suitable for intra-rack connections. Optics become necessary when distances exceed the limits of copper, typically around two meters for 200 Gbps 2NX cables.

Data Center Interconnect:
Cloud data center bandwidth demand grows at a rate of 50% or more per year, leading to increased power consumption by the network, including optics. Silicon process technology improvements, with 5nm and 3nm generations on the horizon, offer significant power reduction for digital logic. Predictable power reduction trends in switch chips can be used to forecast power per bit or peak power per bit for future generations. Both client optics and long-reach optics for data center interconnect are expected to see a factor of two reduction in power consumption. Co-packaging and other approaches are being explored to further reduce power consumption and improve interconnect efficiency.

Original Goal and Electrical Power Savings:
CPO initially aimed to reduce electrical power by switching from LR to a lower power service to drive co-packaged optics on the multi-chip carrier. This would result in power savings at the wall plug level, but required a dedicated 5-nanometer tape-out, which was not economically feasible. Instead, a lower power VSR service was designed, achieving a 200-watt power saving, but not the original 20% goal.

Power Comparison:
The VSR channel leveled the playing field between CPO and pluggable optics, as it can be used with a fly-over copper cable to reduce PCB losses. The power consumption is currently identical for CPO and pluggable solutions using the VSR channel.

Reliability Challenges:
External light sources were adopted to achieve high reliability for CPO, but this introduces additional coupling losses compared to pluggable solutions. These losses increase laser power requirements, offsetting some of the power savings. The use of multiple connectors contributes to higher optical losses and the need for expanded beam connectors to avoid dust contamination.

Alternative Approaches:
Eliminating the DSP and using an enhanced SIRTIS on the switch chip could reduce power consumption. Alternative modulation technologies with lower insertion losses may also be explored.

Other Issues with CPO:
Reliability, serviceability, manufacturability, testability, and business model issues need to be addressed for successful CPO adoption. Expanding beam connectors are crucial to prevent dust contamination and costly switch returns. Extensive internal testability is essential to ensure product functionality during manufacturing. A scalable serviceability model is required to avoid removing complete RMA switches.

Conclusion:
Pluggable optics currently offer a more feasible solution than CPO due to power savings, ease of deployment, and lack of design, serviceability, and manufacturability issues. Lower power solutions are needed to make CPO more attractive. Future innovations in DSPs, SIRTIS designs, optics modulators, active gain materials, and laser sources hold the potential to reduce optical power consumption by half by 2025.

Technology:
The benefits of co-packaged and pluggable optical modules are identical, as they share the same underlying optics technology. Pluggable optical modules offer the advantage of being decoupled from the underlying switch platform, allowing for faster integration of optical advances without waiting for a new switching system release.

Power and Time to Market:
The industry should focus on reducing power consumption and improving time to market for all technologies, from silicon and high-speed optics to better modulators. Aligning these technology transitions at the 51.2 and 102.4 levels is ideal, but avoiding time-to-market risks is paramount.

Development Timelines:
Developing new optics typically takes two or more years from the time a specification is finalized. The process involves chip-level design, fabrication, and silicon photonics development, making it challenging to bring new optics to market quickly.

51.2T Switch Projections:
A 51.2T switch, currently in design, will be available in the lab next year and shipping in 2023. It will feature 64 800-gig plugs, each supporting two 400-gig or 100-gig interfaces, offering significant bandwidth in a compact and cost-effective package.

Optics and Switch Chipsets Transition:
800G optics in early production, volume production expected before product launch. Transitioning to 100G SerDes: majority of bandwidth shipped by 2023 will be 100G, increasing system performance and reducing cost per bit. 200G SerDes expected in 2024-2025.

Optics Forecast:
400G optics peaking in 2023, declining thereafter. 800G optics ramping up in 2023, taking over quickly.

800G Optics Technologies:
800G-CR: next-gen data center interconnect technology, silicon expected next year, productization in 2023. CR-Lite/LR: for 10km data center range, where multiple lower cost 800G DWM can be used. 200G PAM-4: focused on 1-2km data center reach, in market by 2023-2024. Existing honey kick lambda optics: up to 10km reach. VXL multimode fiber solution: up to 50-100m reach.

Product Flexibility:
Same switch port can accommodate 100G/400G/200G/800G native or 800G data center interconnect. Early testing in the lab with different 800G optics from multiple vendors, ensuring interoperability.

Cost-Efficiency of Tuner Geek Optics:
Tuner Geek optics offer significant cost reduction per bit, with at least a 20-30% power saving compared to current solutions. The 200 gig lambda generation will lower the cost per bit even further due to the increased efficiency of the optics.

100 Geek Lamdba and Beyond:
Tuner Geek optics are expected to be available in the lab this year and in production next year. The vast majority of 800 gig ports will likely be utilized in breakout modes, such as dual quad or octal hundred geek configurations, to connect servers in clusters.

OSFP Module Advantages:
The OSFP module supports breakout applications with multiple connectors, including dual LC, mini LC, or MPO, without obstructing airflow. This flexibility enables efficient use of the 810G port capacity in various aggregate use cases.

Key Features of the OSFP-XD:
The OSFP-XD is a quad-density version of the OSFP, featuring a dual-row connector with 16 electrical lanes. It supports 1.6 terabits with 100 gig service or 3.2 terabits with 20 gig service while maintaining backward compatibility with existing OSFP modules. This allows system vendors to fit a full 51.2 terabits in a 1U chassis, offering exceptional density.

1.6 Terabit Generation Modules:
Dual 800 gig CR modules are likely to be adopted for the 1.6 terabit generation, essentially combining two channels of 800 gig CR in a single chip. LR2 modules, optimized for the 10-kilometer range, will also be available using the same silicon. Data center optics will employ 20 gig PAM4, with either eight channels (CWMAT) or dual four channels (FOR), supporting 20 gig records. Higher channel counts are advantageous for lowering the cost per bit compared to 4-channel modules and reducing rack space requirements.

Increasing Density of Optics Modules:
Increasing the density of optics modules can reduce power consumption, cut costs, and enable denser network systems.

Data Center Interconnect:
Microsoft, Amazon, Google, and Facebook have been using data center interconnect to connect data centers. Formic CR, the first true multi-vendor standard for DWM technology, has been a significant success. It matches the density of the client form factor, allowing for easy integration into existing networks without sacrificing density.

Deployment of IP Over DWM:
The power reduction of the 7nm technology enabled the integration of the 400GE CR module into a standard client optics plug, allowing for native IP over DWM deployment.

Example Implementation:
In a running installed system, 36 ports or 400 gig CR per line card can be achieved, enabling the utilization of the entire C-band with just two line cards.

Variations and Applications:
OEF standard application code OXO is designed for 100-kilometer focused datacenter connections. CR+ versions provide additional OSNR improvements and forward error correction for extended reach. Open ROADM supports OTN for Telco applications.

Ethernet CR Design:
The design teams are working on the Ethernet CR design, which involves doubling the baud rate from 60 to 120 gigabytes per second while maintaining 16-QAM for backward compatibility. This will double the density per module and reduce the cost and power per bit.

Backward Compatibility and Advanced Modes:
The Ethernet CR design will have backward compatibility modes with Fornicate CR and CR+ and will support advanced Fornicate QPSK for long-distance connections with native Fornicate.

Solving the Campus Problem:
A cost-reduced version of the Ethernet CR design can potentially be used to address the campus problem.

Matching Next-Generation Routers and Switches:
The 800-gig CR will match the port definition of the next-generation routers and switches, enabling efficient utilization of 800-gig ports.

Data Center Trends:
Data center cloud data center bandwidth demand is consistently escalating by about 50% each year. At the chip level, an increase in the need for bandwidth is observed due to more powerful CPUs and GPUs requiring higher bandwidth. Higher density of chips is observed at the rack level, compelling a demand for increased bandwidth. Power consumption is declining at a rate of approximately half that of bandwidth growth for both copper and optics.

Important Areas for Progress:
Multi-chip packaging such as 2D, 2.5D, and 3D with improved surface area and power efficiency. Integration of various technologies in silicon photonics, including DML, EML, and materials like lithium niobate and titanium oxide.

Copper vs. Optics:
Copper remains a viable option for short-distance connections such as on PC boards and within racks due to its cost-effectiveness. Active and passive copper solutions can extend reach, with passive copper capable of reaching up to 1 meter and active copper reaching 2-3 meters within a rack. For longer distances beyond a few meters, optics is the preferred choice due to its superior performance and scalability.

Pluggable vs. Co-Packaged Optics:
Pluggable optics currently dominate the market due to their advantages in serviceability, manufacturability, and reliability. Co-packaged optics offer potential benefits in terms of power reduction and performance improvements, but challenges in manufacturability and reliability need to be addressed.

Photonic Integration:
Fully integrated silicon photonics, including lasers, is challenging due to manufacturing complexities. Current solutions often rely on assembly approaches, limiting the full realization of integration benefits.

Challenges in Integrated Optics:
Manufacturing integrated silicon photonics with embedded lasers remains a challenge, and the industry is still in the early stages of realizing the full potential of this technology. Although integration offers significant power savings, manufacturability remains a critical hurdle.

Beamforming for Loss Reduction:
Ongoing efforts are focused on reducing losses in optical interconnects, including the exploration of beamforming techniques. Beamforming can reduce losses, but it introduces additional challenges and is not yet widely adopted in production. Expanding beam connectors are seen as an enabling factor for co-packaged optics by reducing dust sensitivity issues.

Copper’s Role in Data Centers:
Copper solutions, including passive and active copper cables, are expected to continue playing a role in data centers, especially for short-reach applications. Passive copper can reach up to 1 meter, while active copper can extend the reach to 2-3 meters within a rack, offering cost-effective connectivity. Beyond 200 gigabit per second, the limitations of copper become more pronounced, and optical solutions become necessary for longer distances.

Fiber Infrastructure:
Existing fiber infrastructure remains valuable, driving the need for solutions compatible with it.

Power Density Limitations:
Data centers face power density constraints, limiting the number of servers per rack based on power budgets. Liquid-cooled racks offer higher power capacity but come with increased overhead.

Scaling and Power Consumption:
Scaling factors such as CPU speed, PAM4 availability, and cluster sizes align well with current trends. Power consumption poses a significant challenge, requiring transitions to smaller process nodes (e.g., 5nm and 3nm) to reduce power.

Data Center Network Topologies:
Quality ECMP with radix-type topologies (e.g., 32-way or 64-way) are widely adopted in cloud data centers. Leaf-spine topologies remain the standard for implementing high-speed networks due to their scalability and reliability.

Potential Innovation Areas:
Silicon transitions are predictable, with aggressive roadmaps from TSMC and Intel. Optics progress is slower due to manufacturing complexities and lack of extreme ultraviolet benefits.

Mesh Architectures for HPC and AI Clusters:
HPC and AI clusters may adopt mesh architectures for minimal latency and high-bandwidth requirements. Copper cables are preferred for cost-effectiveness in such scenarios.