Andreas von Bechtolsheim, founder of Sun Microsystems and chairman of Arista, presents on the latest developments in optics for data center access and long haul. Bechtolsheim emphasizes the importance of Moore’s Law and the ongoing progress in chip design and manufacturing. He discusses the challenges and opportunities of adopting denser process technologies for networking chips and the implications for the industry.

Moore’s Law and Chip Design Trends
Moore’s Law continues to hold, with transistor counts doubling every two years, despite concerns about its limits. The roadmap for further miniaturization, down to 2.1 nanometers and beyond, is well-defined. Chip design costs are skyrocketing, with a single 7-nanometer chip costing around $50 million to design.

Implications for Networking Chips
Networking chips have traditionally lagged behind CPUs in process technology, but are now catching up. This has led to rapid progress in networking performance, with each new process generation bringing significant improvements in density, speed, and power efficiency. The high design costs favor a merchant silicon business model, where chip designs are amortized over large volumes.

Merchant Silicon and the Future of Networking
Merchant silicon chips have become the norm for switching, routing, and optical transport due to their rapid innovation and cost-effectiveness. Custom designs by routing companies are becoming less common as merchant silicon chips offer superior performance and features.

Moore’s Law and Merchant Silicon:
Rapid improvements in networking chip performance are a result of Moore’s Law and the use of increasingly powerful merchant silicon. Custom chip designs are rarely economically viable due to the limited market size for networking chips.

Electrical Lane Speed Improvements:
Throughput per chip has increased significantly thanks to faster electrical lanes, going from 10 gigabits per second to 100 gigabits in the last decade. Next-generation silicon chips will all support 100 gigabit speeds, enabling 400 gig ports with four channels and 800 gig ports with eight channels.

100 Gigabit Service Adoption:
100 gigabit service is gaining traction and predicted to dominate the market by 2023, exceeding the total bandwidth of the entire market in 2022. Its adoption will be driven by cost-effectiveness and better technology compared to previous generations.

Optical Choices for 100 Gigabit Service:
4×25, CWM4, and PSIM4 optics, which use 100 gig lambda per channel, are the optimal choice for 100 gigabit service. Gearboxes are required for other optics to convert between 100 gig electrical channels and 4×25, adding cost and power consumption.

Market Outlook for 100 Gigabit Optics:
100 gigabit service is expected to have a long life and high volume, with a market forecast of 1 billion channels shipped between 2021 and 2031. This substantial market opportunity has attracted significant investments from the industry.

100 Gigabit Optical Module Form Factors:
100 gig SFP: Same form factor as 10/25 gig SFP, available in various single-mode and multimode versions. 400 gig QSFP: Built with four lanes of 100 gig service, compatible with legacy QSFPs. 800 gig OSFP/QSFP-DD: Supports eight lanes of 100 gig lamb or 800 gig optics, offering the highest density and lowest cost per bit.

Progress in Module Speeds:
Modules have kept pace with increasing service speeds, from 10 gigabit to 25, 50, and now 100 gigabit. 800 gig OSFP samples are already available in the lab, expected to enter full production in less than a year.

Variety of Optical Solutions:
No single optical solution fits all distances in networking.

Introduction:
Discussion on various types of optics used in data centers, including multimode, datacenter, longer reach, extra-long reach, and DWM optics.

Speeds and Optics:
Each speed (100 gigabit, 400 gig, 800 gig) has different optics matching specific technologies and reaches.

SR Optics:
SR in the 100 gig SIRTES environment will support up to 100 meters on OM4 fiber (IEEE spec goal).

DR, LR, and ER Optics:
DR specs are complete, supporting two kilometers for 100 gig DR, 400 gig DR4, and 800 gig DR8. LR specs target 10 kilometers, with practical reach on good fibers. ER specs will extend the reach up to 40 kilometers (or perhaps 30-40 kilometers).

CR Optics:
CR optics use high-order modulation, such as 16-QAM, for longer reach.

Other Optics:
BIDI optics will offer the same reach as Honegeek ER over single-mode fiber, saving on fibers. Honegeek optics will have SR4, DR4, FR4, LR4, ER4, and CR variants. 8-Honegeek optics will include SR8, DR8, and dual 400 gig FR4 or LR4 in one module.

Conclusion:
The focus on 800-gig optics is driven by cost considerations, leveraging 100 gig lambda optical engines and high-order modulations.

800 Gig Optics Cost Savings:
800 gig optics are predicted to be one-third to 40% cheaper per bit compared to current 400 gig modules. The cost reduction comes from having one DSP chip for 800 gig optics instead of two chips needed for 400 gig optics. Additional savings come from fewer module connectors, PC boards, and smaller system sizes.

800 Gig Optics and 400 Gig Interoperability:
800 gig optics can be plugged into 400 gig ports and vice versa, allowing for easy integration with existing infrastructure.

Ethernet Technology Alliance and 800 Gig Ethernet:
The Ethernet Technology Alliance, including companies like Microsoft, Broadcom, Cisco, and Aristos, has completed the 800 gig Ethernet specification. The IEEE is expected to finalize the 800 gig Ethernet standard in four years, with chips shipping two years before the standard’s completion.

Optical Interoperability:
Optical interoperability is achieved when the optics from one device can match the optics from another device. Future SFP 100 gig DR optics with 100 gig electrical channels will be interoperable with first-generation QSFP 100 gig DR optics that use a gearbox. Multi-speed 400 gig optics can be configured to support different speeds, including 200 gig FR4 and 100 gig C11-4, without changing the optics.

Physical Interoperability:
New connectors like the SN or MDC connector allow for four 100 gig ports on a single QSFP or QSFP-DD, eliminating the need for MPO-MTP splitter cables. Future dual 400 gig FR4 modules may support regular LC connectors, the most popular connector type in the industry.

Data Center Optics:
Future silicon will have 100 gig SIRTIS, compatible with 100 gig LAMPTA optics. The transition from 25 gig LAMPTA to 100 gig lambda optics is expected to be predictable and long-term. New form factors like 100 gig SFP, 400 gig QSFP, and 800 gig OSFP or QSFP-DD are enabled by these developments.

Multi-Speed Optics:
Multi-speed optics are backwards compatible with lower-speed grades, including Honegeek C74, which can communicate with Honegeek FR4.

Tunageek Optics:
Early discussions and efforts are underway to develop Tunageek SIRTIS and Tunageek optics, but the technology is still in its early stages and requires new forward error correction algorithms.

800-Gig Optics and Lambda Optics:
Initial goal: 800-gig module with four 200-gig lambda optics and 800-gig SIRTUS electrical lanes. Project aims to achieve lower cost than DR8 or dual FR4 modules. First samples anticipated in 2024. 200-gig lambda optics adoption tied to 800-gig ethernet. 400-gig ethernet likely to remain with 100-gig lambda optics.

Extended Reach Technologies:
Two main categories: 10-40 km and 400-gig CR. 100-gig ER: Up to 8 channels in one fiber, passive multiplexing, reaching 30-40 km without amplification. 400-gig CR and CR++: Up to 64 channels per fiber, enabling 25.6 terabits per fiber, requiring external amplification. ZR can be deployed but might be more expensive for 10-40 km applications.

100-Gig ER for Access Links:
Newly proposed at the Lambda Consortium. Targets 5G cell towers, commercial customers, and internet exchange points. Simplified, reliable, and power-efficient solution.

Deployment and Product Roadmap:
100-gig ER in QCP form factor: First product with samples available now. 400-gig ER4 in OSFP and QCF PDD form factor: Next product. 800-gig ER8 aggregating eight 100-gig channels: In two years.

Technical Details and Comparison:
100-gig ER4 module power consumption: 3.5 watts. Mature technology: EML laser, germanium silicon, APD, and strong equalizer. Comparison to ZR: Twice the power and cost for ZR in access links.

Introduction:
Andy Bechtolsheim presents an informative overview of Vornegeek CR and CR+ optics, highlighting their revolutionary impact on optical networking. These next-generation optics offer significant advantages in cost performance, multi-vendor interoperability, and power efficiency.

Benefits of Vornegeek CR and CR+ Optics:
Multi-vendor standard with interoperable designs from multiple vendors ensures lower costs and eliminates vendor lock-in. Utilizes same small pluggable form factor as client optics, eliminating the need for special line cards or configurations. Low power consumption of around 20 watts, ideal for the OSFP or QSFPDD form factor. Revolutionary price performance, approximately one-tenth of previous costs for optical shelves with optical transponders.

Impact on Optical Networking:
Enables the obsolescence of optical shelves and allows for direct IP traffic routing over optical ports. Mix-and-match capability with client ports on any router, eliminating the need for special cards. Projected to transform the market by 2024, replacing most other solutions for high-volume use cases like DCI and metro interconnect.

Current Status and Availability:
Initial modules were tested in January 2023 and successfully established links between the same vendor. Firmware and configuration issues between different vendors resulted in an eight-month delay in achieving interoperability among all well-known DSP vendors. Expected availability of samples in late 2023, with increased volume in Q1 2024.

Specific Applications:
Application code 0x01: 80 to 120-kilometer external amplification high-volume DCI use case. Application code 0x02: Gray optics version with single 400 gig lambda, non-tunable, and higher output power for 40-kilometer point-to-point links without amplification. Application code 0x03: Fornicate CR+, reaching 300 to 600 kilometers, covering traditional metro reach distances.

Standardization and Interoperability:
OpenCR+ specification for CR+ interoperability among two out of three DSP vendors, with the third expected to meet the standard soon. Multi-vendor standard ensures interoperability among all vendors.

Product Roadmap:
O2 module: Increased output power to 0 dB. Improved receiver sensitivity. Better link budget. 40 km reach with 0.25 dB/km loss. ZR+ module: Focus on getting ZR into production. Qualify ZR+ interoperability. Two out of three silicon chips expected to work in ZR+ mode. Gray optics module also available. CR+ module: Projected availability to customers in the second half of 2021.

800G Roadmap:
800G CR module: Under development. 5-nanometer process technology. Same 16-QAM coding as 400G CR. Doubled baud rate (120 gigabaud). Doubled spectrum. Similar bits per fiber as 400G CR. Lower cost per bit (half of 400G CR). Supports FlexE CR backwards compatibility modes. Lower power consumption than 7-nanometer Fornegeek modules. First samples expected in 2022, production in 2022.

Other Details:
Arista supports all DSP vendors for customer qualification. Support for OSFP and DD form factors. Arista’s operating system supports IP or DWM with monitoring functions, OpenConfig, and Google RPC. Lab trials underway. Endless forecast to transform the transport market.

Optical Line Systems for 800, 400 Gig CR and CR Plus:
Conventional transponders and Photonic Coherent require different characteristics. Installing Photonic Coherent Receiver with traditional optical line systems is not part of the standard.

Enhanced Photonic Coherent Receiver Plus Module:
Increased output power and improved specs enable compatibility with existing optical line systems. Support for 50 GHz grids and extended reach with FEC modes. Simplification of DCI use cases with Arista’s optical line system in OSFP form factor.

Ease of Use and Auto-Configuration:
Plug-and-play solution with auto-configuration via EOS operating system. Future systems will have dedicated optical line system ports to avoid losing network-facing ports.

Addressing Legacy Systems:
Enhanced versions of fornicate zero will work with existing optical line systems with vendor support. Brand new OCP form factor modules for vanilla fornicate CR.

Cost Benefits and Performance:
Merchant silicon routers offer an order of magnitude lower cost per port compared to legacy routers. CR modules are significantly lower in cost compared to traditional DWM and optical line systems. A rare occurrence of a 10x cost reduction in one generation.

Benefits of 400-Gig Pipes:
Easier management compared to multiple smaller pipes. Cost-effective when compared to existing smaller pipes.

Market Availability:
Field trials in late 2020 and production in the first half of 2021 for DCI use cases. Production for longer-reach C++ use cases in the second half of 2021.

Power Consumption:
Power per port for 400 gigabit ports is higher than 100 gigabit ports. 7 nanometer (7nm) process technology offers significant power reduction compared to 6 nanometer (6nm) chips. 7nm chips and 7nm DSP optics reduce power consumption. 400 gigabit ports are still higher power than 100 gigabit ports, but power constraints in data centers may limit port support.

Cost:
TSMC’s cost per function for chips made in 6nm, 7nm, and 5nm does not decrease as the chip size shrinks. 7nm chips are more expensive to produce than 6nm chips due to increased manufacturing costs. The high cost is attributed to the limited number of companies with advanced process technology and the significant investment required for new fabs.

Yield:
Information on the yield per wafer in 7nm and 5nm is not disclosed.

Future of Co-Packaged Optics:
Information on the future of co-packaged optics versus regular plugables is not provided in the given text.

Optics Reliability and Co-Packaging
Co-packaging optics modules with switch chips is not practical due to reliability concerns. Optics modules have a low Mean Time Between Failure (MTBF), and co-packaging would increase the failure rate of the entire system. Reliability needs to be significantly improved for co-packaging to become viable.

Multi-Speed Optics Configuration
The Cheshire switch chip allows for multi-speed optics configuration via software. The first-generation optics modules were speed-locked, but newer versions support multi-speed modes. Configuration is done manually through the Command-Line Interface (CLI).

Physical Transceiver Formats: OSFP vs. DD
OSFP has a larger installed base than DD and offers better thermal performance. DD modules run hotter and have a higher failure rate compared to OSFP modules due to temperature differences. OSFP is preferred for high-power modules like Ethernet Geek, which will have a projected power consumption of 24 watts.

Gray Slash CVDM 100G ER Dispersion Limited Distance
The dispersion-limited distance for single-lane gray slash CVDM 100G ER is between 30 and 40 kilometers. It is not suitable for 80-kilometer distances, unlike 100G ER4, due to increased dispersion.

Port Density and Server Infrastructure
With 800G chips, it is possible to achieve 25.6 terabits per 1U, providing extremely high port density. Server infrastructure is catching up, with the introduction of PCI Gen 4 and new server chips that can support dual 100-gigabit uplinks. Dual 100-gigabit NICs are expected to become common in the next two years for improved throughput and cost-effectiveness.

Closing Remarks
The speaker, Andreas Bechtolsheim, concluded the presentation by expressing his hope that the discussion was interesting and informative. He showed his enthusiasm and willingness to engage in future discussions on these topics.