VP Nvidia Americas. Former Silicon Valley CEO (2x) and VP/GM Intel and IBM alum. AI and HPC technologist. Outdoor and business enthusiast. Family first!
#ISC24 No.1 again (in HPCG and Graph500) for #Fugaku, for nine consecutive times over 4 years. We ourselves did not expect to sustain this long which is both honorable and alarming.
Asked ChatGTP why optical I/O is better than electrical:
Optical I/O (Input/Output) offers several advantages over traditional electrical I/O, particularly in the context of large GPU clusters for AI applications. Here are key reasons why optical I/O is considered superior:
1. **High Bandwidth:**
- Optical fibers can transmit data at significantly higher speeds compared to electrical conductors. This high bandwidth is crucial for the massive data transfers inherent in AI applications within GPU clusters.
2. **Low Latency:**
- Optical signals travel at the speed of light, resulting in lower latency compared to electrical signals. In AI scenarios where real-time processing is essential, reduced latency enhances overall system performance.
3. **Longer Distances:**
- Optical signals can travel over longer distances without significant signal degradation. This feature is advantageous in large GPU clusters, where data needs to traverse extensive interconnects.
4. **Immunity to Electromagnetic Interference (EMI):**
- Optical fibers are immune to electromagnetic interference, a common issue with electrical conductors. This immunity ensures data integrity and reliability, especially in environments with high electromagnetic noise, such as data centers.
5. **Reduced Signal Attenuation:**
- Optical signals experience less signal attenuation over distance compared to electrical signals. This characteristic allows for the efficient transmission of data across the interconnected GPUs in a cluster.
6. **Security:**
- Optical communication is inherently more secure than electrical communication. Optical signals do not radiate electromagnetic fields, making them less susceptible to eavesdropping and providing an added layer of data security.
7. **Thinner and Lighter Cables:**
- Optical fibers are thinner and lighter than their electrical counterparts, contributing to a reduction in overall system weight and complexity. This is especially relevant in situations where space and weight considerations are crucial.
8. **Future-Proofing Technology:**
- The demand for higher data rates and more efficient communication continues to grow. Optical I/O provides a scalable solution that aligns with the increasing requirements of AI and large-scale GPU clusters, offering a level of future-proofing.
In summary, optical I/O's advantages in terms of high bandwidth, low latency, reliability, security, and scalability make it a preferred choice for supporting AI GPU to GPU communication in large GPU clusters, contributing to overall system performance and efficiency.
To quote myself: “future improvements are likely to focus less on CPUs and GPUs and more on enhancing the performance of the memory and the interconnect fabric. The most promising avenues for reducing energy consumption per data movement mainly lie in memory technologies and packaging of chips, as well as advanced photonics”
CEO of @AyarLabs Charlie Wuischpard (@Wuischpard) shared insights on how Ayar Labs is creating #optical I/O technology in light of increasing demands for #AI. Read more from @WSJ about the goals associated with optical-enabled #chips.
Check out our new demo running @AyarLabs optical I/O solution integrated with @intel Agilex FPGA technology in a PCIe card form factor.
See the demo live at the @hotchipsorg or @GlobalFoundries Technology Summit 2023.
https://t.co/DgeuZx4WDy
@IntelFPGA #HotChips2023#GTS2023
It's happening! We're getting RGB in processors. Kind of. We're talking optical interconnects that use colors. The colors are in the cable. In this vid, we go through @hotchipsorg 2023 in August - a showcase what silicon of the future.
https://t.co/d9Le74QByP
The 31st Annual Cushing Crossing went down this past weekend, and it was one for the books! One of our most challenging Cushing Crossing setups to date treated us to some hilarious moments, big spills, and the occasional victory celebration 🌊 ⛷️
#SVB has been a super partner to the VC and innovator community. I've been super disappointed at the panic in this same community that brought us to this point. It will accelerate the demise of bad ideas and companies, but it will hit good ideas and innovation, too.
@NicoleHemsoth That’s easy! Full optical interconnect between processors and between processors and memory. Save 30% total power as baseline. Then take bandwidth/latency benefit to build new architectures to deliver huge perf/watt improvements.
California-based startup Ayar Labs is developing a technology called silicon photonics, where data is transferred among computer chips by optical rays https://t.co/OLiUKuArFR