Can you expand on this static-vs-dynamic thing? Are hyperscalers actually modifying the fabric topology dynamically, in response to job layout on nodes?

Risk-management, because IB is a safe and well-understood fabric for clusters. And arguably until recently ethernet simply didn’t scale easily.

Another interesting point is that IB has traditionally carried aspirations of smarter networking – verbs, offloading collectives, etc. Is it true that IB efforts in this direction have failed? I still see marketing about smarter IB NICs – or is it just that that kind of offload isn’t leveraged by the central AI frameworks?

InfiniBand had low latency in a port to port hop inside the switch and that often translated into a very big performance difference across hundreds to thousands of endpoints. But now we are at tens of thousands of endpoints moving on to hundreds of thousands to perhaps a million, and the game is different. It is a statically configured network (InfiniBand) versus a dynamically configured one (Ethernet) and it is 3X the cost versus 1X the cost. There was a time when InfiniBand was cheaper and better and equally scalable. I remember it. Now it is more expensive, and not as scalable for a given number of layers in a network, and soon it will not have lower end-to-end latency for real applications. Nvidia would do well to license whatever Microsoft did with InfiniBand and add it to the Quantum line to add multitenancy and security.

It sure looks like it to me.

It’s been like the Hundred Years’ War between The Ethernets and the Infinibands, but it looks like the high-command field strategists and generals of the Ultra Ethernet Consortium have finally developed the needed low latency packet spraying dynamic load balancing and congestion control weaponry needed for their ultimate takeover of AI and HPC comms territories … but by 2025 this time (almost right away, but not quite, due to tail latency?). Will the “confederates” eventually win this one out against the “secessionists”? History will tell:

Greasing The Skids To Move AI From InfiniBand To Ethernet (2024)
Ethernet Consortium Shoots For 1 Million Node Clusters That Beat InfiniBand (2023)
CISCO Guns for Infiniband (2023)
Broadcom Takes on Infiniband (2023)
Google abandons ethernet to outdo infiniband (2022)
The Eternal Battle Between InfiniBand And Ethernet In HPC (2021)
Infiniband, still setting the pace for HPC (2020)

In the early days of computing, you could throw what would be the equivalent of a few million to transition from binary coding to assembly to using higher level languages for efficiency improvements of several thousand if not hundreds of thousands of percent in coding time.

Now it’s tens to possibly hundreds of billions, for nebulous promises of tens of percent, in both performance of the system itself and of what the output can do.