Your options for building out the most cost-effective AI fabric are not limited to IB vs DDC. In fact, you can successfully leverage the huge investment in the massive, multi-vendor front-end DC fabric that already exists today by focusing the solution to emulating a lossless, low latency network on the endpoints. Of course, it requires the architects of said DC fabric to have the ingenuity and foresight to do so. More importantly, it’s already being done at one very notable hyperscaler.

The architecture, and how it got there, only matters inasmuch as it beats InfiniBand on real-world AI applications and if it is also cheaper, then there ya go. Sauce for the goose, Mr Savik.

If a fully scheduled fabric offers (near) perfect LB and congestion-free operation then why exactly do I need deep buffers?

Note: Those deep buffers come with an additional cost due to the off-chip memory requirement.

It seems to me they’re just trying to repurpose an existing chip with a 15+ year old architecture (i.e., you’re paying for the additional packet memory whether you need it or not), throw “AI” on the end of the name (because we all know the mere mention of “AI” and your stock increase 20% overnight), and inevitably force you into a vendor lock-in strategy.

Correct.

“does not prohibit” -> prohibits

Patience young gracehopper, post-exascale kung-fu gastronomy cannot be rushed; to wit, AI-formulated plant-based cheeses will not hit shelves until 2024 (Bel Group and Climax Foods). According to nVidia dev blog meditations (Yamaguchi & Busato, 2021):

“sparse linear algebra […] does not provide competitive performance [yet …] when sparsity is below 95% […] due to […] scattered memory accesses”

300 PhDs a Shaolin Temple did not hence make(?) … and is a bird in the hand really worth two stoned (or vice-versa — as in the next TNP article)?

We all heard before the theoretical scale of Jerico2 – in reality it wasn’t even close!
Looks like another chip with great marketing but not really optimised.

I think that it is because during training, one deals mostly with dense (full) weight matrices for the NN layers. As with HPC’s HPL dense-matrix benchmark; latency is less critical for this than bandwidth as you can pretty much batch-burst the needed data into caches or buffers, for the computational units, karate-style. Once the ANN is trained on its humongous dataset, one may want to prune it (esp. for weight storage efficiency) resulting in sparse weight matrices, to be used for inference. The inference situation may then be similar to that of the sparse-matrix HPCG benchmark of HPC, where latency is more critical, and some form of memory-access kung-fu becomes valuable. NVIDIA has developed some special sparsity-support-hardware for this purpose if I’m not mis-mixmetaphoring (along with adaptive 4-bit quantization, and more, as discussed in recent TNP pieces).

It must be the case or this would be a Tomahawk variant.