Comments on: The First AI Benchmarks Pitting AMD Against Nvidia https://www.nextplatform.com/2024/09/03/the-first-ai-benchmarks-pitting-amd-against-nvidia/ In-depth coverage of high-end computing at large enterprises, supercomputing centers, hyperscale data centers, and public clouds. Thu, 05 Sep 2024 01:18:49 +0000 hourly 1 https://wordpress.org/?v=6.5.5 By: Calamity Jim https://www.nextplatform.com/2024/09/03/the-first-ai-benchmarks-pitting-amd-against-nvidia/#comment-233420 Wed, 04 Sep 2024 19:27:44 +0000 https://www.nextplatform.com/?p=144643#comment-233420 In reply to Slim Albert.

Alrighty then, I’ll bite … so, from that Token-to-PeakFP16 ratio:
1) H100-SXM*8 to H200-SXM*8 gives 4.14/2.73 = 1.5x advantage to H200 from 76% more HBM in its saddlebags;
2) H200-SXM*1 to B200-SXM*1 is a 4.78/4.25 = 1.1x advantage for taming the Blackwell bronco (with 28% more HBM), and;
3) MI300X*8 to H100-SXM*8 means a 2.73/2.11 = 1.3x advantage for TensorRT+CUDA vs PyTorch+ROCm, which seams believable at this here outside leg juncture of the barrel racing cloverleaf …
And if that’s a sensible enough hold up for you, I reckon horses ain’t quite left the barn yet, AMD’s got its work cut-out for itself, it’s in that there BBQuda secret sauce, just got to ROCm’it and SOCm’it, real good now!

]]> By: Slim Albert https://www.nextplatform.com/2024/09/03/the-first-ai-benchmarks-pitting-amd-against-nvidia/#comment-233408 Wed, 04 Sep 2024 16:05:51 +0000 https://www.nextplatform.com/?p=144643#comment-233408 Magnificent analysis! The ratio of Server-Tokens-per-second (perf on workload) to peak-FP16-Teraflops (max theoretical perf) is a wonderful tool to use here. As you astutely note, it gives a rating of how well-balanced the architecture is (eg. memory-wise, as in H200 vs H100), and how well the software stack manages to take advantage of that arch (eg. PyTorch+ROCm vs TensorRT+CUDA). Additionally, in Blackwell, the chip (and/or model weights, and/or software) have the possibility of FP4 computation that can boost perf by 2x vs FP8 (possibly 4x vs FP16), and this is not available in either MI300X/325X nor H100/200. The FP4 and FP6 in next year’s MI350X should give it the wherewithal to address that B200 feature.

It’s great to see AMD finally coming out of the MLPerf closet with results for the MI300 family. Beyond pricing for an AI perf similar to the H100/200 competition, I think that their value proposition also includes the possibility of running cloud-HPC FP64 workloads at the highest performance, which can help hedge one’s bets, if desired.

]]> By: Patrice https://www.nextplatform.com/2024/09/03/the-first-ai-benchmarks-pitting-amd-against-nvidia/#comment-233361 Wed, 04 Sep 2024 02:14:13 +0000 https://www.nextplatform.com/?p=144643#comment-233361 I expect the MI325X to surpass the H200 and the B100 (at worst on par?). The B200 having higher wattage consumption will be hard to beat.
Like you said, if AMD is pricing correctly the MI325X (around $30K?) it can be a great $/perf card and fit perfectly in the market.
I expect both AMD and NVIDIA to be supply constraints though, so little relief on $/perf for the H100/H200.
The Silo and ZT acquisition will probably help AMD a lot on the software side, software optimization is one thing that AMD has been doing really well recently if given some time.
Lisa said in a previous interview that she has a lot of supply from the manufacturer; we will see soon if that is true.
Last one, I hope AMD will launch a 1000W MI350X (or watercooled one), otherwise, NVIDIA will own the higher end of this market until Q1 2026 at least.

]]>