Comments on: He Who Can Pay Top Dollar For HBM Memory Controls AI Training https://www.nextplatform.com/2024/02/27/he-who-can-pay-top-dollar-for-hbm-memory-controls-ai-training/ In-depth coverage of high-end computing at large enterprises, supercomputing centers, hyperscale data centers, and public clouds. Thu, 13 Jun 2024 05:57:15 +0000 hourly 1 https://wordpress.org/?v=6.5.5 By: Dr. DG https://www.nextplatform.com/2024/02/27/he-who-can-pay-top-dollar-for-hbm-memory-controls-ai-training/#comment-225511 Thu, 13 Jun 2024 05:57:15 +0000 https://www.nextplatform.com/?p=143720#comment-225511 In reply to Timothy Prickett Morgan.

really astute observations and article quality refreshingly high !
– from the inventor ( 30 yrs ago, all in AZ ) of the technology they now use in SoK or Idaho to stick one HBM die on top of another

]]> By: Dr. DG https://www.nextplatform.com/2024/02/27/he-who-can-pay-top-dollar-for-hbm-memory-controls-ai-training/#comment-225510 Thu, 13 Jun 2024 05:53:24 +0000 https://www.nextplatform.com/?p=143720#comment-225510 In reply to Slim Albert.

working on it ! Have to wait for 3 yrs

]]> By: Timothy Prickett Morgan https://www.nextplatform.com/2024/02/27/he-who-can-pay-top-dollar-for-hbm-memory-controls-ai-training/#comment-223981 Sun, 05 May 2024 11:50:18 +0000 https://www.nextplatform.com/?p=143720#comment-223981 In reply to ABHIJIT K RAO.

And that is not what Nvidia is paying for the memory, but an allocation of the cost of the memory at the end user level. Just to be clear.

]]> By: Timothy Prickett Morgan https://www.nextplatform.com/2024/02/27/he-who-can-pay-top-dollar-for-hbm-memory-controls-ai-training/#comment-223980 Sun, 05 May 2024 11:49:16 +0000 https://www.nextplatform.com/?p=143720#comment-223980 In reply to ABHIJIT K RAO.

Here were my best guesses:

https://www.nextplatform.com/2024/02/27/he-who-can-pay-top-dollar-for-hbm-memory-controls-ai-training/

$10,600 for HBM2e for 96 GB, $8,800 for HBM3 at 80 GB, $16,500 for HBM3E at 96 GB. Call it a little less than half of the cost of the SXM module.

We have long known that the cheapest part of an SXM is the Nvidia GPU, and that the packaging and the HBM memory represent most of the cost.

]]> By: ABHIJIT K RAO https://www.nextplatform.com/2024/02/27/he-who-can-pay-top-dollar-for-hbm-memory-controls-ai-training/#comment-223968 Sun, 05 May 2024 06:48:19 +0000 https://www.nextplatform.com/?p=143720#comment-223968 What would be the approximate cost percentage of HBM in the H100/H200 BOM? Could it be 40% of the cost?

]]> By: Slim Albert https://www.nextplatform.com/2024/02/27/he-who-can-pay-top-dollar-for-hbm-memory-controls-ai-training/#comment-220921 Fri, 01 Mar 2024 01:12:38 +0000 https://www.nextplatform.com/?p=143720#comment-220921 Outstanding piece (very nuanced)! One thing I’ve been wondering about recently, is whether Intel’s transparent glass substrate (for chips) could be used to ease chip-stacking and interposer (eg. CoWoS) challenges by making it possible to transmit signals optically through it (rather than with copper vias). I can’t remember reading anything on that possibility (maybe far-fetched?), as the tech was pitched at thermal stability I think, but if it also allows for light-signalling, it may help with having to route 2,048 wires from each HBM stack to all nearby CPUs …

]]>