Comments on: The Art Of System Design As HPC And AI Applications Diverge https://www.nextplatform.com/2022/10/06/the-art-of-system-design-as-hpc-and-ai-applications-diverge/ In-depth coverage of high-end computing at large enterprises, supercomputing centers, hyperscale data centers, and public clouds. Mon, 24 Oct 2022 23:48:24 +0000 hourly 1 https://wordpress.org/?v=6.5.5 By: Barry Bolding https://www.nextplatform.com/2022/10/06/the-art-of-system-design-as-hpc-and-ai-applications-diverge/#comment-199803 Mon, 24 Oct 2022 23:48:24 +0000 https://www.nextplatform.com/?p=141319#comment-199803 Nice article TPM. Look forward to seeing you at SC22

]]> By: HuMo https://www.nextplatform.com/2022/10/06/the-art-of-system-design-as-hpc-and-ai-applications-diverge/#comment-199021 Sun, 09 Oct 2022 18:49:22 +0000 https://www.nextplatform.com/?p=141319#comment-199021 In reply to EC.

I see, “Fourier neural operator” (for PDEs), that may be what Tim was hinting at: “nature itself doesn’t actually do math […] maybe […] more intelligent guessing” — an alternative to domain and equation discretization (maybe ADI), reordering (reverse Cuthill-McKee), linear algebraic system solution (matrix). Interesting research (if it works…)!

]]> By: EC https://www.nextplatform.com/2022/10/06/the-art-of-system-design-as-hpc-and-ai-applications-diverge/#comment-198931 Fri, 07 Oct 2022 16:23:17 +0000 https://www.nextplatform.com/?p=141319#comment-198931 Exactly. The idea of modeling say the earth’s entire weather system in SOTA FP64 models is impossible with today’s technology, or tomorrow’s for that matter. So these models compromise both reach and fidelity. OTOH, the Earth-2 digital twin model and system Nvidia is building will do just that at both higher fidelity and provide a global reach. (https://www.forbes.com/sites/karlfreund/2022/04/03/nvidia-earth-2-leveraging-the-omniverse-to-help-understand-climate-change/?sh=4d904625491f) One can only get there with mixed precision computation and ML. The number of HPC environments succumbing to ML will only increase in time.

]]> By: Art Scott https://www.nextplatform.com/2022/10/06/the-art-of-system-design-as-hpc-and-ai-applications-diverge/#comment-198926 Fri, 07 Oct 2022 13:48:14 +0000 https://www.nextplatform.com/?p=141319#comment-198926 Don’t mitigate. Eliminate.

No hotspots!
HotGauge: A Methodology for Characterizing Advanced Hotspots in Modern and Next Generation Processors
https://sites.tufts.edu/tcal/files/2021/11/HotGauge_IISWC_2021.pdf
NNShim: Thermal Hotspots Simulation on ML Accelerators
https://sites.tufts.edu/tcal/files/2022/06/ISCA22_HSSB_paper_5.pdf

]]> By: HuMo https://www.nextplatform.com/2022/10/06/the-art-of-system-design-as-hpc-and-ai-applications-diverge/#comment-198904 Fri, 07 Oct 2022 04:56:31 +0000 https://www.nextplatform.com/?p=141319#comment-198904 Very nice piece! I think that this three-pronged pitchfork in the road (combining fork and knife) has been sparking innovation in HPC algorithm research, to take advantage of GPU (accelerators) capabilities, beyond pure parallelism, towards mixed-precision speedups. It is challenging, because, for example, single precision Runge-Kutta doesn’t generate closed orbits in phase-space for solutions of the Lotka-Volterra system (it produces spirals, which is incorrect). But mixed-precision Runge-Kutta (FP32+FP64), applied to the Van Der Pol system (nonlinear, cubic) is reportedly accurate and faster than a purely FP64 implementation. Mixed-precision iterative refinement solvers (FP16 to FP64) also seems quite accurate, faster, and more energy efficient, than pure FP64 methods. There is definitely productive cross-fertilization between hardware advancements for AI/ML/gaming (where the bulk of financial rewards lie) and algorithm development for HPC (where most benefits to humanity are located) I think. Mixed-precision HPC is still mostly research today, but has great potential for mainstreaming (the benefits are huge where it is both stable and accurate). Quantum computing, with its half- and 3/4-bits, may yet further revolutionize HPC, but that will be in a future that cannot be fully observed, without modifying it!

]]>