Right on! Lambos don’t get great performance on dirt roads … In the SCREAM paper, the coarse problem’s (110km) strong-scaling looks best at around 8 nodes (GPU speedup approx 3.5) which gives a compute density near 700 spectral elements per node (Table 1), that obtains also at the upper end of the scaling done on Frontier for 3.25km (6.3 million spectral elements, over 8,192 nodes). Neat! Can’t wait for the 10 EF/s box that will run the 1km-grid version of this (with great comps & comms)!

“Running at the strong scaling limit, the performance is dominated by communication costs, and GPU speedups are diminished”

The CPU-only system actually overtakes the GPU-accelerated one at this limit (40 nodes, 110km coarse config.) because networking can’t keep up (this is for Perlmutter; Frontier’s networking is just fine as shown later in Fig. 4)! All this to say that the user-available performance of the gargantuan machines being fielded these days (for HPC and AI) relies in no small way on equivalently performant networking (not to be forgotten as we copiously salivate over the specs of mammoth CPUs and elephantine accelerators!)!