Strong Scaling of OpenACC enabled Nek5000 on several GPU based HPC systems

Jonathan Vincent, Jing Gong, Martin Karp, Adam Peplinski, Niclas Jansson, Artur Podobas, Andreas Jocksch, Jie Yao, Fazle Hussain, Stefano Markidis, Matts Karlsson, Dirk Pleiter, Erwin Laure, Philipp Schlatter

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

We present new results on the strong parallel scaling for the OpenACC-accelerated implementation of the high-order spectral element fluid dynamics solver Nek5000. The test case considered consists of a direct numerical simulation of fully-developed turbulent flow in a straight pipe, at two different Reynolds numbers Reτ = 360 and Reτ = 550, based on friction velocity and pipe radius. The strong scaling is tested on several GPU-enabled HPC systems, including the Swiss Piz Daint system, TACC's Longhorn, Jülich's JUWELS Booster, and Berzelius in Sweden. The performance results show that speed-up between 3-5 can be achieved using the GPU accelerated version compared with the CPU version on these different systems. The run-time for 20 timesteps reduces from 43.5 to 13.2 seconds with increasing the number of GPUs from 64 to 512 for Reτ = 550 case on JUWELS Booster system. This illustrates the GPU accelerated version the potential for high throughput. At the same time, the strong scaling limit is significantly larger for GPUs, at about 2000 - 5000 elements per rank; compared to about 50 - 100 for a CPU-rank.

Original languageEnglish
Title of host publicationProceedings of International Conference on High Performance Computing in Asia-Pacific Region, HPC Asia 2022
PublisherAssociation for Computing Machinery
Pages94-102
Number of pages9
ISBN (Electronic)9781450384988
DOIs
StatePublished - Jan 7 2022
Event5th International Conference on High Performance Computing in Asia-Pacific Region, HPC Asia 2022 - Virtual, Online, Japan
Duration: Jan 12 2022Jan 14 2022

Publication series

NameACM International Conference Proceeding Series

Conference

Conference5th International Conference on High Performance Computing in Asia-Pacific Region, HPC Asia 2022
Country/TerritoryJapan
CityVirtual, Online
Period01/12/2201/14/22

Keywords

  • Benchmarking
  • Computational Fluid Dynamics
  • Nek5000
  • OpenACC
  • Scaling

Fingerprint

Dive into the research topics of 'Strong Scaling of OpenACC enabled Nek5000 on several GPU based HPC systems'. Together they form a unique fingerprint.

Cite this