HOMMEXX 1.0: a performance-portable atmospheric dynamical core for the Energy Exascale Earth System Model
Open Access
- 11 April 2019
- journal article
- research article
- Published by Copernicus GmbH in Geoscientific Model Development
- Vol. 12 (4), 1423-1441
- https://doi.org/10.5194/gmd-12-1423-2019
Abstract
We present an architecture-portable and performant implementation of the atmospheric dynamical core (High-Order Methods Modeling Environment, HOMME) of the Energy Exascale Earth System Model (E3SM). The original Fortran implementation is highly performant and scalable on conventional architectures using the Message Passing Interface (MPI) and Open MultiProcessor (OpenMP) programming models. We rewrite the model in C++ and use the Kokkos library to express on-node parallelism in a largely architecture-independent implementation. Kokkos provides an abstraction of a compute node or device, layout-polymorphic multidimensional arrays, and parallel execution constructs. The new implementation achieves the same or better performance on conventional multicore computers and is portable to GPUs. We present performance data for the original and new implementations on multiple platforms, on up to 5400 compute nodes, and study several aspects of the single- and multi-node performance characteristics of the new implementation on conventional CPU (e.g., Intel Xeon), many core CPU (e.g., Intel Xeon Phi Knights Landing), and Nvidia V100 GPU.This publication has 26 references indexed in Scilit:
- Kokkos: Enabling manycore performance portability through polymorphic memory access patternsJournal of Parallel and Distributed Computing, 2014
- Optimization-based limiters for the spectral element methodJournal of Computational Physics, 2014
- Towards a performance portable, architecture agnostic implementation strategy for weather and climate modelsSupercomputing Frontiers and Innovations, 2014
- The Community Earth System Model: A Framework for Collaborative ResearchBulletin of the American Meteorological Society, 2013
- Progress towards accelerating HOMME on hybrid multi-core systemsThe International Journal of High Performance Computing Applications, 2012
- CAM-SE: A scalable spectral element dynamical core for the Community Atmosphere ModelThe International Journal of High Performance Computing Applications, 2011
- Conservation of Mass and Energy for the Moist Atmospheric Primitive Equations on Unstructured GridsPublished by Springer Science and Business Media LLC ,2011
- EARLY EXPERIENCES WITH THE 360TF IBM BLUE GENE/L PLATFORMInternational Journal of Computational Methods, 2008
- High-Resolution Mesh Convergence Properties and Parallel Efficiency of a Spectral Element Atmospheric Dynamical CoreThe International Journal of High Performance Computing Applications, 2005
- The Piecewise Parabolic Method (PPM) for gas-dynamical simulationsJournal of Computational Physics, 1984