The Science

Increasing evidence shows that groundwater regulates water and energy fluxes in the land-atmosphere system and thus is critical in Earth System Models (ESMs). To fully understand the subsurface hydrologic processes and reasonably upscale them to the scales and resolutions of ESMs, modelers need large-scale particle tracking that account for the groundwater flow paths and their connections with the land-atmosphere system. In a new study, a multi-institutional team of scientists developed and tested a parallel framework on distributed, multi-graphics processing unit (GPU) platforms for the EcoSLIM code, thereby enabling large-scale particle tracking with high spatio-temporal resolutions.

The Impact

Large-scale groundwater models configured with lateral groundwater flow were developed a decade ago, but this type of modeling mainly focused on water quantity. Few studies were conducted on water quality and age. Recent studies highlighted that the terrestrial water cycle might have a period much longer than one year when researchers identified water pathways in the annual water balance. This longer period is attributed to the contribution of groundwater to the Earth’s surface processes. Communities of hydrology and Earth surface process modelers lacked a particle tracking tool that could handle cross-scale simulations. By parallelizing the EcoSLIM code, there is now a promising tool for the hydrologic community and ESM developers for scientific exploration.

Summary

EcoSLIM is a Lagrangian particle tracking code that works seamlessly with the integrated hydrologic model ParFlow-CLM to simulate subsurface advection and diffusion of water parcels. EcoSLIM was developed to calculate water ages (e.g., groundwater, evapotranspiration, and outflow), and diagnose source water composition (e.g., snow, rainfall, and historical groundwater).

The team decomposed the modeling domain into subdomains, considered the particle transfer and load balancing between subdomains, and further accelerated the code on GPUs. Tests (4 NVIDIA A100 GPUs relative to 128 AMD EPYC cores) based on the Little Washita watershed showed a significant speedup of 25.49-fold; 8-fold is the basic requirement. Tests based on the Little Washita watershed in Oklahoma and the North China Plain (NCP) showed excellent parallel scaling. Tests based on the NCP and continental United States demonstrated EcoSLIM’s ability to handle regional- to continental-scale simulations with reasonable wall-clock time. While this study uses EcoSLIM as an example, the parallel framework is portable for other particle tracking models in Earth systems research.

Principal Investigator

Reed Maxwell
Courtesy Princeton University.
reedmaxwell@princeton.edu

Program Manager

Paul Bayer
U.S. Department of Energy, Biological and Environmental Research (SC-33)
Environmental System Science
paul.bayer@science.doe.gov

Funding

This work was supported by the U.S. Department of Energy’s (DOE) Office of Science Advanced Scientific Computing Research program and the Biological and Environmental Research (BER) program’s IDEAS-Watersheds project and Watershed Function Science Focus Area under Award Number DE-AC02-05CH11231.

References