Speaker
Description
The PALM model system [1] is an open-source, HPC-enabled, extensible large-eddy simulation (LES) modelling system which includes components for many related processes that are necessary for a complex environmental modelling in micro-scale, such as resolved urban climate simulations in the street level [2]. These simulations are vital for studying the adverse effects of urbanization and climate change such as air pollution and heat stress and they enable the urban planners to take into account and mitigate these effects [3]. One of the most important processes in micro-scale atmospheric modelling is radiation and its interaction with surfaces, plant canopy etc. which provides thermal energy that powers most atmospheric processes, especially in the boundary layer.
The radiative transfer model (RTM, [4]) in PALM provides fully resolved, explicit 3-D simulation of short-wave and long-wave radiation that includes multiple reflections, shading, absorption and emission, interaction with semi-transparent plant canopy and other processes, and it is fully integrated in the model, working on a common 3-D grid, time-steps, data structures and parallelization via horizontal domain decomposition. Such simulation is computationally expensive and it presents challenges to parallelization, because the immediate interactions are not limited to neighbouring sub-domains.
The RTM is centred around the radiosity approach and it uses pre-calculated view factors as the basis for interaction among discretized surface elements, plant canopy grid cells and other objects of radiation. The most computationally intensive task is the calculation of the view factors, which is performed as part of the pre-processing step and it uses a custom column-integrated raytracing algorithm, which is optimized with respect to the discretization scheme and the horizontal domain decomposition.
Over the last few years, the RTM has received numerous improvements which greatly enhanced its applicability by adding new simulated processes and subjects of radiation. As a result of substantial improvements of algorithms and representation schemes, it features excellent scalability and it has been successfully used for extremely large simulations with tens of thousands of parallel processes. Many of these improvements have been tested and validated on the IT4I supercomputers Salomon and Karolina [5]. An important and significant improvement is the recently developed new parallelization scheme for the raytracing algorithm. By reorganizing the parallel computation and using new patterns for data exchange among the MPI processes, it allows to avoid certain problematic MPI calls as well as some large arrays, thus greatly enhacing scalability. It is currently being tested on several HPC systems including Karolina. First tests show that it can speed up raytracing by an order of magnitude in certain scenarios. This presentation shows the new paralelization scheme and other new computational improvements in the RTM.
Literature
[1] https://doi.org/10.5194/gmd-13-1335-2020
[2] https://doi.org/10.5194/gmd-10-3635-2017
[3] https://doi.org/10.1016/j.buildenv.2022.109484
[4] https://doi.org/10.5194/gmd-14-3095-2021
[5] https://doi.org/10.5194/gmd-14-4797-2021
