3–4 Nov 2022
IT4Innovations
Europe/Prague timezone

ARC-CE+HyperQueue based submission system of ATLAS jobs for the Karolina HPC

Not scheduled
2h
atrium (IT4Innovations)

atrium

IT4Innovations

Studentská 6231/1B 708 00 Ostrava-Poruba
Poster Poster session Conference Dinner and Poster Session

Speaker

Michal Svatos (FZU, AV CR)

Description

For several years, the distributed computing of the ATLAS experiment at the LHC has been granted opportunistic use of computing resources of the Czech national HPC centre, IT4Innovations. With the introduction of Karolina HPC, resources provided to ATLAS significantly increased, but with lower efficiency. The inefficiency arose because ATLAS jobs, designed for 8 cores, have a rather short multiprocessing phase on the 128 cores of Karolina's worker nodes in comparison with initialization and finalization running on a single core. To ensure efficient usage, HyperQueue was implemented in the ARC-CE based submission system. This enables four 32-core jobs to be sent to each worker node, significantly improving CPU efficiency without leaving empty resources.

Primary authors

Michal Svatos (FZU, AV CR) Jiří Chudoba (FZU AV ČR) Petr Vokáč (FJFI, CVUT)

Presentation materials