[ONLINE] SPACE CoE: CI/CD with git

The webinar is organised by the SPACE CoE and NCC Czechia.

Annotation

In this seminar, we will show how to set up a Continuous Integration (CI) pipeline in Gitlab on a local system as well as on an external cluster, starting with a simple example of a larger, full, functional pipeline of a real project. We will present a flexible and robust solution to set up CI that can work in various scenarios. The solution is built using a set of widely available technologies such as Apptainer, SSH, and Tmux. The seminar will contain a live demo showing common pitfalls, solutions, and best practices. At the end, we will show how this approach works under the hood.

This seminar will cover

• Installation, creation, and registration of GitLab runners.
• Creation of a simple pipeline within GitLab.
• Additional options will be introduced using examples from the OpenGadget3 CI pipeline.
• Technical requirements for setting up a pipeline on a supercomputer.
• Running Slurm jobs in a CI pipeline.
• Integration of GitLab-runner with Slurm using Apptainer.
• Common pitfalls and best practices.

Language

English

Prerequisites

This seminar is intended for users with limited to no experience with GitLab CI pipelines and for more experienced users who want to learn about the specifics of running a pipeline on a larger cluster.

Tutors:

Kristian Kadlubiak is a researcher at the INFRA lab of IT4Innovations National Supercomputing Center, where he is responsible for designing and developing various acceleration and optimization techniques for the flagship application ESPRESO. He specializes in parallel and vector processing, accelerator offloading, and performance tuning in general. He holds a master's degree in embedded and computer systems from the Brno University of Technology, where he is partially involved as a Ph.D. student. In his studies, he is developing modifications of the Local Fourier Basis (LFB) method to adapt it for efficient use on HPC systems.

Geray Karademir is a post-doctoral researcher working at the Ludwig-Maximilians-Universität Munich on the N-Body/SPH code OpenGadget3. During his PhD in Melbourne, Australia, Geray worked on the statistical analysis of data sets from extensive galaxy surveys that consisted of millions of objects. This experience made him appreciate the potential of High-Performance Computing (HPC) to provide scientific insights. He is mainly interested in optimizing code to utilize available resources efficiently and making them accessible for any researcher to understand and develop further, thus advancing scientific research.

Acknowledgements

This work was supported by the SPACE project under grant agreement No 101093441. The project is supported by the European High-Performance Computing Joint Undertaking and its members (including top-up funding by the Ministry of Education of the Czech Republic ID: MC2304).

This project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 101101903. The JU receives support from the Digital Europe Programme and Germany, Bulgaria, Austria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, Greece, Hungary, Ireland, Italy, Lithuania, Latvia, Poland, Portugal, Romania, Slovenia, Spain, Sweden, France, Netherlands, Belgium, Luxembourg, Slovakia, Norway, Türkiye, Republic of North Macedonia, Iceland, Montenegro, Serbia. This project has received funding from the Ministry of Education, Youth and Sports of the Czech Republic.

This course was supported by the Ministry of Education, Youth and Sports of the Czech Republic through the e-INFRA CZ (ID:90254).