Speaker
Description
Biomedical workflow management systems enable clinicians to use grid, cloud and high performance computing (HPC) services easily. These systems describe complex problems, such as treatment planning, screening, and diagnosis, using workflows. Workflows can be seen as directed weighted graphs providing a formal way to define and automate multi-step computational procedures. The graph nodes present individual tasks that may differ in their nature, performance and computational demands. They also encapsulate lower level details about the task specific parameters. Since HPC environments are highly dynamic and heterogeneous, efficient manual task execution, tuning to the specific computational machine, monitoring and dealing with various types of failures is very tedious and time consuming. Therefore, achieved cluster throughput may be very limited.
The presented framework, called k-Dispatch, mainly focuses on computational problems related to biomedical environment and uses only predefined workflows. Although the workflow structures are predefined, they have a level of adaptivity
based on the provided input data. The end users (clinicians) only submit input data (MR and CT scans) and workflow specific parameters. To be compliant with medical security policies, only certified executables for given HW can be used.
k-Dispatch provides automated task scheduling, execution, monitoring, and fault tolerance. It is being developed as a module of the k-Plan system. k-Plan performs a model-based treatment planning for ultrasound(US) therapy such as tissue ablation, neurostimulation and targeted drug delivery.
Since the task run configuration strongly affects the final tasks mapping on the computational resources, cluster throughput and computational cost, the execution planning is of the highest priority. Currently, k-Dispatch only allows a static task planning similar to many related tools. For each task in the workflow, a default binary for a specific code type and a default run configuration are selected. This approach enables users to run their workflows without any advanced knowledge of the current HPC service, however, the tasks are likely not to be executed efficiently and may spend a long time in queues waiting for computational resources.
Thus, k-Dispatch is getting at considering a variable amount of computational resources per individual tasks. Since the scaling of the individual HPC codes is never perfect, k-Dispatch may find such a good mapping even an experienced user would miss. The proposed adaptive execution planning is based on collected performance data and the current cluster utilization monitoring.
