Harnessing the power of supercomputers using the PanDA Pilot 2 in the ATLAS Experiment

Paul Nilsson, Alexey Anisenkov, Doug Benjamin, Wen Guan, Tomas Javurek, Danila Oleynik

Результат исследования: Публикации в книгах, отчётах, сборниках, трудах конференцийстатья в сборнике материалов конференциинаучнаярецензирование

Аннотация

The unprecedented computing resource needs of the ATLAS experiment at LHC have motivated the Collaboration to become a leader in exploiting High Performance Computers (HPCs). To meet the requirements of HPCs, the PanDA system has been equipped with two new components; Pilot 2 and Harvester, that were designed with HPCs in mind. While Harvester is a resource-facing service which provides resource provisioning and workload shaping, Pilot 2 is responsible for payload execution on the resource. The presentation focuses on Pilot 2, which is a complete rewrite of the original PanDA Pilot used by ATLAS and other experiments for well over a decade. Pilot 2 has a flexible and adaptive design that allows for plugins to be defined with streamlined workflows. In particular, it has plugins for specific hardware infrastructures (HPC/GPU clusters) as well as for dedicated workflows defined by the needs of an experiment. Examples of dedicated HPC workflows are discussed in which the Pilot either uses an MPI application for processing fine-grained event level service under the control of the Harvester service or acts like an MPI application itself and runs a set of job in an assemble. In addition to describing the technical details of these workflows, results are shown from its deployment on Titan (OLCF) and other HPCs in ATLAS.

Язык оригиналаанглийский
Название основной публикации24TH INTERNATIONAL CONFERENCE ON COMPUTING IN HIGH ENERGY AND NUCLEAR PHYSICS (CHEP 2019)
РедакторыC Doglioni, D Kim, GA Stewart, L Silvestris, P Jackson, W Kamleh
ИздательUniversity of Lueneburg, Department of Economics and Social Sciences, Research Institute on Professions
Число страниц6
DOI
СостояниеОпубликовано - 2020
Событие24th International Conference on Computing in High Energy and Nuclear Physics (CHEP) - Adelaide, Австралия
Продолжительность: 4 ноя 20198 ноя 2019

Серия публикаций

НазваниеEPJ Web of Conferences
ИздательE D P SCIENCES
Том245
ISSN (печатное издание)2100-014X

Конференция

Конференция24th International Conference on Computing in High Energy and Nuclear Physics (CHEP)
СтранаАвстралия
ГородAdelaide
Период04.11.201908.11.2019

Предметные области OECD FOS+WOS

  • 1.03 ФИЗИЧЕСКИЕ НАУКИ И АСТРОНОМИЯ

Fingerprint

Подробные сведения о темах исследования «Harnessing the power of supercomputers using the PanDA Pilot 2 in the ATLAS Experiment». Вместе они формируют уникальный семантический отпечаток (fingerprint).

Цитировать