Global heterogeneous resource harvesting: The next-generation PanDA Pilot for ATLAS

A. Anisenkov, D. Drizhuk, W. Guan, M. Lassnig, P. Nilsson, D. Oleynik

Research output: Contribution to journalConference articlepeer-review

1 Citation (Scopus)

Abstract

The Production and Distributed Analysis system (PanDA), used for workload management in the ATLAS Experiment at the LHC for over a decade, has in recent years expanded its reach to diverse new resource types such as HPCs, and innovative new workflows such as the Event Service. PanDA meets the heterogeneous resources it harvests in the PanDA Pilot, which has embarked on a next-generation reengineering to efficiently integrate and exploit the new platforms and workflows. The new modular architecture is the product of a year of design and prototyping in conjunction with the design of a completely new component, Harvester, that will mediate a richer flow of control and information between Pilot and PanDA. Harvester will enable more intelligent and dynamic matching between processing tasks and resources, with an initial focus on HPCs, simplifying the operator and user view of a PanDA site but internally leveraging deep information gathering on the resource to accrue detailed knowledge of a site's capabilities and dynamic state to inform the matchmaking. This paper will give an overview of the new Pilot architecture, how it will be used in and beyond ATLAS, its relation to Harvester, and the work ahead.

Original languageEnglish
Article number032031
Number of pages5
JournalJournal of Physics: Conference Series
Volume1085
Issue number3
DOIs
Publication statusPublished - 18 Oct 2018
Event18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research, ACAT 2017 - Seattle, United States
Duration: 21 Aug 201725 Aug 2017

Fingerprint Dive into the research topics of 'Global heterogeneous resource harvesting: The next-generation PanDA Pilot for ATLAS'. Together they form a unique fingerprint.

Cite this