Fermilab Computing Division

CS Document 6763-v1

Use of the dCache Resilient Pool in Grid Jobs (uboone)

Document #:
CS-doc-6763-v1
Document type:
Presentation
Submitted by:
Anna Mazzacane
Updated by:
Anna Mazzacane
Document Created:
15 May 2019, 13:18
Contents Revised:
15 May 2019, 13:18
Metadata Revised:
15 May 2019, 13:18
Viewable by:
  • Public document
Modifiable by:

Quick Links:
Latest Version

Abstract:
In early 2018, the FIFE (FabrIc for Frontier Experiments) group recognized that the number of user jobs accessing large tarballs - usually consisting of user code - on the scratch dCache pools was putting a significant strain on the storage resources. This issue affected not only the user's jobs in question, but many other users and experiments, as these dCache pools are shared between experiments. To that end, FIFE worked with the Data Movement and Storage groups to create a new set of dCache pools, resilient dCache. Furthermore, the job submission software used by FIFE experiments, jobsub, was consequently updated with new features. In this talk, we discuss the reasons behind creating resilient dCache pools, how to use jobsub to upload files and directories to resilient dCache, the advantages of allowing jobsub to manage resilient dCache usage, and the challenges we still face.
Authors:
DocDB Home ]  [ Search ] [ Authors ] [ Events ] [ Topics ]

DocDB Version 8.8.9, contact Document Database Administrators