Fermilab Computing Division

CS Document 5435-v0

JobSub: Scaling a Job Submission Infrastructure using a 3-Tier Architecture

Document #:
CS-doc-5435-v0
Document type:
Conference
Submitted by:
Gabriele Garzoglio
Updated by:
Gabriele Garzoglio
Document Created:
16 Oct 2014, 09:37
Contents Revised:
16 Oct 2014, 09:37
Metadata Revised:
16 Oct 2014, 09:37
Viewable by:
  • Public document
Modifiable by:

Quick Links:
Latest Version

Abstract:
The FIFE (Fabric for Frontier Experiments) community has been using the JobSub tool for over two years to run scientific workflows in the Open Science Grid (OSG) and local clusters at Fermilab. JobSub provide user communities with a collection of easy to use tools that automate several job submission tasks, such as facilitating site selection using the GlideinWMS Workload Management System, managing submission credentials, handling data transfers, etc. With the increase in diversity of the FIFE community, the previous simplistic architecture of JobSub provided several operational challenges and limitations. There was a need to provide the community with a shared, scalable, reliable and robust job submission system to meet their computing needs. This paper describes the initial challenges of the simplistic 1-tier architecture, the diverse set of requirements from the FIFE and operational communities, and how these challenges and complex requirements are addressed in the new 3-tier architecture of JobSub.
Files in Document:
None
Keywords:
CHEP15 JobSub FIFE OSG GCS
Associated with Events:
held on 13 Apr 2015
DocDB Home ]  [ Search ] [ Authors ] [ Events ] [ Topics ]

DocDB Version 8.8.9, contact Document Database Administrators