Fermilab Computing Division

CS Document 5437-v0

Dark Energy Survey computing on FermiGrid

Document #:
Document type:
Submitted by:
Gabriele Garzoglio
Updated by:
Gabriele Garzoglio
Document Created:
16 Oct 2014, 10:10
Contents Revised:
16 Oct 2014, 10:10
Metadata Revised:
16 Oct 2014, 10:10
Viewable by:
  • Public document
Modifiable by:

Quick Links:
Latest Version

The Dark Energy Survey (DES) uses a CCD camera installed in the Blanco telescope in Cerro Tololo, Chile. The goal is to study the effect known as Dark Energy. To get science results, several different kinds of processing must be done on the raw exposures that come from the camera. Among other resources, DES has an allocation on Fermigrid used for this purpose. So far an average of only about 1.5k CPU hours per day have been needed. The difficulties for DES are the complexity of the software and the requirements for disk space, RAM, and network bandwidth.

The data processing has been integrated with a variety of software services to implement the distribution of the computation. A Processing Framework has been developed to handle the workflow of the scientific applications and plans have been devised to identify suitable units of data processing and coordinating data storage. Processing of the computational units is managed through a high-throughput computing platform (HTCondor) and relies on a global caching system (the CERN VM File System - CVMFS) for software distribution. Output data is transferred to the central archive at NCSA through the http protocol. For applications requiring memory footprints beyond the statically allocated computing slots on FermiGrid, appropriately sized Virtual Machines on FermiCloud have been used.

DES started running production with the new system in August of 2014. As more resources are considered for integration, such as more leadership class machines and the Open Science Grid, we will present how we addressed some notable problems with the current system, including slow propagation of files stored in CVMFS, low CPU efficiency, and jobs hanging while finishing.

Files in Document:
Associated with Events:
held on 13 Apr 2015
DocDB Home ]  [ Search ] [ Authors ] [ Events ] [ Topics ]

DocDB Version 8.8.9, contact Document Database Administrators