Fermilab Computing Division

CS Document 5433-v1

Cloud services for the Fermilab scientific stakeholders

Document #:
CS-doc-5433-v1
Document type:
Conference
Submitted by:
Gabriele Garzoglio
Updated by:
Gabriele Garzoglio
Document Created:
16 Oct 2014, 09:20
Contents Revised:
08 May 2015, 13:19
Metadata Revised:
08 May 2015, 13:19
Viewable by:
  • Public document
Modifiable by:

Quick Links:
Latest Version

Other Versions:
CS-doc-5433-v0
16 Oct 2014, 09:20
Abstract:
As part of the Fermilab/KISTI cooperative research project, Fermilab has successfully run an experimental simulation workflow at scale on a federation of Amazon Web Services (AWS), FermiCloud, and local FermiGrid resources. We used the CernVM-FS (CVMFS) file system to deliver the application software. We established Squid caching servers in AWS as well, and used the Shoal system to let each individual virtual machine find the closest squid server. We also developed an automatic virtual machine conversion
system so that we could transition virtual machines made on FermiCloud to Amazon Web Services.
We used this system to successfully run a cosmic ray simulation of the NOvA detector at Fermilab, making use of both AWS spot pricing and network bandwidth discounts to minimize the cost. On FermiCloud we also were able to run the workflow at the scale of 1000 virtual machines, using a private network routable inside of Fermilab. We present the details of the technological improvements that were used to make this successfully.
Files in Document:
Associated with Events:
held on 13 Apr 2015
DocDB Home ]  [ Search ] [ Authors ] [ Events ] [ Topics ]

DocDB Version 8.8.9, contact Document Database Administrators