Fermilab Computing Division

CS Document 5745-v1

Fermilab's Scientific Computing Storage Architecture (for HTC)

Document #:
Document type:
Submitted by:
Gerard B Altayo
Updated by:
Gerard B Altayo
Document Created:
11 May 2016, 20:40
Contents Revised:
11 May 2016, 20:40
Metadata Revised:
11 May 2016, 20:40
Viewable by:
  • Public document
Modifiable by:

Quick Links:
Latest Version

High Energy Physics experiments record and simulate very large volumes of data and the trend in the future is only going up. All this data needs to be archived, and accessed by central processing workflows as well as a diverse group of scientists to extract physics results. Fermilab supports a wealth of storage technologies for the experiments for very different tasks, from NFS mounted appliances to mass storage systems handling transparent access to files on tape through large disk caches. To prepare for the future, Fermilab recently developed a Scientific Computing Storage Architecture to consolidate and to prepare for a future evolution of the supported storage forms. A big aspect was to remove POSIX access to the storage systems from the worker nodes, necessary to be able to transparently support workflows on different resources like the local batch systems, grids and commercial clouds.
In this presentation, we will discuss the architecture and describe how experts categorizes workloads accessing storage, files and access patterns. We are currently in the process of implementing this strategy together with Fermilab’s experiments and projects and will report on the progress.
Files in Document:
DocDB Home ]  [ Search ] [ Authors ] [ Events ] [ Topics ]

DocDB Version 8.8.9, contact Document Database Administrators