Fermilab Computing Division

CS Document 4701-v1

Challenges and opportunites in large-scale storage systems

Document #:
CS-doc-4701-v1
Document type:
Presentation
Submitted by:
Gabriele Garzoglio
Updated by:
Gabriele Garzoglio
Document Created:
02 Apr 2012, 11:03
Contents Revised:
02 Apr 2012, 11:03
Metadata Revised:
02 Apr 2012, 11:03
Viewable by:
  • Public document
Modifiable by:

Quick Links:
Latest Version

Abstract:
Exascale computers will enable the unraveling of significant scientific mysteries. Predictions are that 2019 will be the year of exascale, with millions of compute nodes and billions of threads of execution. The current architecture of high-end computing systems is decades-old and has persisted as we scaled from gigascales to petascales. In this architecture, storage is completely segregated from the compute resources and are connected via a network interconnect. This approach will not scale several orders of magnitude in terms of concurrency and throughput, and will thus prevent the move from petascale to exascale. At exascale, basic functionality at high concurrency levels will suffer poor performance, and combined with system mean-time-to-failure in hours, will lead to a performance collapse for large-scale heroic applications. Storage has the potential to be the Achilles heel of exascale systems. We propose that future high-end computing systems be designed with non-volatile memory on every compute node, allowing every compute node to actively participate in the metadata and data management and leveraging many-core processors high bisection bandwidth in torus networks. This presentation discusses this revolutionary new distributed storage architecture that will make exascale computing more tractable, touching virtually all disciplines in high-end computing and fueling scientific discovery.
Files in Document:
Authors:
Associated with Events:
Challenges and Opportunities in Large-Scale Storage Systems held on 29 Mar 2012 in FCC1 Conference Room
DocDB Home ]  [ Search ] [ Authors ] [ Events ] [ Topics ]

DocDB Version 8.8.9, contact Document Database Administrators