Investigation of Storage Systems for use in Grid Applications

Document #:

CS-doc-4662-v3

Document type:

Presentation

Submitted by:

Gabriele Garzoglio

Updated by:

Gabriele Garzoglio

Document Created:

16 Feb 2012, 14:04

Contents Revised:

24 Feb 2012, 10:10

Metadata Revised:

27 Feb 2012, 10:08

Abstract:

In recent years, several new storage technologies, such as Lustre, Hadoop, OrangeFS, and BlueArc, have emerged. While several groups have run benchmarks to characterize them under a variety of configurations, more work was needed to evaluate these technologies for the use cases of scientific computing on Grid clusters and Cloud facilities. This paper discusses our evaluation of all these technologies as deployed on a test bed at FermiCloud, one of the Fermilab infrastructure-as-a-service Cloud facilities. The test bed consists of 4 server-class nodes with 40 TB of disk space and up to 50 virtual machine clients, some running on the storage server nodes themselves. With this configuration, the evaluation compares the performance of some of these technologies when deployed on virtual machines and on "bare metal" nodes. In addition to running standard benchmarks such as IOZone to check the sanity of our installation, we have run I/O intensive tests using physics-analysis applic!
ations. This paper presents how the storage solutions perform in a variety of realistic use cases of scientific computing. One interesting difference among the storage systems tested is found in a decrease in total read throughput with increasing number of client processes, which occurs in some implementations but not others.

Files in Document:

• Storage-Evaluation-iSGC-Feb-2012-v1.4.pdf (2.1 MB)
• Storage-Evaluation-iSGC-Feb-2012-v1.4.ppt (2.5 MB)

Topics:

• Shared Systems and Services:Grid

Authors:

• Gabriele Garzoglio

Keywords:

Storage Investigation Lustre Hadoop OrangeFS BlueArc Grid Cloud

Associated with Events:

ISGC 2012 held on 26 Feb 2012 in Academica Sinica, Taipai, Taiwan