Fermilab Computing Division

CS Document 5063-v2

CHEP 2013 - Big Data over a 100G Network at Fermilab

Document #:
CS-doc-5063-v2
Document type:
Presentation
Submitted by:
Gabriele Garzoglio
Updated by:
Gabriele Garzoglio
Document Created:
21 Mar 2013, 01:21
Contents Revised:
31 Oct 2013, 09:34
Metadata Revised:
31 Oct 2013, 09:34
Viewable by:
  • Public document
Modifiable by:

Quick Links:
Latest Version

Other Versions:
CS-doc-5063-v1
08 Oct 2013, 10:40
CS-doc-5063-v0
31 May 2013, 13:50
Abstract:
As the need for Big Data in science becomes ever more relevant, networks around the world are upgrading their infrastructure to support high-speed interconnections. To support its mission, the high-energy physics community as a pioneer in Big Data has always been relying on the Fermi National Accelerator Laboratory to be at the forefront of storage and data movement. This need was reiterated in recent years with the data taking rate of the major LHC experiments reaching tens of Petabytes per year. At Fermilab, this resulted regularly in peaks of data movement on the WAN in and out of the laboratory of about 30 Gbits/s and on the LAN between storage and computational farms of 160 Gbits/s. To address these ever increasing needs, as of this year Fermilab is connected to the Energy Sciences Network (ESNet) through a 100 Gbit/s link.

To understand the optimal system- and application-level configuration to interface computational systems with the new high-speed interconnect, Fermilab has deployed a Network R&D facility connected to the ESNet 100G Testbed. For the past two years, the High Throughput Data Program has been using the Testbed to identify gaps in data movement middleware when transferring data at these high-speeds. The program has published evaluations of technologies typically used in High Energy Physics, such as GridFTP, XrootD, and Squid. This work presents the new R&D facility and the continuation of the evaluation program.

Files in Document:
Associated with Events:
CHEP 2013 held on 14 Oct 2013 in Amsterdam, The Netherlands
Publication Information:
Acceptance Type: Oral presentation
DocDB Home ]  [ Search ] [ Authors ] [ Events ] [ Topics ]

DocDB Version 8.8.9, contact Document Database Administrators