Fermilab Computing Division

CS Document 5073-v1

Public Storage for the Open Science Grid

Document #:
Document type:
Submitted by:
Tanya Levshina
Updated by:
Tanya Levshina
Document Created:
28 Mar 2013, 14:43
Contents Revised:
28 Mar 2013, 14:43
Metadata Revised:
30 Oct 2013, 17:24
Viewable by:
  • Public document
Modifiable by:

Quick Links:
Latest Version

OSG doesn’t provide efficient means to manage public storage offered by participating sites. A Virtual Organization (VO) that relies on opportunistic storage has difficulties finding appropriate storage, verifying its availability, and monitoring its utilization. The involvement of the Production Manager, site administrators and VO support personnel is required to allocate or rescind storage space. One of the main requirements for Public Storage implementation is that it should use SRM or GridFTP protocols to access the Storage Elements(SE) provided by the OSG Sites and doesn’t put any additional burden on sites. No new services related to Public Storage could be installed and run on OSG sites.
Opportunistic users also have difficulties in accessing the distributed storage during the execution of jobs. The typical users' data management workflow includes pre-staging common data on sites before a job's execution, then somehow storing output data produced by a job on a worker node for a subsequent download to a local institution. When the amount of data is significant, the only means to temporarily store the data is to upload it to one of the Storage Elements. In order to do that, a user's job should be aware of the storage location, availability, and free space. After a successful data upload, users should somehow keep track of the data's location for future access. In this presentation we proposed solutions for storage management and data handling issues in the OSG.
We are investigating the feasibility of using the integrated Rule-Oriented Data System (iRODS) developed at RENCI as a front-end service to the OSG SEs. The current architecture, state of deployment and performance test results will be discussed. We will also provide examples of current usage of the system by beta-users.
Files in Document:
Associated with Events:
CHEP 2013 held on 14 Oct 2013 in Amsterdam, The Netherlands
Publication Information:
Acceptance Type: Oral presentation
DocDB Home ]  [ Search ] [ Authors ] [ Events ] [ Topics ]

DocDB Version 8.8.10, contact Document Database Administrators