Fermilab Computing Division

CS Document 1264-v1

CHEP06 abstract - A case for application-aware grid services

Document #:
Document type:
Submitted by:
Stuart C. Fuess
Updated by:
Stuart C. Fuess
Document Created:
13 Nov 2005, 23:31
Contents Revised:
07 Mar 2006, 09:01
Metadata Revised:
19 Oct 2006, 15:57
Viewable by:
  • Public document
Modifiable by:

Quick Links:
Latest Version

In 2005, the DZero Data Reconstruction project processed 250 tera-bytes of data on the Grid, using 1,600 CPU-years of computing cycles in 6 months. The large computational task required a high-level of refinement of the SAM-Grid system, the integrated data, job, and information management infrastructure of the RunII experiments at Fermilab. The success of the project was in part due to the ability of the SAM-Grid to adapt to the local configuration of the resources and services at the
participating sites. A key aspect of such adaptation was coordinating the resource usage in order to optimize the typical access patterns of the DZero reprocessing
application. Examples of such optimizations include database access, data storage access, and worker nodes allocation and utilization.

A popular approach to implement resource coordination on the grid is developing services that understand application requirements and preferences in terms of
abstract quantities e.g. required CPU cycles or data access pattern characteristics. On the other hand, as of today, it is still difficult to implement real-life resource optimizations using such level of abstraction. First, this approach assumes maximum knowledge of the resource/service interfaces from the users and the applications. Second, it requires a high level of maturity for the grid interfaces. To overcome these difficulties, the SAM-Grid provides resource optimization implementing
application-aware grid services. For a known application, such services can act in concert maximizing the efficiency of the resource usage. This paper describes what
optimizations the SAM-Grid framework had to provide to serve the DZero reconstruction and montecarlo production. It also shows how application-aware grid services fulfill
the task.

Files in Document:
Associated with Events:
CHEP2006 held from 13 Feb 2006 to 17 Feb 2006 in Mumbai, India
DocDB Home ]  [ Search ] [ Authors ] [ Events ] [ Topics ]

DocDB Version 8.8.9, contact Document Database Administrators