Fermilab Computing Division

CS Document 449-v2

Application of the SAMGrid Test Harness for Performance

Document #:
CS-doc-449-v2
Document type:
Conference
Submitted by:
JoAnn Larson
Updated by:
JoAnn Larson
Document Created:
03 Aug 2004, 11:49
Contents Revised:
24 Sep 2004, 17:48
Metadata Revised:
20 Jan 2005, 09:21
Viewable by:
  • Public document
Modifiable by:
  • Same as Viewable by

Quick Links:
Latest Version

Other Versions:
CS-doc-449-v1
24 Sep 2004, 17:00
CS-doc-449-v0
20 Sep 2004, 17:41
Abstract:
The SAMGrid team has recently refactored its test harness suite for greater flexibility and easier configuration. This makes possible more interesting applications of the test harness, for component tests, integration tests,
and stress tests. We report on the architecture of the test harness and its recent application to stress tests of a new analysis cluster at Fermilab, to explore the extremes of analysis use cases and the relevant parameters for tuning in the SAMGrid station services. This reimplementation of the test harness is a python framework which uses XML for configuration and small plug-in python modules for specific test purposes. One current testing application is running on a 128-CPU analysis cluster with access to 6 TB distributed cache and also to a 2 TB centralized cache, permitting studies of different cache strategies. We have studied the service parameters which affect the performance of retrieving data from tape storage as well. The use cases
studied vary from those which will require rapid file delivery with short processing time per file, to the opposite extreme of long processing time per file. We also
show how the same harness can be used to run regular unit tests on a production system to aid early fault detection and diagnosis.These results are interesting for their implications with regard to Grid operations, and illustrate the type of monitoring and test facilities required to accomplish such performance tuning.

Files in Document:
  • Paper (test harness paper.pdf, 411.2 kB)
  • Talk (test harness talk.pdf, 711.2 kB)
Notes and Changes:
Fermilab Publication number CONF-04-454-CD
Associated with Events:
CHEP2004 held from 27 Sep 2004 to 01 Oct 2004 in Interlaken, Switzerland
DocDB Home ]  [ Search ] [ Authors ] [ Events ] [ Topics ]

DocDB Version 8.8.9, contact Document Database Administrators