Fermilab Computing Division

CS Document 5227-v1

A Fresh Perspective on Large-Scale Distributed Cyberinfrastracture: Abstractions, Models and Interoperable Tools

Document #:
Document type:
Submitted by:
Ruth Pordes
Updated by:
Ruth Pordes
Document Created:
17 Oct 2013, 16:41
Contents Revised:
17 Oct 2013, 16:41
Metadata Revised:
17 Oct 2013, 16:41
Viewable by:
  • Public document
Modifiable by:

Quick Links:
Latest Version

A Fresh Perspective on Large-Scale Distributed Cyberinfrastracture:
Abstractions, Models and Interoperable Tools

We begin with a brief survey of large-scale distributed computing -- applications and infrastructure, and offer insights into the state-of-the-art. Using pilot-systems --- which have arguably been one of the most successful abstractions in distributed computing, as an exemplar, we motivate the importance of an abstractions-based and model-oriented approach to distributed computing. We discuss how the P* Model -- a conceptual model of pilot-systems, provides a unifying framework to understand and extend (by symmetry) the pilot-concept, as well as separate implementation details from core conceptual aspects. We then discuss examples of applications composed of many simulations, but with varying degrees of coupling that have used an extensible and interoperable implementation of the pilot-system to achieve "extreme" scales. Interoperability in of itself is not an end-goal, nor a design objective, but it is the foundational principle upon which higher-level innovation becomes possible. We conclude by synthesizing ideas about how the next generation of middleware for large-scale distributed systems might be designed, with special consideration on the characteristics of future high-energy physics workloads.
Admittedly, the technical issues associated with the design and implementation of the next-generation of distributed systems will be intertwined with funding, historical and social considerations, but if we dare to imagine the former without the distractions of the latter, what would and should it look like?

Shantenu is an Assistant Professor at Rutgers University, and a Visiting Scientist at the School of Informatics (University of Edinburgh) and at University College London. Before moving to Rutgers, he was the lead for Cyberinfrastructure Research and Development at the CCT at Louisiana State University. His research interests lie at the triple point of Applied Computing, Cyberinfrastructure R&D and Computational Science. Shantenu is the lead investigator of the SAGA project (http://www.saga-project.org), which is a community standard and is part of the official middleware/software stack of most major Production Distributed Cyberinfrastructure -- such as US NSF's XSEDE and the European Grid Infrastructure. His research is funded by multiple NSF and US Department of Energy and recently by US National Institute for Health (NIH) as well as the UK EPSRC (OMII-UK project and Research theme at the e-Science Institute). He is the recipient of the NSF CAREER Award in 2013 and has won several prestigious awards at ACM/IEEE Supercomputing and the International Supercomputing Series. He seeks fearless and revolutionary young minds to join the RADICAL (http://radical.rutgers.edu) group. Away from work, Jha tries middle-distance running and biking, tends to be an economics-junky, enjoys reading and writing random musings and tries to use his copious amounts of free time with a conscience.

Associated with Events:
A Fresh Perspective on Large-Scale Distributed Cyberinfrastracture held on 15 Oct 2013 in Racetrack
DocDB Home ]  [ Search ] [ Authors ] [ Events ] [ Topics ]

DocDB Version 8.8.9, contact Document Database Administrators