Developer-recommended Software for TeraGrid Science Gateways

From TeraGrid Wiki

Jump to: navigation, search

Contents

Page info

  • Last updated 4/28/2009
  • Contact Nancy Wilkins-Diehr
  • Parent page: http://www.teragrid.org/gateways/developers (in development) and TeraGrid Science Gateways Primer
  • Content: Developer and TeraGrid staff recommendations for software for building gateways and connecting them with TeraGrid resourcs
  • Notes: Parent page is in TG Web site; this page is maintained in the TG Wiki for ease of contribution

Introduction

The software below has been recommended by TeraGrid Science Gateway Developers or TeraGrid staff for developing gateways, enhancing interfaces, and connecting them to TeraGrid resources and includes everything from Google Gadgets to Web Services. For a detailed survey of current software usage per-gateway, see Science Gateway Use Cases. For information about the development process where this software is used, refer to the Developers section of official TeraGrid Science Gateway site.

Software List

Cloud Computing

Frameworks

Cloud Computing
Name/Topic URLs for downloads & documentation Description & Comments
Nimbus http://workspace.globus.org/ Nimbus is an open source toolkit that allows you to turn your cluster into an Infrastructure-as-a-Service (IaaS) cloud. Working clouds can also be accessed at http://workspace.globus.org/clouds/
Eucalyptus http://www.eucalyptus.com/ Eucalyptus is an open-source system for implementing on-premise private and hybrid clouds using the hardware and software infrastructure that is in place, without modification.

Portlets Development

Portal Framework Software
Name/Topic URLs for downloads & documentation Description & Comments
Gridsphere http://www.gridsphere.org
OGF19 presentation

http://www.gridsphere.org/gridsphere/gridsphere?cid=download
Gateway developers may be able to make use of capabilities included in or developed for the TeraGrid User Portal. The user portal is available at https://portal.teragrid.org/gridsphere/gridsphere. Portlets that may be of interest to gateway developers are listed at the Gridsphere download page.
OGCE http://www.collab-ogce.org OGCE is in use by several TeraGrid gateways.
Clarens http://clarens.sourceforge.net/ The Clarens Grid-Enabled Web Services Framework is an open source, secure, high-performance portal for ubiquitous access to data and computational resources provided by computing grids
SimpleGrid https://www.cigi.uiuc.edu/doku.php/projects/simplegrid/index (project & download page)

Gateway-in-a-Day tutorial (TG 06) (PDF)
A simple gateway framework for learning the fundamentals of connecting to the TeraGrid, forms the building blocks for a gateway.
Hub Zero http://www.hubzero.org HUBzero allows you to create dynamic web sites that connect a community in scientific research and educational activities. HUBzero sites combine powerful Web 2.0 concepts with a middleware that provides instant access to interactive simulation tools. These tools are not just Java applets, but real research codes that can access TeraGrid, the Open Science Grid, and other national Grid computing resources for extra cycles. In use by several TeraGrid gateways.
Django http://www.djangoproject.com "The web framework for perfectionists with deadlines, makes it easier to build Web apps more quickly with less code", in use by several TeraGrid gateways.
Portlet Development Software
Name/Topic URLs for downloads & documentation Description & Comments
RENCI's Science Desktop https://portal.renci.org/RENCIDesktop/ Generates science desktop interface portlets from XML description
Gridder http://gridder.sourceforge.net/ Portlets that simplifies the use and administration of grid services, extends and documents the functionality of the OGCE Bundle.

Application Hosting

Application Hosting
Name/Topic URLs for downloads & documentation Description & Comments
P-GRADE http://www.lpds.sztaki.hu/pgrade
Application Hosting Environment (AHE) http://www.realitygrid.org/AHE/

CA Certificates, CRLs, Authentication, Authorization, Usage

CA certificates and CRLs
Name/Topic URLs for downloads & documentation Description & Comments
TeraGrid CA certificates distribution http://security.teragrid.org/TG-CAs.html
https://repo.teragrid.org/cgi-bin/viewcvs.cgi/security/
Maintaining CRLs gx-map
gx-ca-update
fetch-crl
Credential Management http://teragridforum.org/mediawiki/index.php?title=Science_Gateway_Credential_Management
Authentication, Authorization, Usage
Name/Topic URLs for downloads & documentation Description & Comments
GridShib SAML Tools proxy credential with user attributes used by a gateway to create a proxy credential with user attributes, signed by a gateway's community credential; the attributes help to increase the information flow from the gateway to the resource provider (RP); the RP uses the attributes in the proxy for auditing, incident response, and access control
Portal-based User Registration System (PURSe) http://www.grids-center.org/solutions/purse/ a system for registering users of Web-based applications that use the Grid Security Infrastructure (based on PKI and X.509 certificates); PURSE coordinates the process of establishing Grid security certificates for new users when logging in to an application web site
Grid Account Management Architecture (GAMA) http://grid-devel.sdsc.edu/gridsphere/gridsphere?cid=gama omplete GSI credential management and integration solution tailored for use in emerging CyberInfrastructure through web portals or web service-based clients. GAMA consists of two components: a backend security service that provides secure management of credentials, and a front-end set of portlets that provide tight integration into web/grid portals

Information Services, Web Services

Information Services
Name/Topic URLs for downloads & documentation Description & Comments
TeraGrid Information Services http://info.teragrid.org/
Information_Services_Access_Using_WS/REST
Use any HTTP client software to access IIS RESTfull APIs
Introduction to TeraGrid Information Services for Developers http://info.teragrid.org/presentations/200804Developers.htm Presentation with links to TeraGrid's information services
TeraGrid Profile Service https://info.teragrid.org:8444/web-apps/html/profile-v1/usage User-specific information on the TeraGrid (profile, resources, accounts, allocations, jobs)
MDS (will be deprecated Summer 2011) Querying Data from MDS 4 on the TeraGrid (repo/requires password)
Custom MDS Java Query client (will be deprecated Summer 2011) Source
tarball
Globus comes with a command line client called wsrf-query for querying MDS with arbitrary XPath expressions; written by Eric Roberts
Web Services
Name/Topic URLs for downloads & documentation Description & Comments
Opal http://nbcr.net/software/opal/ Opal is a toolkit to wrap legacy applications as Web services. Opal facilitates application deployment of applications as Web services in a matter of hours. Optionally, you can set up scheduling with the help of Globus and SGE/Condor, security with the help of GSI-based certificates, and service state management with the help of a PostgreSQL database.
gRAVI Introduce
gRAVI introduction
Technical Details
email lists : gravi-user@globus.org, gravi-dev@globus.org
An Introduce plugin that can be used to generate an application specific WSRF compatible Web Service. This is used by CaBig
Soaplab2 (or Soaplab) Soaplab a tool that can automatically generate and deploy Web Services on top of existing command-line analysis programs
QBETS QBETS information in TeraGrid Wiki

Job Submission, Monitoring, and Accounting

Job Submission, Monitoring and Accounting
Name/Topic URLs for downloads & documentation Description & Comments
"Common TeraGrid Software Stack" (CTSS) client toolkit https://www.teragrid.org/web/user-support/sso_tg_client_toolkit Includes remote login (myproxy and gsissh) and remote data movement (uberftp and globus-url-copy)
Qstat monitor http://wiki.cogkit.org/index.php/Java_CoG_Kit_Qstat It also contains some APIs that may be useful for gateway and portal developers, e.g. it can also generate XML and HTML output.
Client desktop grid job submission framework http://projects.arcs.org.au/trac/grisu/ Grisu is an open source framework to support end users in a grid environment. It is built on GRAM (WS GRAM and Pre-WS GRAM), GridFTP and MyProxy.
SAGA - a simple API for grid applications http://saga.cct.lsu.edu/ SAGA is a high-level API, and which is roughly for distributed computing, what MPI is for parallel computing. SAGA distributes jobs across grids where MPI distributes tasks across processors.

Tools to launch tens of thousands of jobs

Tools for large numbers of jobs
Name/Topic URLs for downloads & documentation Description & Comments
Swift http://www.mcs.anl.gov/research/project_detail.php?id=56
SWARM http://www.collab-ogce.org/ogce/index.php/Swarm

Workflow Tools

Workflow Tools
Name/Topic URLs for downloads & documentation Description & Comments
Gateway Workflow Survey, December 2009 http://teragridforum.org/mediawiki/index.php?title=Gateway_Workflow_Survey Presentations on:
  • Taverna
  • OGCE/XBaya
  • Apache/ODE
  • Swift
  • Condor DAGman
  • Pegasus
  • Kepler
Java COG http://wiki.cogkit.org/index.php/Java_CoG_Kit Features include:
  • part of the Java CoG Kit
  • abstraction of the Grid and non Grid through providers
  • scripting language in XML
  • additional functional programming scripting language
  • supports graphs, loops, iterations, ..., has result caching
  • the language is extensible
  • workflow repository
  • prototype of graphical input forms that can be expressed through the XML language (e.g. they become part of the workflow)
  • allows users to integrate their own schedulers and concrete resource mappers
  • ability to launch MPI jobs on the TeraGrid
Kepler http://kepler-project.org
XBaya http://www.extreme.indiana.edu/xgws/xbaya Graphical client program for workflow composition, monitoring, and more; provides an easty-to-use GUI for users to compose workflows from Web Services described in WSDL

Miscellaneous: Interface Enhancements, Instruments, Globus, Compilations, and Gallimaufry

Instruments
Name/Topic URLs for downloads & documentation Description & Comments
none listed
Interface Enhancements
Name/Topic URLs for downloads & documentation Description & Comments
Google Gadgets (Tutorial) Google Gadgets Tutorial (6 MB; zip) Tutorial presented at the Web 2.0 workshop, March, 2009, Edinburgh by Mark Borkum (M.I.Borkum@soton.ac.uk, University of Southampton) and Jiten Bhagat (jits@cs.man.ac.uk, University of Manchester) on handy tools to add to gateways
Miscellaneous/Compilations
Name/Topic URLs for downloads & documentation Description & Comments
NSF's Software Development for Cyberinfrastructure awards
Globus Toolkit http://www.globus.org/toolkit/downloads/4.0.2

How to Add New Software to the Recommended Software List Above

Gateway Developers may request a TeraGrid Wiki account and add their recommendations and comments. Tutorials, links to examples, and software downloads are all appropriate here.

Markup for Adding a New Software Title

Use Wiki markup to add a new software title; paste in the following row into the table above and add appropriate text.

   COPY AND PASTE WHAT YOU SEE READING THE PAGE, NOT FROM THE EDIT FIELD
|-
| Type in name of software or software topic here
| [http://www.yourlink.org title-of-page]
| Type in a description or comments about this software

Markup for Adding a New Software Category

Use Wiki markup to add a new software category and a new title; paste in the following Title, Header, and Content rows inside the table above.

   COPY AND PASTE WHAT YOU SEE READING THE PAGE, NOT FROM THE EDIT FIELD

Markup for Adding a New Subcategory in Existing Section

|-
! colspan="3" style="background-color:yellow;" | Replace This Category Name 
|-
! width="125" | Name/Topic
! width="300" | URLs for downloads & documentation
! width="375" | Description & Comments
|-
| Type in name of software or software topic here
| [http://www.yourlink.org title-of-page]
| Type in a description or comments about this software

Markup for Adding a New Table (A New Category)


=== Replace This Category Name === 
{| border="1"
|-
! colspan="3" style="background-color:yellow;" | Replace This Category Name 
|-
! width="125" | Name/Topic
! width="300" | URLs for downloads & documentation
! width="375" | Description & Comments
|-
| Type in name of software or software topic here
| [http://www.yourlink.org title-of-page]
| Type in a description or comments about this software
|}

Personal tools