Spicing up the use of federated Grids for computational biology

Dresden 29 June 2006Peter Coveney and Shantenu Jha presented their ISC Award Winning Paper on the SPICE project which stands for "Simulated Pore Interactive Computing Environment". The project uses a federated Grid for grand challenges in computational biology.


The UK National Grid Service (NGS) and the US TeraGrid provided the federated Grid for the SPICE team to compute free energy for biomolecular systems. There is also the EU DEISA Grid infrastructure to which the team hopes to have access in the near future.

Peter Coveney explained why scientists need computational Grids. The research problems are too large to fit on any one resource so there is a problem of quantity. What is large is time dependent so there is a need for large systems, that are time invariant. In addition, a computational Grid provides more of the same and efficient utilization; the solution scales economically. There is also a need to use different and new resources which involves a qualitative factor allowing scientists to insulate code and use a new paradigm. The intellectual challenge plays its role too: we use a computational Grid "because we can".

Peter Coveney also addressed the complexity of biosystems. There are regions where computational modelling can be employed today versus the goals to cover. The calculation of free energy is computationally expensive and scientists need accurate results fast. The algorithmic advances are necessary but unfortunately not sufficient. Computational Grid science which approaches can be adapted to exploit Grid-based computation. An interplay of physical algorithms and Grid architectures will be established.

The SPICE project is handling RNA translocation through protein pores. It is a molecular biology critical and ubiquitous process. SPICE is a model for gene expression in eukaryotic cells to detect viral infection. The size complexity and timescale computations are expensive. The millions of CPU hours that are using vanilla MD are not good enough.

The free energy profile is extremely challenging but yields maximal insight and understanding of the translocation process, according to Professor Coveney. It also involves a novel algorithm. SMD+JE needs to determine optimal parameters before simulations are performed at the optimal values. This requires interactive simulations and distributing many large parallel simulations. The interactive live coupling uses visualization to steer the simulation and reduces computational cost.

The process replaces the single long running vanilla MD simulation with the following scheme:

  • step 1: understand structural features using static visualization
  • step 2: interactive simulations for dynamic and energetic features
  • step 3: simulation to compute optimal parameter values
  • step 4: use computed optimal values to calculate

We are talking a whole novel analytical approach here, according to the speaker. SPICE requires interactivity but not with fixed datasets, involving computational steering, Grid infrastructure and a library to determine the destination.

The interactive simulations are used to determine the optimal value, stated Shantenu Jha, who is working on the SPICE project together with Professor Coveney. The steered simulations involve bidirectional communication both qualitatively and quantitavively.

Interactive simulations are used to determine the optimal value of force-constant and pulling velocity; there is the choice of the sub-trajectory length and the location for optimal value simulations. SPICE uses visualization to provide input to the running simulation. This requires 256 px or more of HPC for interactivity in the steady-state data stream up and down.

Interactive simulations perform better when using optical lightpaths between the simulation and visualization due to the network charateristics. This is a typical legacy app NAMD, not written for network I/O. An unreliable transfer can stall the simulations, according to the speaker.

What do lambdas provide that cannot be accomplished on best effort networks? They offer a fixed latency with no jitter at all, next to a high bandwidth and QoS providing what you need when you need it. Lambdas allow to provide virtual networks a la UCLP. Shantenu Jha also mentioned the ability to use non standard transport. Scientists need new porgramming models and scientific algorithms to exploit this increasingly powerful resource.

At the Global Lambda Integrated Facility, performance profiling experiments are being set up. Experts are investigating how differing network characteristics affect the performance of an interactive NAMD job. The NAMD simulation sends the simulation state over the network to VMD.

The speaker addressed the performance profiling results. One can define performance as wall time per simulation timestep. The interactive mode performance is sensitive to packet loss and the performance is sensitive to latency. On the other hand the performance degradation is insensitive to the processor count.

The scientists are performing a simulation of molecular dynamics using Jarzynsky's identity scalable non-equilibrium solutions. The free energy differences in large structures are calculable. The simulations are useful to understand diseases and to develop drugs, according to Professor Coveney.

At the end of 2004, the joint NSF UK Research Council issued a call for proposals to use the UK high performance computing services and the TUS extensible terascale facility.

Next to SPICE there are two other projects. Nektar is studying the blood flow in the human arterial tree and Vortonics addresses vortex dynamics on transatlantic federated Grids. The enormous problem sizes, the memory requirements, the long run times and the largest runs require a geographically distributed environment.

As far as the challenges are concerned, scientists aim at an interface application code to Grid middleware through well defined user-level APIs. No code refactoring would be required, according to the speaker. It hides the heterogeneity of the software stacks and the site-specific details.

SPICE uses a RealityGrid steering library. The machine configuration issues were the following:

  • variants of the same problem faced a hidden IP issue for MPICH-G2 and RealityGrid steering
  • the same problem occurred on differnet resources e.g. PSC and HPCx
  • the PSC=qsocket + Access gateway node are vital
  • performance issues still remain due to protocol constraints
HPCx involves the same solution but does not work for RealityGrid Steering.

New policies are required, according to Peter Coveney. Three different scientific applications all had a common requirement: the co-scheduling of resources. There are three levels of scheduling complexity:

  • advance single resource reservation
  • advance co-ordinated multiple reservations across a Grid
  • advance co-ordinate reservations across distinct Grids

The first breaks the standard HPC usage model. The second requires lots of effort and the third, cross-Grid scheduling is very hard today. The current levels of human intervention are too high: we need automation, the speaker stressed.

There are four essential common services:

  • authentication, authorization, identity managment
  • jobs submission and auditing tracking
  • data management
  • resource discovery and information services

More needs to be done and quickly too. The current barrier to utilise federated Grids is still high and many degrees of freedom need co-ordination. The speaker stated that collective inter-Grid debugging is required.

The interoperability should be considered in practice. A stress test is needed for that, using real applications. This requires additional user level middleware, concluded Peter Coveney.

Leslie Versweyveld

[Medical IT News][Calendar][Virtual Medical Worlds Community][News on Advanced IT]