Mr. Zwieflhofer explained that the Earth system science covers a wide range of temporal and spatial scales from short-term weather to long-term climate evolutions but also from the local to the regional up to the global level.

The key goal of these efforts is to develop and enhance the capability to monitor and predict how the Earth system is evolving, according to the speaker.

As far as the temporal scales are concerned, the art of weather forecasting is dominated by the atmosphere's initial conditions which can change in only a couple of hours or a few days. Seasonal and inter-annual forecasting spans a period ranging from two weeks till two years. Here, it are the oceans' initial conditions which have the largest impact on the forecasting. Climate change prediction in turn heavily depends on the forcing factors that can occur in nature or that are caused by human activity. This goes over centuries.

Mr. Zwieflhofer indicated that the major model components used in Earth system modelling are those visualising the atmosphere coupled with waves and other surface effects to forecast the weather. For the seasonal and inter-annual forecasting, the atmosphere is couples with the ocean up to about 300 m. In climate change prediction, it is not only the ocean which is coupled with atmosphere but also the cryosphere, carbon cycle, aerosols, and many more elements.

In weather forecasting the data is assimilated in real time but requires often more resources than the forecast models can provide, according to the speaker. Chemistry details are therefore very desirable, that is if the component resources allow it. For insight into historical climate evolutions, it is necessary to produce consistent data sets to validate the model. Here, the researchers have to monitor the evolution of the key variables. Elements such as CO2 are essential in this regard.

In order to re-analyse historical observations, researchers have to be able to produce data sets that are as consistent as possible. Mr. Zwieflhofer stressed the importance of international collaborations that are required to incorporate the maximum of the available data. From 1957 to 2002, ERA-40 was active in this respect and performed conventional observations from 1957 and satellite data from 1973. The analysis system used a 125 km grid and a coupled wave model.

The validation and production phase took no less than three years on ECMWF's HPC systems but it had to be completed within the life-time of one HPC system to avoid the overheads caused by migration. Now, the resulting data set is close to 40 TB in size.

Walter Zwieflhofer also addressed the problems related to the data management. Space-based instruments and high-resolution models produce large volumes of data. To use this data effectively, the data needs to be carefully managed.

The archives held by centres such as the National Center for Atmospheric Research (NCAR) in Boulder, Colorado, USA, containing over 1000TB of data and ECMWF storing more than 800TB, count as some of the most valuable assets.

The NCAR and ECMWF data management systems are clearly separated from the HPC resources. Metadata-based access and increasingly faster wide-area network links have opened these archives to the wider research community. The speaker is convinced that the data problem is not insurmountable. All that it requires is attention and the availability of dedicated human resources.

Questions which need to be asked here at SCI 2003, according to Mr. Zwieflhofer, are whether the vector architecture still remains indispensable and if there is an interaction between the Earth system modelling and the TOP500. Summarised, it comes down to one issue: which architecture is best suited for the job?

In relation to vector architecture, the speaker told the audience that ECMWF has 25 years of experience with vector machines and its IFS model is over 99 percent vectorised. Great care has been taken however to ensure that the model runs efficiently both on vector and scalar architectures. A high-level blocking scheme which is adjustable at run time, supports both the use of long vectors for vector systems and the cache requirements of scalar systems.

It has been claimed that vector machines handle the dynamical parts of the models better and that scalar machines handle the physics calculations more effectively. Mr. Zwieflhofer referred to recent optimisation efforts and profiling work which have proven that these claims do not need to be taken for granted as such.

For a half year up to March 2003, the ECMWF institute operated three vector parallel systems from Fujitsu and two large scalar Cluster 1600s from IBM. Differences are as follows: the Fujitsu VPP5000 has 100 vector processors against 2x960 scalar processors for the IBM Cluster 1600. The Fujitsu peak performance is 9.6 Gflops per processor in comparison with the 5.2 Gflops per processsor with 8 processors per shared-memory node for IBM. The Fujitsu systems have 4GB memory per CPU compared to IBM's 8GB memory per node.

Recent experiences within ECMWF have shown that advantages of vector-based machines include a clear feedback on the efficiency of the codes being run and better environmental characteristics than systems built using general-purpose SMP servers. The pros of scalar-based machines involve that front-end machines for scalar-only tasks are not required and that the system is more tolerant if a medium level of efficiency is acceptable. ECMWF application programmers agree that vector and scalar system can equally well provide the HPC resources for high-end modelling work, according to the speaker.

Looking at the relationship between the Earth system modelling and the TOP500 of supercomputing, Mr. Zwieflhofer noted that for most scientifically valuable applications of this type of modelling, the peak/sustained ratio is heavily influenced by the machine architecture. Vector machines show typically 20 to 50 percent of peak - a factor of 3-4 - while for scalar machines this is 5 to 15 percent of peak - a factor of 1.4. If we study the TOP-10 systems in the November 2002 TOP500 list, we can see that vector machines have an average Rmaxis of 88 percent of Rpeak. For scalar machines this is an average Rmaxis of 62 percent of Rpeak.

Mr. Zwieflhofer therefore thought that Linpack results are not an accurate reflection of a system's capability for running Earth system codes - and this is neither the TOP500 list producers' intention. However, there is an affinity between Linpack and high-resolution spectral models. The efficiency of Linpack lies in its compute intensive part, which is a matrix multiply operation, e.g. BLAS DGEMM, as the speaker explained.

For spectral models, the LegendreTransform (LT) can be a relatively expensive component. LT costs increase as a cube of resolution: O(m3). FFTs costs increase: O(m2*log2m) and Grid computation costs increase as well: O(m2). But as is the case for Linpack, the compute intensive part of the LegendreTransform is a matrix multiply operation (DGEMM). This property of spectral models has been well documented and demonstrated by the Earth Simulator team, as the speaker noted.

With Linpack, very large problem sizes can be chosen, thus reducing the relative importance of communication bandwidth. The global communication requirements are substantial and the very high sustained performance shown can only be achieved on a system with excellent communication bandwidth. Walter Zwieflhofer however thought users of spectral models do not have to worry that their compute power soon will be consumed solely by LegendreTransforms.

In fact, most models have a very different starting point. They devote a much higher proportion of the resources to the calculation of physical and chemical processes. In addition, numerical methods such as for instance semi-Lagrangian that allow longer time steps and shorter run times, reduce the relative importance of the LTs.

To know which architecture is best suited for the job, the major criterium is the relationship cost/performance but for Earth system modelling, peak performance, Linpack results and peak/sustained performance ratio do not constitute ideal measures for cost/performance evaluations, and neither the sustained Gflops that can be achieved on the user's application as measured by HW counters. According to Mr. Zwieflhofer, the most reliable measure for performance is the wall-clock time needed to solve a given task such as for instance a 100-year simulation that is representative for the scientific work planned.

As far as scalability issues are concerned, the Earth system codes do not scale linearly if the same size problem is run on a larger number of processors. The planned increases in the problem sizes however mitigate this effect. Generally, a high ratio of parallel instruction streams in the application and their length is advantageous, as well as are the number of parallel instruction streams required to keep the hardware busy, according to the speaker. In this respect, an 8-way M&A vector pipe will require as many independent instruction streams as 8 scalar M&A units. The sustainable flop count in absolute terms per "hardware thread" is essential for application scalability whereas the ratio of peak/sustained flops is not.

It is clear though that the time to solve must not exceed certain limits. For operational weather forecasting, this means a few hours. For an ensemble of hight-resolution climate runs, it can mean several months. High standards of system stability and reliability are mandatory for many production-oriented HPC centres and the system must be reasonably easy to use. Mr. Zwieflhofer also observed that lately the power, cooling and space requirements of large systems built out of commercial servers have been growing steadily but he does not think that this is sustainable.

Earth system modelling is very demanding with regard to HPC requirements. The speaker mentioned increases in resolution for atmosphere and ocean, the use of ensemble techniques for probabilistic forecasting, and the introduction of more chemistry and the modelling of clouds which requires more processes and more realistic formulations. The architecture though should not be too closely tied to a particular balance of these growth drivers since the scientific progress can be unpredictable, concluded the speaker.