Most of the 25 presentations were from vendors, describing their own products, on topics such as hardware, compilers, graphics, storage and networking. They focused on cluster solutions, based on commodity chips, interconnect networks and associated file storage systems. An important component of the workshop is the availability of systems for benchmarking evaluation purposes.

There were exhibitions and presentations from eighteen companies, keen to promote their readymade products including those based on AMD Opteron and the Intel Itanium2 processor. A strong presence of AMD Opteron and Intel Beasley, early dual core systems as well as various models of Blade products, were on display and available for demonstrations.

Ron Bell, from the UK Atomic Weapons Establishment (AWE), started the workshop with a presentation titled: "The AWE HPC Benchmark". This was an interesting talk in that it discussed AWE's benchmark based on workload profile characterization and projections of future user requirements. They are currently running an IBM P3 system with 1856 processors rated at 2.88Tflop/s peak performance. The requirement is to buy a new system delivering 10 to 25 times greater performance than their current system, which translates to a system in the range of 30 to 70Tflop/s peak performance.

The benchmark contains a combination of codes from the whole AWE user community, physicists, engineers and material scientists, weighted to reflect their workload. It measures both capacity (throughput) and capability turnaround.

The physicists contributed plasma physics and hydrodynamic codes plus visualization. The engineers codes for solving explicit and implicit (100MDOFs) models with as much as 30Million elements. In materials two molecular dynamics codes were used, DL-Poly from Daresbury and WARP from Sandia.

Ron went on to cogently discuss the pitfalls of benchmarking when dealing with systems of differing speeds. He gave an example of two systems A and B with system A twice as fast as B and B having twice the number of processors than A. He then presented the usual scenario where capacity can be achieved by partitioning system B into two, running both partitions in parallel giving the illusion that both systems have the same throughput.

He went on to say: "The answer is not system A, gives better turnaround times but has the same throughput. The correct answer is system A has higher throughput than system B because system B must scale further in order to achieve the required job turnaround and system B may be unable to give the required turnaround for some capability jobs. In conclusion, don't compare N-way on A with N-way on B. Adjust N so that turnaround is about the same on A and B".

In running the AWE benchmark they found that: "There is a problem with measuring throughput of capability jobs.

At modest levels of parallelism, scalability is largely unaffected by interconnect. Scalability is intrinsic to application. The ratio between systems is constant with PE count. At higher PE counts where performance 'turns over' relative throughput varies wildly and becomes meaningless."

Also, it was very difficult to perform the capability test. There were very few benchmark data available, up to "turn over" point. To measure the job turnaround at a point just before "turn over", the best the system can do irrespective of number of PEs became difficult. Generally, a system, scoring better on this measure would need more PEs to achieve it - so throughput was probably lower. For this reason, capacity (throughput) and capability figures were presented as separate measures, with a warning about the large uncertainties on the capability figures.

Ron Bell wanted to be able to say things like: "System A has 10% higher throughput than system B for modestly parallel work, but system B has better scalability - so capability jobs show 20% higher throughput on B. If we assume half of the system will be dedicated to capability jobs, then System B gives more overall throughput". This was not possible from the benchmark results.

Ron Bell concluded: "The benchmark was too complex for vendors. Throughput test is not really useful as a benchmark. Multiple jobs don't usually interfere much. There was inadequate I/O subsystem on benchmark systems, but this should be excellent as acceptance test. Capability was very difficult to quantify."

In short, Ron Bell was saying that for large-scale applications (30Million elements), time to completion (turnaround) must be heavily weighted in the selection criteria of a system. Capability systems can always deliver capacity where capability is often beyond capacity systems, but capability is difficult to measure accurately as vendors do not usually have systems of that size for benchmarking.

Of course other factors often dominate the selection of large-scale systems. For example, one trade-off is price/performance; another are perceived hidden benefits from having the same system as your collaborators. One suspects, it depends on how one constructs the Total Cost of Ownership (TCO) integral. For my money I would also include ease of use and Mean Time Between Failures (MTBF), especially with systems with tens or hundred thousand processors.

As in previous years, of great interest were the Daresbury Benchmark results. These consisted of a plethora of distributed memory benchmark results, compiled by Martyn Guest and his team from Daresbury, from many systems including the latest products from vendors using their latest two core chips. The Daresbury benchmark suite, used to obtain these results, consists of many computational chemistry kernel codes, molecular dynamics, Quantum Monte Carlo, Jacobi Solver, STREAM - measured sustainable memory bandwidth in HPC (TRIAD), the Ab Initio molecular electronic structure DL_POLY and the parallel molecular dynamics benchmark. The results from SPECfp2000, SPECInt2000 and other well-known benchmarks were also presented.

Martyn Guest gave a similar talk as previous years, using this year's performance results normalised on last year's best PC based system, the HP RX5670 Itanium2/1.5GHz. Martyn emphasised that single processors are complex and often provide misleading results as they are almost always used in n-way nodes. If they are dual core, use both processors, if four-way cores, use all four so that interactions of cache memory and communications are accounted in the performance measure. For example, in the case of the 32-way IBM P4 this normalises availability of L3 memory, which reduces performance compared to using all L3 memory for one CPU. This applies to other vendors' products too. When more cores are added on a chip the interconnect fabric for transfer rates (bandwidth) to L3 memory need to be increased proportionately to handle the extra computational power from the extra cores. Otherwise memory paths if shared with the additional cores are likely to degrade the overall system performance.

Note that the results presented below, reflect performance in computational chemistry and this does not necessarily reflect the performance of these computers in other application domains. Also, the results given are not normalised on price/performance, so no specific value-for-money comparisons are made or implied.

Looking at SPECfp 2000 relative to the HP RX5670 Itanium2/1.5GHz, the SGI Altix 3700 Bx2/1600-9M Itanium2 system is 26% faster and the IBM e-Server P5 570/1900 is 22% faster, while the HP RX4640 Itanium2/1600-9M is 29% faster. These systems deliver around 40% of their peak performance on the SPECfp 2000 metric.

The benchmark results compiled by the Daresbury team indicate that the Intel Itanium2 systems are still the bright stars of today, but the IBM e-server P5 570/1900 is not far behind. They fare well as far as performance is concerned compared to other super-scalar chips, although one can detect a trend towards convergence. Performance of a particular chip tends to vary on different benchmarks and the version of compiler is run on, but one can see a pattern emerging. To give you a flavour, the summary index, normalised on the AMD Opteron 852/2600 using the PathScale compiler EKO2.2 - for the Matrix-97, Chemistry Kernels, GAMES-UK and DL-POLY benchmarks - was found to be as follows:

• The AMD Opteron 852/2600 (EKO2.2) (100)
• The AMD Opteron 852/2600 (PGI5.2) (88)
• The Dell PowerEdge 1850/3600 2MBL2 (72)
• SUNFire V40z 2.2GHz (76)
• The Bull NovaScale 4040 Itanium2/1500 (98)
• SGI Altix3700 Itanium2/1500 (98)
• SGI Altix3700 Itanium2/1600 (105)
• The HP RX5670 Itanium2/1500-H (99)
• The HP RX1620 Itanium2/1600-L (106)
• The HP RX1620 Itanium2/1600-H (121)
• The IBM p-Series 690/1700 (80)
• The IBM e-Server p5 575/1500 (78)
• The IBM e-Server P5 570/1900 (105)

Note that the HP compiler delivers about 15% improvement compared to its Linux counterpart on the HP RX1620. A similar pattern is also evident on the AMD Opteron 852/2600, when using PathScale's EKO(Ver.2.2) and the Portland Group's PGI(Ver.5.2) Fortran compilers.

The rest of the workshop consisted of presentations from vendors, with a strong contingent from the FPGAs fraternity, Mitrion, Celoxica and Nallatech, users sharing their experience and presentations from a number of companies, specialising in providing tailored system solutions from commodity components on demand. Instead of buying pre-packaged products from traditional vendors, a contract is placed with a small computer integration company, ClusterVision, Streamline, to built a cluster from favoured chips and an interconnect network, such as Gigabit, QsNet, Infiniband or Myrinet.

The presentations focused on cluster systems affordable by academic departments and associated components, which make the departmental computing environment. For example, Fiona Burgess and Michal Harasimiuk from ClearSpeed, presented their new co-processor, the CSX600, suitable for offloading compute-intense math library functions from serial CPUs. Michal claimed that: "Each CSX600 co-processor can sustain 25Gflop/s on DGEMM, while consuming only 5Watts of power. The CSX600, now available, is a SIMD array of processors. Each PE is a VLIW processor with a multiple execution floating point adder and floating point multiplier in both 32bit and 64bit IEEE754 standard."

A two-chip board delivers 50Gflop/s peak and uses 10Watts in total. It has up to 1GB shared DRAM for local processing. A dual CSX600 using PCI-X, 20Watts/card accelerator board gives 100Gflop/s peak on DGEMM, BLAS, LAPACK, GROMACS, CHARMM NAND and so on. It is suitable for computation in finance, Monte Carlo, Generic math, providing transparent acceleration for packages such as Matlab, Mathematica, NAG, Maple and entire applications in biochemistry.

Chris Brown gave the "IBM HPC systems perspective" talk emphasizing that one size does not fit all. He explained that in addition to the IBM P5 line, IBM is exploring the low power consumption Cell technology. The speaker claimed that because IBM is a large company with a strong track record of innovation and in control of component developments, it is able to leverage these innovations across the whole design spectrum. For example, take the game processor developed from cell technology for the large consumer market, add a couple of floating point pipes and what one has is the Blue Gene. Both the Blue Gene and the MareNostra at the Technical University of Catalonia based on the Power PC 970 FX Processor are examples of experimental technologies. IBM believes that next generation chip designs are focusing on high performance/power consumption ratios and that semiconductor power trends are driving future systems. With hundreds of thousands of processors, the challenges are from software developments for an easy to use environment. I may also add, reliability such as MTBF.

Joerg Stadler from NEC HPC Europe described the NEC SX-8 vector parallel system whose raison d'être is capability supercomputing. NEC is committed to continue to develop products at the top end of HPC using the latest technologies. Their next generation system is likely to be heterogeneous with a strong vector processing component. He also explained that they are offering total solutions to customers, from high computation to data management infrastructure, based on their super-scalar Itanium2 system. He cited as an example the DKRZ climate centre in Germany where the computation element is delivered by the NEC SX-6 and the data management of some 220Tbytes using the NEC TX7 and Oracle. They also deliver tailored made scalar systems in collaboration with other vendors. The 100Tflop/s scalar system for the Tokyo Institute of Technology, where NEC is acting as the integrator, but collaborating with ClearSpeed, AMD and Sun Microsystems, was cited as an example.

There are some very positive trends in multi-core chip developments, but the workshop also had some surprises. These are too sensitive to be printed here. For those of you keen to add the latest 'gismos' to your facilities, remember the Latin adage: "Caveat Emptor". As the season of goodwill is upon us: Wishing you all, Seasons Greetings and a Peaceful Happy New Year.