The HPC Challenge benchmark suite includes the same single test, called Linpack, that is used as the basis for the semi-annual TOP500 ranking, and substantially augments Linpack with six additional tests to better predict how HPC systems will perform on a broad range of real-world applications.

"Linpack is useful, but no single test can accurately reflect the overall performance of HPC systems", stated Jack Dongarra of the University of Tennessee, a co-publisher of the TOP500 list who assembled the new test suite with colleague Piotr Luszczek. "The HPC Challenge benchmark test suite stresses not only the processors, but the memory system and the interconnect. It is a better indicator of how an HPC system will perform across a spectrum of real-world applications."

On the individual synthetic Linpack test used to produce the TOP500 list, a Cray X1 system with 504 applications processors achieved a speed of 11.7 billion calculations a second (gigaflops) per processor, or 5.9 trillion calculations per second (teraflops) in total. This was the highest per-processor speed of any currently available HPC system in the rankings.

The Cray X1 system also achieved efficiencies - actual speed as a percentage of theoretical "peak" speed - of 90 percent or better in all cases. These efficiencies are substantially better than those reported for high-performance computing (HPC) systems from major U.S. competitors.

"Despite the Linpack test being a purely processor-centric measurement versus the more system-centric approach of the HPC Challenge tests, a Cray X1 system at Oak Ridge National Laboratory with 504 applications processors ranked number 20 on the list of 500 supercomputers, outperforming competing supercomputers with double and triple the number of processors", stated Peter Ungaro, Cray's vice president of worldwide sales and marketing.

Based on test results reported by customers including the Army High Performance Computing Research Center (AHPCRC), Oak Ridge National Laboratory (ORNL) and the U.S. Army Engineer Research and Development Center (ERDC), a Cray X1 supercomputer had the highest overall test scores of any system listed on the new HPC Challenge benchmark test suite, as of June 10, 2004.

The new set of tests, co-sponsored by the DARPA HPCS (High Productivity Computing Systems) program, the U.S. Department of Energy and the National Science Foundation, was introduced at the SC2003 conference in November 2003.

The HPC Challenge benchmark suite is intended to test multiple attributes that can contribute substantially to the real-world performance of HPC systems.