The Top500 list - an analytical and critical viewMunich 31 July 2001 Since 1973 the top team Jack Dongarra, Hans Meuer and Erich Strohmaier publish the top 500 list twice a year, the odd number in June at Meuer's Supercomputer Conference, now in Heidelberg located, and the even lists at the IEEE SC conference. In the beginning it was quite interesting to analyse the list. But during the years it developed a life of its own, driven by the "marketers" of the HPC vendors. This view highlights two sides, the political and the analytical aspects. It is a personal view and summarises some discussions and a talk given by me at the Hewlett-Packard HPC users group meeting in Barcelona last year.
Professor Hans Werner Meuer started in June 1986 his - at that
time - German based Supercomputer Conference in the halls of
University in Mannheim - by the way earlier than the IEEE
Supercomputer. He presented the 15 German Supercomputer Centres
with their machines. They fitted without problems on one slide.
During the years this list grew steadily. Then the question
raised up, what is a supercomputer and what characterises it?
At that time Jack Dongarra's Linpack benchmark was well known
and accepted. Every computer vendor used it and published the
result in Jack's report. It started with a matrix size of the
linear equation of 100x100. But soon Jack extended the dimension
to 1000x1000. Rumours say that the idea of the Top500 grew, when
Hans Meuer and Jack Dongarra drank beer on one of these
conferences. The computers should be measured following a
maximal possible linear equation size and then the first 500
ranked, based on the Linpack performance. In June 1993 they
published in Mannheim the first Top500 list.
This year Hans Werner Meuer presented the 17th list in
Heidelberg and supplied the reader with an important
contribution to the history of supercomputing since 1993. In the
first June list we find the Thinking Machines CM5 with 1024
processors and a Linpack performance, named Rmax in the table,
of 59.7 GFlop/s (billions of floating point operations per
second) and a peak performance (Rpeak) of 131 GFlop/s. In the
fall list the numerical wind tunnel from Fujitsu with 140
(vector) processors, an Rmax of 124.5 GFlop/s and an Rpeak of
236 GFlop/s. This computer demonstrates the optimisation
possibilities of the Linpack Program. In 1993 Rmax of the
Fujitsu achieved 53% of the peak performance. In 1996 the
computer has been extended to 167 processors and gained 281
GFlop/s Rpeak and 229 GFlop/s Rmax, 81% efficiency. Number 500
in June 1993 was a Fujitsu Siemens computer VP-200, in 1985 at
IABG in Ottobrunn and 1990 outsourced to debis Systemhaus with
0.422 GFlop/s Rmax und .533 GFlop/s Rpeak. At the time of its
installation it was the first Japanese vector supercomputer in
Europe and one of the fastest with 64 MB main memory. By the
way, the author took care for the industrial users of this
machine for seven years, advisory service, vectorisation,
optimisation, education, marketing and sales.
Today IBM's ASCI White is ahead with 7. 226 TeraFlop/s Rmax - a
factor of 121 faster than the CM5 from 1993. It uses 8193 Power3
Processors with 375 MHz and has an Rpeak of 12 TeraFlop/s, 60%
efficiency. Last November the same computer only achieved 4.9
TeraFlop/s, only 41% efficiency. IBM had done a good job in
optimising the code.
The Top500 table in HTML-format contains the rank, the vendor,
the computer, the Linpack performance (Rmax), the installation
site - mostly with web addresses, the country, the year of
installation or upgrade, the number of processors, the peak
performance (Rpeak) - this is the performance vendors guarantee
not to exceed, NMAX und N1/2. The last two figures are for
experts or specialists but have a high entropy. NMAX is the
dimension of the solved linear equation and N1/2 is the
dimension of the system, at which the computer achieves half of
the peak performance. Both figures are optional. As an example,
I look at the Hitachi SR8000 at Leibnizrechenzentrums of the
Bavarian Academy of Sciences in München. With a matrix size NMAX
of 120 000 the computer gets 1035 GFlop/s (Rmax), with a matrix
size of only 15160 (N1/2) he achieves 672 GFlop/s. Thus one can
conclude that this machine has a good performance with "minor"
problems. Last November IBM's ASCI White did not got half of the
peak performance, with a dimension of 430 000 it reached "only"
4938 GFlop/s. But this year, as mentioned earlier, the IBM team
optimised the Linpack. Now, with a dimension of 518 000 it gets
7226 GFlop/s and with a dimension of 179 000 (N1/2) should get
6144 GFlop/s.
Technically the list is very interesting and contains a lot of
information. One example is the efficiency, the relationship of
Rmax and Rpeak in the Linpack benchmark. Looking for the first
systems one gets this list:
An other important aspect is the scaling. Does the computers in the list scale with higher processor numbers. The Rmax results show whether the increasing processor number results in a similar increase in the performance. Top500 and the science politicsThe Top500 list is extremely easy to understand, as the computers are ranked following the Rmax value. Only one number is important, the rank! This pleases the scientific boards or the Father of the Country and the science politics. This happened for example, when Germany got rank 5 in the world list with the Leibniz Computer Center of the Bavarian Academy of Sciences in Munich one year ago. The Wissenschaftsrat in Germany, Scientific Council, regularly makes use of this list in his recommendations and papers. Thus he mentioned in his recommendations from 1995 that in Germany the top systems are missing, as Germany had no computer in the first fifty ranks. On the other hand, politicians are happy to mention that their system is highly ranked. Top500 and the vendorsThe computer vendors send their Linpack results to the Triumvirat, Dongarra, Meuer, Strohmaier. In the meantime - within these eight years - they recognised the huge marketing potential. All computers are presented, which get the performance and which are allowed by their customers to be mentioned. There will never run a Linpack on most of the commercial computers. An example are the computers in the banking scene, where they run risk analyses or SAP R/3 problems. An other field is the Telecommunication industry. Thus the Top500 list developed its own way of life, never expected and thought by the authors. On the other hand, the integration of commercial systems in this list underpins the importance of those computers for scientific high-performance computing. The clusters and parallel computers are now used in both arenas. The Top500 list over timeAn important information delivers the time. Eight years are a long time, where one can see the different developments in computer architecture and processor structure. In the beginning life was so easy, most of the systems have been vector supercomputers, 340 out of 500. About 100 have to be counted to MPP systems like CM5 or paragon, nearly 40 belong to the SIMD architecture, today gone like the dinosaurs. This year there are only 47 vector computers and 453 MPPs. The 500 computers divide into 314 MPPs, 113 Systems based on constellation nodes, 40 SMPs and 33 clusters. Other topics of this historic view are the vendors - a lot of them have gone, they are no longer with us. Others, not so active in the supercomputer arena - like IBM - have become the leader in the Top500 list. Interesting is the look to the applications, the move from engineering to database and other commercial usage. The Top500 Web SiteThe website have been redesigned some time ago and now allows sublists, the user can specify. The access to old lists and analyses are possible. The new site delivers a lot of information and the interested reader can sit hours at the computer and view all the possible information. Linpack BenchmarkThe program solves a dense linear equation. For the Top 500 the vendors can scale the problem to the maximum memory size and optimise the software to gain the best performance. The method must be identical in the number of floating point operations as the LU factorisation: number of floating point operations: = 2/3 N exp 3 + O (N exp 2). This excludes the usage of fast matrix multipliers like the Strassen algorithm. As the vendors like top Rmax, some used the Strassen algorithm, which needs less floating point operations. Then they divided the results by the number of LU operations. This lead to the curious situation that the supercomputer exceeded in the Linpack the peak performance. When I received the list some years ago before publishing, I noted this fact. The vendor then revised the results that then lay below Rpeak.
Uwe Harms |
||||||||||||||||
 |