US Climate Research, damaged by US policy on HPCLondon 26 January 2001 A recent report on Climate Research in the USA complains that the US policy on HPC which exclusively supports distributed commodity scalable computing under the ASCI programme, is having a devastating effect on their ability to deliver state-of-the-art analysis, and that their European counterparts using Japanese computers are at least five years ahead of them.
Last November I reported the ECMWF workshop on meteorology, describing how weather and climate research require
Teracomputing to realistically run the current somewhat limited models. Introducing
more chemistry in the model and reducing the grid size from 65 kilometres to 3
kilometres, computing requirements spiral to exaflop/s. More importantly the memory
size and memory bandwidth grows even faster. This is why the MPPs developed via
the ASCI programme are totally inadequate for this type of problem. Using tens of
thousands of commodity chips, may provide the capacity, (peak flop rates), but not the
capability because of lack of memory bandwidth to a very large shared memory. For
rationale: (see HPCwire article 17756, "Limits and Expectations of HPC systems", 26
May 2000).
When one looks at the requirements at ECMWF for running their spectral model, they
use 400Gflop/s sustained running at 30% peak performance on their Fujitsu VPP5000.
What they want for their next system in a 18 months time is at least a 3 fold increase;
i.e. 1.2Teraflop/s sustained, 4-5 Teraflop/s peak performance.
At the ECMWF workshop many of the presentations were given by US researchers
from the main Federally funded meteorological and atmospheric centres. As is usual
in these presentations they discussed the merits of PVP Vs SMP architectures and
their software implications.
Many presenters made a plea for keeping vector parallel systems in production
because they felt that systems such as the NEC SX-5 with their well balanced
architecture and high memory bandwidth, are well suited for weather simulation
applications. They were aware that Fujitsu was pulling out of the Vector machine
market and recognised that this paradigm is under intense commercial pressure. What
I did not realise then is how desparate they were.
Dr. M. O'Keefe, University of Minnesota, with academic freedom to speak his mind,
gave a spirited defence of PVP systems such as the NEC SX-5, exposing the fallacy of
the 10 myths often used in the USA to pursue a scaleable architecture paradigm at the
total exclusion of vectors. He proceeded to show that PVPs are more efficient and
because of Amdahl's Law effects on load balancing, they outperform "infinite"
scaleable SMPs. He went on to say that: "the US lost its way in promoting only Scalar
systems in the ASCI programme and myopic in its crude restrictive policy of
excluding Japanese systems from the US market".
What I did not know then was that, as a step toward defining a U.S. climate modelling
strategy, the Environment Division of the White House Office of Science and
Technology Policy (OSTP) in January 2000 commissioned an ad hoc Working Group
on Climate Modelling Implementation and charged it with preparing a plan for
USGCRP climate modelling activities. The Working Group was chaired by Dr.
Richard Rood of the National Aeronautics and Space Administration and included a
representative from each of the Federal agencies with a significant investment in
global weather and climate modelling. Their 45 page report and recommendations to
the USGCRP are on the WEB and can be accessed from: http://www.usgcrp.gov
It is an excellent report, making a convincing case to streamline climate research in
the USA. It also articulates the shortcomings of US policy on HPC in a frank and
incisive way.
I have taken the liberty to use extracts from this report to wet your appetite, but I
strongly recommend that anyone interested in Climate and HPC should read the actual
report.
"The U.S. Global Change Research Program was initiated in 1989 as a high-priority
effort to address key uncertainties about changes in the Earth system, both natural and
human-induced; monitor, understand, and predict global change; and provide a sound
scientific basis for national and international decision making on global change issues.
Congress codified the USGCRP in the Global Change Research Act of 1990, in order
to provide for development and co-ordination of a comprehensive and integrated
United States research program which will assist the Nation and the world to
understand, assess, predict, and respond to human-induced and natural processes of
global change increasing the overall effectiveness and productivity of Federal global
change research efforts.
Since its inception, the USGCRP has been directed toward strengthening research on
key scientific issues. The Program has supported research that has led to substantial
increases in knowledge, improved predictive understanding, and documented
evidence of global environmental change, including major scientific advances in the
understanding of stratospheric ozone depletion, the El Nino-Southern Oscillation
phenomenon, global climate change, tropical deforestation, and other issues.
The interagency Subcommittee on Global Change Research of the Committee on
Environment and Natural Resources, a component of the National Science and
Technology Council, provides overall direction and executive oversight of the
USGCRP. Within this framework, agencies manage and co-ordinate Federally
supported scientific research on global change".
What Dr. R. Rood et al, found after their deliberations is that: "The requirements and expectations placed on the climate community have grown to the point that the U.S. requires the service of a dedicated organization, which is referred to here as the Climate Service". In short they want a climate institute to be set up with at least 150 scientists at an estimated cost of 70-120 million dollars a year for the next five years. The report goes on to say: "The Climate Service must be cognizant of and responsive to foreign centres that are defining the state-of-the-art in assessment and simulation capabilities and, increasingly, in scientific research". But it is the section dealing with issues related to high performance computing which makes fascinating reading, and I quote: "Shared-memory, vector computers manufactured in Japan, and essentially unavailable to U.S. researchers, have a combination of usability and performance that gives them far more capability than computers available to U.S. scientists. Parallel computers manufactured in the U.S., often with distributed memory, are difficult to use. In addition, there are intrinsic limitations to the ability of climate-science algorithms to achieve high levels of performance on these computers". And again: " Japanese-manufactured computers already delivered to foreign centres assure that U.S. scientists will have significantly less computational capability for at least three to five years". It then goes on to say: " With the delivery of the next generation of Japanese computers, and continuation of current approaches to computing in the U.S., the gap between the U.S. and foreign centres will increase and exist for longer than five years. The purchase of Japanese vector computers would have an immediate impact on climate and weather science in the U.S. and offers the only short-term strategy for closing the computational gap between U.S. and foreign centres". The story does not end here........ " Bottom line: the usability of Japanese supercomputers is much higher than that of U.S. computers, and they are therefore, pragmatically, "faster." The U.S. climate-science community is faced with a difficult, perhaps intractable, problem. With the present national strategic focus on distributed-memory computers, a tremendous expenditure on software is required. This includes not only applications software, but also the systems software needed to make the computer systems run. There is, increasingly, evidence that the ability of climate-science applications to utilize distributed memory computers is limited". Recommendation 7 spells it out even further: "The U.S. policy on high performance computing adversely affects the climate-science community and places U.S. centres at a competitive disadvantage with centres in other countries. At the least, the usability and performance of Japanese vector computers already delivered to non-U.S. centres assure that by many metrics the U.S. will lag non-U.S. centres for five years. There is substantial risk that this deficit will last longer with pervasive negative impact on U.S. climate-science"........ In addition, the growth of high-quality climate-science capabilities outside the U.S., especially the European Centre for Medium-range Weather Forecasts (ECMWF), The Hadley Centre, and The Max Planck Institute for Meteorology (Hamburg), places the U.S. climate-science community in a difficult position. ...... "In many cases weather and climate products from these centres are held up as the standard, and there is little hope of the U.S. meeting these standards in the immediate future".... Idle Reflections on what the future may holdIt is ironic, when the UN Intergovernmental Panel on Global Warming warns that Florida and many other lowlands could be drowned by rising sea levels in the next fifty years, US climate researchers are ignored by miss-advised politicians and left high and dry. One can't help but recall that when the Egyptians built the Aswan Dam, the rest of the world had to dismantle ancient temples and reconstruct them on higher ground. One wonders whether in 50 years time the White House and Abe Lincoln would get the same treatment, thanks to current political actions.
Chris Lazou |
 |