Supercomputing goes in and out of fashion - An interview with Justin Rattner - part 3

Heidelberg 07 April 2003 High-end supercomputing is much more depending on politics than most of those active in the field want to admit. It is always going in and out of fashion, depending on how important the national security and defense issues are perceived. At least in the USA that is the case, according to ISC2003 keynote speaker Justin Rattner. There is a transition from main stream supercomputing towards more cluster computing. A new breed of companies, system integrators focusing on cluster computing, is emerging as a result. According to Justin Rattner, however, SMP will remain a viable architecture for many years to come, building on single-chip multiprocessors.

Primeur: If you look at supercomputing and how it is used you see a difference in manufacturing and deployment of supercomputers in different parts of the world. Is there a difference in how people perceive supercomputing in Europe, the US and Japan?

Justin Rattner: Two things. First I have not been actively involved in supercomputing for seven or eight years. My work these days is in the microprocessor research area. But what I would observe in the US, having been around the field for nearly two decades, is that supercomputing goes in and out of fashion. In the US it is very much driven by politics. I cannot comment on whether it is the same in Europe or Japan. When the Cold War ended there was a huge loss of interest from the US government in high performance computing. If it was not for the ASCI programme supercomputing pretty much would have come to a dead stop in the US. Even with ASCI, the drop in government purchasing had the effect of chasing most vendors from the market. When the new administration came in and after 9/11 things seemed to be picked up again. There is a lot more interest from the government side in high performance computing. It's now viewed as integral to homeland security.

On the commercial side it is not all that different. In the late 80's and early 90's, there was this big swing away from large high performance machines towards high performance desktop machines because people could get comparable throughput from a high performance workstation - sometimes even better throughput - than they could from a larger computing resource. That also had an influence on the market for large high performance machines.

There now seems to be a swing back. But for most part that is driven by these self-assembled clusters like Beowulf systems. People are able to build 16, 32, 64-node machines for a specific application and get significant speed-up.

Also new applications have "come on-line" so to speak, like the genomic applications where highly parallel computing is quite useful. So at least in the US there seems to be a resurgence of interest in high performance computing solutions. But it is by no means a runaway success. We will continue to see a loss in the total number of vendors.

Primeur: But there are also a number of new small companies that are active in cluster computing?

Justin Rattner: Yes, the need has changed and these new companies are serving that need. We supply a number of board-level or even box level integrators. These companies use them in rack-mounted server products, but they are not like the traditional supercomputer vendors who could and would do everything on their own. They are integrators. I think a significant portion of the market has moved in that direction.

Today it is much easier and less expensive to build large SMP machines. You can get 16, 32 and even larger SMP machines. There is still a reasonable market for those machines and they go both into the technical and commercial applications.

Some of the processor trends we see, particularly chip-level multi-processing, are going to push the number of processors you can assemble as an SMP very much higher. In the next few years, we will see machines with 256, or 512 in an SMP configuration. With four CPUs per chip in a few years and maybe 8 CPUs in the second half of the decade, interconnecting 32 or 64 of those in SMP is going to be attractive. Large SMP is going to remain a viable architecture.

Primeur: And the software is ready to support that?

Justin Rattner: The SMP software always has been ahead of the clusters, although it is definitely improving on the clusters.

Justin Rattner is presenting a key note at ISC2003: http://www.isc2003.org.

At the Intel web site (http://www.intel.com) information on the latest chip technology and position papers of Intel on issues as Grid computing and cluster computing can be found.

For an overview of the complete HPC/Grid arena, visit http://EnterTheGrid.com.

News on supercomputing can be found on: http://www.hoise.com/primeur.