Parallel computing is here to stay

Delft, Friday 20th August 1999 A year ago, there was a lot of uncertainty in the parallel processing arena. No apparent progress was visible, people were leaving the field and a decreasing interest of companies in high-end parallel processing led to some pessimism. According to Professor Henk Sips parallel processing has now come into quiet waters. It is definitely here to stay. Perhaps it is not as exciting as a few years ago, but there lies much work ahead. We talked to Henk Sips at the Parco Parallel computing conference in Delft, of which he was co-organiser.

According to Henk Sips the parallel processing industry has changed considerably during the past year. It entered into a consolidation phase. In shared memory hardware for instance, there is a convergence to a common architecture, based on integrated clusters of NUMA SMP's. Also the difference between SMP and MPP is disappearing. He pointed at the industrial session, were the major vendors all outlined the same basic architecture for their supercomputers.

At the next lower level of performance, one finds clusters of work stations, that still give a considerable computing power.

What is becoming increasingly important, according to Sips, low latency interconnections and the locality of data. You may not want to know it, but parallel programmes run much faster when the data to be processed is near the processor that has to do the calculations.

Three of the main improvements Sips sees are:

Compiler technology: For example HPF has become more efficient during the past year
The performance of message passing software is improving
Network speed is becoming better

Also during the recent past, commodity has sneaked in. Off-the-shelf component s are now indispensable for building parallel systems.

This marks the end of a turbulent era in parallel processing. We have now entered a phase of convergence, both in hardware and software architecture and of consolidation, were we built more optimal and maintainable parallel software. Parallel processing has become main stream: there a set of methods and tools one can use.

So all in all, Sips is optimistic when it comes to the future of parallel processing.

However, that we know how to do parallel processing, does not mean it is easy. As David Womble pointed out in his Keynote Presentations, especially parallelising legacy codes takes time. One of the reasons, is that we do not have depreciate on software, says Sips. For buildings, it is custom to do that. It means making reservations so that there is money to build something new afterwards. For software it is the same. It takes time to bring it to a state-of-the-art level.

In some related areas, there are still some very new developments. On is for instance the "Computational Grid" in the US, that is modelled after the power grid. In fact, the grid is more of a political than technical project. It aligns funded projects in a way they tend to co-operate within the framework defined by the grid. However, from a technological point of view, it does not work smoothly yet. Connecting co-operating distributed systems from different organisations in a seamless way, is still a bridge to far. Unfortunately, in Europe, there is no comparison to the grid project. There are some national initiatives, but they are on a smaller scale.

The high-performance computing market is becoming more and more split into two. In the very high-end, there are the top-of-the line machines. Their number is slowly declining, as is the number of users that really need the power. Here the Japanese vendors are doing pretty well.

Below the top-level, you have a growing segment of mainly cluster based systems that are cheaper and satisfy all but the most demanding users.

Building a cluster is easy, Sips explains. You go to the computer store and buy a number of computer boards. Then you put Linux on them, and off you go. They have done this at Sips' institute at the Delft University of Technology without any problems and there are many other examples.

Linux is becoming increasingly popular, especially for scientific and technical computing. Apart from Linux being a stable and reliable operating system, the level of support one can get is important. There are several good and supported distribution available in the market. In Delft they use Redhat, but you also have for instance Suse and Caldera. Also the increasing number of applications available on Linux is important.

Compared to Solaris, that is now also freely available to universities, Linux is smaller. However, for building large parallel systems, even Linux is much to heavy to have a complete copy on each node. There is need for a light-weight kernel for the computation nodes of a large parallel machine.

The biggest Linux cluster machine in the Netherlands probably the DAS distributed parallel computer, that is distributed over four universities and three cities, including Delft.

Overeseeing all what is happening in the parallel processing field, Sips notices the uncertainty has gone, and there is a clear road ahead.

Ad Emmen