|
The basic architecture consists of a number of computational nodes, a number of visualisation nodes, an Infiniband interconnect, and FC Raid disks. The system can support several visualisation devices, including a rear projection stereoscopic wall and Inifinity wall, and a tiled display. It will also be connected to the Grid, allowing remote visualisation and computational steering.
The system is designed to support interactive visualisation (computational steering) of large simulations. Therefore there were minimum requirements on the number and performance of the visualisation nodes, the storage capacity, and the interconnect speed. For the number of computional nodes the choice was: as many fast processors as we can afford.
The 8 visualisation nodes consist of 2,4 Ghz dual AMD Opterons with nVidia Quadro FX 4500 PCI-e graphic cards and 12 Gbyte of main memory. The 200 compute nodes each have 3,2 GHz dual Intel Xeon processors with 2 Gbyte of main memory. Infiniband is connected through HCA PCI-e boards. There is a 216 port Infiniband switch. The system has 5 Tbyte of FC-Raid and a 10 Tbyte parallel filesystem.
The stereoscopic visualisation system has been built in one of the PC2's conference rooms. It consists of a 1,80m by 2,40m projection screen, with back side projection. The resolution is 1400 x 1050 pixels and tracking is through infrared, that can be on the VR-glasses or the fly-stick. Two of the hpcLine's visualisation nodes are needed for the visualisation projection system.
Cooling the large hpcLine system was a design problem. At 2,6 Tflop/s of computing speed, the system needs 70 kW of electricity power. And as we know that has to come out as heat one way or the other. The computer room at PC2 had only 60kW of air cooling power. Hence the choice was to install a special water cooling system, just for the new computer, or come up with some other idea. The latter was done. The processors generate about 60% of the heat. They are directly cooled and that cooling system is through a heat exchange linked to the buildings climate system. Although the latter has a relatively high temperature (as it is not designed to deliver low temperatures) of 24-30oC, this proves to be enough to let the hpcLine's residual cooling requirements fit into the air cooling's capacity.
The compute node design allows for later inclusion of other special processors, like FPGA's, GPU's or Cell processors.
The system has a large collection of software and tools. The access is through the CCS management software: the computer centre management software is designed by the PC2. The operating system is Linux Redhat AS release 4. Compilers include the usal GNU gcc compiler and the Intel C/C++ and Fortran compilers. Both Scali MPI and MPICH are supported. Scientific libraries include Intel MKL and Atlas. padViz and Amira are available for scientific visualisation.
Simon concluded his presentation by giving a number of benchmarks results, including SPEC results, NAS parallel benchmark and Linpack benchmark results.
The main purpose of the system is in exploring interactive visualisation and computational steering, which are difficult to capture in simple benchmarking figures. PC2 and its users are already setting up a number of projects to use the system's capacity.
|