Pittsburgh PCS transfers large archives to SDSC

San Diego, 20 March 98 Pittsburgh Supercomputing Center (PSC) moved part of its data and program files to the San Diego Supercomputer Center and the US National Center for Supercomputing Applications. The transfer of files to SDSC will allow researchers to continue their NSF-funded research projects on NPACI's Cray T90, T3E, and IBM SP-2 supercomputers. PSC has approximately 40 Tbytes of data in its archives, of which a quarter is recent and still being used by NSF researchers.

The files are being transferred to NPACI's HPSS archival storage systems, which uses robots to access thousands of tapes. The files transferred from PSC will be contained in an IBM 3494 Automated Tape Library with IBM 3590 tape drives and a capacity of 6000 tapes. Each 3590 tape holds 10 Gbyte of data without compression, or about 13 Gbyte at typical compression ratios.

Until the advent of the vBNS, only experimental networks had carried information ovmer long distances at high data rates. The only solution to the problem of moving Tbytes of information was to dump the data onto hundreds -- or even thousands -- of tapes, physically transport the tapes to a new location, and load the computers at the new site from these tapes. The network makes the process simpler, less expensive, and much less labor-intensive and prone to risk.

Setting up such a massive and sustained data transfer is not a trivial task: each link has to be up to spec. The two centers are connected on the vBNS by an OC-3 link at 155 Mbits per second, and fast internal networks and I/O interfaces at PSC and SDSC comprise the rest of the chain.

The end of May is the end of NSF support for computing at PSC, but data in PSC's archives probably will be accessible for some number of months after May. However, the the storage resources inevitably will be reclaimed by PSC for other projects. The Pittsburgh Supercomputing Center is continuing operation under funding from the Department of Energy, the state of Pennsylvania, and other sources, but NSF-sponsored research accounts will no longer be available.

A joint paper on the file transfer process and its results will be submitted by PSC, NCSA, and SDSC for presentation at the SC98 conference in Orlando this Fall.

Sandra Wermer