Department of Electrical & Computer Engineering

Institute of Comunications and Computers
EUROMED Laboratory
9, IROON POLYTECHNIOU Street
Zographou 15773, Athens, Greece.

The Creation of a global Telemedical Information Society

Andy Marsh

Euromed Project Manager

Institute of Communications and Computer Systems National Technical University of Athens 9 Iroon Polytechniou Street, GR-15773 Zografou, Athens, Greece. Tel: +301 7722287 Fax: +301 7723557, email: marsa@phgasos.ntua.gr

Abstract

Healthcare is a major candidate for improvement in any vision of the kinds of "information highways" and "information societies" that are now being visualized. The medical information management market is one of the largest and fastest growing segments of the healthcare device industry. The expected revenue by the year 2000 is US$21 billion. Telemedicine currently accounts for only a small segment but is expanding rapidly. In the united States more than 60% of federal telemedicine projects were initiated in the last two years. The concept of telemedicine captures much of what is developing in terms of technology implementations, especially if it is combined with the growth of the Internet and World Wide Web (WWW). It is foreseen that the World Wide Web (WWW) will become the most important communication medium of any future information society. If the development of such a society is to be on a global scale it should not be allowed to develop in an ad hoc manner. For this reason, the Euromed Project has identified 20 building blocks resulting in 39 steps requiring multi-disciplinary collaborations. Since, the organization of information is therefore critical especially when concerning healthcare the Euromed Project has also introduced a new (global) standard called "Virtual Medical Worlds" which provides the potential to organize existing medical information and provide the foundations for its integration into future forms of medical information systems. Virtual Medical Worlds, based on 3D reconstructed medical models, utilizes the WWW as a navigational medium to remotely access multi-media medical information systems. The visualisation and manipulation of hyper-graphical 3D "body/organ" templates and patient-specific 3D/4D/and VR models is an attempt to define an information infrastructure in an emerging WWW-based telemedical information society.

1. Introduction

Telemedicine is the interactive audiovisual communication between healthcare providers and their patients or other healthcare providers regardless of geographic distance. The first use of telemedicine dates back to 1969 when X-ray images were transmitted across telephone lines. Nowadays, the uses vary widely, for example:

The potential benefits for telemedicine are apparently well defined and justifiable. However, with the introduction of a new technology there are potential problems. Two abstracts taken from Medical Equipment International (February - March 1997) clearly illustrates the dilemma of adopting a new healthcare technology. Despite its numerous problems telemedicine is slowly becoming adopted as an alternative method of healthcare. During December 1996 U.S. and Israel physicians participated in an international Tele-collaboration by conducting a telemedicine exchange. An all day event on the campus of Israel's largest medical centre (Rabin Medical Centre) located new Tel Aviv, with Texas children's Hospital and Baylor College of Medicine (Houston, TX, USA) and Duke University Medical Centre (Durham, NC, USA). The physicians participating in the exchange presented adult and pediatric cardiology cases for multisite medical collaboration including long distance transmission X-rays and ultrasound images. Also, in December 1996, during the Arab health exhibition held in Dubai, United Arab Emirates and two companies; VSI Enterprises (Noscross GA, USA) and interclinical Ltd (London UK) demonstrated live surgical procedures from Holland to physicians in the Middle East. Live interactive demonstrations of coronary artery ballon angiography and coronary artery implanations were transmitted from Katherina Hospital in Eindhoven. Six cases were presented via video conferencing enabling the surgical team in Holland to demonstrate the clinical applications and techniques. There are many other numerous projects such as emergency telemedicine, Medical care in remote areas, home telemedicine and trans-national projects such as Telehealth Africa.

The usage of telemedicine is becoming a global interest. For example in China, the Chinese Medical Information Network (CMINET) announced in March 1996 that CMINET will connect the Chinese Academy of Medical Sciences, Beijing Medical University, Peking Union Medical College and medical schools in Shangai, Zhejaing, Hunan, Xian, Huaxi, and Tibet. In June 1996 Guangzhoa Huamei Communications (GHC) involved a memo of understanding for very fast telecommunications and internet connections with the health division of the people's liberation Army (PLA) of China. GHC is affiliated with Chinese defense suppliers intends and plans to provide the PLA hospitals with ATM connections up to 155 Mbps. The PLA hospital intends to use the ATM connections to provide high-quality video telemedicine, combining education, radiology, sonography and nuclear medicine. So it can be seen that a global telemedical information society is a realistic possibility. The problem now is to develop a global standardized use of telemedicine to also encorporate the new advanced imaging techniques and sequentially the usage of supercomputing support.

New advanced imaging techniques developed in the USA are behind new revelations about the mechanics of human thought. So promising are these techniques that the United States Congress declared the nineties the decade of the Brain. Researchers are on the verge of accurately charting different brain functions (the US project NEUROSCOPE - Real Time Acquisition, Control and Processing environment for Studies of brain functions using MRI, lead by University of Illinois). Modern neurology and neurosurgery make extensive use of medical images for both diagnostic and therapeutic purposes. Imaging modalities which are quite complementary may be used to display various anatomical structures. For example, X-ray Computed Tomography (CT) is relevant for a skull and ventricular system, Magnetic Resonance Imaging (MRI) is suitable to visualise cerebral tissues, Angiography (film-bakol or digital) is used to display blood vessels and Nuclear medicine (PET, SPECT) for functional imaging. Nevertheless there are potentially many situations where a clinician would like to inspect an area of intent using more than a single image system. Viewing different images side by side provide little detailed information of the similarities and the differences between them. Accurate alignment of such images provide anatomical and functional information in the set of superimposed data (such as the US project ANALYZE [5], lead by the Mayo Clinic which comprises of over 60 programs allowing fully interactive display, manipulation and measurement of multidimensional image data). Not restricted to computational neurology this technique can be used in application fields such as radiation oncology, plastic and re-constructive surgery, medical diagnosis and mammography forming the basis for developing computationally intensive virtual surgery and surgery planning techniques, such as VRASP being developed at the Mayo clinic [8,9].

One important consideration for the 21st century is to create a telemedical information society whereby patient's medical data can be accessed globally and transparently of any storage and communication medium. The medical data related to a particular patient may be in various mediums from textural reports to a surgery planning video. This information may also be located in various databases and hospital information systems. When all the patient information can be stored in a computer accessible medium and all the computers are connected in a global network, the most important element of such a society becomes the navigation of information. Additionally, when related to the medical discipline this navigation should be an abstraction from computerized jargon and communication pragmatics. The essence of a 21st Century medical information system, in an emerging telemedical information society is that multi-media systems will be accessible remotely via a homogeneous communication protocol. It is therefore envisaged that such navigation will be aided by realistic hyper-graphical medical models, such models are shown in Figures 1 and 2.

Figure 1:

courtesy of Dr. Richard A. Robb, Biomedical Imaging Resource,
Mayo Foundation, Rochester MN.

Figure 2 :

courtesy of Dr. Richard A. Robb, Biomedical Imaging Resource,
Mayo Foundation, Rochester MN.

2. The elements of telemedical information society

Whatever form of an information society related to healthcare we can imagine, it will be based on three basic components, as shown in Figure 3, namely, raw medical data, reconstructed medical data and derived medical data. The raw medical data will consist of the medical data that can be ascertained directly from the patient for example X-RAY, CT, MRI etc.

There is now a defacto standard for this data called DICOM 3.0. The ACR-NEMA Digital Imaging and Communications in Medicine (DICOM) Standard has been developed to meet the needs of manufacturers and users of medical imaging equipment for interconnection of devices on standard networks. Its multiple parts provide a means of expansion and updating, and the design of the standard was aimed at allowing simplified development for all types of medical imaging. DICOM also provides a means by which users of imaging equipment may assess whether two devices claiming conformance will be able to exchange meaningful information. The future additions to DICOM include support for creation of files on removable media (such as optical disks or high-capacity magnetic tape), new data structures for x-ray angiography and extended hard copy print management. For further information the reader is referred to the Web page: http://www.xray.hmc.psu.edu/dicom/ dicom_intro/DICOMIntro.html. The derived medical data will be the subsequent diagnosis and in various formats including text. The reconstructed medical data will consist of, for example, computer generated models. All the medical data will have to be archived, as shown in Figure 4. The raw medical data will be stored in DICOM PACS and multi-media databases. The derived medical data will also be stored in multi-media RIS and HIS databases. The reconstructed medical data will reside on some computing device, we envisage this to be a WWW server. It is envisaged that that the communicating protocol will be the World Wide Web (WWW). The unique property of the WWW is that it provides a uniform meta operating system allowing computer platforms of various topologies to communicate. With the introduction of JAVA and JAVA Script it is now possible to run the same Application Programmes (Applets) on various computing platforms without any porting problems. Schematically, this can be represented as Figure 5.

Figure 5: Multi modal medical Information Systems

Based on this methodology, the accessibility of patient information is reduced to the notion of navigating or surfing the Web space attributed to the medical domain.

There are currently a number of software providers offering WWW Dicom Interfaces. One system is that implemented at the University of Joensuu within the framework of the Euromed Project. To use WWW-DICOM system, the user first runs a WWW browser such as Mosaic. Netscape, or Lynx and specifies a URL on one of DICOM PACS Unix workstations. This URL refers to an HTML file that contains a query form, for example as shown in Figure 8.

Figure 8 :

This query form contains a number of fields such as patient name and medical record number. The user may specify any or all fields as well as wildcards in fields such as the name field. Once the form is completed, the user presses a button to submit the request. The HTML form submits the query to a CGI (Common Gateway Interface) program that executes on the DICOM PACS server. This program accepts as input the form field values that the user specified. This program then communicates with the archive via DICOM requests to determine those patients that match the search criteria. The user may then choose a patient which in turn causes the studies for this patient to be displayed. Finally, the user may select a study which causes those images to be retrieved from the archive and displayed via the Web browser. The result of this system is an easy to use interface to a DICOM PACS with the option to query and move images from the PACS.

Alternative public DICOM implementations can be found at Pennsynvalnia State University, the University of Oldenburg and Mallinckrodt Institute of Radiology which is the premier of publicly available Dicom implementations. There are also a number of hospitals currently using WWW browsers to access a Dicom PACS system. One such implementation is at the Medical Imaging Unit Center of Medical Informatics, Geneva University hospital where the conventional PACS environment was replaced by a prototype of the WWW browser that directly triggers a specific program for displaying medical images from the conventional Netscape or Mosaic browser. Especially designed interface written in HTML can be used from any conventional WWW browser or any platform.

3. The building blocks of a telemedical information society

The development of a global telemedical information society should not be allowed to develop in an ad hoc manner. It is the objective of a European Commission funded project called Euromed [3, 5, 15, 16] to standardize the foundational elements of such a society. The project has identified 20 building blocks schematically represented in Figure 7.

Figure 7 :

Consequently from the building blocks 39 steps have been defined. Each step is well defined and modular in nature. It is only when all the steps and therefore building blocks are put together that a telemedical information society becomes a realistic possibility. The 20 building blocks listed below are briefly explained in the next subsections:

  1. Communication
  2. Accessibility
  3. Storage
  4. Privacy and Security
  5. Navigation
  6. Transmission
  7. Telecollaboration
  8. Visualisation and Manipulation
  9. Interaction and Prediction
  10. Computing
  11. Compression
  12. Archival
  13. Data Warehousing
  14. Education and Training
  15. Knowledge discovery
  16. Standardization
  17. Promotion
  18. Marketing
  19. Dissemination
  20. Usage

3.1 Communication

It is envisaged that in any image of the information society of the future, the computer will play an important part. It is therefore possible to conceive that everybody will have access to a computer, especially in a working environment. Therefore, it is not unrealistic to assume that all medical practitioners will have access to a computer. In addition, with the advancement of telecommunications infrastructures and the growth of the Internet it is therefore possible to imagine that all computers will be connected to Internet. That is all computers that are in a static workplace. However, with the development of modern technology and satellite technology the concept of a mobile connection to the Internet is also conceivable. Mobile connections could be used to support cases of emergency, natural disasters and temporary sites.

3.2 Accessibility

The advantages of using the WWW to the Accessibility protocol for a telemedical information society is that it is platform independent and therefore accessibility form any platform. Additionally, interpreted languages such as JAVA have been developed to run on the WWW. The World Wide Web can then be though of as the generic operating system.

The concept of telemedicine captures much of what is developing in terms of technology implementations, especially if it is combined with the growth of internet and World Wide Web (WWW). The Web is a simple, yet ingenious system that allows users to interact with documents stored on computers across the Internet as if they were parts of a single hypertext. The Internet is a "network of networks" that link computers around the World. These computers range from PCs to High Performance Computing platforms, but they all use a set of rules called TCP/IP to exchange information. A major reason for the accelerated growth of the Internet in the last few years is the WWW which began in 1992 at CERN, the European Laboratory for Particle Physics, as a means of distributing and annotating specific research. Technical standards are now defined by the World Wide Web consortium, specifying four sets of rules creating, publishing and finding documents:

3.3 Storage

The storage of data (medical information) is one of the most critical corner stones of the societies building blocks. For medical purposes this is becoming to be well-defined in that all medical imaging data should be stored in Dicom 3.0 format in a picture archiving system referred to as PACS. If the World Wide Web is to be adopted as the accessibility medium and the Internet as the communication medium then there should be a WWW interface to the storage components. An example is shown in Figure 8 where snapshots have been taken of the University of Joensuu WWW-dicom archiving system. It is therefore envisaged that Hospital information systems, PACS and any other storage mediums shall be Web accessible. The integration protocols between PACS/HIS and other information systems such as Radiology information systems should be based on well defined HL7, Dicom 3.0 and Web protocols such as HTTP. Consequently, further HIS implementations should be based on Web interfaced multi-media systems, whereby all the data should be viewable by a Web browser. Alternative databases such as video archiving systems should also be designed with Web interfaces.

3.4 Privacy and Security

Clearly, by making the medical information systems accessible by the Web raises problems of unlawful access. Therefore, another building block in the society should control the Privacy and Security of the stored data. A European funded project called Euromed-ETS is dealing with the issues of security of WWW-based telemedicine. The recommendations of this project is to adopt Trusted Third Party services for the management of unique keys, such as the client certificate server developed by Netscape and the usage of http for the communication protocol. Clearly, a lot of work needs to be undertaken regarding privacy and the issues of data protection, reliability and integrity. However, many of these issues can be borrowed from the commercial commerce society.

3.5 Navigation

Assuming that the medical data can be stored and accessed in a confidential manner, the next important building block becomes navigation. It is assumed that the medical data related to a patient will be dispersed around the Web space attributed to the patient. An example is shown in Figure 9.

Figure 9 :

Figure 10 :

To reduce the burden of surfing the Web for related patient information, EUROMED has defined the concept of Virtual Medical Worlds. Schematically, Virtual Medical Worlds is shown in Figure 10 whereby hyperlinks are used to link the patients' medical record. The electronic health card record of the patient is now dispersed over the network but accessible from an initial html page.

Virtual Medical Worlds is built upon the components; medical images (in Dicom 3.0 format) stored in PACS, reconstructed medical pictures (in VRML 2.0 format) stored on WWW servers and medical application packages (that can utilize the X windows protocol). It is also assumed that every medical institution in a telemedical information society will have a computer on the Internet (i.e. have an IP address) running as a WWW server. Additionally, major health centers will have PACS and medical application packages. The concept of healthcards is now being defined so it is not unrealistic to image that every member of the community will have a unique Virtual Medical World Personal home Page (PHP). An example is shown in Figure 11.

Figure 11 :

Figure 12 :

The PHP will be located on a unique WWW "Virtual Medical Worlds" server. From the PHP a practitioner can trace all the medical history related to the patient. The PHP is an html page that will be divided into four sections. An example is shown in Figure 12. The first section will point to general administrative information related to the patient for example account status e.t.c. The second section will contain general personal details such as a hyperlink to the patient's address, historical details Figure 17 such as a hyperlink to the diagnosis reports and a hyperlink to the Virtual Medical Worlds environment related to the patient. The Virtual Medical Worlds environment is hypergraphical and is broken down into whole body templates. Atlas templates, organ templates, patient specific models and medical images/reports. The latter may be any information related to the patient that can be accessed by any Web browser. The former templates and patient specific models are in VRML format. Typically, The VMW link from the PHP will point to a VRML world containing whole body templates.

Figure 13 :

An example is shown in Figure 13. At this level the whole body is represented in 3D VRML and annotations can be added to hyperlink to a more detailed model such as the torso (Figure 14).

Figure 14 :

The notation of hypergraphical templates is consistent from all parts of the body annotations can be in various formats (Figures 15, 16) . To access an image from a PACS server using the WWW interface. The pragmatics of the accessibility and location of the medical need to be known by the practitioner.

Figure 15 :

Figure 16 :

The navigation of a patient's healthcare record now consists of following hyperlinks around the Web space attributed to the telemedical information society. To avoid following unnecessary links shortcuts have been added in the PHP link to the respective levels of templates and patient specific information. A practitioner from the PHP could then trigger the request to accept a specific medical image instead of following the hypergraphical route. For example, from the PHP the practitioner could follow a link directly to the heart, as shown in Figure 17 and/or to the raw medical image, for example Figure 18.

Figure 17 :

Figure 18 :

From the organ templates the raw medical data, for example an ultrasound, as shown in Figure 18, could be obtained via the intermediate HTML catalogue as shown in Figure 19. Figure 19

Figure 19 :

In order to make the Virtual Medical Worlds as general as possible, the following types of general procedures, actions, entities and data were identified: Examination, Examination method, Specific finding, Parameter, Diagnosis and Therapy. Each of these will be added as a hyperlink in each of the Virtual Medical Worlds if applicable dynamically. We defined examination as a collection of actions defined by some examination methods selected by a physician. Examination method is a collection of actions performed to collect some medical parameters. An examination method can include examination submethods and/or specific findings. Specific finding is a collection of medical parameters that have something in common. Medical parameters represent in some form information about the medical condition of some part of the human body. A parameter can be a numerical value, text, image(s), audio signal, video, 3D or 4D model, or some other form of information. The meaning of diagnosis and therapy is obvious.

Each instance of examinations, examination methods, specific findings, parameters, diagnosis or therapy is defined as a Personal Medical Resource. A physician should have an opportunity to reach data related to any Personal Medical Resource of any patient existing in medical databases throughout the world. So far, we defined that each Personal Medical Resource [17] has a corresponding Personal Medical Resource Locator to make that task possible. This locator makes it possible to reach the PMR by referring it from any WEB page on any Internet connected computer. Obviously, the personal medical resource locator must be unique.

Personal Medical Resources are stored in Personal Medical Resource Databases, and the basic rule is that the PMR is stored in the database of the hospital where it was generated. That is how a distributed database of medical data is generated. With the PMRLs we have the potential to reach PMR(s) throughout the medical world on Internet. The basic question is where can we find PMRL(s) of a patient. We defined a Personal Home Page site as the place where a person has a collection of all of his PMRL(s). This collection is actually a Personal Medical Resource Locator Database that (conceptually) exists only at the Personal Home Page site. To find any PMR, we need to collect information about the corresponding PMRL from the Personal Home Page site. This is transparent to the user. To easily reach the PHP site, there should be some mapping between the personal identification of the patient and his personal home page. Finally, we decided that there actually should be no mapping - the PHP home page URL of a patient is equal to the personal identification since both are unique!

The crucial point is how to generate WEB pages with mutually related PMR(s), what descriptions should exist on those pages and how to make surfing through WEB pages a process where the physician can easily reach all logically related PMR(s). This is the point where the expert knowledge of medical doctors is necessary. This information should be incorporated in a personal medical resource description database. It should exist as identical copies at each node in the Medical Worlds 1.0. As the representation of expert knowledge through the PMRDD database is improved, the WEB pages and surfing options will change, but these modification need not to be done simultaneously throughout the network. Obviously, new versions of PMRDD databases can be easily distributed through Internet.

3.6 Transmission

Identifying the locality of medical information is the first step towards its usage. The transmission of this data over the network becomes critical when considering the bandwidths of today's networks. The issues of transmission of medical data and compression are closely linked. For the purposes of the paper the topic of compression relates to the storage of the medical data and minimizing the required space. Whereas transmission relates to minimizing the time the practitioner has to wait in order to view the medical data. Of course if the data to be transmitted is compressed, the amount of data to be transmitted is therefore reduced and hence less time is taken in the transmission. However, since the compression of medical data is a complex issue at present an alternative method maybe to transmit medical images in a progressive manner or even in a selective manner. For example, as shown in Figures 20, 21, 22 and 23, general information maybe transmitted at first and then progressively more detail.

Figure 20 :

Figure 21 :

Figure 22 :

Figure 23 :

If the practitioner wishes to wait the complete time all the data as presented in the original format will be acquired. However, at any instance the practitioner may have enough information to make diagnosis. Additionally, the practitioner could indicate at the general level a specific area of interest what could be transmitted first and then the remaining data progressively. The topic of progressive transmission is an ongoing area of research but it is hoped that it will be enclosed in the definition of Virtual Medical Worlds for both 2D and 3D medical images.

3.7 Telecollaboration

The real benefit of telemedicine is that more that one practitioner can telecollaborate (e.g. view the same medical data simultaneously). With Web based tools this becomes easy by the using plug-ins such as Cooltalk. Cooltalk has a shared whiteboard, talk facilities and can even be combined with video conferencing facilities. The shared whiteboard can be used to view and annotate the same image simultaneously on two Web connected workstations therefore facilitating the tele-aspect of telemedicine. An example is shown in Figure 24.

Figure 24 :

Clearly, furthermore sophisticated plug-ins and video conferencing tools can be used but Cooltalk provides the first easy solution.

3.8 Visualisation & Manipulation

For Visualisation purposes a VRML browser can be used to view Virtual Medical Worlds and reconstructed images from-raw medical data. For example CT slices to VRML, could be done on the fly by a JAVA applet. JAVA could also be used to manipulate the VRML models, performing tasks such as intersection and measurement analysis. Due to its characteristics JAVA is platform independent therefore the developed applets could run on any Web browser. An example of the same applet, for region growing , is shown in Figures 25 and 26.

Figure 25 :

Figure 26 :

A JAVA applet could be used to reconstruct the VRML representation from MRI images as shown in Figure 27. These models could be fused to produce a new model shown in Figure 28 whereby the movement of endocardium is denoted by color.

Figure 27 :

Figure 28 :

3.9 Interaction, prediction and computing

The ultimate objective for imaging is the interaction and prediction with the medical models. The interaction is in the form of surgery planning and surgery assistance, making use of Virtual Reality techniques, such as VRASP developed at Mayo clinic. Virtual endoscopic techniques can then be developed to examine the body, even remotely, for example as shown in Figures 29, 30, 31 and 32 on endoscopic investigation can be made of the lungs.

Figure 29 :

Figure 30 :

Figure 31 :

Figure 32 :

The prediction aspect includes medical physics for determining blood flow analysis and tumor growth. Clearly, both interaction and prediction require a vast amount of computing. This should be provided either by a connection from a hospital to a computing site or by using the hospital computers in a collaborative network. Within EUROMED, the WWW can be regarded as the generic operating system providing a transparent access to a variety of computing platforms (referred to as the HCFI), ranging from PC's to High Performance Computing (HPC) platforms.

Figure 33 :

The subject of the HPC centers involved in the EUROMED project may be considered as a first attempt to build up a "meta-center" similar to what is currently being developed in the US through several initiatives such as the CASA Gigabit network (including CALTECH, Jet Propulsion Lab, Los Alamos NEC lab and San Diego Supercomputing Center). The EUROMED "meta-center" as shown in Figure 33 consisting of University of Calabria, University of Amsterdam, University of Joensuu and the National Technical University of Athens connected currently by Internet can be considered as a virtual computing centre supplying heterogeneous computing platforms. The physical location of the HPC platforms need not be known to sites accessing the "meta-center", only the type of HPC platform required and the collective services available need to be known in advance currently within Euromed the meta-centre is used to create complex VRML medical models. However, the purpose of the meta center is to support activities such as the medical imaging requirements (e.g. Volume rendering, 3D reconstruction) of the Virtual Assisted surgery.

VRASP [8] is intended to provide support for preoperative planning and rehearsal, but also on-line access to the preoperative data during the surgical procedure in the OR, and real time qualitative and quantitative comparison between OR data and the preoperative plan. The project is designed in three components and phases:

Figures 34 and 35 illustrate these respective operational phases and component development for each phase of VRASP.

Figure 34 :

Figure courtesy of Dr. Richard A. Robb, Biomedical Imaging Resource, Mayo Foundation, Rochester MN.

Figure 35 :

Figure courtesy of Dr. Richard A. Robb, Biomedical Imaging Resource, Mayo Foundation, Rochester MN.

In the initial phase, the surgeon plans the surgical procedure, using prescanned data from Computed Tomography (CT) and Magnetic Resonance Imaging (MRI), on a powerful computer workstation using comprehensive, interactive 3-D image rendering and image analysis software called ANALYZE [5]. This component is currently operative, and has been used to plan hundreds of procedures on craniofacial surgery patients, many orthopedic surgery patients, and several neurosurgery patients. In the second phase, the surgeon reviews and rehearses the procedure developed during the planning stage using VR equipment and high-performance computer, namely a head-mounted display, an interactive data glove, and customized surgeon interface. A demonstration version of the rehearsal system is currently being evaluated by craniofacial, orthopedic, neurological and urological surgeons at the Mayo Clinic.

Customised software interfaces are being designed for each surgical procedure by the respective surgeons working with experienced system analysts and programmers. In the third and final phase, the surgical team performs the procedure in the OR using the VRASP system. The surgeon wears the VR equipment and uses the customised interface to see preplanned and ad hoc volumetric images in virtual space locked into world-surgeon-patient frames of reference. The virtual image can be manipulated hands-free or, if desired, using interactive input devices. This final phase of the program will be implemented in two Mayo Clinic hospitals following laboratory/clinical evaluation of the second phase.

While the software components of VRASP will include extensions to existing software tools (ANALYZE and AVW), the VR interface is being designed as a set of asynchronous interacting agents so as to distribute the computation tasks over multiple processors and to decouple the computational and visual update rates. Work done by Kaltenborn, Shaw and others have shown that the use of distributed interacting agents is an effective way to reduce the overall latency of complex simulations [18,19]. Clearly, the usage of the HPC becomes critical if photorealistic models are to be produced. Within EUROMED the usage of the established telemedical infrastructure, based on the WWW, is being investigated to provide remote computation power to support VRASP in a number of European hospitals.

Virtual reality technology is posed to make significant improvements and contributions to healthcare, particularly in planning, rehearsal, and execution of surgical procedures. Practising surgeons are committed to assisting with the development, evaluation, and deployment of the VR systems. The VRASP package, developed at Mayo Clinic, represents the state-of-the-art in applying VR to surgery planning and assistance but as mentioned in the previous sections it requires state of the technology. It is conceivable that in a telemedical society of the future that specialised/general hospitals will have access to the VR equipment, but not the required HPC power. It is the objective of the EUROMED project to define the super-structure of such a society. A direct high speed data link will connect the hospitals and centralised HPC centres. Figure 352: Diagram of HPCC support for VRASP The communication protocol will be WWW. The work of the VRASP will be extended to not only use these computing centres but also make use of WWW interface to promote surgery planning and education on a remote site, or any site with a Web browser. The VRASP code will also be parallelised to use the full potential of the HPC platform. In-line with EUROMED objectives, the use of VR will be integrated with Virtual medical worlds.

Figure 36 : Diagram of VRASP hardware as developed in 3 phases

As shown in Figure 35 Phase II the VRASP package requires the usage of vast computational power. The collaboration work of Mayo Clinic and the EUROMED project will firstly allow the HPCC power through a CGI-bin based WWW interface to be located at a remote site, diagramatically represented in Figure 36, Phase II. Secondly, the surgery planning aspect of the operation can also be undertaken at a site outside the hospital. This is particular crucial when applying this concept to support the underdeveloped regions of the community such as the Greek islands where the locality of specialised doctors and clinics and geographically dispersed. The on-going work is also looking at the concepts of the remote accessibility to the VR models for educational purposes to reduce professional isolation.

3.10 Archival and Compression

Returning to the corner stone of storage, a general hospital over a one year period will require on average 6,000,000 MB of disc storage. An example is shown in Figure 38.

Figure 38 :

The idea of an archive is to store images for as long as they might be needed. Generally, this is much longer than can be cost-effectively supported by the technologies used in the on-line storage system. If a PACS should provide digital archival it must be decided how many images should be made accessible. Since any archive system will in principle overflow in time most systems provide for an on-line and an off-line archive.

Figure 37 :

In terms of an optical disk jukebox, on-line pallets are those located on a shelf in a storeroom, ready to be manually inserted in the juke box on request. The on-line storage capacity is best thought of in terms of time-depth. Studies have shown that less than 10% of images are accessed ever again after the first year. An example study has shown the following statistics as presented in Figure 37. In terms of disc space this represents approximately 10% of 6,000,000 MB. An example study has shown the following storage requirements in a general hospital over a one year period.

Taking into consideration time-depth and storage capacity the requirements of an archival system might read:

With current technology and the evolution of data warehousing there appears to be a thin line between where "on-line" archival in a PACS system ends and on a data warehousing environment begins. It is envisaged that archiving sites will be set up so that a hospital can download the requirements of vast amount of storage to an external site. In conjunction with Archival is the compression of the data. Compression is a current topic with a large amount of interest due to its legal implications. However, techniques such as a wavelet compression could provide the answer.

3.11 Data Warehousing

In a telemedical information society the collection of vast amounts of medical data will not only support the requirements of archiving but also provide a platform for the application of data mining and knowledge discovery to determine possible medical trends and the real data to support educational training. New technology in the form of data warehousing provides the potential to store the required amounts of medical data. It is envisaged however that a hierarchical approach will be adopted for the storage of data. This also has implications for the networking infrastructure.

Figure 39 :

The hierarchical infrastructure as presented in Figure 39 will consist of hospital PACS, regional PACS, National PACS and a European PACS. Each PACS system outside of the hospital PACS can be regarded as a data warehouse and consists only of the archiving component of a complete hospital PACS environment. It is envisaged that the medical imaging data related to a particular patient will reside on the hospital PACS up to an expiring date, before the end of this expiring date the medical imaging data will be copied to the Regional PACS. The medical imaging data will then reside on the Regional PACS up to an expiring date. Again before the end of the expiring date it will be copied to the National PACS, and so on. After a period of maybe up to a year the data will finally reside on the National PACS with a copy on the European PACS. At any moment in time after the initial period the patient medical imaging data will reside on two PACS systems therefore facilitating the concept of navigating the data. It is also envisaged that the unique names (Web addresses) given to each PACS system on the Euromed network will reflect the hierarchical native of the data warehouse for example: Hospital . Region . National . European It is also envisage that the expiring times will be uniform for every level of the data warehouses and therefore from the data the medical data is created it is possible to calculate the two PACS systems where the data shall reside. For example:

If the medical data was required at Week 8 it can be found on both the hospital PACS where it was created and at the regional PACS where the hospital is located.

3.12 Knowledge discovery and educational training

When vast amounts of medical data become available due to the data warehousing then, knowledge discovery techniques can be applied to identify pathologies and disease trends. The data can also be used for educational and training purposes because maybe one of the unique cases can be identified and used in expert system like applications to advise practitioners.

3.13 Standardization

In order that the core building blocks cooperate in a collaborative manner their interfaces and individual components must be standardized . It will be the role of European Commission funded programmes such as ISIS (Information Society Industrial Standardization) and the standardized bodies such as CEN TC 251 to coordinate such interactions.

3.14 Promotion, marketing, dissemination and usage

The remaining four building blocks relate to how this new society will be used. The society must be promoted through various marketing channels so that medical vendors take an integral part in the development of the society. Then via varying initiatives the implications on the health care community should be presented so the results can be disseminated to the end users.

4. Summary

A global telemedical information society has be envisaged that can be built upon 20 building blocks. Each building block is well defined and its interaction well understood. The society should be built upon the existing well defined medical standards such as Dicom 3.0 and HL7 and the computing networking standards of the World Wide Web such as VRML, html, http etc. By building the society with this modular approach allows for the independent development of independent elements. The consequence of defining the 20 building blocks is the following 39 steps:

  1. Static Communication using Internet.
  2. Dynamic Internet Communications for emergencies.
  3. WWW as Access protocol.
  4. Store medical images in Dicom PACS.
  5. Store non-medical images on WWW Server.
  6. Use Trusted - Third party services for security.
  7. Support Patient - doctor privacy.
  8. Use hypertext to navigate documents.
  9. Use hypergraphics to navigate models.
  10. Transmit images progressively.
  11. Communicate by Netscape mail.
  12. Discuss with Netscape news.
  13. Collaborate with Cooltalk.
  14. Use VRML browser for visualisation.
  15. Use JAVA for model creation.
  16. Use Java for model manipulation.
  17. Support mathematical modeling.
  18. Batch-processing surgery planning.
  19. High speed links surgery assistance.
  20. Cgi-bin common interface for meta-computing.
  21. Multi-platform meta-computing.
  22. Workstation clusters for computing resources.
  23. Wavelets for archival compression.
  24. Intelligent medical image compression.
  25. Hierarchical archival procedure.
  26. Collection of medical information in Data Warehouses.
  27. On-line Web-based anatomy atlas.
  28. On-line Web-based medical databases.
  29. Knowledge discovery identify pathologies.
  30. Data mining identity statistics.
  31. Standardized data interfaces.
  32. Standardized user interfaces
  33. Hard copy promotion of Web-based telemedicine.
  34. On-line promotion of telemedicine.
  35. Integrated Web-based telemedical products.
  36. On-line magazine.
  37. Use of patient data locally.
  38. Use of patient data remotely.
  39. Use of patient data instantly.

For the envisaged healthcare society of the future to be a realistic possibility, a global co-operation needs to be established combining on a World Wide scale academia, industry and government. The 20 building blocks have led us to the 39 steps to be able to create the healthcare community of the 21st century.

5. Acknowledgments

The authors would like to thank the European Commission DGIII/B under the ISIS 95 program for funding support for the EUROMED Project. The authors would also like to thank all the EUROMED partners for their constructive advice and Miss I. Christogianni, for the production of this paper.

6. References

[1]. Williams and M. Moore, "Telemedicine: Its place on the information Highway", to appear in Impact: How IC2 Research Affects Public Policy and Business Markets. Edited by W.W. Cooper, F. Phillips, D.V. Gibson, S. Throe, and A.B. Whinston. Volume in honour of George Kozmetsky.

[2]. Logan , "Whole Frog Technical Report", Imaging and Distributed Computing Group, Information and Computing Sciences Devision, Lawnence Berkley Laboratory, Berkley, CA 94720, Publication Number: LBL-35331, http://george.lbl.gov/ITG/Whole.Frog/frog/Frog.TechRep.html.

[3]. A. Marsh, "EUROMED - Coupling High Performance Computing and 3D Visualisation Techniques for Telemedical Diagnosis, presented at the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA '96), Sunnyvale Hilton, California, USA, August 9-11, 1996.

[4]. A. Marsh "EUROMED - Visualisation suites for medical imaging" Presented at the 3rd International Workshop on Community Networking, Antwerpen, Belgium, May 23-24, 1996.

[5]. R. A. Robb and D.P. Hanson, "The ANALYSE software system for visualization and analysis in surgery simulation", in Computer Integrated Surgery, Lavalle, et al., Eds., MIT Press, 1996.

[6]. Hudson, "Communication satellites: Their Development and Impact", New York: The Free Press, 1990.

[7]. Kuhn, "Frontiers of Medical Information Sciences", New York, 1988.

[8]. R.A.Robb and B.M. Cameron, "Virtual Reality Assisted Surgery Program", Medicine Meets Virtual Reality III: interactive Technology and the New Paradigm for Healthcare, San Diego, California, Jan. 1995.

[9]. R.A Robb, J.J. Camp and D.P. Hanson, "Computer-aided surgery and treatment planning at the Mayo clinic" Special issue on CAS of Computer and Computer Graphics and Application.

[10]. Moore, "Elements of Success in Telemedicine Projects". Report of a Research grant from AT&T; Graduate School of Library an Information Science, the Austin, Texas, 1993.

[11]. R. Holloway, H. Fuchs, W. Robinette, "Virtual worlds research at the University of North Carolina at Chapel Hiils as of February 1992", In: Computer Graphics - International Visual Computing, T. L. Kunii, Ed, Springer, 1992, pp 109-128.

[12]. Hohne, K.H., Bomans, M., Pommert, A. Riemer, M., Schubert, R. Tiede, U. Lierse, W. "A 3D anatomical atlas based on a volume model". IEEE Comput. Graphics Appl. 12, 4 (1992), 72-78.

[13]. Spritze, V., Ackerman, M. J. Scherzinger, A. L., Whitlock, D. "The Visisble Human Male: A Technical Report. J. Am. Med. Inf. Ass. 3, 2 (1996), 118-130.

[14]. Pommert, A. Schubert, R. Riemer, M. Schiemann, T. Tiede, U. Hohne, K. H. "Symbolic modeling of human anatomy for visualisation and simulation, In Robb, R.A. (Ed.): Visualisation in Biomedical Computing 1994, Proc. SPIE 2359. Rochester, MN, 1994, 412-423.

[15]. A. Marsh, "EUROMED - Combining WWW and HPCN to support advanced medical imaging", to appear at the International Conference and Exhibition on High Performance Computing and Networking (HPCN 1997), Vienna, Austria, 28th-30th April, 1997.

[16]. F. Simistira, A. Marsh, R. A. Robb "Coupling HPCN and Virtual Reality in a telemedical information society", to appear at the International Conference and Exhibition on High Performance Computing and Networking (HPCN 1997), Vienna, Austria, 28th-30th April, 1997.

[17]. I.Milojevic, N. Djukanovic "Distributed Personal Medical Data Storage and Access", Information Technologies, Zabljak, Yugoslavia, 15th March 1997.

[18]. C. Shaw, M. Green, J. Liang and Y. Sun, "Decoupled simulation in virtual reality with MR Toolkit", ACM Transactions on Information Systems, 11(3), 287-317, 1993. [19]. K. F. Kaltenborn and O. Rienhoff, "Virtual reality in medicine", Meth. Inform. Med., 32(5), 407-417, 1993. 1 Figure 34 courtesy of Dr. Richard A. Robb, Biomedical Imaging Resource, Mayo Foundation, Rochester MN. 2 2