Dr. Robb defined interactive imaging as a nearly instantaneous response of the image modality system to any user stimulus, requiring a rate of at least 10 to 20 frames per second. Ideally, the visualisation is multi-modal, combining images in a synergistic way through spatio-temporal fusion. This is achieved by registering the images with sophisticated mathematical algorithms. The actual visualisation consists of rendering and displaying multi-dimensional objects, preferably in 3D, but also involves the whole image pre-processing procedure. In turn, image analysis comes down to both measurement and characterisation of the visual material.
Endoscopic examinations within the body change our place in the universe, as Dr. Robb explained. When shifting from the macro to the micro levels of scale, we can visualise organs, tissues, cells, and organelles, to finally enter the world of the neuron and study it. This requires a need for volume imaging in 3D at the least. The clinician can recur to a broad range of imaging modalities, such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI), micro CT, spiral CT for the chest, ultrasound, Positron Emitter Tomography (PET), Single Photon Emission Computed Tomography (SPECT), confocal imaging, and microscopy. Since 1975, high temporal resolution imaging is being used as the first multi-source, multi-detector, real time 3D imaging system to produce simultaneous image volume slices.
Computer technology thus has become the enabler of modern biomedical imaging. The GHz PC, mass memory, and the Internet - "for better or worse, for it allows you to send e-mail and transmit large images" - have provided us with big opportunities. Richard Robb also briefly referred to the proprietary ANALYZE tool, a comprehensive visualisation and analysis software to surmount the insurmountable, which was developed at Mayo Clinic. Several procedures in visualisation and analysis are commonly used and can be made interactive. Dr. Robb mentioned fast 3D registration in Nuclear Medical Imaging (NMI); images that are segmented and classified in just a few seconds on a simple PC; surface and volume rendering where the volume images are being dissected interactively; and anatomy modelling and tissue mapping.
The big deal is to reach beyond anatomy and to add function to structure. Dr. Robb showed how this is done for instance in elastic deformation where the lung epithelial cell is "stretched" to watch the cell breath. Deformation is performed through texture mapping. Advanced in-vivo 5D visualisation even allows to display structure and function in real time. Virtual Reality forms a further possibility to find ourselves in a 3D world without having to imagine it, as Dr. Robb described. We suspend our disbelief to become fully immersed and generate a taxonomy with generations of virtual anatomy and functions, that ranges from geometry to biochemistry. Virtual Reality is now frequently used to rehearse complex surgery on a patient and train anaesthesiologic procedures by means of simulation.
Another upcoming technology is virtual endoscopy, a new method of diagnosis that processes 3D image data sets to deliver simulated visualisations of patient specific organs similar or equivalent to those produced by standard endoscopic procedures. Concrete examples are virtual gastroscopy and virtual colonoscopy. Which brought Dr. Robb to the point of giving a number of current applications in visualisation to the audience, such as craniofacial surgery planning; neuro-surgery planning in which 3D MRI and SPECT images are subtracted and fused to localise brain regions causing epileptic seizures in paediatric patients; intra-operative image-guided neuro-surgery consisting in on-line multi-planar display of MR images with segmented 3D tumours and blood vessels; and image-guided diagnosis of coronary artery disease using intra-vascular ultrasound.
Stenting or on-line image-guided treatment of coronary artery disease is not yet done routinely but limited to experiments only. Other procedures based on 3D visualisation are image-guided cardiac ablation or E.P.-mapping; and interactive visualisation of the prostate in which a real time trans-urethral ultrasound image is registered to a patient-specific 3D model. Until recently, implanting seeds with CT in prostate brachytherapy still happened blind but now it can be visualised. Very spectacular was the pre-operative planning for the separation of conjoined twins, since a pre-calculation was necessary for the amount of skin that had to grow again.
In order to make interactive medical image visualisation acceptable for daily clinical routine, some current needs and issues still have to be addressed by research. Dr. Robb only named a few including an increased 4D resolution in space and time; automatic and accurate anatomic segmentation; fast and robust multi-dimensional registration; faithful tissue and function classification; and realistic real time volume rendering. The guidelines for clinical assessment have to be based on physician requirements and expectations. Solutions need to be clinically relevant, reliable, accurate and precise, according to Dr. Robb, who stressed that the tools should improve outcome and patient survival, meaning a reduction of both morbidity and cost.
First choice are those imaging systems with a modular concept and an easy-to-use interface. Designers have a task to focus on the user right from the start via continuous user testing. This involves they have to do their homework properly by performing an application analysis and a usability evaluation. Dr. Robb also strongly pleaded for an integrated design and iterative prototype design cycles. Standards are everywhere, so in visualisation as well. A sensitive issue though since we face an inflation of standards. Maybe it would not be all too crazy to create a standard for standards, as Richard Robb put it, next to a standard for common sense, of course. We definitely need to agree on standards for 3D image acquisition, image storage and transfer, segmentation and registration, analysis and interpretation, visualisation parameters, and validation of algorithms.
To make diagnosis and treatment based on advanced 3D medical visualisation come true as we have seen it happen in Star Trek, Dr. Robb thought it appropriate to list a few objective as well as subjective considerations for evaluation of new technologies in clinical applications. The objective factors to be taken into account are sensitivity of resolution and texture; object specificity; inclusion of artifacts; local relationships between objects; reproducibility; time; cost; and outcome. Physician and patient acceptance; trust in computer versus human performance; insurance by third party payers; referential research studies; and the weighting of the relative value added by unique features are the subjective categories in the validation process.
Richard Robb ended his talk by indicating that the need for support and training should not be omitted. Technical support for visualisation tools is perfectly deliverable via the Web through the offering of frequent patches and upgrades. If you want to have a dedicated, technically knowledgeable clinical staff, give them the opportunity to keep abreast with the latest developments via tutorials, regular context-sensitive help, and both on- and off-site training sessions. Of course, this will bring on substantial costs but Dr. Robb used another one-liner to ward off this critical remark: "If you think education is expensive, consider the cost of ignorance."
You can find more details and information on the work of Dr. Robb at Mayo Clinic in the following VMW articles: