Dr. Richard Robb is one of the leading experts in the Biomedical Imaging Resource at Mayo Clinic in Rochester, involved with the progressive development of virtual endoscopy (VE) as a diagnostic and clinical tool. He actually prefers the term "endoscopic or computed virtuality" since it specifies more accurately the essence of simulated visualisation of patient specific organs based on computer processing of 3-D image datasets. The current availability of the Visible Human Datasets (VHD) from the National Library of Medicine offers a tremendous opportunity to turn VE into a reliable alternative for real invasive endoscopy, allowing viewing angles that up till now were inaccessible. True, a number of technical problems still need to be solved concerning image resolution and rendering, segmentation, registration and appropriate preparation of the data in order to refine and validate the simulation for routine clinical use. Very soon however, physicians will boldly enter regions of the human body which no eye has ever beheld.
Virtual endoscopy is based on the idea to represent real world objects as spatial information. Basically, there are two methods of proceeding. The first one starts out with three-dimensional images acquired from a scanner which are prepared for modelling, meaning that single anatomic objects should be segmented from the 3-D images and their surfaces extracted. This surface is converted into a meshwork of polygons, a process referred to as "tiling". The addition of colour, lighting or textural patterns enhances the details in the general structure which is rendered for visualisation by means of computer algorithms. The resulting endoscopic display can either be simulated on-line, in real time through an interactive simulator with use of a head-mounted display, head tracking and 3-D input devices, or through a pre-determined "flight path" in order to compute sequential frames of views which are turned into an animated video sequence.
In the second approach, the voxels of the appropriately segmented 3-D images are mathematically being ray-casted through, a process called perspective volume rendering which generates different surface views. In contrast to parallel volume rendering, the rays cast at divergent angles, originate at a finite eye position, allowing voxels in close proximity to appear larger than those situated at a distance. Trilinear interpolation smoothes the rather "blocky" visualisation by sub-sampling the data to match the sampling rate of the rays cast through the volume. At present, large images are difficult to fully visualise at real-time rates so pre-determined flight paths and animation are performed to generate cine sequences. Still, it is possible to preserve the intrinsic 3-D richness of the volume data, including depth layers displaying for instance blood vessels in the luminal wall. These details cannot be captured in surface models.
The Visible Human Dataset (VHD) is playing a capital role in the validation of virtual endoscopy (VE). Next to mathematical simulations and phantom studies, both in vitro and in vivo research is required. The main difficulty consists in the provision of a "ground truth" measure. The VHD cryosection data precisely offers the standard against which the image processing effects can be calibrated and judged whereas the very same VHD data forms the basis to produce realistic simulations for testing procedures. The VHD models serve as a relevant framework for medical education, anaesthesiology training, surgery rehearsal and endoscopic simulations.
Regular clinical use of VE however implies a more accurate and subtle level of 3-D resolution and surface rendering than currently available. And there are more challenges to meet, such as the essential step of segmentation "on the fly" to immediately distinguish one organ from another and not merely after visualisation. Fusion of static images from different imaging sources constitutes no problem anymore but frameless registration between a set of dynamic images is still limited to a few selected applications, like stereotactic neurosurgery. Finally, an extension of preparatory methods for certain organs or tissues is needed to obtain adequate and effective visualisation.
Dr. Richard Robb describes the progressive transformation of virtual endoscopy into a visualising tool of realistic perfection as a five generations process. Initially, 3-D geometric anatomic shapes were computed resulting in simple interactive fly-throughs. Next, more realistic graphics were generated including physical properties of tissue, such as stretching and deformation, and allowing interactive positioning of anatomic structures. On the brim of the third generation, physiologic characteristics like breathing, bleeding, leaking liquids and motility are being integrated for the first time in the VHD or the patient specific data. The following step will be to add microscopic level detail to the anatomy to visualise miniature size structures such as neurovascular bundles, glandular structures or even individual cells. The last stage will insert complex biochemical parameters with multi-organ system integration to represent systemic functions, such as neuro-endocrine and immunologic functions or pathologic states like shock.
The Biomedical Imaging Resource team at Mayo Clinic finally aims at developing a virtual endoscopic procedure which might become indistinguishable from the real patient and permit a continuous, seamless fly-through from gross anatomic, through macroscopic and eventually to microscopic realms. For specific information on visualisation techniques, we refer to the article From multiplanar reformation to virtual endoscopy: a survey of medical visualisation techniques in this very same issue.