Virtual Reality, Medical Informatics, and EHR Systems

Shepherdstown 08 January 2008This article provides a high level overview for management on virtual reality (VR) solutions and their application in the field of healthcare at this point in time. Virtual Reality (VR) has real applications in healthcare, especially in education, surgery, therapy, and 3D visualization of medical information. The article also specifically touches on the potential relationship of VR to electronic health records (EHR) systems in the future. Finally, recommendations are offered on next steps healthcare organizations should consider taking with regards to this technology.


Virtual Reality (VR)

The field known as virtual reality (VR) is a multi-disciplinary field of computing technology that emerged from research on three-dimensional interactive graphics and pilot/vehicle simulations in the 1960s and 1970s. See

According to Stephen Ellis at NASA's Ames Research Center, "The technology of the 1980s was not mature enough." Virtual Reality (VR) helmets and their optics were too heavy, computers were too slow, and touch-feedback systems often didn't work. However, twenty years later, things have vastly improved. Computers are thousands of times faster; VR peripherals are more portable and lightweight; and systems deliver a greater sense of feedback and immersion to their users. Researchers better understand crucial human factors and the development and use of virtual reality systems are on the march again. See


Virtual Reality (VR) - A three dimensional (3-D) technology which allows a user to navigate around and interact with a computer-simulated environment.

Virtual Reality - The use of computer technology to create the effect of an interactive three-dimensional world in which the objects have a sense of spatial presence. (Steve Bryson - NASA Ames)

Selected Virtual Reality (VR) Concepts and Technologies

Holomers - There are modeling and robotic technologies currently available that can feed into a "holomer", serving as a virtual representation of an individual. The "holomer" is a desired format for EHR and clinical imaging systems of the future.

Cave Automatic Virtual Environment (CAVE) plays an important role in the 3-D visualization of medical data and in the creation of simulation techniques for certain clinical applications. The CAVE is a projection-based VR system. Projecting stereoscopic computer graphics into a cube composed of display-screens that completely surround the viewer creates the illusion of immersion. It is coupled with a head and hand tracking system to produce the correct stereo perspective and to isolate the position and orientation of a three-dimensional input device. A sound system is often included to provide audio feedback to the system user.

Examples of Virtual Reality (VR) Technologies in Healthcare

In the examples below, you will see a variety of ways that virtual reality (VR) technologies are being applied in healthcare settings.

Surgery & VR Systems

Kylie is a virtual reality surgical simulator invented by a team at Monash University. "The system allows surgeons to practice by standing in front of a simulated patient's abdomen and working at a computer screen in real time", according to Professor Healy. "They will have instruments in front of them and actually be able to feel the weight and texture of tissue and organs as they work." Just like a real operation, the surgeons insert a tiny video camera into the body and then study graphics on a computer screen representing the anatomy.

VRASP is being developed at Mayo Clinic to specifically assist surgeons during craniofacial, orthopedic, brain and prostate surgery. VRASP will enable surgeons to interactively visualize 3-D renderings of CT and MRI data with hands-free manipulation of the virtual display. The clinical goal is dynamic fusing of 3-D body scan data with the actual patient in the operating room. The customized interface will permit ready, on-line access to the preoperative plan and to update measurement and analysis based on the real-time operating room data.

The Dutch company DelltaTech has developed the SIMENDO Virtual Reality (VR) Trainer to train surgeons in the basic skills of Minimally Invasive Surgery (MIS) in a challenging way at a reasonable cost. See

Surgeons at Mayo Clinic in Rochester have also been developing, using and assessing the ANALYZE software system for interactive visualization, manipulation and measurement of 3-D medical images. Over 3000 medical scientists, physicians and surgeons in more than 200 institutions throughout the world are working with ANALYZE for Computer Aided Surgery (CAS) and Radiation Treatment Planning (RPT) in various applications such as craniofacial, orthopedic and neurosurgery planning as well as radiation therapy planning.

Montefiore Medical Center has opened one of the most advanced minimally invasive surgical training centers in the United States. The Montefiore Institute for Minimally Invasive Surgery (MIMIS) includes a Virtual Reality & Skills Laboratory, surgical workstations and a state-of-the-art conference room with global teleconferencing and telemedicine capabilities. One key advantage of the virtual reality workstations is that they produce an ongoing video of the student's abilities and progression in left and right hand orientation.

The Integrated Environment for the Rehearsal and Planning of Surgical Interventions (IERAPSI) project, was funded by the European Commission's IST Programme. The IERAPSI consortium, consisting of eight partners, focused particularly on the planning, simulation, and training of surgical procedures performed on the petrous bone, a common site with a complex anatomy. The system was to be designed to address frequently applied interventions, such as mastoidectomy, cochlear implantation and acoustic neuroma. In a later stage, IERAPSI would also be used for other surgery specialties. See and

VR Systems & Health Education and Training

While its use is still not widespread, virtual reality (VR) is finding its way into the training of healthcare professionals. Use ranges from anatomy instruction to surgery simulation. Virtual reality is now frequently used to rehearse complex surgery on a patient and train anesthesiology procedures by means of simulation. Annual conferences are held to examine the latest research in utilizing virtual reality in the medical fields, e.g. Medicine Meets Virtual Reality (MMVR) Conference.

The French company Simulife Systems is now marketing 3D Dental, the first virtual reality dental training simulator for students. The virtual reality dental training simulator includes a screen on which the patient with dental problems is displayed. Students select their instrument from the tool kit proposed on the screen and take hold of the haptic system's stylus. Their action is immediately translated on the screen through the real-time representation of the instrument. At the same time, the haptic system transmits to the operator's hand the same sensory information that he would experience with the real instrument. See

The TOUCH Project is a collaborative effort of the University of Hawaii and the University of New Mexico Schools of Medicine. Its goal is to explore the impact of advanced computational and telecommunications technology on human comprehension. The approach has been to develop three-dimensional immersive virtual environments that place students in medical educational contexts where they may pursue exploratory and experiential learning. These learning contexts include scenarios where patients can "practice" treating acutely ill virtual patients without endangering human life. Other contexts allow students to learn body function and physiology by exploring and interacting with environments that are physically inaccessible to them. The ability to use virtual environments in a collaborative mode that allows students, colleagues and teachers to interact with learning objects and each other is seen as central to the projects mission. See and

According to a November 2006 article in the San Antonio Express, Texas A&M and commercial game developer Break Away Ltd. are working on a state-of-the-art VR tool for teaching medical students, doctors and nurses how to recognize particular medical conditions and respond appropriately. The 'Pulse' project utilizes VR technology and a virtual patient named Sam who can be programmed to simulate just about any condition in the medical books - from acute appendicitis to major trauma from an improvised explosive device. A female patient will follow simulating the myriad uniquely feminine conditions. In January 2007, the Navy, Johns Hopkins University and Yale University began testing the prototype computer programs. See

There is increasing pressure on many fronts (e.g. ethical, financial, etc.) to find suitable alternatives to using animals to teach physiology and surgical skills. Advances in simulator technology have provided some exciting potential alternatives. For example:

  • VIRGIL, a simulator for chest tube insertion training, and
  • SIMPL, a simulator for teaching diagnostic peritoneal lavage.
The results of studies related to the effectiveness of these solutions were presented by Telemedicine and Advanced Technology Research Center (TATRC) staff at the MMVR 2006 Conference. See

Therapeutic Uses of VR Systems

Virtual reality systems are starting to be used in a number of therapeutic situations. For example:

Virtual Reality Treatment (ViRT) System - As reported in an article in Virtual Medical Worlds, ViRT uses chromokey technology and gesture control software to insert a patient into a virtual game environment allowing him to have fun exercising. The patient is stimulated to exercise for longer periods and more frequently whereas the system monitors the progress of the patient. This evolutionary physiotherapy treatment method which applies technologies borrowed from the entertainment and broadcast industries was successfully tested at the Riverside Campus of the Ottawa Hospital. For more information see

MOTEK, a leading developer and provider of high-quality motion capture technology, implemented the Computer Assisted Rehabilitation Environment System (CAREN) at the University of Groningen. CAREN allows a therapist to place the patient into a virtual environment to help diagnose medical disorders like Parkinson's Disease as well as other neurological abnormalities. In addition, CAREN enables therapists to introduce dynamic corrections to a patient's virtual and physical realities, creating an interactive relationship that can reduce the rehabilitation times for neuro-muscular and skeletal injuries by about 50 percent. In turn, the shorter rehabilitation times cut treatment costs. See

Women with breast cancer have fewer adverse effects from chemotherapy and less fatigue when using virtual reality as a distraction intervention during treatments, according to a study from the Duke University School of Nursing and Case Western Reserve Comprehensive Cancer Center. In the study, published in the January 2004 issue of Oncology Nursing Forum, the researchers described how chemotherapy patients eased their fatigue and discomfort by solving a mystery, touring an art gallery or going deep-sea diving in a virtual environment as they underwent treatment. See

In an article entitled "A Dose of Virtual Reality", Carlos Bergfeld writes that "one of the primary uses of virtual reality (VR) in a therapeutic role is its application to various forms of exposure therapy, ranging from phobia treatments, to newer approaches to treating pst-raumatic stress syndrome (PTSD)." The Virtual Reality Medical Center in San Diego, funded by the Office of Naval Research (ONR), has designed a VR system to help personnel returning from the wars in Iraq and Afghanistan cope with so-called acute post-traumatic stress disorder (PTSD). A version of the system is being distributed to various facilities, including academic institutions and military healthcare installations. More information on this can be found in the following BusinessWeek article -

Virtually Better creates virtual reality environments for use in the treatment of anxiety disorders such as fear of flying, fear of heights, fear of public speaking as well as post-traumatic stress disorder (PTSD). Virtually Better designed "Virtual Iraq" to support exposure therapy for veterans of anxiety disorders resulting from high-stress battleground environments. See and

Researchers from the University of Ulster and the stroke unit at the Royal Hospitals have developed revolutionary techniques to help people with stroke regain use of their upper limbs with the help of virtual reality. They have launched a pilot study employing a low cost, virtual reality system, which allows people with stroke to be immersed in a virtual world. Patients use a head-mounted display and wear a flexible glove connected to position and orientation sensors. These enable the patient's hand and arm movements to be tracked in the virtual environment, providing visual feedback to the patient. Audio feedback in the form of a "virtual physiotherapist" is also possible, offering encouragement and motivation during the tasks. See

In May, 2007, it was reported that a Multiple Sclerosis and Parkinson's Disease virtual reality device that combines a wearable, cell phone-sized audio component and a visual feedback apparatus to improve walking speed and stride length in patients suffering from multiple sclerosis has been developed at the Technion-Israel Institute of Technology. The integrated device has been in use at a number of medical centers in Israel and the United States, including the University of Cincinnati and the State University of New York. See

Virtual Reality (VR) & EHR Systems

Future electronic health record (EHR) systems may see a move away from text displayed on a flat screen, using a chart metaphor, that clinicians use today to more interactive three-dimensional computer-simulated environments. For example:

The PREPaRe System is a client-server-system based on the "Sanagate-Virtual Hospital" Internet portal. A standard Internet browser displays the user interface. The system offers the users access to a 3-D information system embedded in a virtual reality model of a real hospital. The distributed system of specialized data and media servers runs on different computer platforms. The implementation uses 'open source' and other non-commercial software and is based on current standards. The data contained in EHRs are dynamically collected from specific clinical information systems, pre-processed, and composed according to user specific access permissions and personal preferences. The user is able to use 3-D visualization and interaction for these "micro-worlds". The PREPaRe system allows access to all medical data contained in the EHR. See

Virtual Reality Avatars as Health Advocates is a new project proposed by the University Healthcare Infusion Services, affiliated with the University of Utah. They claim that by developing avatars as health advocates in a virtual reality environment, patients would become capable of having individual control over their design and interact with the avatars as their personal health advisors based on knowledge served up by decision support engines in existing electronic health records (EHR) and 'Health Banks' of the future. The virtual health advocates would be created in accordance with the patient's own preferences and specifications and integrate the functions of electronic health records with secured messaging using cell phones to communicate with healthcare providers. Feedback loops containing the patient's actual documented health behavior would provide the virtual reality avatar world with positive information for supporting the person's health status. See


In a recent article in Healthcare IT News, a new report from research firm Kalorama Information was referenced stating that the 2010 U.S. market for virtual reality in surgery, medical education, therapy and other areas will grow to $290 million. According to the Kalorama report, while still in the early stages of commercialization, VR technologies are being widely used by the Department of Defense (DoD) and a growing number of medical schools. The report also notes that the establishment of industry standards should lead to rapid commercialization of products. See

As we have seen from the examples given, VR can be used for numerous medical applications from pre-operative planning and robot-assisted surgery, to medical curricula to teach anatomy of body parts. VR applications provide opportunities to perform medical tasks in a risk-free environment and make training assessable to large numbers of students. It can also be used for the visualization of medical data that can be integrated and simulated into 3D models. It can even be used as an alternative therapeutic for treatment of pain and depression.

Papers on many other projects involving virtual reality and medicine from the last Medicine Meets Virtual Reality (MMVR) Conference can be found online at

Recommendations & Next Steps

The following are some recommendations and next steps healthcare organizations should consider taking with regards to Virtual Reality (VR) systems.

  • Consider establishing a workgroup to identify functional requirements and/or potential uses of VR systems for use by your healthcare organization.
  • Conduct a detailed literature search annually and obtain lessons learned from healthcare VR projects underway at other institutions.
  • Identify potential organizations to collaborate with on the research, development, testing and use of VR systems in healthcare, e.g. medical schools, vendors.
  • Conduct a feasibility study into the use of VR and select potential pilot projects.
  • Investigate changes in clinical and IT practices that may need to be made in anticipation of utilizing VR technology in healthcare.
  • Initiate and fund pilot project(s) and complete a detailed cost benefit analysis.

Selected References & Resources

The following are some key resources to explore when seeking more detailed information about VR activities and progress in the field of medical informatics.

The authors recently completed a book on collaboration, open solutions and innovative technology entitled "Medical Informatics 20/20", published by Jones & Bartlett in 2007. See


Peter J. Groen is on the faculty of the Computer & Information Science Department at Shepherd University in West Virginia and is one of the founders of the Shepherd University Research Corporation - see

Douglas Goldstein is an "eFuturist", author, speaker, and President of Medical Alliances, Inc. Visit or contact him at

Marc Wine is an advisor to the U.S. Department of Defense, Telemedicine & Advanced Technology Research Center (TATRC) at Fort Detrick, Maryland and is on the faculty The George Washington University, Health Services Management and Leadership Department, where he teaches Health IT Systems Management. You can contact him at

Peter Groen, Douglas Goldstein, Marc Wine

[Medical IT News][Calendar][Virtual Medical Worlds Community][News on Advanced IT]