Professor Ewart Carson from the City University of London sheds a light on telemedicine from the perspective of built-in evaluation criteria. At ITIS-ITAB'99, he illustrated his viewpoints on telemedical methodology with the Fourth Framework project HOMER-D, a homecare haemodialysis service for patients with end-stage renal failure. HOMER-D was set up by the Greek Ergo company in order to develop, apply and validate innovative telemedical monitoring services to support isolated patients who perform haemodialysis treatment at home or at satellite centres outside the hospital. Through use of state-of-the-art multimedia and networking technologies, the functionality of an existing dialysis machine is enhanced, and monitoring and supervision services are implemented between the hospital-based control station and the patient's site. Remote advice and decision support should provide the patient with more confidence.
Telemedicine is an aspect of health care technology, that is based on information management and decision support. Its relevance in the light of a methodological approach for a broader range of applications therefore is obvious. As a professor of systems science, Dr. Ewart Carson is interested in modelling, design, and evaluation methodology, based on feedback. Models help us in the process of understanding and design and provide a framework for evaluation. The approach to evaluation, adopted by the HOMER-D team, includes the three ingredients of evaluability, formative evaluation as well as summative evaluation. In telemedicine, as in physiology, the modelling starts with the perception of a problem to end up in a fully validated model. In the validation process, the model's consistency and its correspondence with the laws of theory are verified.
In the HOMER-D telemedical service, a simple but powerful feedback model is used, consisting of the decision maker or doctor, the effector system or the dialysis machine, and the patient state. The monitoring information system influences the patient as well as the machine, and provides feedback to the physician in the hospital. This generic decision making model is applied to help the team understand the treatment cycle of haemodialysis sessions, in order to design and evaluate the telemedical system. The various feedback loops have been converted into hard- and software implementations between the remote unit and the patient's site. After having created a prototype, the team focuses on evaluation. The evaluability phase consists in the definition of appropriate evaluation criteria, input and output, objectives as to benefits and costs, which amounts in a matrix of all the stakeholders and evaluation criteria.
Formative evaluation serves to describe systems design and implementation, analyze the stakeholders' matrix, collect additional data, as well as refine the system. Summative evaluation is performed to determine the ultimate effects on health outcomes and make recommendations for modifying the process of health care delivery, seen in the light of Information Technology. Central in the evaluation is the stakeholders' matrix. Professor Carson starts from the Donebedian based telemedicine evaluation matrix for the qualitative analysis of the programme and its impact on the patient's satisfaction, but broadens the range of criteria, as to involve patients, family, clinical staff at the remote and local site, technical staff, and health system management purchasers. In addition, a necessary set of criteria relating to the system's safety, reliability, acceptability, and ergonomic design is defined, as well as a subset of criteria involving its effectiveness, efficiency, and affordability.
Next to stakeholders and criteria, the matrix implies a third dimension which deals with the different perspectives from which the criteria can be looked at, namely the medical, technical, organizational, and social viewpoint. As such, we note that economic considerations don't appear in terms of the necessary criteria. Each cell of the model is evaluated through a mix of qualitative and quantitative data. First, the necessary set of criteria is analysed and met in an appropriate way, before moving over to the subset. The HOMER-D team so far has carried out a number of preliminary trials at four sites in Athens, Milano, Padova and Newcastle to build a reasonable body of clinical sessions to collect data from. The formative evaluation has led to system adjustments in each site. The different data types relate to specific sessions, the state of the dialysis machine and the alarm systems, and to questionnaires filled in by nephrologists, nurses, and technical staff in the language of the country.
At the different sites, the results of the questionnaires were largely positive. The HOMER-D system thus has become acceptable to the clinical users as far as the four necessary criteria are concerned. Perhaps, some elements of effectiveness should be included as necessary conditions. Dr. Carson in this regard considers the system's impact on the physical status of the patient. The subset of criteria has not yet been assessed in HOMER-D but might be the subject of a future project under the Fifth Framework. In conclusion, we can define a systems-based methodology to be useful for both the design and evaluation of a telemedical system, like the HOMER-D haemodialysis service. The modelling methodology also has wider applicability but there is need for further methodological development.