As people approach middle age, their cognitive abilities begin to decline, and forgetfulness permeates nearly every aspect of daily life. This problem is especially acute among the aging, a generation where not everyone is comfortable with new hi-tech "brainboosting" gadgets and may even shy away altogether from any new technology devices.
"HERMES is helping realize the vision of lengthening the stage of independent living and helping aging populations remain active", explained Professor Manfred Tscheligi, Director of CURE - Center for Usability Research & Engineering - and co-ordinator of the HERMES project. "Developing innovative yet non-intimidating technology to address the needs of this valuable population is a challenge that this community of researchers is eager to embrace."
The HERMES project brings together experts ranging from gerontology and speech processing, to hardware integration and user-centred design to achieve the common goal of cognitively supporting older people. The first stage, a joint effort of the user partners CURE in Austria and INGEMA in Spain, will work on better understanding the memory needs of the elderly through a range of user-centred methods. The technology partners in the consortium, including the IBM Research Lab in Haifa, Israel, Athens Information Technology in Greece, Bradford University in the United Kingdom, and TXT e-Solutions in Italy will work to develop the sensory and perceptual technologies.
The HERMES "home of the future" prototype will be equipped with microphones and video cameras to record conversations and experiences at the user's discretion. The elderly will be equipped with a mobile device that records conversations, experiences, location co-ordinates, dates and times outside the home, for example at the doctor's office, the bank, or with acquaintances. All the information will be stored, processed and analysed to help augment the person's memory.
"With HERMES, our research into multimedia technologies is taking a real leap forward in transforming how people will go about their daily lives", noted Ron Hoory, manager of speech technologies at the IBM Haifa Research Lab. "The project will enable us to target more complex dynamic environments that demand new innovative approaches. These new approaches can work equally well for business environments, to help analyse telephone calls in a call centre, broadcast news and video, or any scenario where speech is of essence."
The project's ultimate goal is to build a prototype system that will offer three main services:
- The first service will help the user remember what happened in the recent past. For example, Alice can ask the system, "What did my daughter say to me yesterday when we discussed the snow storm in Montréal?" The system will search its repository of recordings based on the time window and the keywords of the content, and then let Alice choose to play back the relevant conversations recorded in audio or on video.
- The second service will give the person reminders or prompts to help manage their daily schedule. For example, if George notices that the jar of coffee is almost finished, he can tell the system to add a reminder to buy coffee. The system will connect the words "coffee", "buy" and name of the local store where George usually shops. When George passes the store and his mobile device notices the store location, the system will remind him to buy coffee using a text-to-speech synthesis in a natural human voice.
- The third service provided will be memory fitness exercises based on actual personal experiences. For example, at any given time, the system will have a list of appointments, some from the past and others that are yet to take place. The system can present the appointments and ask the user to organize them by category or quiz the user by asking the exact time for next week's doctor's appointment. Another question might be: From the following three appointments which is first, second, and third? These games help jog the user's memory and are based on actual conversations or experiences that were recorded or entered as appointments.