Introduction

Assistive technologies (AT) have great potential to enhance the lives of older adults and their caregivers. These technologies are becoming crucial given the growing proportion of older adults in the population and the simultaneous decrease in the number of working age caregivers (Cranswick & Dosman, 2008). Recent developments in artificial intelligence and robotics are increasingly being applied to create advanced AT for older adults. These AT are aimed to compensate for functional changes associated with aging or chronic health conditions such as sensory loss, decreased cognition, mobility limitations and to enable autonomy and participation in meaningful activities (Broekens, Heerink, & Rosendal, 2009; Gillespie, Best, & O'Neill, 2012). In addition, AT are intended to assist caregiving activities. Designing and evaluating AT to address these multiple and complex concerns is challenging. Creating AT for older adults with dementia (OAwD) entails developing for a highly heterogeneous group and their caregivers, supporting dynamic conditions, and encouraging sustained AT use if health changes. The design process necessarily involves an interdisciplinary team of engineers, computer scientists, clinicians, and end users such as the OAwD themselves, caregivers and health care providers in a user-centred design approach (Bharucha et al., 2009; Boger, Wang, & Taati, 2012; Czarnuch & Mihailidis, 2011). As our team has experienced, developing AT for these users and involving them in design and evaluation processes can be challenging.

This paper summarizes some of the lessons we have learned through developing advanced AT for use by OAwD and their caregivers. These lessons are illustrated by examples encountered through our work. It is hoped that insights from our experiences will be useful to others developing AT for these users.  

Overview of Projects

The projects described include the following:

1. Modified or intelligent power wheelchairs to enable the mobility independence of older adult long-term care (LTC) home residents with physical, sensory and cognitive impairments;
2. COACH, an intelligent supportive environment to assist OAwD to complete activities of daily living (ADL) more independently; and
3. Personal robots, including the human-like social robot Brian, to assist OAwD in performing ADL and engage them in social interactions.

Lessons Learned

1.  Explore, understand and value the perspectives of OAwD and those involved in their care.

It is essential to remember that in spite of changes in cognitive, communication and functional abilities and roles, OAwD are people with distinct and genuine perspectives and opinions. Akin to any user group, understanding and incorporating the preferences of OAwD will help align the AT to their wants, needs and abilities, making the result more acceptable and usable. However, it is not always easy to solicit information from OAwD because of changes in memory (especially explicit and declarative memory), language abilities (e.g. word finding or comprehension difficulties), and abstract thinking (American Psychiatric Association, 2000). OAwD may also have hearing or vision changes that make communication and self-expression difficult. Nevertheless, many older adults with mild and moderate Alzheimer Disease, for example, can describe their perspectives on life, showing a maintained sense of personal identity (Westius, Kallenburg, & Norberg, 2010). Furthermore, studies have shown that older adults with moderate or even severe cognitive impairment can report on aspects such as unmet needs, sense of well-being and quality of life (QoL) (Beer et al., 2010). Proxy's responses are different than ones from OAwD themselves; for example, caregivers' estimates of QoL of OAwD were found to be lower than self-reported QoL from OAwD (Beer, et al., 2010). The ability of OAwD to communicate their opinions was demonstrated when we tested a prototype anti-collision power wheelchair in a LTC home (Wang, Kontos, Holliday, & Fernie, 2011). The three participants (mild cognitive impairment according to the Mini Mental State Exam) who tested the prototype and who had potential to use it did not accept it, reporting that it was large, unattractive and not useful to them. This illustrates the need to include OAwD in the development of AT because their opinions on need, acceptance, and ways to improve technology can and should be heard.

2.  Present prototypes to users early and often, and make use of negative feedback.

Waiting too long before getting feedback from users leaves design requirements and assumptions unchecked. Evaluating designs early and often can greatly improve AT development by capturing and incorporating requirements before resources are invested in a suboptimal design. Presenting conceptual prototypes (that demonstrate concepts, functions and form factors) or Wizard-of-Oz systems (where prototypes appear to operate autonomously but are partially/fully operated by a human (Green & Wei-Hass, 1985)) can elicit valuable ideas from representative users to guide development. For example, in a project to develop a personal smart-home robot, a tele-operated robot was used to explore the feasibility and usability of a mobile robot to deliver audio and video prompts to assist 10 OAwD to perform a tea making task. The robot was extremely useful to gather evidence on feasibility and acceptability from OAwD and their caregivers and feedback to improve the prompting system, as well as social interactiveness and physical attributes of a robot, prior to investing resources on an autonomous robot and integrating it into a smart home system  (Begum, Wang, Huq, & Mihailidis, in review). 

While positive feedback can indicate features or functions that are perceived to be useful, negative feedback can be used to refine or redesign the AT. When evaluating one version of the anti-collision power wheelchair (Wang, Gorski, Holliday, & Fernie, 2011), all six users who tested the device were unable or chose not to use it, which led to abandonment of the design. In another wheelchair study, two participants were frustrated when the wheelchair did not allow them to make safe maneuvers towards an obstacle within a pre-specified distance (Viswanathan, 2012). This feedback has led to ongoing work in obstacle recognition and more advanced control strategies to allow users to move closer to obstacles in situations such as docking under a table. Although it may be difficult at times, developers should maintain an objective and constructive attitude toward design feedback as this can foster creativity and lead to a more useful AT.    

3.  Small scale studies are a good place to begin when evaluating prototypes.

Prototype evaluation studies are often descriptive and observational or may use single subject research designs that involve a small number of participants. While randomized controlled trials are often considered to be the gold standard in intervention evaluation research, they are not an option for the early stages of AT development, as the cost is prohibitive, there are difficulties with recruiting sufficient participant numbers to allow for homogenous comparisons, and there are difficulties with identifying or controlling influencing factors (Brandt & Alwin, 2012).  

Small scale studies have enabled us to better understand our users and match designs to users and their environments. For example, descriptive and observational studies ranging from 10-40 participants in LTC facilities have been used with the social robot, Brian, in both controlled (McColl & Nejat, 2013, in press) and uncontrolled environments (Louie, McColl, & Nejat, 2012; McColl, Louie, & Nejat, 2013, in press). These studies have provided valuable information regarding how participants interact with such a robot, including the impact on activity engagement and compliance with the robot’s requests.

4.  Participant recruitment and retention can be complex and time consuming, but clinical collaborators can greatly help.

It should be kept in mind that recruitment can take a long time and retention for longer term studies can be challenging. Even with broad inclusion and exclusion criteria, is difficult to access and recruit from the OAwD population. The process of acquiring informed consent can often involve several intermediary steps since most researchers are unable to contact potential participants directly and must rely on others to gain access. OAwD often have substitute decision makers who need to participate in the informed consent process. OAwD may also have chronic disease conditions, and delays in participation and drop outs due to poor health are not uncommon.

Successful recruitment can be greatly aided by clinical collaborators. Identifying collaborators and “champions” for a research project and investing the time and effort needed to maintain these working relationships are essential. Collaborators typically have access to pools of potential study participants, and can help to promote studies and identify suitable candidates. For example, we have had many clinical collaborators assist us in the past, such as with COACH studies (Czarnuch, Cohen, Parameswaran, & Mihailidis, 2012, in review; Mihailidis, Barbenel, & Fernie, 2004). Clinical partners are often willing to participate in the recruitment process. For example, in the latest study with COACH, collaborators from a local memory clinic were very supportive and helped to screen and contact potential OAwD and caregivers for participation, thus expediting the enrollment process.

5. Mixed methods data collection and analysis approaches work well. 

When evaluating prototype performance, user experiences with using an AT or an AT’s affect on users, mixed methods approaches that include quantitative and qualitative data are useful as they provide complementary types of data. Quantitative data can be extremely important for several aspects of evaluation that require measurement and comparison. For example, in studies with intelligent power wheelchairs we evaluated sensor performance (Viswanathan, Boger, Hoey, & Mihailidis, 2007) and user performance (e.g., number of collisions) (Viswanathan, Little, Mackworth, & Mihailidis, 2011). Studies with COACH have examined the device’s efficacy in terms of COACH’s ability to correctly recognise and respond to different events and the users’ responses to COACH’s prompts (Mihailidis, Boger, Craig, & Hoey, 2008; Mihailidis, Fernie, & Barbenel, 2001). During one-on-one interactions with Brian, we have evaluated the robot’s sensing and behavior selection capabilities in addition to user engagement and acceptance (McColl & Nejat, 2013, in press).

Quantitative data, however, cannot present a complete picture of the interaction and experiences of OAwD with AT. While a review is beyond the scope of this paper, there is a dearth of reliable, valid and sensitive tools to measure the satisfaction, acceptability and impact of AT for OAwD. We have employed a variety of observational methods (e.g., documenting/coding observations during trials or using video recordings) and informal interviews (e.g., asking questions during AT use) to gain access to aspects that do not readily lend themselves to measurements. On the whole, data from observations and informal interviews are more valid in some situations compared to post-trial interviews or questionnaires if short term memory and recall are concerns.

In general, qualitative data complement quantitative assessment when performing an in-depth evaluation. For example, we used observations and informal interviews during trials and questionnaires and interviews after trials when examining the usability of a multi-modal user interface for a collision-avoidance power wheelchair (Wang, Mihailidis, Dutta, & Fernie, 2011). The observations of use (e.g., completion of mobility goals), user comments, and facial expressions corroborated post-trial questionnaire and interview data. In another wheelchair study, conflicting results were found as facial expressions and comments made by some users during wheelchair use showed evidence of frustration, which remained unreported in their questionnaire ratings for feelings of frustration (Viswanathan, 2012). Using both quantitative and qualitative data allows for an understanding of AT impact that would not be possible using either type of data alone.

Conclusions

This review highlights some of the challenges of developing advanced AT for OAwD, such as the need to design support for a heterogeneous group whose needs will likely change over time and whose caregivers and health care providers need to be included in development. Moreover, many of these lessons learned are applicable and useful to developers for other populations and applications.