INTRODUCTION

Measuring outcomes are crucial to any assistive technology (AT) service program. But the seriousness of this discussion is now more important than ever. The world level initiatives by WHO, the China Belt and Road, and others have elevated the attention to begin meeting a world level need. As new programs embark, this becomes a critical time for discussing how outcomes are best measured and systems developed. There is no better time to introduce new assistive technology outcomes systems as when new programs have the potential of collecting naturally occurring baseline data prior to service provision that can be compared longitudinally as services and device interventions are implemented.[1]

However, the value of assistive technology is not the same to everyone and this value is core to outcomes measurement. For example, to the user of technology it is only as valuable as what the person gains from using it, the benefits of use compared to the expenditures of procuring it, time learning to use it, fatigue in using it, embarrassment of using it, and so on. When a product or system meets standards of good design and usability, its use and realized benefit from use depends heavily on initial expectations of benefit, involvement in product and feature selection, and adequate training for use.

"AT users may be satisfied with the clinic's services, have the necessary funding for the device, received a product that is usable, looks good, functions well and meets all safety standards, and helped them achieve functional gain -- but if it is a hassle to use, set-up and maintain, if it doesn't fit with their needs/preferences/lifestyle, if they feel self- conscious using it, insecure with use even though it is safe, if they are socially and physically and emotionally uncomfortable with use, then they are not realizing benefit from use and will not use it. It is not a good match of person and AT. Ultimately, it is the User Experience (UE) and realization of benefit that drives and determines whether or not a device is used, for how long, what percent of the time and in which environments" (Scherer, 2017, p. 1).

As more and more technologies are becoming indispensable to people with disabilities, use is becoming less and less optional. This makes it even more important to assess realization of benefit, and whether use was prematurely or inadvisably stopped and why that occurred. But other AT stakeholders hold different perspectives and value different outcomes. [2,3]

Different Views of Desirable AT Outcomes

These distinct perspectives of what outcomes are even extends to the point where some do not even have it in their vocabulary. In the U.S. where the concepts of "outcomes" are commonly used, it was found that consumers of AT devices and services may not even conceptualize the word for their device use, suggesting that perhaps the term is conceptualized by administrators, managers, funders and academics, not AT end users. [4]

Table 1 contains examples of key assistive technology stakeholders and what they view as key outcomes. While they do share common goals, they differ in the weight they place on the person, the technology, and situations of use.

Table 1. Examples of Various Assistive Technology (AT) Outcomes According to Stakeholder*

Stakeholder	Focus	View of AT	Desired AT Outcome	Sample Outcome Measures
AT user	Comfort, function	Functional gain, use worthiness	Realization of benefit from use, enhanced well-being/QoL	ATD PA, ATUFS
Clinic director	Continued operation and funding	Functional gain	User satisfaction, cost containment	User feedback surveys, financial reports, SCAI
Funding agencies and payers	Minimization of financial losses	Functional gain	Functional gain, profit, cost containment	Accounting reports
Physical therapist	Mobility, movement, seating and positioning	Functional gain	Mobility, comfort, functional gain	FIM, WST
Occupational therapist	Task performance	Functional gain	Comfort, functional gain, personal well-being, underutilization/ overutilization	log of AT use, SWBS, ATD PA, OTFACT
Speech Language Pathologist	Communication	Speech communication support or alternative	Communication	(Same as for OT), AAC TOM
Rehabilitation counselor, social worker, psychologist	Employment, personal factors, goal achievement,	Functional gain	Use worthiness, personal well-being, realization of benefit, underutilization/ overutilization	(Same as for OT)
Special Educators	Academic achievement, social interaction	Academic and functional performance	Learning, class participation	MATCH-ACES, ET PA
Rehabilitation engineer, computer scientist, manufacturer, supplier	Device, system, and components, product sales	Functional gain, safety, operability, affordability	Usability, performance, technology adoption/acceptance/ diffusion, user satisfaction	QUEST, log of AT use, TAM, UTAUT
* all emphasize enablement and the performance of activities and participation, but vary in and weigh differently attention to the person, milieu/environments of use, and technology functions and features Source: Authors as modified from Scherer, 2017 [5]

 

INTERNATIONAL RELEVANCE

Different geographic areas, economies and cultures value some outcomes more highly than other outcomes and this can make global data sharing and strategizing challenging. Consequently, the identification of the most common elements of outcomes measurement systems and databases will be essential to use as an initial way to communicate across regions and disciplines.

OUTCOMES MEASURES

Many lists of measures exist. A few AT assessments are listed here. Another compilation of instruments, all free when catalogued, exists on the R2D2 Center ATOM Project website: http://www.r2d2.uwm.edu/atoms/idata/ [6] and in a scoping review of AT evaluation tools [7].

AAC TOM: Enderby P. (2014) Introducing the therapy outcome measure for AAC services in the context of a review of other measures, Disability and Rehabilitation: Assistive Technology, 9:1, 33- 40, DOI: 10.3109/17483107.2013.823576 Functional Independence Measure (FIM): Hamilton BB, Granger CV, Shervin FS, et al. A uniform national data system for medical rehabilitation. In: Further MJ, ed. Rehabilitation outcomes: analysis and measurements. Baltimore: Paul H Brooks, 1987 MATCH-ACES: Zapf, SA, Scherer, MJ, Baxter, MF & Rintala, DH. (2016) Validating a measure to assess factors that affect assistive technology use by students with disabilities in elementary and secondary education, Disability and Rehabilitation: Assistive Technology, 11(1), 38-49. PMID: 26696460. Matching Person & Technology (MPT) Portfolio: Scherer, M. J. (1998). Matching person & technology: A series of assessments for evaluating predispositions to and outcomes of technology use in rehabilitation, education, the workplace & other settings. Institute for Matching Person & Technology. https://sites.google.com/view/matchingpersontechnology/
	ATD PA: Assistive Technology Device Predisposition Assessment (ATD PA), see Matching Person & Technology (MPT) Portfolio
	ATUFS: Assistive Technology Use Follow-up Survey (ATUFS), see Matching Person & Technology (MPT) Portfolio ET PA: Educational Technology Predisposition Assessment (ET PA), see Matching Person & Technology (MPT) Portfolio
OTFACT: Smith, R. O. (2002). OTFACT: A multi-level performance-based software instrument with an assistive technology outcomes assessment protocol. Technology and Disability, 14(3), 133-139. Psychosocial Impact of Assistive Devices Scale (PIADS): Jutai, J., & Day, H. (2002). Psychosocial impact of assistive devices scale (PIADS). Technology and Disability, 14, 107-111. Quebec User Evaluation of Satisfaction with assistive Technology (QUEST): Demers, L, Weiss- Lambrou, R, Bernadette Ska B. (1996) Development of the Quebec User Evaluation of Satisfaction with assistive Technology (QUEST), Assistive Technology, 8:1, 3-13, DOI: 10.1080/10400435.1996.10132268 SIVA Cost Analysis Instrument (SCAI): Andrich, R. (2002). The SCAI instrument: Measuring costs of individual assistive technology programmes. Technology and Disability, 14(3), 95-99. Subjective Well-Being Scale (SWBS): Diener, E. (1984). Subjective well-being. Psychological Bulletin. 95 (3): 542–575. doi:10.1037/0033-2909.95.3.542. PMID 6399758. Technology Acceptance Model (TAM): Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 319–340. doi:10.2307/249008 Wheelchair Skills Test (WST): Kirby RL, Dupuis DJ, MacPhee AH, Coolen AL, Smith C, Best KL, Newton AM, Mountain AD, MacLeod DA, Bonaparte JP. The Wheelchair Skills Test (version 2.4): measurement properties. Arch Phys Med Rehabil 2004;85:794–804.

DISCUSSION

We have an opportunity like never before and maybe never again as AT programs rapidly proliferate globally. While a local AT outcomes system will be of benefit to programs as they document their challenges and success, national [8] and indeed, global reporting and related databases are of immense benefit for comparison and informing each other. The recent COVID-19 is a clear example of how global data can be of mutual international interest. While AT is an intervention and COVID-19 is the challenge, the need to collect data around the need for intervention and the success of interventions is actually the same. We all learn from the experiences of others around the globe. We need common terminology, common data elements, common databases to best benefit from the work of our international colleagues and achieve global .