RESNA Annual Conference - 2020

Development, Validation And Impact Of A Suite Of Outcomes Tools To Give Triangulated Data On The Appropriateness Of Wheelchairs Designed For Use In Low-Income Settings

Karen RispinBegin Superscript1End Superscript, Vicki SheaferBegin Superscript2End Superscript, Natasha LaytonBegin Superscript3End Superscript, Larry RaymondBegin Superscript4End Superscript

Begin Superscript1End SuperscriptAssistive Technology Catalyst Project, Begin Superscript2End Superscript Monash University Begin Superscript3End Superscript LeTourneau University, Begin Superscript4End Superscript John's Hopkins


Over the past ten years, we have been involved in research with the goal of providing data to wheelchair developers and manufacturers to improve the effectiveness of their chairs in low-resource settings. We focused on field studies on the function of wheelchairs designed for use in low-resource areas. We found that it was important to discriminate between different aspects of function. For example, a wheelchair could be durable and in good condition, but not at all appropriate for its current user. Or the seating was adequate, but the chair did not roll well on surfaces commonly encountered. In addition, wheelchair users have a different point of view than do wheelchair professionals, but both are essential. To that end, over a period of time, we developed and validated four outcomes tools. All are wheelchair specific and are designed so that feedback is explicit enough to spark beneficial changes to the design of wheelchair parts. To increase discriminatory ability, all use a mixed methods design including qualitative explanatory comments along with quantitative data. The quantitative data is suitable for powerful parametric analysis. Two of the tools are completed by wheelchair users: The Wheelchair Satisfaction Questionnaire and Aspects of Wheelchair Mobility. The other two are professional report tools: The Wheelchair Interface Question enables informed professional data on how suitable a wheelchair is to its user's needs; the Wheelchair Components Questionnaire enables feedback on the maintenance condition of a wheelchair. These outcomes have been validated and used in studies and are available open source on-line along with supporting research papers. Our hope is that these outcomes tools will enable feedback to improve wheelchair design and services globally.


Research directly comparing different health-related interventions is essential to confirm that the goals of those interventions are achieved. Laboratory studies that test wheelchairs to ISO standards are an important part of the picture, but they are incomplete. Field studies are necessary because the lab doesn't perfectly model the conditions in which wheelchairs are used, and especially because direct feedback from people involved in wheelchair use and provision is of key importance [1,2,3].

There are significant challenges to collecting data in the field. Wheelchair professionals have time constraints and are focused on the immediate needs of their clients Any tool that is actually going to be of use must be brief and simple to use. Collecting data over time places a heavy load on busy clinicians and is less likely to be complete than data sets that can be obtained at one sitting. Outcomes tools which are not wheelchair specific are necessarily vague and unlikely to produce data about specific problems with design in a given wheelchair component or aspect of use. The design of questions and protocols also matters. If the results of a study are intended to spark evidence- based changes in wheelchair design, studies must produce data likely to serve that purpose. The type of data set produced by the tool also matters. Studies designed to produce data suitable for parametric statistical such as analysis of variance (ANOVA) are known to be more likely to discern differences. In addition, the power to discriminate meaningful difference is increased in mixed methods studies which include qualitative as well as quantitative data [4,5].

High variation among wheelchair users presents a challenge. Ideally, there should be a level playing field for all wheelchair users. If parts of the wheelchair are described as they relate to human body regions, almost everyone can complete all questions. In studies involving rolling on different surfaces, differences between participants may mask differences due to wheelchair design. A wheelchair user with a very strong upper body may be able to roll farther and more quickly in almost any chair on any surface than a wheelchair user with reduced upper body function. This can be solved by a within-subjects study design in which each participant uses two or more devices and is only compared to themselves [6,7,8].

Another challenge to providing data able to spark positive change is lack of clarity on the specific aspect of function addressed by an outcomes tool. Data to do with one aspect of function may mask data to do with another aspect of function. For example, data on durability is needed but should not obscure data on clinical appropriateness. Data from wheelchair professionals is needed, but it will likely differ from data provided by wheelchair users [1,9,10].

We hypothesized that wheelchair specific tools designed to provide clear data on specific aspects of wheelchair function would enable positive design changes on the part of manufactures and enable the provision of more appropriate wheelchairs.


Beginning in 2010, comparative studies were done in partnership with BethanyKids at JoyTown School for students with disabilities in Kenya with the goal of providing feedback to manufactures of wheelchairs intended for use in low-resource settings. Undergraduate teams prepared in the US and then traveled to Kenya on short trips. Given those limitations, we felt we could provide meaningful data on four topics: 1) Is it broken? 2) Does it roll well? 3)Does it suit its user? 4) Is the user satisfied?

Figure 1 is a question from the Wheelchair Components Questionnaire showing the user oriented clear wording and the format of questions utilized in these tools. The question says, “Rate the seat include cushions and other parts supporting the hips, buttocks and thighs.” It illustrates a visual analogue scale with emoticons as anchors and a comment line to explain the rating.
Figure 1. Format of the questions used in each of the tools described in this paper.

Four outcomes tools were developed. Each was designed to provide a level playing field to all wheelchair users. Visual analogue scale (VAS) format questions were used to produce data suitable for ANOVA. To provide a shared understanding of the rating scale, emoticons were used as anchors underneath the visual analogue line. Each VAS question also solicited an explanatory qualitative comment (Figure 1)

Each outcomes tool was brief so that it could be completed in a short time period, and it focused on only one of the four topics. It was designed to provide a snapshot of function at one moment in time. For the four outcomes tools validation studies have been published covering construct validity and reliability. Comparative studies on different wheelchair types have also been published and in those studies ideally 30 wheelchairs of each type were included. The tools are very briefly described below in the order of their development. For more information, related research publications are available at All four outcomes tools are available open source on the same website.

Wheelchair Components Questionnaire (WCQ)

The WCQ addresses the "Is it broken?" topic. It is a professional report tool for wheelchair professional to assess the maintenance condition of a wheelchair. The WCQ does not require the tracking of repairs over time and is brief with only eight questions. Questions are written to relate wheelchair regions to regions of the human body so that all questions can be completed for almost all wheelchairs in use. Figure 1 is a question from the WCQ [10,11,12,14]

Aspects of Wheelchair Mobility Tests (AWMT)

The AWMT addresses the "Does it roll well?" topic. If the AWMT is used in a comparative study, 25 or more participants each roll in two or more wheelchair types on tracks that model three commonly encountered rolling environments: smooth surfaces, rough surfaces, and tight spaces. Each participant is compared only to themselves so that the impact of wheelchair design on mobility is not obscured by individual variation. After completing a track in each type of wheelchair, a participant answers a feedback [6,7,8,13,15]

Wheelchair Interface Questionnaire (WCQ)

The WIQ addresses the "Does it suit its user?" topic. Therapists with wheelchair experience and training are the population most able to answer this question. The WIQ provides a moment in time assessment based on informed professional opinion on the quality of the interface between individual users and their wheelchairs.

Because of the difficulties in interacting with non-verbal wheelchair users, or small children, or those who only speak a mother-tongue, interaction with the wheelchair user is encouraged but not required [3,9].

Wheelchair Satisfaction Questionnaire (WCQ)

The (WSQ) addresses the "Are the users satisfied?" topic. It collects data on a wheelchair user's satisfaction with their own wheelchair at a given moment in time. This tool has 16 questions that cover satisfaction with regions of the wheelchair supporting different bodily regions. It also includes questions on other aspects of satisfaction with a wheelchair [1,15].


Responses to our studies indicate that our goal of providing feedback specific enough to spark positive change in wheelchair design and provision was met. The AWMT and the WCQ were the first two outcomes tools to be developed, and these have been used in several comparative studies. There were responses on the part of the wheelchair companies whose wheelchairs had been involved in studies. Free Wheelchair Mission modified the attachment of footplates on their Gen 2 wheelchair. A designer who had been working with Hope Haven, has indicated that our study results were a key factor in his development of the concepts behind the BeeLine wheelchair initiative. Wheelchairs for Kids modified the design of their lateral trunk supports and castor bearings. Several manufacturers indicated that problems revealed in our studies had been due suppliers providing inadequate parts; for example, Whirlwind altered the subcontractor for the caster material when our study revealed failure of castor wheel material. A few manufacturers have not responded to significant problems revealed by our studies. Even in those cases, wheelchair provision was locally improved because our results confirmed a strong hesitancy on the part of several organization to order or use those wheelchair types.


Although the snapshot like nature of these outcomes tools does facilitate quick and complete data collection, there are of course limitations. For example, WCQ data does not reveal exactly when the wheelchair was last repaired. However, to some extent, comments can give insight on the long-term reasons for the snapshot responses of those completing the WCQ, WIQ and WSQ. With the AWMT, participants sometimes mention other situation in which they've encountered problems rolling in a certain rolling environment.

Many studies have used visual analogue scale (VAS) format questions to produce data which was then found to be suitable for parametric analysis. However, the lack of an understanding of the scale can be a problem. Early on in the development of these measures, we had tried to use a VAS line without grades. Without anchors, we found that data was obscured For example, for the WCQ, the score for maintenance condition of the push handles was scored higher than other wheelchair components for our study wheelchairs by all raters; however, this finding was obscured because of high variation in the way raters responded to the VAS line. Some raters marked all parts lower on the scale or higher on the scale, or closer together. The push-handles were consistently scored higher, but the actual scores were highly various, therefore ANOVA did not indicate statistical significance. For VAS format pain assessment questionnaires, 'anchors" under the scale help to give a shared understanding of what is meant [17]. However, emoticons indicating grief or pain seemed inappropriate for our purpose. When we paced symbols for school grades as anchors under the line, this solved the problem, presumably because of the shared understanding of these symbols. We have found that school grade symbols also provide a powerful communications tool when addressing the wheelchair manufactures. The statement, "your footplates consistently received an average mark equivalent to a failing grade," has more punch than "your footplates received a mark on the lower end of the VAS scale". Local culturally appropriate grading symbols need to be used as anchors to provide this intuitive understanding. Although we have done validation studies in Kenya and in the US which have slightly different grading symbols, additional validation with grading symbols from different cultures is still to be done.

The data for the WSQ and AWMT come directly from wheelchair users. This is of course essential for any assessment of the function of a wheelchair. Unlike the other measures, the AWMT provides hard data on distance traveled along with accompanying response questions. It may be a bit more difficult to implement because it requires following a protocol to set up the tracks and collect data. With the professional report data, we wanted to answer the "is it broken?" question separately from the "does it suit its user?" A wheelchair that is durable and in good condition may not be at all clinically appropriate. Therapists and other wheelchair professionals with wheelchair experience and training are the people who have base of knowledge to provide data on the maintenance condition of the wheelchair, and on the appropriateness of a wheelchair for its user Professional report data from the WIQ and WCQ is also easier to collect than data directly from users. Wheelchair users may be non-verbal, or children, or only fluent in small mother-tongue languages. In contrast, wheelchair professionals almost universally speak the local language of education which is often a worldwide language. In our areas of study, wheelchair professionals are almost universally fluent in English.

The WCQ and the WIQ could be quickly completed by wheelchair professionals as part of a follow-up wheelchair assessment. At the same time, the wheelchair users who are verbal and able to complete a questionnaire could complete the WSQ. The three tracks for the AWMT could be part of a wheelchair skills training area, and a four- minute roll test on each track could become part of wheelchair selection or follow-up assessments. Data from these validated outcomes tools could be used to as supporting evidence for wheelchair repair, modification or replacement.


For a full picture of the function of a wheelchair or wheelchair type, it is helpful to be able to answer four questions: is it broken? Does it suit its user? Does it roll well? Is the user satisfied? The first two are completed by wheelchair professionals, the last two by wheelchair users. All are wheelchair specific and are designed so that feedback is explicit enough to spark beneficial changes to the design of wheelchair parts. To increase discriminatory ability, all use a mixed methods design including qualitative explanatory comments along with quantitative data. All four outcomes tools can and have been used successfully in studies on specific types or makes of wheelchairs, they can be used to improve local wheelchair services.


  1. Toro-Hernández, M. L., Kankipati, P., Goldberg, M., Contepomi, S., Tsukimoto, D. R., & Bray, N. (2019). Appropriate Assistive Technology for Developing Countries. Physical Medicine and Rehabilitation Clinics, 2019 30(4), 847-865.
  2. Bane, H. M., Sheafer, V., & Rispin, K. Face and content validity for the Wheelchair Satisfaction Questionnaire. Disability and Rehabilitation: Assistive Technology, 2019, 1-5. DOI: 10.1080/17483107.2019.1684579
  3. Rispin, K., Davis, A. B., Sheafer, V. L., & Wee, J. Development of the Wheelchair Interface Questionnaire and initial face and content validity. African Journal of Disability (Online), 2019 8, 1-8.
  4. Kamaraj, D. C., Bray, N., Rispin, K., Kankipati, P., Pearlman, J., & Borg, J. A conceptual framework to assess effectiveness in wheelchair provision. African Journal of Disability (Online), 2017 6, 1-5.
  5. May, L., The challenge of measuring change: Responsiveness of outcomes measurements', Canadian Journal of Rehabilitation. 1997 10(1), 15–24.
  6. Rispin, K., & Wee, J. Comparison between performances of three types of manual wheelchairs often distributed in low-resource settings. Disability and Rehabilitation: Assistive Technology, 2015 10(4), 316- 322.
  7. Rispin, K. L., Huff, K., & Wee, Test-retest reliability and construct validity of the Aspects of Wheelchair Mobility Test as a measure of the mobility of wheelchair users. African Journal of Disability (Online), 2017 6, 1-6.
  8. Rispin, K. L., Hamm, E., & Wee, J. Discriminatory validity of the Aspects of Wheelchair Mobility Test as demonstrated by a comparison of four wheelchair types designed for use in low-resource areas. African Journal of Disability (Online), 2017 6, 1-11.
  9. Davis, A. B., Sheafer, V., Rispin, K., & Layton, N. (2019). The inter-rater reliability of the Wheelchair Interface Questionnaire. Disability and Rehabilitation: Assistive Technology, 1-6.
  10. Rispin, K., Riseling, K., & Wee, J. (2018). A longitudinal study assessing the maintenance condition of cadres of four types of wheelchairs provided in low-resource areas. Disability and Rehabilitation: Assistive Technology, 13(2), 146-156.
  11. Rispin, K., DiFrancesco, J., Raymond, L. A., Riseling, K., & Wee, J. (2018). Preliminary inter-rater reliability of the wheelchair components questionnaire for condition. Disability and Rehabilitation: Assistive Technology, 13(6), 552-557.
  12. Rispin, K., Dittmer, M., McLean, J., & Wee, J. (2017). Preliminary reliability and internal consistency of the wheelchair components questionnaire for condition. Disability and Rehabilitation: Assistive Technology, 12(8), 852-856.
  13. Rispin, K., & Wee, J. (2014). A paired outcomes study comparing two pediatric wheelchairs for low-resource settings: The regency pediatric wheelchair and a similarly sized wheelchair made in Kenya. Assistive Technology, 26(2), 88-95.
  14. Michael, M., Riseling, K., Rispin, K. (2019). "Inter-rater Reliability of the Wheelchair Components Questionnaire between Multicultural Wheelchair Professionals," Rehab Week and RESNA, Toronto, June 24-28, paper 78.
  15. Rispin, K., & Wee, J. (2014). A paired outcomes study comparing two pediatric wheelchairs for low-resource settings: The regency pediatric wheelchair and a similarly sized wheelchair made in Kenya. Assistive Technology, 26(2), 88-95.
  16. Bane, H., Rispin, K., Sheafer, V., Reliability of the Wheelchair Satisfaction Questionnaire, Proceedings of the 35 International Seating Symposium, Pittsburg, March 20-22, 2019, PS 5.2
  17. Huguet, A., Stinson, J. N., & McGrath, P. J. (2010). Measurement of self-reported pain intensity in children and adolescents. Journal of psychosomatic research, 68(4), 329-336.