INTRODUCTION

Research directly comparing different health-related interventions is essential to confirm that the goals of those interventions are achieved. Laboratory studies that test wheelchairs to ISO standards are an important part of the picture, but they are incomplete. Field studies are necessary because the lab doesn't perfectly model the conditions in which wheelchairs are used, and especially because direct feedback from people involved in wheelchair use and provision is of key importance [1,2,3].

There are significant challenges to collecting data in the field. Wheelchair professionals have time constraints and are focused on the immediate needs of their clients Any tool that is actually going to be of use must be brief and simple to use. Collecting data over time places a heavy load on busy clinicians and is less likely to be complete than data sets that can be obtained at one sitting. Outcomes tools which are not wheelchair specific are necessarily vague and unlikely to produce data about specific problems with design in a given wheelchair component or aspect of use. The design of questions and protocols also matters. If the results of a study are intended to spark evidence- based changes in wheelchair design, studies must produce data likely to serve that purpose. The type of data set produced by the tool also matters. Studies designed to produce data suitable for parametric statistical such as analysis of variance (ANOVA) are known to be more likely to discern differences. In addition, the power to discriminate meaningful difference is increased in mixed methods studies which include qualitative as well as quantitative data [4,5].

High variation among wheelchair users presents a challenge. Ideally, there should be a level playing field for all wheelchair users. If parts of the wheelchair are described as they relate to human body regions, almost everyone can complete all questions. In studies involving rolling on different surfaces, differences between participants may mask differences due to wheelchair design. A wheelchair user with a very strong upper body may be able to roll farther and more quickly in almost any chair on any surface than a wheelchair user with reduced upper body function. This can be solved by a within-subjects study design in which each participant uses two or more devices and is only compared to themselves [6,7,8].

Another challenge to providing data able to spark positive change is lack of clarity on the specific aspect of function addressed by an outcomes tool. Data to do with one aspect of function may mask data to do with another aspect of function. For example, data on durability is needed but should not obscure data on clinical appropriateness. Data from wheelchair professionals is needed, but it will likely differ from data provided by wheelchair users [1,9,10].

We hypothesized that wheelchair specific tools designed to provide clear data on specific aspects of wheelchair function would enable positive design changes on the part of manufactures and enable the provision of more appropriate wheelchairs.

METHODS

Beginning in 2010, comparative studies were done in partnership with BethanyKids at JoyTown School for students with disabilities in Kenya with the goal of providing feedback to manufactures of wheelchairs intended for use in low-resource settings. Undergraduate teams prepared in the US and then traveled to Kenya on short trips. Given those limitations, we felt we could provide meaningful data on four topics: 1) Is it broken? 2) Does it roll well? 3)Does it suit its user? 4) Is the user satisfied?

Four outcomes tools were developed. Each was designed to provide a level playing field to all wheelchair users. Visual analogue scale (VAS) format questions were used to produce data suitable for ANOVA. To provide a shared understanding of the rating scale, emoticons were used as anchors underneath the visual analogue line. Each VAS question also solicited an explanatory qualitative comment (Figure 1)

Each outcomes tool was brief so that it could be completed in a short time period, and it focused on only one of the four topics. It was designed to provide a snapshot of function at one moment in time. For the four outcomes tools validation studies have been published covering construct validity and reliability. Comparative studies on different wheelchair types have also been published and in those studies ideally 30 wheelchairs of each type were included. The tools are very briefly described below in the order of their development. For more information, related research publications are available at www.atcatalyst.org. All four outcomes tools are available open source on the same website.

Wheelchair Components Questionnaire (WCQ)

The WCQ addresses the "Is it broken?" topic. It is a professional report tool for wheelchair professional to assess the maintenance condition of a wheelchair. The WCQ does not require the tracking of repairs over time and is brief with only eight questions. Questions are written to relate wheelchair regions to regions of the human body so that all questions can be completed for almost all wheelchairs in use. Figure 1 is a question from the WCQ [10,11,12,14]

Aspects of Wheelchair Mobility Tests (AWMT)

The AWMT addresses the "Does it roll well?" topic. If the AWMT is used in a comparative study, 25 or more participants each roll in two or more wheelchair types on tracks that model three commonly encountered rolling environments: smooth surfaces, rough surfaces, and tight spaces. Each participant is compared only to themselves so that the impact of wheelchair design on mobility is not obscured by individual variation. After completing a track in each type of wheelchair, a participant answers a feedback [6,7,8,13,15]

Wheelchair Interface Questionnaire (WCQ)

The WIQ addresses the "Does it suit its user?" topic. Therapists with wheelchair experience and training are the population most able to answer this question. The WIQ provides a moment in time assessment based on informed professional opinion on the quality of the interface between individual users and their wheelchairs.

Because of the difficulties in interacting with non-verbal wheelchair users, or small children, or those who only speak a mother-tongue, interaction with the wheelchair user is encouraged but not required [3,9].

Wheelchair Satisfaction Questionnaire (WCQ)

The (WSQ) addresses the "Are the users satisfied?" topic. It collects data on a wheelchair user's satisfaction with their own wheelchair at a given moment in time. This tool has 16 questions that cover satisfaction with regions of the wheelchair supporting different bodily regions. It also includes questions on other aspects of satisfaction with a wheelchair [1,15].

RESULTS

Responses to our studies indicate that our goal of providing feedback specific enough to spark positive change in wheelchair design and provision was met. The AWMT and the WCQ were the first two outcomes tools to be developed, and these have been used in several comparative studies. There were responses on the part of the wheelchair companies whose wheelchairs had been involved in studies. Free Wheelchair Mission modified the attachment of footplates on their Gen 2 wheelchair. A designer who had been working with Hope Haven, has indicated that our study results were a key factor in his development of the concepts behind the BeeLine wheelchair initiative. Wheelchairs for Kids modified the design of their lateral trunk supports and castor bearings. Several manufacturers indicated that problems revealed in our studies had been due suppliers providing inadequate parts; for example, Whirlwind altered the subcontractor for the caster material when our study revealed failure of castor wheel material. A few manufacturers have not responded to significant problems revealed by our studies. Even in those cases, wheelchair provision was locally improved because our results confirmed a strong hesitancy on the part of several organization to order or use those wheelchair types.

DISCUSSION

Although the snapshot like nature of these outcomes tools does facilitate quick and complete data collection, there are of course limitations. For example, WCQ data does not reveal exactly when the wheelchair was last repaired. However, to some extent, comments can give insight on the long-term reasons for the snapshot responses of those completing the WCQ, WIQ and WSQ. With the AWMT, participants sometimes mention other situation in which they've encountered problems rolling in a certain rolling environment.

Many studies have used visual analogue scale (VAS) format questions to produce data which was then found to be suitable for parametric analysis. However, the lack of an understanding of the scale can be a problem. Early on in the development of these measures, we had tried to use a VAS line without grades. Without anchors, we found that data was obscured For example, for the WCQ, the score for maintenance condition of the push handles was scored higher than other wheelchair components for our study wheelchairs by all raters; however, this finding was obscured because of high variation in the way raters responded to the VAS line. Some raters marked all parts lower on the scale or higher on the scale, or closer together. The push-handles were consistently scored higher, but the actual scores were highly various, therefore ANOVA did not indicate statistical significance. For VAS format pain assessment questionnaires, 'anchors" under the scale help to give a shared understanding of what is meant [17]. However, emoticons indicating grief or pain seemed inappropriate for our purpose. When we paced symbols for school grades as anchors under the line, this solved the problem, presumably because of the shared understanding of these symbols. We have found that school grade symbols also provide a powerful communications tool when addressing the wheelchair manufactures. The statement, "your footplates consistently received an average mark equivalent to a failing grade," has more punch than "your footplates received a mark on the lower end of the VAS scale". Local culturally appropriate grading symbols need to be used as anchors to provide this intuitive understanding. Although we have done validation studies in Kenya and in the US which have slightly different grading symbols, additional validation with grading symbols from different cultures is still to be done.

The data for the WSQ and AWMT come directly from wheelchair users. This is of course essential for any assessment of the function of a wheelchair. Unlike the other measures, the AWMT provides hard data on distance traveled along with accompanying response questions. It may be a bit more difficult to implement because it requires following a protocol to set up the tracks and collect data. With the professional report data, we wanted to answer the "is it broken?" question separately from the "does it suit its user?" A wheelchair that is durable and in good condition may not be at all clinically appropriate. Therapists and other wheelchair professionals with wheelchair experience and training are the people who have base of knowledge to provide data on the maintenance condition of the wheelchair, and on the appropriateness of a wheelchair for its user Professional report data from the WIQ and WCQ is also easier to collect than data directly from users. Wheelchair users may be non-verbal, or children, or only fluent in small mother-tongue languages. In contrast, wheelchair professionals almost universally speak the local language of education which is often a worldwide language. In our areas of study, wheelchair professionals are almost universally fluent in English.

The WCQ and the WIQ could be quickly completed by wheelchair professionals as part of a follow-up wheelchair assessment. At the same time, the wheelchair users who are verbal and able to complete a questionnaire could complete the WSQ. The three tracks for the AWMT could be part of a wheelchair skills training area, and a four- minute roll test on each track could become part of wheelchair selection or follow-up assessments. Data from these validated outcomes tools could be used to as supporting evidence for wheelchair repair, modification or replacement.

CONCLUSION

For a full picture of the function of a wheelchair or wheelchair type, it is helpful to be able to answer four questions: is it broken? Does it suit its user? Does it roll well? Is the user satisfied? The first two are completed by wheelchair professionals, the last two by wheelchair users. All are wheelchair specific and are designed so that feedback is explicit enough to spark beneficial changes to the design of wheelchair parts. To increase discriminatory ability, all use a mixed methods design including qualitative explanatory comments along with quantitative data. All four outcomes tools can and have been used successfully in studies on specific types or makes of wheelchairs, they can be used to improve local wheelchair services.