Introduction

For the 5.5 million US adults who use a wheelchair,[1] their device represents their ability to Interact with the world and to perform many of their activities of daily living. Since mobility is so closely tied to quality of life,[2] it is important that these individuals receive an assistive technology device that meets their needs. The process of matching an individual with an assistive technology device is complex, multifaceted, and requires specialized clinical knowledge.[3] The Rehabilitation Engineering and Assistive Technology Society of North America (RESNA) Wheelchair Service Provision Guide[4] is a process framework that describes the complex system of providing a wheelchair to individuals with mobility impairments. The process includes eight components: 1) Referral; 2) Assessment; 3) Equipment Recommendation and Selection; 4) Funding and Procurement; 5) Product Preparation; 6) Fitting, Training and Delivery; 7) Follow-up, Maintenance and Repair; and 8) Outcome Measurement. Taking place over the course of years, these steps require coordination between several distinct groups of stakeholders with wildly different backgrounds and conflicting goals/incentives. It is the very nature of complex systems, such as this, to have many potential points of failure, and determining priorities for system improvement is challenging.

The Disability and Rehabilitation Research Projects (DRRP) Program: Research on Healthcare Policy and Disability, housed at the University of Pittsburgh, is a collaboration of many stakeholder groups (i.e. academia, manufacturer, supplier, clinician, disability & advocacy groups). They are conducting a series of interrelated research projects to inform discussion on what policy changes might be most appropriate and impactful in improving wheelchair (i.e. Complex Rehabilitation Technology (CRT)) service provision. Investigators have completed a scoping review of the literature examining the barriers and facilitators to obtaining CRT under current policies.[5] This review discovered several themes that seemed stratified by stakeholder group. The second phase of the project was to further investigate how to affect policy to improve wheelchair service provision and to examine the differences and similarities between stakeholder groups. The purpose of this paper is to describe the development of the online questionnaire to be administered to the different stakeholder groups and report a preliminary analysis of a subset of the data.

Questionnaire development

Methods

The questions for the questionnaire were based on themes that emerged from the scoping review of the literature completed previously, which examined the barriers and facilitators to obtaining CRT under current policies.[5] Questions were targeted at elucidating attitudes of identified stakeholders regarding the overall satisfaction of the process of obtaining a wheelchair.  The questions were initially written by a subset of the project team and refined over the course of 3 escalating reviews of subject matter experts (SMEs). Each round of reviews included a greater number of SMEs who were increasingly peripheral to the study team. Details of these reviews are in Table 1.

Table 1 : Rounds of question review

Review Round	1	2	3
Group consulted	Project Team	DRRP Team	External SMEs
Group makeup	4 Clinicians 2 Students	4 Manufactures 3 Clinicians 2 Suppliers 2 Undisclosed 1 Consumer/Caregiver 1 Payer	8 Clinicians 4 Consumer/Caregivers 4 Manufacturers 2 Consultants 2 Suppliers 2 Payers
Format	Questions rated for clarity and relevance. Free-response comments	Questions rated for clarity and relevance. Free-response comments	Questions rated for clarity and relevance. Free-response comments
Question review criteria	Consensus	Clarity/relevance: < 80% of respondents rating below 3/5 Relevance: Respondents indicating relevance to less than 70% of stakeholder groups on average	Clarity/relevance: < 80% of respondents rating below 3/5 Relevance: Respondents indicating relevance to less than 70% of stakeholder groups on average

Prior to Round 1, 3 SMEs from the DRRP Executive Team who were knowledgeable of the themes from the scoping review on CRT service provision, drafted the initial set of questions for the online survey. For Round 1, the project team reviewed the initial set of statements for the online survey created by the executive team in the domains of clarity and relevance. Clarity was defined as avoiding jargon, direct and easy to understand and relevance defined as appropriate, important, and applicable. The responses to the domains were 'Yes' and 'No' and if 'No', a text box was present for open ended comments (e.g, rationale and proposed changes to statements). These domains and definitions would be carried on into the next two rounds. The purpose for round 1 was to provide free response comments along with consensus of which statements to include in the 2nd review round, In rounds 2 and 3, the project team delivered the proposed survey questions in a Qualtrics (SAP SE, Walldorf, Germany) form and asked SMEs to rate the statements on the clarity and relevance on a 4-point numerical Likert scale, and to identify which stakeholder groups they believed that the statement was relevant to. For analysis, the Likert responses were dichotomized to "clear/relevant" for responses of 3 or 4, or "unclear/irrelevant" for responses of 1 or 2. All responses were explored using descriptive and visual analysis. The Likert responses were compared against established benchmarks[6] that were set a priori. In preparation for further analysis, an author who was not involved in the initial development of the questions mapped the fully developed statements to the components of the RESNA Wheelchair Service Provision Guide.[4]

Results

Questions 5, 6, 7, 13, and 18 were flagged as being not clear but relevant based on our criteria and were re-worded, based on the comments from the participants, for additional clarity. No questions were identified as having low relevancy across all stakeholder groups. Questions 11, 17, and 18 were identified as having low relevancy for specific stakeholder groups. The authors discussed the potential for using branching logic to hide questions from irrelevant groups but felt that the potential perspective outweighed the potential for irrelevancy. The 19 finalized questions and their mapping to the RESNA wheelchair provision process are listed in Table 2. Responses are plotted on a 6-point Likert scale with the labels "completely agree, mostly agree, somewhat agree, somewhat disagree, mostly disagree, and completely disagree."

 

Table 2 : Finalized statements mapped to RESNA wheelchair service provision guide

Survey Questions	RESNA Provision Process
1. The process for obtaining a wheelchair is easy to understand.	Overall
3. There is good communication and collaboration between all parties (consumer, family, clinician, physician, manufacturer, and/or supplier) involved in the wheelchair provision process.
5. Wheelchairs are typically provided in a reasonable amount of time.
7. Advocacy efforts for wheelchairs are well coordinated across stakeholder groups.
8. Most clinicians are aware of the service delivery process for wheelchairs.
	Referral
9. There is sufficient availability of knowledgeable clinicians who address wheelchairs.	Assessment
4. Equipment for trial and demonstration purposes is readily available prior to choosing a specific type of wheelchair.	Equipment Recommendation and Selection
10. There is sufficient availability of knowledgeable suppliers who address wheelchairs.
11. There is sufficient availability of knowledgeable manufacturer representatives who address wheelchairs.
2. Insurance coverage policies for wheelchairs are complicated.	Funding and Procurement
6. When there is an appeal for a wheelchair, the process is easy to navigate.
15. Wheelchair coverage policies (e.g., insurance) consider a person's context (i.e., natural and built environment, community, and culture).
	Product Preparation
19. Set-up and training on the use of a new wheelchair is routine.	Fitting, Training and Delivery
16. The process for maintaining (i.e., preventative/ongoing) a wheelchair is easy.	Follow-up Maintenance and Repair
17. The process for repairing wheelchairs is easy.
14. Current wheelchair outcome measures (i.e., satisfaction & effectiveness) are well defined and established	Outcome Measurement
12. Most wheelchairs are durable.	Device
13. Most wheelchairs are high quality in design and manufacturing.
18. Wheelchairs are frequently in need of repair.

Preliminary data collection

Sampling strategy

We utilized a snowball sampling strategy in which the project team directly contacted 42 distinct organizations representing each of the targeted stakeholder groups. Representatives of these groups were emailed a standardized script inviting participation in the questionnaire and requesting that the email be forwarded to the members of the respective organization. These requests were followed up by personal communications to verify that the email had been passed on or to remind them to do so. The online questionnaire was also advertised at various professional conferences, webinars, and social media outlets. Data collection began on November 27, 2021 and will close on or around March 4, 2022.

Analysis

For analysis, responses were dichotomized as feeling that a given aspect of the wheelchair provision process works well, or not. This was, generally, operationalized as responses of either "completely agree" or "mostly agree." In the case of question 18, in which a negative response was favorable, "completely disagree" or "mostly disagree" were coded as a positive perception. Trends in the data were explored through descriptive and visual analysis.

Results

Preliminary data were pulled on January 10, 2022 and included 736 valid responses, 38% from clinicians, 30% from consumers/caregivers, 27% from suppliers, 4% from manufactures, 1% from payers, and less than 1% did not disclose their stakeholder group. Responses were relatively evenly spread across the United States, representing all states except Vermont, Rhode Island, and Alaska. In addition, 12 responses were received from Canada, 2 from Mexico, 1 from El Salvador, and 1 from Sweden. Averaged across all questions, 3% of responses were "completely agree," 14% "mostly agree," 22% "somewhat agree," 22% "somewhat disagree," 21% "mostly disagree," and 18% "completely disagree." Across all questions, payers responded with "completely agree" or "mostly agree" 28% of the time, suppliers 22%. clinicians 17%, manufacturers 17%, and consumers/caregivers 12%.

Discussion

We describe the development of an online questionnaire based on a scoping review of CRT provision and report results from a portion of the full dataset. We intend to perform a more robust analysis of potential significance in the findings once data collection has completed. Even then, the project team believe that our preliminary analysis of this partial dataset has yielded meaningful and valid insights. For example, only 17% of all stakeholders in the wheelchair provision process have a generally positive impression of that process and the outcomes that it is capable of achieving in its current state. We also note that consumers/caregivers have a meaningfully lower perception of the process, being the only group that is well below the overall average. The field has to do a better job with a person-centered approach to make sure that the consumer/caregiver have an understanding of what is expected. In contrast, the payers report a meaningfully higher perception of the process than the other groups, but the limited sample size (10 payer responses) make it difficult to fully trust. Anecdotally, these findings align with stakeholder feedback that the SMEs on the team have heard for decades with respect to the wheelchair service delivery process.

Conclusions

There is clear need for extensive reform in the CRT service delivery process within the United States. No group of stakeholders who are a part of the process has a positive perception of how it plays out in their lives. Further research is necessary to identify highest priority target for improvement efforts but the current system is untenable and requires change.