INTRODUCTION

An assistive robotic manipulator (ARM) is a viable solution for assisting daily manipulation activities for electric-powered wheelchair users with severe upper limb disabilities such as people with spinal cord injury, muscular dystrophy, stroke, or multiple sclerosis. Research studies have shown the benefit of ARM in assisting with daily tasks involving reaching, object handling, and manipulation [1]. ARMs can be mounted on an electric-powered wheelchair to support long-term daily use. One such example is the JACO/JACO2 manipulators (Kinova Robotics, Canada). Most power wheelchairs today feature integrated control, i.e., using a single control interface (e.g., joystick or head array) to operate two or more devices (e.g., power wheelchairs, mobile devices, or seating functions) [2,3]. Wheelchair-mounted ARM could be one of such devices [4]. The ARM users would need to switch between different control modes in order to use a 2-axis wheelchair joystick to control 7 degree-of-freedom (DOF) ARM and the pre-recorded ARM positions [5][6]. Research about ARMs has largely been on developing alternative user interfaces such as touchscreen, voice, gesture, electromyography (EMG), and brain-computer interface (BCI). There has been no research to date that examined how long-term owners of wheelchair-mounted ARMs are controlling their wheelchairs and ARMs daily, nor have the challenges they face, or their experiences been explored. This paper presents the results from an interview study conducted via Zoom with 11 long-term owners of wheelchair-mounted ARMs, where we inquired about and observed their practice in operating their ARMs and gathered their feedback on their perceived effectiveness.

METHODS

ARM interfaces

Figure 1a shows the connection from existing power wheelchair control interfaces [2] to control the wheelchair, seating system, mobile devices or computers, and the ARM via the R-Net, a programmable wheelchair control system, colored in yellow. The proportional control, colored in dark gray, includes the wheelchair and attendant joystick. The non-proportional control, colored in green, includes a head array, Penta switch, sip & puff, and chin joystick which are connected to the Omni display with two environmental control unit (ECU) DB9 connectors, shown in green lines. The Omni display translates the non-proportional inputs to the steering R-Net commands. The seating functions, colored brown, are controlled either by a joystick or the additional toggle switch box often mounted at the end of the armrest. The purple switches are used for changing modes for the wheelchair and ARM. These switches are connected to the wheelchair controller or ARM universal interface via the 1/8" (3.5 mm) mono jack, purple lines. The R-Net ECU input/output module (IOM) translates the R-Net signals to ECU signals to the ARM universal interface, which translates to ARM movements.

Figure 1b shows the control modes that could be used for ARM users. The wheelchair circles through four or more modes: drive, seating functions, Bluetooth, and ARM by pressing the mode switch. The Bluetooth mode allows the user to control mobile devices or computer by using the joystick as a Bluetooth mouse. The ARM mode is the most complicated, which has two modes: basic (B mode) and advanced (A mode). The B mode is further divided into several modes depending on the control interface. For a typical 2D joystick, there are four B modes with B1 for forward/backward and left/right movements, B2 for up/down movements and wrist left/right rotations, B3 for the wrist up/down and left/right movements while keeping ARM fingertips staying at the same location, and B4 for fingers open/close. The A mode is typically divided into A1 for moving the ARM to the four preset positions and A2 for memorizing four ARM positions. The ARM A/B mode switches are also used to move to the retract or extend to home/ready positions via a long press. Other control modes such as drinking mode are included in the user guide [7].

Study Protocol

The study was approved by the Institutional Review Board (IRB) of the University of Pittsburgh (STUDY 22020152). The inclusion criteria were 1) 18 years or older; 2) using a power wheelchair as the primary means of mobility; 3) having upper limb impairments that prevent completion of daily manipulation tasks; 4) owner of an ARM for at least 3 months. After informed consent, they were asked to answer a questionnaire about their demographics, functional limitations, power wheelchair and ARM experience, and general technology attitudes. They were also asked to complete the capabilities of upper extremity (CUE) [8,9]. The CUE is a 17-item self-reported measure of upper limb function and disability and assesses the individual's perceived ability to perform a wide range of activities, including fine and gross motor tasks and activities of daily living (ADLs) such as eating, dressing, and grooming. Participants will rate each item using a 5-point scale (0-4) with 4 indicating no difficulty and 0 indicating unable to complete. They would rate 15 items for left and right hands, respectively, and 2 bi-manual tasks for both arms. The maximum score is 128 with higher scores indicating better upper limb functions. We then scheduled a 1.5-hour interview via Zoom with each participant. During the interview, we requested the participants to demonstrate how they use their ARMs for a grasping task while verbally describing the procedure of transitioning between various modes. As the other topics discussed during the interview are unrelated to this paper, we have not included them in this paper. Two investigators independently extracted the ARM operations from observations as well as themes from participant comments. They then convened to discuss their findings and achieved consensus. 

RESULTS

Eleven ARM owners participated in the study and Table 1 displays their information. The average CUE score is 14 (range: 0-42) indicating very limited upper limb functions. The average ARM use is 3.2 years (range: 3 months to 8 years and 5 months). Eight participants used a typical hand-operated joystick to control their wheelchairs, and the other three used a foot joystick, a chin joystick, and a head array, respectively.

Table 1. Demographics of the participants

ID	Disability	Wheelchair Model	Driving Wheel	Joystick Location	ARM	ARM location	ARM use (years)	CUE Score
								Total	Right	Left	Both
1	Multiple Sclerosis (MS)	Invacare Aviva FX 20	Front	Right	Jaco	Left	3.5	3	3	0	0
2	Spinal Muscular Atrophy (SMA)	Quantum Edge Q2	Mid	Right	Jaco 2	Right	1.2	9	6	3	0
3	Muscular Dystrophy	21st Century Electronics Bounder	Rear	Right	Jaco	Left	8.4	36	25	11	0
4	Arthrogryposis	Invacare TDX SP	Mid	Right	Jaco	Left	1.3	42	21	21	0
5	Spinal Cord Injury	Permobil F5	Front	Right	Jaco 2	Left	1.1	3	1	2	0
6	Amyotrophic Lateral Sclerosis (ALS)	Permobil F3	Front	Foot	Jaco	Right	0.4	1	1	0	0
7	Muscular Dystrophy	Rovi A3	Mid	mouth	Jaco	Left	5	0	0	0	0
8	Cerebral palsy	Invacare Aviva	Front	Head	Jaco	Left	3.5	24	12	11	1
9	SMA Type 3	Permobil F3	Front	Right	Jaco	Left	0.25	17	9	8	0
10	Spinal Cord Injury	Permobil F5	Front	Left	Jaco	Right	8	2	0	2	0
11	Muscular Dystrophy	Permobil F3	Front	Left	Jaco	Right	3	17	7	10	0

Table 2 lists the integrated control interface for the wheelchair and ARM. Nine participants used a proportional joystick as the control interface as ARM control. Five participants used advance mode for preset ARM positions. Two participants were satisfied with the current control interface setup. One participant had two separate control for the wheelchair and ARM.

Table 2. Control interfaces of all participants

ID	Wheelchair	ARM			Problems
	Method to ARM Mode	Direction Control	Basic (B) Mode	Advance (A) Mode
1	WC Mode Switch with a Fingerbot (a Bluetooth button pusher) controlled by iOS voice control	WC Joystick –Proportional	Head Switch	Head Switch	iOS voice control is not responding reliably. Grid accessibility control Longer process to get ARM ready to use
2	WC Mode Switch (left hand)	WC Joystick – Proportional	Switch (right)	Tried but not using	Long learning process on wrist mode
3	Two controls No need to switch	Penta Switch – Non-proportional	Middle Button on Penta Switch	Long Press on B Mode Switch	Satisfied
4	WC Mode Switch (right)	WC Joystick –Proportional	Switch (right)	Not Used	Difficulty in switching between WC driving and ARM control
5	WC Mode Switch (right)	WC Joystick –Proportional	Toggle Switch (right)	Switch (left)	Confusion in wrist mode if flipping upside down
6	Unknown	Foot Joystick –Proportional	Unknown	Unknown	Joystick type limitations
7	Head Switch (left)	Chin Joystick –Proportional	Head Switch (right)	Not used	Not well integrated Could reduce number of mode switch buttons
8	Egg Head Switch (left)	ATOM Electronic Head Array – Non-proportional	Egg Head Switch (left)	Not used	Not reported
9	Long Press on WC Mode Switch (right)	WC Joystick –Proportional	Switch (right)	Long Press on B Mode SW while in B1	Satisfied
10	WC Mode Switch (left)	WC Joystick –Proportional	Micro Switch	Micro Switch	Cumbersome in switching between WC driving and ARM control
11	WC Mode Switch (left thumb palm)	Mini Joystick –Proportional	Switch (left index finger knuckle)	Not used	Learning wrist mode is hard

DISCUSSION

More than half of the participants have been using ARMs for more than 3 years (n=6). Most participants in previous studies [5,10] are either new or first-time ARM users who may not be familiar with the control interface. The CUE score shows that most participants were unable to do meaningful manipulation tasks without the ARM. Due to the severity of the upper limb impairment, a lower percentage of regular wheelchair joystick users than average is expected. The percentage of participants on their power wheelchair control interfaces (joystick: 72.7%, chin: 9.1%, head or foot: 18.2%) is slightly lower than the clinical study from 46 wheelchair user surveys (joystick: 81%, chin: 9%, sip & puff: 6%, other – head or foot: 4%) [11]. During the demonstration, participants were able to complete simple tasks successfully such as picking and placing, drinking, rotating blinds, and adjusting body posture.

The most used modes are the XY and Z/wrist rotation. It requires three ARM basic (B) mode switches to change from the Z/wrist rotation to the XY mode (Figure 1b). Contrarily, it is faster to switch from XY to Z/wrist rotation with only one press. Three users reported using advanced (A) mode and home position could save time on switching between B modes. For example, one participant saved several preset locations for fully left and right as shortcuts used for picking objects or reaching on the side. Another participant has a few preset locations for scratching the head, adjusting the shoulder, or resting the head. The other user pressed and held the home/ready button right after grasping the cup on the table to quickly lift the cup and move the ARM back. Using preset positions reduces B mode changes.

While integrated control interface technology built on the R-Net system, shown in Figure 1a, shows the advantages of a single input device controlling multiple assistive devices. However, as the number of devices and functions increases, mode switching becomes more frequent and sometimes requires additional buttons. Slower mode changes would dramatically reduce the manipulation task performance. Table 2 shows that ARM users have more than two mode switches to circulate the modes for the wheelchair, seat, computer access, and ARM in Figure 1b. If the user accesses the smartphone through switch control, four buttons are required including the A mode. Two users use long-press for switching to A mode on the ARM control interface. Two participants have long-press and double-press as a shortcut to the ARM mode. But this setup requires certificated wheelchair suppliers to modify the R-Net settings. One user has a Fingerbot to press the wheelchair mode button through iOS voice control, which sometimes fails to respond, and the user has to use the grid, another accessibility function on iOS, to change modes.

In comparison to wheelchair driving on flat surfaces which could be performed without switching to other modes, the ARM manipulation requires frequent mode changes due to the limitation on the 2 DOF joystick. Moreover, users reported that for ADL such as opening doors, mode changes are more complicated because of the switching between wheelchair driving and ARM control. One participant using Penta switch for the ARM and a joystick for the wheelchair can control them simultaneously which saves time for switching modes. However, this two-input control may not be a solution for other participants due to their disabilities.

Three users reported the wrist mode is "confusing to begin with" and takes "a really long time to learn how the wrist works." This confusion could be alleviated after more operation time. Due to the small ARM user community, information exchange between users is limited. One user's problem could be solved by another user's setup. For example, the long-press or double-press shortcut to ARM mode can be a solution for the user who has trouble using Fingerbot. One user suggested a single button push to "get my chair into Jaco mode and get my arm into ready mode" which would make the ARM "a lot more appealing." Another participant also suggested a similar method to combine the B mode button and wheelchair mode switch. This indicates that preparing the ARM ready for use may require a significant amount of time and effort for some users. One participant said that it would be "so much easier to move chair without switching back and forth" or during "doing something with the JACO." The inability to simultaneous control of ARM and wheelchair would be one of the barriers for improving efficiency.

STUDY LIMITATIONS AND FUTURE WORK

The study reveals the challenges of control interface complexity that ARM user faces with integrated control interface technology. Because participants demonstrated the ARM under unstructured home environments, the performance was unable to be compared statistically. However, it still shows the potential to incorporate A modes and challenges in B modes. Future studies could focus on the improvement of the integrated control interface technology and the development of preset positions and mode loops for more efficient ARM performance.