BACKGROUND

Several approaches have been used to combat the public health concern of physical inactivity in the general population (Kirwan, 2012; Kooijmans, 2013; Godin, 2011). Interventions that focus on behavior modification have identified many techniques that have an overall larger effect on increasing activity levels. One tactic, in particular, utilizes self-management skills to monitor activity levels and decrease sedentary behavior (Dishman, 1996; Michie, 2009).

Using technology to objectively monitor physical activity (PA) is a popular and useful tool amongst the ambulatory population; however, most activity monitors that are currently commercially available cannot accurately measure the activity of wheelchair users (Hiremath, 2011; Nightingale, 2015). Investigators at the Human Engineering Research Laboratories developed a custom Physical Activity Monitoring and Sharing System (PAMS) that enables manual wheelchair users (MWUs) to track personal PA parameters in daily life.

PAMS consists of a tri-axial accelerometer worn on the upper arm over the triceps (Wocket), a gyroscopic-based wheel rotation monitor (GWRM) mounted on the wheel (Hiremath, 2013), and a smartphone application that communicates data between the two devices. The GWRM holder is zip-tied to the spokes of the wheel, and the single buckle design enables users to insert and remove the GWRM with one hand. Figure 1 depicts the Wocket, GWRM, and the smartphone application; Figure 2 shows the GWRM being inserted into its holder, zip-tied to the wheel.

In addition, the app has other aspects to promote PA (i.e., goal setting, data summary, social interaction). The algorithms for the following parameters were developed and evaluated for validity: distance (miles), energy expenditure (EE; kcal), time being active (min), push count, and push efficiency (feet/push). The mean signed percent error for distance was 0.61% ± 2.43%, 7.4% ± 31.9% for EE, 37.5% ± 22.1% for the time being active, and -4.54% ± 15.01% for the push count (Wongsirikul, 2014).

PURPOSE

The purpose of this study was to evaluate the field-based usability of a custom physical activity monitoring and sharing system among manual wheelchair users.

METHODS

Subjects

Ten people with SCI were recruited to participate in the study. Subjects used a manual wheelchair as their primary means of mobility (>80% of ambulation), had experience using a smartphone, and were between 18-65 years old. Subjects were excluded if they could not tolerate sitting for longer than 2.5 hours, or if they had an active pelvic/thigh wound.

Procedures

The study consisted of a testing session in the lab and an in-home session followed by a final lab visit to gather comprehensive feedback regarding the home trial. The testing session took place at the University of Pittsburgh’s Rehabilitation Science and Technology department. Upon arrival, subjects signed the consent form and completed questionnaires regarding demographics, PA habits and smartphone usage.

Afterward, subjects viewed three videos related to PAMS, including: the Wocket, the GWRM, and the PAMS app. The first video introduced PAMS, how it works, and what it is used for. The second video explained how to use the Wocket and GWRM. The last video explained how to use the PAMS app on a smartphone.

Upon completion of the lab session, subjects began the 6-day home trial, where they were instructed to use PAMS daily, and the app at least three times per day. At the end of the home trial, subjects completed final questionnaires and an interview regarding their overall experience using PAMS and the app.

 

Data Analysis

Descriptive statistics were used to analyze data from questionnaires and smartphone logs. Content analysis was used to extract common themes and problems identified by subjects during interviews.

RESULTS

Participant Activity Summary

Table 1 summarizes the PA parameters of subjects over the 6-day home trial. Subjects reported perceived accuracy ratings for each parameter. Nine subjects thought the app was accurate at predicting distance (miles) and push count. Seven subjects thought the app was accurate at predicting EE (kcal) and time being active (min.). All ten subjects thought the app was accurate at predicting push efficiency (feet/push).

General Usability

In terms of ease-of-use, after the 6-day home trial, most subjects (n=9) thought it was easy to put on, take off, and recharge the Wocket. Most subjects (n=8) were satisfied with the Wocket size and only one person thought the Wocket felt uncomfortable. In regards to the GWRM, most subjects (n=9) reported that it was easy to put on and remove from its holder. In addition, most subjects (n=9) were satisfied with the size and found that the GWRM was easy to recharge. Finally, in relation to the PAMS app, most subjects (n=9) were satisfied with their overall experience and found the app easy to use on a daily basis, easy to navigate, and easy to understand.

In terms of the perceived usefulness of the PAMS app, the highest rated feature was the Goal Setting feature; all ten subjects thought this feature was useful. Most subjects (n=8) thought the Summary feature was useful. Half of the subjects reported that the Social Feature would be useful, while others rated that it would only be “somewhat useful” (n=2) or “not useful” (n=3). The highest rated PA parameters were Distance, Push Count, and Push Efficiency. All ten subjects thought these parameters would be useful. Most subjects reported that EE would be useful (n=9) and Time Being Active would be useful (n=8). In addition, the majority of subjects (n=7) rated the Distance parameter as their favorite.

 

Table 1: PA Parameters Log For Each Subjects Over 6 Days

	EE (kcal)	Distance (miles)	Push Count	Time Being Active (min)	Push Efficiency (feet/push)
P1	4370.99	5.18	3993	56	6.85
P2	3524.02	3.53	4276	88	4.36
P3	8229	9.09	7243	185	6.63
P4	5377.69	1.90	3174	150	3.16
P5	4021.44	3.52	5388	612	3.45
P6	829.33	0.76	731	91	5.50
P7	6138.5	5.40	10224	22	2.79
P8	6837.37	4.04	3425	912	6.23
P9	1977.18	0.98	2195	10	2.36
P10	9665.78	3.81	3990	101	5.04

App Usage

Across all subjects, the average amount of time that the sensors were connected over 6 days was 35.1 ± 14.0 hours. The average percent disconnection rate during the first 8 hours of use was 25.4% ± 13.4%. The app quit, 2 ± 1 times/day. Based on the data recorded from the smartphone logs, subjects viewed the first page of the app for 12.2 ± 13.4 min/day during the week, and 6.8 ± 10.9 min/day during the weekends. Subjects opened the app 11 ± 9 times/day during the week, and 8 ± 7 times/day during the weekends. When combining the summary and social features, the most commonly viewed PA parameters were EE (16.2 ± 78.8 sec/day) and Distance (12.1 ± 65.8 sec/day).

System Usability Scale

The results from the 10-item System Usability Scale (SUS) are summarized in Table 2. To calculate the final SUS score, each item had a contributing factor ranging from 0-4. All odd-numbered items were subtracted by 1; all even-numbered items were subtracted by 5. All scores were summed, then multiplied by 2.5 to find the overall SUS score. PAMS scored an 85.5 ± 12.9, indicating very high usability and learnability.

Table 2:  PAMS System Usability Scale Ratings

PAMS System Usability Scale
1. I think that I would like to use this system frequently.
SD 0	Disagree 0	Neutral 1	Agree 3	SA 6
2. I found the system unnecessarily complex.
SD 6	Disagree 3	Neutral 1	Agree 0	SA 0
3. I thought the system was easy to use.
SD 0	Disagree 0	Neutral 0	Agree 5	SA 5
4. I think that I would need the support of another person to be able to use this system.
SD 5	Disagree 4	Neutral 1	Agree 0	SA 0
5. I found the various functions in this system were well integrated.
SD 0	Disagree 0	Neutral 1	Agree 4	SA 4
6. I thought there was too much inconsistency in this system.
SD 3	Disagree 5	Neutral 2	Agree 0	SA 0
7. I would imagine that most people would learn to use this system very quickly.
SD 0	Disagree 0	Neutral 1	Agree 2	SA 7
8. I found the system very cumbersome or burdensome to use.
SD 7	Disagree 2	Neutral 1	Agree 0	SA 0
9. I felt very confident using the system.
SD 0	Disagree 0	Neutral 0	Agree 5	SA 5
10. I needed to learn a lot of things before I could get going with this system.
SD 6	Disagree 3	Neutral 1	Agree 0	SA 0
Note. SD = Strongly Disagree; SA = Strongly Agree

Qualitative Analysis

The feedback gathered from the post-home trial interview was classified as positive findings, negative findings, and suggestions. The most common positive response regarding the Wocket was that it was easy to use and understand. Conversely, six users reported that the Wocket armband slipped off when propelling or cycling. The most common suggestion was to offer a variety of armband sizes with stronger Velcro. In regards to the GWRM, the most common positive response included its ease of use and that it did not interfere with propulsion. The majority of subjects provided constructive criticism about the general design. These subjects expressed dissatisfaction with connecting the holder via zip-ties and the color. Finally, the most common positive comments regarding the app was for the Push Count and Push Efficiency parameters. Six subjects expressed strong interest in the social feature. The most common negative comment pertained to the speed/connection of the app. The most common suggestion was to add a “net calories” parameter.

DISCUSSION

Overall, the results from the post-home trial indicate that the Wocket, GWRM, and PAMS app were easy to use. A score above 68 on the SUS is considered to be above average (Brooke, 1996) and the average PAMS score was 85.5 ± 12.9. To corroborate this rating, none of the subjects reported having to reference the instruction manual during their home trial.

The issues found with PAMS were identified by verbal feedback from the subjects and by reviewing the smartphone logs. Subjects found that the Wocket battery life was too short and that the design was too fragile. In addition, subjects suggested redesigning the GWRM in terms of color (black instead of white to blend in better and prevent a “dirty” appearance) and functionality (to make it more robust and to allow people with tetraplegia to handle the device without assistance).

The smartphone logs indicated that subjects spent the majority of their time viewing the main summary page of the app. The summary page provides a comprehensive review of their current activity status and updates every minute. In addition, the summary page provides graphs and trends for each parameter. Subjects viewed the Weekly Distance and EE summaries most often; however, these two graphs are the first two pages within the summary feature and it is possible that subjects either did not want or find it necessary to view the other summary graphs. Despite this finding, subjects reported the usefulness of the summary feature, particularly at the end of the day, which enabled them to compare their current activity parameters to previous days. Additionally, more than half of the subjects expressed strong interest in using the app’s social feature. Within the social feature, users are able to share and compare PA parameters with other users.

Regarding the PA parameters, half of the subjects reported high interest in the push count and push efficiency. One subject stated, “I try to monitor my push rate because if I push too fast or too quickly I would be at risk for repetitive stress and injury…and the fact that I can actually see it in numbers is great!” Another subject described the parameter as “something that makes me think about my pushing on a daily basis… [It’s] kind of using what you were taught a while ago; how to push, sometimes let it glide a little instead of always going and going and overusing your shoulders.”

The home trial also exposed differing subject intentions regarding the use of PAMS. Some subjects viewed PAMS as a performance-measuring tool; for example, they were only interested in using PAMS when engaging in PA and found it unnecessary to use when sedentary. One subject explained, “I would use [PAMS] right before I am ready to do a workout to monitor and see how much improvement I need in that area.” Other subjects viewed PAMS as a general activity monitor resulting in continuous use throughout the day, regardless of active or sedentary behaviors. Furthermore, three subjects indicated that they would like to see a “net calories” app feature (the difference between calories expended and calories consumed), where they would have the option of tracking food intake. Another subject suggested implementing a feedback cue that would remind the user to do more PA based on his/her current status relative to their personal goal. These suggestions particularly reflect the potential use of PAMS as an everyday weight management tool.

Limitations

Subjects were provided with specific guidelines for using PAMS at home. This was to prevent damage to the sensors, and to encourage frequent use of PAMS so that quality feedback could be obtained. Given that subjects may not have been using PAMS as naturally as they typically would have, certain assumptions regarding subject behaviors cannot be made.

CONCLUSION

In general, subjects regarded PAMS positively and had high levels of satisfaction after one week of use at home. The app was easy to use and provided beneficial PA information that future MWUs may be able to use to maintain or attain a healthier lifestyle. Any problems that were identified by subjects or investigators did not affect the overall performance of PAMS. Further refinement based on subject feedback should be done in order to make this a feasible tool for monitoring PA levels for MWUs.