Machine Perception for Occupational Therapy: Toward Prediction of Post-Stroke Functional Scores in the Home

Sonya J. Allin

ABSTRACT

We are developing a novel, relatively low cost system to automatically assess upper body performance of stroke survivors during functional tasks at home. The system makes use of calibrated digital cameras stationed about a desktop, as well as objects instrumented with force sensitive resistors. In this paper, we describe a single motor statistic which we have found to correlate well with functional scores on the AMAT. In addition, we evaluate the measurement of this statistic with our home appropriate device relative to the measurement made with a high end commercial motion capture system.

Keywords:

Computer Vision, Stroke Assessment, Home Therapy

  BACKGROUND

Recovery from stroke is a long term process. Stroke survivors have been shown to make significant motor gains several months, and even years after the acute phase of their care, and long after they have been discharged from a hospital (5). Yet outpatient therapy usually spans a relatively short period of time. There is also ample evidence suggesting that people regain motor skills most readily in environments where they feel comfortable and where tasks have real functional benefit (2). Most outpatient therapy, by contrast, takes place in clinical settings. Finally, relatively long sessions of massed, repetitive movements have been shown to yield significant motor improvements (6); most outpatient sessions, however, are comparatively short. There is therefore a gap between the outpatient therapy available to stroke survivors and the therapy literature suggests is required for optimal outcomes.

Here, we present a novel system which makes use of relatively inexpensive sensors, such as cameras and force sensitive resistors, to help fill gaps in therapeutic care. We specifically envision a simple, cost effective system that can be easily integrated into the households of stroke survivors, enabling long term measurement of functioning at home. Ultimately, we see such a system generating continuous and clinically relevant observations of domestic functioning, quantifying long term trends in performance and providing motivating feedback.

Our current system represents a step toward this vision. We have simplified the measurement problem, and are working to create increasingly "natural" measurement environments.

The Measurement System

Figure 1: The experimental setup d (Click image for larger view)

The tools we are currently using to measure upper body motion are illustrated in Figure 1. They are:

1. 8 synchronized digital cameras. These provide views of upper body motion in context.
2. A tabletop, painted ultra-matte green to make foreground segmentation straightforward.
3. Colored markers, which facilitate localization of joints.
4. Objects that are instrumented with arrays of 6mm square piezo-resistive pressure sensors.

Figure 2. At the top, the system tracking colored markers affixed to the joints. At bottom, a region based tracker following the location of limb segments, each of which corresponds to a different color on a jersey. (Click image for larger view)

All cameras are calibrated so as to enable three-dimensional reconstruction of joint locations via a simple weighted least squares technique. In this way, the vision component of the system is similar to a commercial motion capture system, save that it is less expensive and records more visual information about the environment in which motion takes place. This contextual information can play a critical role when it comes to providing interpretable feedback about functional motion for clients and therapists. In addition, the visual system provides a valuable baseline for markerless, region-based tracking algorithms, which are becoming increasingly accurate and robust (3). We are currently exploring the utility of region-based techniques in an effort to move in this markerless direction. See Figure 2 for an illustration.

METHODOLOGY

In initial work, we relate machine perceptions of an upper extremity motor statistic from stroke survivors' motion to functional scores on the Arm Motor Ability Test (AMAT). The AMAT is an American post-stroke assessment with high inter-rater reliability, sensitivity to change, and concurrent validity with other leading assessments, including the Wolf Motor Function Test (1). The motor statistic we focus on here is torso displacement in the direction of a target during a reach. This statistic is known to correlate with degree of disability after a stroke (4). The specific hypotheses we test are: A) Torso displacement during a reach, as measured by a commercial motion capture system, correlates strongly with expert functional scores on the AMAT; and B) Measurements made with our system match those made by motion capture.

To test these hypotheses, we asked a small set of individuals to wear two upper body sets of markers, one of which was visible to a motion capture system (VICON) and one of which was visible to the system of our design. We then requested subjects perform the following task, from (4): With the right hand, move a cone at one of three initial locations off a 9 cm. post to a location in front of the sternum.

Three individuals with hemiparesis and three controls participated in the study. All stroke subjects were first evaluated while performing the AMAT by an occupational therapist. AMAT scores, as well as demographics of the subject population, are shown in Tables 1 and 2. Estimates of scores on the AMAT are reported; in the future, when scores are verified by multiple therapists, we will report numeric scores.

Subjects with hemiparesis were asked to attempt the reaching task with their impaired hand and arm, but were told to recruit the opposing hand when task demands were found to exceed the capacity of the impaired arm. The task was executed twelve times, three for each starting position with the impaired arm and once at each location with the uninvolved arm. Starting locations were located directly in front of the torso; one 12 cm from the table's edge, one 22 cm and the final one 32 cm.

Optical 3D tracks were recorded simultaneously with a VICON motion capture system and our system. The VICON sampled the position of markers 60 times a second, and was down-sampled to match the speed of the digital video. Data analysis was performed only on portions of motion corresponding to reaches from the start position to the cone. To locate these portions, raw movement data was passed through a median filter of width 15 and a simple segmentation scheme was applied. The beginning of a reach was said to correspond to the point when velocity surpassed 5 percent of its maximum. The end of the reach was located at the next valley in the velocity profile. This corresponded to a zero crossing in recorded acceleration. Torso displacement was then calculated based on the tangential displacement of the markers at the shoulder during the reaching phase of the movement, i.e.:

RESULTS

We first examined the average torso displacement for each subject at each target distance; this is illustrated at the right of Figure 3. As in (4), all subjects exhibit a roughly linear relationship between target distance and the degree of torso motion. The greatest degree of torso motion is consistently used by the most impaired participant in our study, and the slope of the linear fit is slightly steeper for this individual than for anyone else. The individual with a moderate functional score uses slightly less trunk compensation with much less variance. It appears that she has learned a strategy that utilizes the trunk, and is fairly consistent insofar as the amount of trunk she requires for repeated moves. The least impaired individual uses an amount of trunk motion that is virtually the same as the age matched controls. We also correlated torso displacement at the far target with our estimates of AMAT scores; this yielded a coefficient over 0.95.

To test our second hypothesis, we report a comparison between summary statistics based on the VICON and summary statistics based on our system. More specifically, we approximate one result for torso displacement at each target distance for a single subject. Mean deviation between the summary statistics relating to the torso is 10 mm, with a standard deviation of 20 mm. The slope of the linear fit is 0.12 for the VICON; for our system the slope is 0.11. This result is illustrated at the bottom of Figure 3.

DISCUSSION

Our long term goal is a measurement system that can be easily integrated into the household of stroke survivors. This system will generate continuous and clinically relevant observations of functioning as it takes place at home. In this paper we make a first step toward this goal. More specifically, we present a single motor statistic that correlates well with the functional scores of stroke survivors. We also demonstrate the capacity to measure this statistic accurately with a lightweight, cost effective system which utilizes digital cameras instead of IR cameras. This system, as it images the visual spectrum, has the potential to enable easily interpretable feedback, and is appropriate for the home in the way IR systems are not. Moreover, our system has the potential to ultimately enable markerless tracking, which may allow robust, quantitative measurement of functioning in realistic situations.

REFERENCES

1. Kopp, B., Kunkel, A., Flor, H., Platz, T., Rose, U., Mauritz, K, Gresser, K., McCulloch, K, and Taub, E. (1997). The Arm Motor Ability Test: Reliability, validity and sensitivity to change." Arch. Phys. Med. Rehab, 78, 615-620.
2. Ma, H., Trombly, C., and Robinson-Podolski, C. (1999). The effect of context on skill acquisition and transfer." Am J Occup Ther., 53(2), 138-44.
3. Ramanan D. and Forsyth., D. (2003). Finding and Tracking People from the Bottom Up". Computer Vision and Pattern Recognition (CVPR).
4. Roby-Brami, A. Feydy, A. Combeaud, M. Biryukova, E. Bussel, B. and Levin, M. (2003). Motor compensation and recovery for reaching in stroke patients. Acta Neurol Scand., 107(5). 369-81.
5. Taub, E. Miller, N. Novack, Cook, Fleming, Nepomuceno, Connell, and Crago, J. (1993). Techniques to improve chronic motor deficit after stroke." Arch Phys Med Rehabil, 74. 347-54.
6. Taub, E. Uswatte, G. and Pidikiti, R. (1999). Constraint-Induced Movement Therapy: a new family of techniques with broad application to physical rehabilitation". J Rehabil Res Dev, 36, 237-51.