INTRODUCTION

Mirror visual feedback (MVF) was initially utilized by Ramachandran and Rogers-Ramachandran in 996 to alleviate pain and paralysis in amputees [1] and was specifically designed to trick the patient’s brain while transforming their mind [2]. When patients with chronic pain issues anticipate movements to be painful, mirrors help deceive them into thinking that they are not experiencing pain via dynamic feedback to their brains [3]. McCabe, Haigh and Blake have stated that "mirrors and vision are inextricably linked, and the reflected image appears strikingly believable even if deliberately distorted” [4]. Using observation of the uninvolved limb helps to "drive proprioception" in the involved limb, thereby normalizing the "movement process” [5]. Thus, the use of the mirror gives the patient the "impression of having two normal limbs” [6]. The concept behind this "visual input" modality is that it helps patients re-educate, or re-introduce to their altered higher processing neural networkings, a normal relationship between a physical movement and the sensory feedback it provides [2].

Several studies have been conducted looking at the effectiveness of MVF therapy with Chronic Regional Pain Syndrome (CRPS) patients. McCabe et al. studied eight subjects, hypothesizing that a disturbance between motor and sensory cortices was the cause of their CRPS [4]. The investigators found that MVF was helpful for pain reduction in the early stage of CRPS and for stiffness in the intermediate stage; no changes were seen with late stage CRPS [7]. Tichekaar and colleagues reported that mirror therapy, in conjunction with cognitive behavioral therapy, "plays a positive role" in patients with CRPS [8].

A pilot study conducted by Sato and colleagues that employed a virtual reality MVF system to observe pain intensity in patients with CRPS, showed MVF to be a "promising alternative treatment" for this population, though further investigation into such technologies is critical [9]. Altschuler & Jeong described a case report of a patient following a post-operative distal radius fracture who initially was only able to extend her wrist with electrical stimulation; following approximately two months of MVF both in and outside of the clinic, the patient was able to regain thirty-five degrees of active wrist extension.

In order to successfully incorporate MVF into a treatment plan, a mirror and a box are needed. These can easily be constructed from materials found at home, acquired at hardware stores, or purchased as a complete set specifically for MVF from therapy catalogs. Therapy is initiated by asking the patient to describe, with vision occluded, his or her perception of the painful limb. The patient is then asked to sit at a table and position him or herself with the involved extremity behind the mirror (or inside the box) and hidden from view. The uninvolved extremity should be placed in front of the mirror so as to make the reflection look like the contra-lateral limb. Before any movement or exercises, the patient should simply look at the limb in the mirror and focus on engaging in the belief that the mirrored image is in fact his or her contra-lateral limb. The patient is then instructed to perform gentle movements with the uninvolved extremity in front of the mirror while continuing to focus on the mirror This creates the illusion that the movements are occurring bilaterally. At this point, the therapist can watch to see what the hidden involved extremity is doing. McCabe and colleagues have suggested that the way the involved limb is moving is of little importance, as long as it is "bilateral and synchronized” [6].

BACKGROUND AND MOTIVATION

Robotic Based Stroke Therapy (RBST) has been proven to be beneficial for patients to learn the required perception-action skill. Success of using RBST to help patients relearn other task necessary for Activities of Daily Living (ADL) has been limited [10, 11]. Observational learning of motor skills, however, has been shown to produce transfer across limbs and generalization across muscle groups in the same limb [12], as well as transfer to perceptual tasks [13,14]. Therefore, observational learning may offer a greater benefit regarding transfer to ADLs in comparison to RBST. 

Research over the past ten years suggests that action-observation training improves upper-limb function in children with unilateral cerebral palsy [15]. Human imaging work (PET, fMRI) has revealed a mirror-neuron network (pre-motor cortex, parietal lobe, temporal lobe) that supports our ability to learn through action imitation and action observation [16-18]. This direct link between human visual perception and human action execution is diminished [19] or disappears when non-anthropomorphic motion is observed [20-22]. Recent research started to investigate how action-observation protocols linked to the mirror neuron system may benefit recovery of function after stroke and enhance clinical training protocols to produce transfer of recovered function from the clinic to ADLs [23-25]. Some promises regarding the use of action-observation as a means to tap into the mirror neuron system in the clinic have come from training protocols that use video to help patients mimic ADL [26] and virtual reality systems that transfer the motion of the patient’s real arm to a set of virtual arms in real time [27], [28].

Recent research shows that Virtual Reality Based Therapy (VRBT) is feasible and a cost-effective means of bringing therapy to stroke patients. Furthermore, VRBT has a “wow factor” not experienced through conventional therapies and is considerably safer than RBSTs [29]. Use of remotely monitored virtual reality videogames, regular use of tele-rehabilitation appears to produce improved hand function and forearm bone health in adolescents with chronic disability. Improved hand function appears to be reflected in functional brain changes [30]. Additionally, virtual/augmented reality exercise programs have been shown to improve the ambulation ability of subjects with cerebral palsy [31]. The use of virtual or augmented reality in physical therapy has produced some encouraging results which demonstrate the potential of using these techniques for all types of physical therapy, including stroke therapy [31-34].

For stroke, motor recovery of the upper extremity plateaus in the first year after the initial incident according to clinical and biomechanical measures. However, there is some evidence that the time needed for recovery is not limited to one year and that additional practice can improve mobility in both the sub-acute and chronic phases following a stroke [35]. The latter leads naturally to the concept of “functional potential”. The additional recovery is statistically significant and provides a baseline effect with which to work. However, patients often only achieve small levels of improvement to their mobility.

Given the limitations of recovery, it is necessary to find novel tools and methods for retraining the motor system. The current paper explores combining action observation and virtual reality into virtual reality mirror therapy for improving upper limb mobility for post-stroke patients.

DEVELOPMENT OF THE ARWED

Virtual-reality systems aim to integrate additional, computer-generated information within the human perception of the physical world [31]. In our case, human vision is targeted and the aim is to modify visual input of some object within the visual field of the subject. If the original object is an upper limb with a limited range of motion, then the modified object is a realistic 3D model of the subject’s upper limb. In order to perform the virtual/modified perception, the object needs to be identified and tracked within the visual field. Its kinematic structure needs to be determined to calculate its motion, all in real-time. For this research, mapping of the active motion of the unaffected limb onto the affected one was used. In order to do so, the subject’s unaffected, healthy hand and forearm was tracked using a Leap Motion sensor, and the resulting tracking data was mapped onto the affected arm and forearm in the virtual environment (see Figure 1) [36, 37].

The ARWED system can be used as a standard desktop system or as a head-mounted virtual reality system using Oculus Rift. The goal was to have fast tracking and rendering so that the virtual limb tracks seamlessly the movement of a real limb. Some challenges to this are: quick rendering of a photorealistic representation of the subject’s affected limb; motion smoothness; shadow casting; calculation or estimation of appropriate translation, orientation and scaling parameters of the virtual limb to smoothly align with the object(s) in the real environment; discontinued rendering of the virtual limb in the presence of occlusion or sporadic loss of tracking data to reduce/eliminate the loss of persistence for the user, among others. Some of these problems are detailed in [36, 37].  

METHODS AND EXPERIMENTAL RESULTS

Experiment on the 3D model to assess the performance of ARWED

One key point in the development of the ARWED device is to assess whether a 3D model of a limb can create the desired priming effect in the subject, and if this effect is any different depending on the hand representation and accuracy presented to the user. In order to answer this question, a series of experiments were developed to examine automatic elicitation of hand actions in individuals without any known loss of neurological function. The experiments progressed from static priming conditions (Experiment 1 described below) to dynamic priming conditions (Experiment 2 described below) [36, 37].

Experiment 1 required participants to rapidly identify gestures generated by a human model or generated by a 3D model of the human arm. The gestures were presented as static pictures. Since the ARWED device blocks vision of the hand and arm, participants’ hands and arms were blocked from view when producing the action. Reaction time was measured, based on surface EMG and a key pressing action provided a measure of response priming. The three different actions, a pinch, hand open/hand closed, and pointing, shown in Fig. 6, were presented as test stimuli. The task required participants to correctly identify a specific response over a series of 40 trials in a block. Each block consisted of 80% compatible actions and 20% incompatible actions, with participants only responding to compatible actions by pressing a response key. For example, if the block consists of the hand open gesture, catch trials might be two fingers raised or the thumb up. The results from Experiment 1 showed that the reaction times (both EMG and key press) and percent correct responses were similar for all four gestures for both the 3D model and human model. This implies that the 3D model limb primes the human motor system in a manner consistent with the human model.

Experiment 2 utilized the same hand gestures as Experiment 1 but presented as dynamic hand gestures; i.e., participants saw a hand pinch and then were asked to replicate the action. The same four gestures were used and presented as a video. Reaction time was measured with surface EMG (first dorsal interossei muscle), and movement time was measured from hand kinematics (see Figure 2). Hand kinematics was recorded with a 3D motion capture system, and movement time was based on motion of the?

Each action was presented in a block of 40 trials with 10% the trials consisting of catch trials. Within each block of trials, the actions were completed with four different movement times. Participants were required to both imitate the action and the rate of the action. The results from Experiment 2 showed that the reaction time and movement time values between the 3D model limb and the human model are similar between the two conditions. This implies that the dynamic motions of the developed 3D model limbs mimic those of a human model.

Experiment to assess the significance of ARWED as a tool to improve upper-limb mobility for post-stroke patients

After initial testing outside the virtual reality environment, one post-stroke volunteer was recruited for four training sessions (once a week with a duration of 30-40 min.). The patient was 55 years of age and was diagnosed with a right basal ganglia hemorrhagic stroke more than three years ago, causing residual left-sided hemiparesis without residual cognitive deficits. A series of protocols for the application of the ARWED system and experiment on its significance as a tool to improve the upper-limb mobility were developed. The stroke volunteer was required to wear virtual reality goggles and sEMG sensors were attached to the hand and arm (see Figure 3 and Figure 4).

The participant was asked to start with their unaffected hand in a neutral position, palm down resting on a table and aligned with the body midline. This experimental task extends the experiments described in the previous sub-section, by having the patient use their unaffected upper extremity for reaching, grasping and moving objects, as well as pushing away flying objects while wearing the Oculus Rift goggles. The experimental setup allowed for priming to be examined in terms of reaction time, which was measured with surface EMG. This task allowed for a more detailed analysis of the reaction time to determine if the 3D model presentation of the limb in virtual reality can generate similar priming processes in a more complex movements. Within the Virtual Reality environment, the participant was able to see the reflection of his unaffected limb superimposed onto his affected upper extremity performing the required task. The results from the sEMG in Figure 5A and 5B clearly show muscle activation/peaks in the biceps and shoulder muscles during weeks 3 and 4. 

The post-stroke patient was also tested and evaluated on Box and Blocks and Fugl Meyer Motor Scale (FMMS) standardized tests prior to training and at the end of the four training session. All four tests measure arm motor function changes and have been thoroughly investigated and reported in literature on post-stroke rehabilitation. Overall, no major changes were noticed in the assessment score on the preliminary testing with the four sessions of device training. However, the Box and Blocks test showed a slight improvement in the right upper extremity, so a possible success indicator could be the beneficial transfer of training from the unaffected to affected limb or vice-versa.

CONCLUSIONS

This paper describes initial testing of the ARWED, which is a virtual reality system for physical rehabilitation of patients with reduced upper extremity mobility resulting from a stroke. The purpose of this system is to increase limb Active Range of Motion (AROM). To do this, the ARWED maps, in real-time, the patient’s unaffected limb to a virtual representation of the affected limb with the intent of tapping into the mirror neuron which will facilitate the initial learning phase.

The experiments with healthy subjects show that people with reduced joint motions can react to computer animations, link those animations onto joint motions, and learn to move successfully with a constraint. The results on the assessment of the developed virtual photorealistic 3D hand models with healthy subjects imply that the dynamic motions of the 3D models mimic those of a human limb model, as well as that the developed 3D models prime the human motor system in a manner consistent with the human model. The pilot tests with a post-stroke patient suggest that the virtual reality mirror therapy could trigger muscle activation in patients that are more than 3 years post-stroke. This can further serve as an evidence that the time needed for recovery from stroke is not limited to one year and that additional practice can improve mobility in both the sub-acute and chronic phases following a stroke. The future work includes increased frequency and duration of training with the device to assess for any changes in the assessment score.