INTRODUCTION

For individuals with transradial or more distal amputations, it is often possible to place electromyographic (EMG) sensors on residual muscle sites of the forearm for intuitive control of prosthetic wrist and hand functions (Scheme & Englehart, 2011; Roche et al., 2014) . When the amputation is proximal to the elbow, however, the situation is more difficult because of the lack of available muscle sites for control output. Targeted muscle reinnervation (TMR) is the only approach currently offering a truly intuitive control interface for high-level upper limb amputees, whereby nerves from the amputation site are surgically relocated to other muscles of the body, such as those in the chest. EMG sensors can then be used to measure muscle activations of the reinnervated muscles which, to the user, feel like movements of the missing limb. These “gestures” can then be classified for control of prosthetic arm functions (Kuiken et al., 2009).

In this work, we propose a noninvasive alternative to the surgical approach, where the amputee would control an upper limb prosthetic with analogous movements of the lower leg. The idea of controlling a powered upper limb prosthetic in this way is not new and has recently been successfully implemented with inertial measurement unit (IMU) sensors on both feet (Resnik et al., 2014b) for controlling the DEKA prosthetic arm and hand (Resnik et al., 2014a). Both feet are needed to control one prosthetic because the number of functions exceeds the number of control signals provided by one IMU alone; furthermore, all functions are not meant to be intuitive. In our work, we are aiming to keep all upper limb prosthetic control bound to one leg and for all leg movements to be analogous to arm movements in order to minimize cognitive load and training time required to operate the system. To our knowledge, this is the first application of EMG-based gesture recognition techniques to the leg for this purpose and the first use of surface EMG for recording extrinsic toe flexors and extensors.

In this case study, subjects performed hand gestures with EMG sensors placed on the forearm. Then, sensors were placed on the leg and subjects performed an analogous set of foot gestures while seated. All recordings were taken in a single session for each subject, and the data was analyzed offline. This is only the first step toward control of an upper limb prosthetic via EMG-based recognition of leg gestures, however the results are promising and motivate future work wherein subjects would control a simulated or actual prosthetic.

BACKGROUND

As a preliminary task for the experiment, a mapping between the wrist/hand and ankle/foot was developed. The first part of this is a gesture mapping, which is intended to be intuitive assuming a body posture similar to that used while typing on a keyboard. The gestures chosen are shown in Figure 1, with analogous pairs indicated by arrows. The upper limb gestures selected represent most of the desirable functionality of a prosthetic wrist and hand, with the thumb extension (TE) gesture acting as a placeholder for any additional hand function such as key grip or fine pinch. The analogous foot gestures were chosen based on the alignment of the degrees of freedom of the wrist and ankle when the hand is placed in front of the body with palm face down and the foot is oriented in the standard anatomical position

In order to further strengthen the analogy between the leg and the arm, we also identified a set of muscles which have analogous primary actions on the corresponding limb. They are listed in Table 1 and are the muscles targeted by the EMG sensors in our experiment. While it is common in EMG-based gesture recognition research to place a ring of sensors circumferentially around the forearm due to amputation (Scheme & Englehart, 2011), the leg is in general not limited by this constraint, so we are able to record directly at applicable muscle sites.

METHODS

Subjects

The three subjects that participated in this study (two male, one female) were all undergraduate students at UC Davis between the ages 20–21 and were right-hand dominant. Subjects were informed of and consented to procedures approved by the Institutional Review Board at UC Davis (protocol #251192).

Experimental Protocol

The experiment consisted of a single session lasting approximately two hours. In the first part of the session, EMG sensors were placed on the dominant forearm and wrist while subjects performed the seven gestures shown in the top of Figure 1. Each recording trial consisted of the following sequence: two seconds of rest while viewing the prompted gesture, onset of the gesture, three seconds holding the position, then resting for four seconds before the start of the next trial. All gestures were performed three times per cycle in randomized order. Four cycles were run on the arm with approximately one minute of rest in between.

In the second part of the session, the sensors were removed from the arm and placed on the ipsilateral leg (Table 1). Subjects were shown the images of the hand gestures in Figure 1 and were asked to perform analogous foot movements. Before beginning the recording trials, any discrepancies between the subject’s interpretation of the analogous gestures and the gestures in the lower part of Figure 1 were verbally corrected (usually little correction needed if any) and it was ensured that subjects understood the mapping before proceeding. This process typically lasted about two minutes and the recording cycles for the foot gestures began immediately after. The same trial/cycle protocol applied to the foot gesture portion of the session.

Electrode Placement

Twelve disposable Ag/AgCl center snap electrodes (ConMed 1620) were placed in bipolar pairs on the muscles listed in Table 1 with approximately 2.5 cm inter-electrode distance. An additional electrode was placed on the olecranon in the arm recording setup and the medial malleolus in the leg setup, serving as a ground connection. The muscles with sensor location recommendations by SENIAM (Hermens et al., 1999) (i.e. gastrocnemius lateralis, tibialis anterior, peroneus longus) were positioned as recommended. For the other muscles, palpation was used to locate the muscle belly and a movable “EMG probe” with dry electrodes was used to identify a suitable recording site by asking the subject to perform various movements while monitoring the probe signal. For most of the muscles, this procedure was straightforward, however the extrinsic toe flexors/extensors were not as simple to isolate as they lie beneath other muscles. A compromise was found by incrementally moving the probe distally while the subject performed discriminatory movements (e.g. toe extension and dorsiflexion), trading off between crosstalk from the larger muscles associated with ankle movements and small signal amplitude from toe movements. After marking each recording site, the electrodes were placed and all signals were checked for quality before proceeding with the recording trials.

Data Collection and Analysis

 

Table 1: Muscles used in the gesture recognition system.

Arm		Leg
Muscle	Primary Action	Muscle	Primary Action
extensor carpi radialis longus	wrist extension	tibialis anterior	dorsiflexion
pronator teres	forearm pronation	peroneus longus	foot eversion
flexor carpi radialis	wrist flexion	gastrocnemius lateralis	plantarflexion
extensor pollicis longus	thumb extension	extensor hallucis longus	hallux extension
extensor digitorum	finger extension	extensor digitorum longus	lesser toe extension
flexor digitorum superficialis	finger flexion	flexor digitorum longus	lesser toe flexion

A custom graphical user interface program was created for prompting subjects to perform the gestures and collecting the raw EMG waveform data. At the start of each recording trial, an image of a hand gesture from Figure 1 was shown, and a progress bar prompted the participant for gesture onset and offset. Six EMG channels were amplified by Motion Labs Systems Y03 differential EMG amplifiers (×300 gain, 100 dB CMRR, − 3 dB bandwidth from 15 Hz to 2 kHz), sampled at 8 kHz by a Measurement Computing USB-1608G data acquisition system (16-bit), and recorded directly to disk by the user interface program. Subsequent analysis was performed offline.

Signal conditioning included filtering with a fourth-order digital Butterworth bandpass filter with a 10–450 Hz passband then downsampling by a factor of 4 to 2 kHz. The filtered and downsampled signals were segmented into 150 ms windows with 50 ms overlap. The windows from 1–1.5 seconds (just before gesture onset) of each recording were used as instances of the rest class, and windows between 2–4 seconds (just after gesture onset and before gesture offset) were used as instances of the class corresponding to the gesture image prompt shown to the subject during that trial. Four time-domain features, introduced by Hudgins et al. (Hudgins et al., 1993), were extracted from each channel of the recording to form a 24-dimensional feature vector: mean absolute value (MAV), waveform length (WL), number of slope sign changes (SSC), and number of zero crossings (ZC). The linear discriminant analysis (LDA) classifier was chosen due to its simplicity, computational efficiency, and status as a standard with respect to which other classification methods were tested (Lorrain et al., 2011). For both the arm and leg configurations, classification accuracy was determined by a cross validation scheme in which two cycles were used for training the classifier and the remaining two were left for testing. Every possible combination was tested in this way and the total number of correct classifications for all combinations divided by the total number of testing instances produced the average classification accuracy.

RESULTS

The average classification accuracy for each subject in the two experimental configurations is shown in Figure 2. Although the accuracy in the leg configuration is systematically lower than that of the arm configuration, all subjects achieved foot gesture classification accuracy at or above 90%.

Figure 3 is complementary to the average accuracy for each subject, showing the average accuracy for each foot gesture. The rows of the matrix represent the actual gesture class and the columns represent the predicted class. Elements in the matrix are the percentages of testing instances with the label of the corresponding row which were classified as the corresponding column. Hence, diagonal elements are correct classification rates and off-diagonals are misclassification rates. All but two gestures had classification accuracy above 90%. Note that toe extension (TE) was misclassified as dorsiflexion (DF) 9% of the time, indicating that recordings of the toe extensors were contaminated by crosstalk from the much larger tibialis anterior. Also, the hallux extension gesture was notably the most difficult for participants to perform repeatably, which is reflected by its misclassification as several other gestures 3% of the time each.  The hallux extension (HE) and lesser toe extension (TE) gestures were surprisingly distinguishable, despite the close proximity of the sensors measuring the activity of the muscles responsible for these movements (extensor hallucis longus and extensor digitorum longus).

DISCUSSION

The results presented here demonstrate the feasibility of recognizing gestures of the lower leg and foot via surface electromyography. Participants found the mapping between the hand and foot intuitive and achieved classification accuracies that could be sufficient for prosthetic control; however further work is needed to test the usability of such a system explicitly. The next step is to develop an online system wherein the user views a prosthetic arm and receives visual feedback as to the accuracy of their movements. It has been reported that there is considerable variation in online task performance when offline classification accuracy is at or above 90% (Ortiz-Catalan et al., 2013), indicating that the small difference between hand and foot gesture classification accuracy reported here could be overcome by providing feedback to the subject. Furthermore, the foot and toe movements described in this paper are not performed as frequently as their hand and finger counterparts in daily life. Hence, with additional training and online feedback while performing the gestures, a user might be able to execute a task in the leg configuration just as well as with the arm setup.

While there are practical issues involved in using a foot gesture system as a basis for upper limb prosthetic control, Resnik et al. address many of these concerns and show that amputees are generally willing to use movements of the foot for control of a prosthetic arm and hand (Resnik et al., 2014b). One advantage of EMG over inertial measurement sensing is the possibility of using isometric contractions of the leg muscles to control the prosthetic. In addition, control outputs unique to EMG could be combined with IMU-based techniques to achieve increased capabilities for overall control.