RESNA Annual Conference - 2010

BACKGROUND

As consumer technology continues to advance, techniques for assistive technology that were previously not realistic due to cost have come within reach of the average user. This has become possible thanks in part to the market demand for 3-D and interactive technologies, particularly within the realm of video game systems. These products have driven the cost of many technologies down due mostly to the economics of scale, making it possible to take advantage of advanced technologies such as infrared camera systems. With large online communities building up around the re-purposing of these technologies, software has been written to interact with many of these devices at various levels. These strides and improvements, along with the support provided by their respective communities, make the devices a feasible tool for use in aftermarket products. By repurposing these technologies it becomes possible to use them as unique alternative input devices (1).  The strength of this comes however, from the flexibility behind a system built using such devices.

Many alternative input devices on the market use custom proprietary methods for providing input to systems (2) (3).  An example of this might be a joystick device which could be used to control a mouse pointer but, the manufacturer does not include this feature.  While it is possible to use third party software to possibly remap these inputs, these applications are few in number, limited, and often application specific.  When creating an assistive technology it is ideal to make it as broadly customizable as possible to allow for it to cover a large consumer base as well as meet their needs as specifically as possible.  In many instances, a combination of devices is needed to provide the best solution.  The market currently lacks a standard system to interface and customize the input from the multitude of devices that exist.  Therefore, in order to allow an increasing number of low cost devices to enter the market of assistive technology, a system to facilitate their communication and use is needed.  With this system, an increase in these low cost commodity devices entering the market will lead to cheaper more versatile systems than are currently out there.

RESEARCH QUESTION

The objective of this study was to evaluate methods for creating a low-cost system that was able to provide users a viable method for interacting and controlling a computer. As time progressed and the need for additional input from multiple devices was needed, it became clear that a unified software architecture was needed to support this endevor. The objective was thus expanded to be a low cost system which allowed for the use and integration of commodity hardware.

METHOD

The Wiimote from Nintendo (4) was initially selected as the target device for the platform due to it's low entry cost, large support community, and already established open source projects (5). Taking advantage of the Wiimote's ability to track up to four unique IR sources (6), the device was positioned pointed towards the user's head area. By mounting an IR source on a pair of protective glasses, it became possible for a user to provide input to the Wiimote through the movement of their head. With a camera resolution of 1024x768 pixels, the Wiimote provided a fairly accurate resolution allowing for control through minimal head movement. With the addition of a second Wiimote, it became possible to calculate an accurate three dimensional location of the IR source. The data from the devices was used to control the location of the mouse pointer in the Windows Operating System. The system still lacked the ability for a user to produce a Windows mouse-click. Nintendo hardware provided the answer to this problem as well.

Figure 1. Design of a Wiimote-based proof-of-concept for 3-D head-tracking. (Click for larger view)

To implement mouse clicking, Nintendo's Wii Balance Board was integrated into the system. The board has several pressure sensors which allows for the calculation of an individual's center of mass (7). A user would sit on the board and use it to detect changes in their center of mass. This feature was used to generate a mouse click whenever a user shifted their balance in a certain manner. Both right and left clicks were able to be generated using this technique. Combining the two Wiimotes with the balance board gives a user the ability to move a pointer around on a computer screen and right and left click on objects. The mouse could be controlled in both an absolute as well as relative manner, allowing for variety and customization. The experiment proved that a more versatile framework than what currently existed was needed.

The software framework was developed to support the efforts of the hardware team's design. The framework was designed to operated much like a sound board might. Each device connects to a module unique to it, meaning that there is a Wiimote module as well as a Balance Board module. These modules were made such that they can be instantiated multiple times allowing for multiple devices to be used. The modules would then connect to the main server where they pass their data. Any number of client modules can then connect to the server and request a copy of the data being produced by the devices. These client modules could provide functionality such as gesture recognition, integration of the input of multiple devices, signal filtering, and any other functionality that is needed. These client modules route their output to a display module which was written to interact with whatever operating system the software is being run on. This allows for the system to be cross platform compatible as well as take full advantage of any advanced features that the system’s display manager might present.

Photo 1. Implementation of a Wiimote-based proof-of-concept 3-D head-tracking

RESULTS

A hardware system was developed that tracks user's head movements and moves a computer cursor accordingly (8). The cursor can be controlled in two ways: relatively and abosolutely. Relative mode works like a normal mouse by moving relative to the last position. Absolute mode means that wherever a user's head is, the cursor will be at a proportional position on the screen regardless of how much the user moves around.

The software developed to support our hardware system resulted in a general purpose input device framework. . The cross-platform framework facilitates the communication between various input devices to allow combinations of devices to operate in unison to create an advanced human interface device. This framework makes the integration of future commodity devices an easy process, opening the door to the development of more low cost systems to help revolutionize the assistive technlogy field.

DISCUSSION

The results of this project illustrate that repurposing of commodity hardware can be used to assist individuals with disabilities. Due to low cost of components, the resulting systems become affordable to an average user. The use of software framework allows rapid system development because it simplifies application programming by presenting a common API for all hardware inputs and removing the interaction with the input hardware from the application. The software framework also enables efficient code re-use by allowing previously created modules to be used with new input devices. Our present hardware platform uses Nintendo Wiimote but it can be extended to other commodity devices such as Microsoft’s Project NATAL.

Figure 2. An example comfiguration of the modular software framework that manages data flow and connections between client applications and client input devices.

On the hardware side, the use of software framework simplifies the hardware-software interface by not requiring the developer to write operating system specific device driver code. The framework interacts with the operating system and presents developers with a standardized API. The frameworks’ modularity allows for customization of processing functions (e.g. gesture recognition, signal processing, etc) so that modules can be added and removed as needed, filters and other transformations can be added into the data flow. For example, a low pass filter may be useful for a person with disorders involving shaking during movement and requires smoothing on their data set to remove high frequency noise. The modularity also allows for complete control over the parameters for all of the framework features, giving the user the ability to tune or change parameters according to their personal needs.