RESNA Annual Conference - 2010

BACKGROUND

Cochlear implanting is a relatively new technology that is rapidly growing. The cochlear implant uses electrodes implanted on the various auditory nerves to allow the wearer to receive incoming signals with differing frequencies.   With an estimated 150,000 people worldwide using cochlear implants, it is important that we are able to provide the best functionality out of the electrode implants.  The problem often encountered with the implants is that background echoes and chatter interfere with the user listening to the television or another audio signal.  The excess noise picked up by the implants can cause the user to misinterpret the signal that they need to hear.

With an increasing number of cochlear implants, the creation of an Assistive Listening Device (ALD) becomes an essential need for hearing-impaired individuals to hear audio clearly.  Our ALD will allow the user to switch between audio signals using their PDA with ease.  Fast Fourier Transform procedures will use clipping techniques to filter the signals without background interference.  Another benefit of our device is the existence of a “universal” control of the volume of the input signal.  An additional benefit that could be implemented with this device is the capability of talking on a landline phone from anywhere in your house.  When using our ALD, a hard of hearing individual can listen to the television audio or radio audio at the desired volume (even when it differs from another person in the room).  The ultimate goal is to bestow a hearing-impaired individual with control over any input signal they desire in an optimal listening environment.

The ALD we constructed consists of  three main components: gumstix PCBs: Verdex Pro, which contains a powerful, but low power usage CPU, AudioStix with audio I/O capabilities, and NetPro, which contains a Wi-Fi chip. Using these gumstix products we made an ALD capable of processing, receiving, and transmitting signals in a DSP format.  The final ALD will have additional component in order to enable switching between multiple audio sources. 

In next couple of points we will describe the main accomplishments, and our future goals. In addition, we will enumerate our planned approach to complete the project and offer alternate ways to accomplish our goals. Within our approach, we will show diagrams of the hardware and the functionality of the product.

RESEARCH QUESTION

We are designing a solution which uses currently available Pocket PCs and an embedded Linux platform to handle audio signal processing and wireless communications. Our custom designed solution will deliver the ability to easily and seamlessly switch between multiple audio signals such as TV/radio, music listening, and telephony, with optional voice-command based control. The goal is to eliminate cumbersome switching of devices and integrate the design with currently available hearing instrument platforms.

The client requested that the design be able to seamlessly switch between the different signals and be integrated with currently available hearing instrument platforms.  To address this, in the next semester we will prototype a controller which will handle the selection and wireless broadcasting of multiple signals

Through the use of currently available Pocket PCs and a user friendly interface, the user should be able to comfortably control the selection of their audio signal. This interface will give the user freedom to roam their home while still receiving a good quality signal and control. Since this involves wireless communications, we have investigated the quality of service provided by the device.

A prototype model, which is able to transmit and receive a single audio signal between the ALD and the PDA, has been produced and thoroughly tested. Compatibility tests with current hearing instrument platforms will be provided. All design drawings will be provided, and consultation for further platform improvements will be available.

METHOD

The exact plan of the project considered two distinct hardware parts, which are programmed to work synchronously. The two hardware parts are (as shown in Figure 1): Assistive Listening Device (ALD) and the PDA. The third hardware part shown in Figure 1 is the Hearing aid with Bluetooth Module, which will be implemented in the next stage of our project. As Figure 1 shows the flow of information, you can easily understand the system concept. Audio signals coming from different home audio equipment are going to the home device (or ALD), then the right signal is chosen and sent wirelessly to the personal device (PDA). Finally the desired information (audio signal) will go to the hearing aid wirelessly (using Bluetooth protocol).

The idea is to multiplex input audio signals (up to eight different inputs) into one which would be selected based on the user’s command. After multiplexing, this signal will go into the ALD where it gets transmitted to the PDA via Wi-Fi. There are two main components of our ALD device and those are the multiplexer and the ALD prototype. After intensive research and our mentor’s suggestion, we decided on gumstix products, Verdex Pro XM4 COM motherboard with the NetPro-VX and Audiostix 2 expansion boards, for our module’s boards. Those products’ spec sheets promise high performance, good efficiency, and low power.

However, a major concern about this project is the software element. The brain of the whole project had to be programmed using available software. Gumstix-boards are easy to deal with because they run a Linux distribution, which is easy to program for. The PDA programming part is already well developed with the PJSIP project, and it will be properly used to control the flow of the wanted signal. The PJSIP project has the capability of streaming audio signals on multiple platforms, including Windows Mobile and Linux. Our current prototype is able to stream an audio signal, sampled from a 3.5 mm headphone jack on the ALD, from the ALD to the PDA with minimal noise and packet loss.

First we examined the hardware options available for a demonstration platform and decided on the gumstix boards for use. Then we chose the SIP communications platform for transmission of signals because it meets all the requirements for our implementation and allows for future expansion. After extensive research, we chose the PJSIP libraries due to its platform-specific applications. We then split into two groups: one to work on the PDA application and one to work on the ALD application.

The PDA group wrote an implementation of PJSIP that dials a specific IP address using the SIP protocols through a simple GUI and creates a VoIP connection with another device. The ALD group got a Linux distribution running on the gumstix board, including Wi-Fi capability, and implemented a PJSIP communication application, which can communicate with other SIP-enabled devices. Next, the two teams worked together to get the ALD and the PDA communicating with each other over an ad-hoc Wi-Fi network.

RESULTS

During the assessments of our product using the ALD as a transmitting wireless sensor node and the PDA as the receiving node we did not drop any packets.  The transmission was successful in that our average latency was only 12 milliseconds which is well within our original goal of 20 milliseconds.  The maximum jitter we experienced was 16.5 milliseconds, and this result falls within the 50 millisecond detectable range of human auditory capacity.  Finally, after testing the range of our prototype system, we were able to confirm a range of around 150 feet including plaster wall interference.  The results give us confidence that our system relying on a Wi-Fi b/g protocol network would be beneficial to a hearing impaired user.

DISCUSSION

As we mentioned above, the Assistive Listening Device (ALD) is an essential device to assist persons with hearing impairments in their daily lives. A well designed ALD can empower the same hearing experience regardless of hearing impairment without complicated procedures

Our design goals are to have a low power, versatile implementation with a good user interface so that it could significantly improve the usability of the device and easily give users freedom to switch to any audio source (up to eight) in a home environment.

The proposed solution will be composed of 3 major components: an ALD for multiplexing audio source signals and transmitting them wirelessly; a PDA as a client for receiving audio signals from the ALD and transmitting user control signals back to the ALD; and a BT headset for the user to listen to various audio sources without the burden of a wired connection. The ALD is our major design component, and is currently working, minus the signal switching capability. On the suggestion of Doug and Dr. Loizou, we used the gumstix Verdex line of boards as our hardware development platform and Linux as our software development platform for the ALD.