INTRODUCTION

Communication requires the ability to transmit information through different codes and systems governed by general rules, so that the subject can interact with others and integrate into society. Through speech, associated with gestures, facial and body expressions, we interact with other people, communities and cultures, forming social bonds, thus characterizing the human condition [1].

The area of assistive technology which is specifically targeted to the expansion of communication skills is called Alternative and Augmentative Communication (AAC). AAC helps people without speech or writing ability to speak and/or write [2].

As a result of such communication limitations imposed to people with disabilities, both written and oral, methods and alternative communication systems are required [3].

Sensory, cognitive or motor difficulties can compromise all or part of human communication skills, which can be complemented with a support system and Alternative Augmentative Communication (AAC). These alternative mechanisms are less efficient than the natural ones, with significantly less typing speeds [4]; however, in many cases, they may be the unique alternative for those with reduced mobility to access the computer and communicate with the world.

To meet this demand, we developed a keyboard and mouse emulator software with low cost sensors, able to meet a large portion of the population with some kind of physical disability that impairs their communication.

Among the various compromising diseases of human communication, we can mention Cerebral Palsy, which causes disorders in motor development, without necessarily affecting the intellectual capacity of the individual, but greatly compromising their communicative capacity [5].

ETM is different from other solutions by the adaptability to meet the particularities of each person through its customization features that will be described in this article.

So, in this paper we present and discuss ETM development and application.

MATERIAL AND METHODS

ETM [5] was originally developed to assist people with cerebral palsy. Due to its adaptability to different types of situations, its use has spread to many different types of disabilities as quadriplegics and people with ALS. The software has undergone several changes since its first version. Originally an electronic circuit was connected to the mouse and keyboard inputs of the computer. To make the most affordable solution, the hardware had been replaced by software and sensors directly connected to computer devices entries as USB (Universal Serial Board) [7] and Serial Port (RS232).

The program acquires signals from sensors connected to the computer inputs, turning on the keyboard and mouse actions, allowing the user to use all features of the keyboard and mouse. These sensors are attached to the wheelchair or the user's body. For example, to use the software, the person with paralysis that only moves the head presses a key coupled to the back of the wheelchair. With the movement of the head, a signal is sent to the computer that executes the command indicated on the virtual keyboard or the virtual mouse displayed on the screen.

Figure 1 shows the application architecture. The software consists internally of a layouts manager, responsible for the visual part of the virtual keyboards, a script processor for processing chained commands, one device manager to acquire signals from the sensors and the command processor that is the central part responsible for emulation of keyboard and mouse. Emulation means to declare to the operating system to type characters as if received from the keyboard and to click or move the cursor as if received from the mouse. All combinations are allowed like Ctrl-C or Ctrl-V.

Input Devices

The layer responsible for obtaining the signals from the sensors was developed to require a minimum of hardware. The possible input sensor devices are:

USB Joystick:

currently, it is the most widely used input device due to its simplicity, ease of acquisition and low-cost. One advantage of Joysticks is that they use the HID protocol (Human Interface Device) [7], which is a universal connection, not requiring any additional driver installation, which makes it compatible with most operating systems on the market. When connected to the USB port, ETM recognizes the command buttons, and one can set them individually for each type of action. If the automatic scanning system is disabled, the user can choose a command for each action. For example, one can configure the A Button to skip the lines, the B button to jump the columns, and the C button to choose the selected layout key. As a joystick is not suitable for a person with physical disability, we must open it and weld wires to the corresponding contacts. Once connected, these wires are attached to keys adapted for wheelchairs or body.

Serial Port:

although it is obsolete, the serial port is present in many desktop computers. It is very common to find older computers on schools, often the result of donation.

Serial Port, or RS232C, is very suitable to be transformed on ON/OFF sensors, just connecting two wires to DTR pin (Data Terminal Ready) and CTS (Clear to Send). By software, the user can poll (test) the condition of CTS and state whether the contact is made or not. The same may be done for some other pins.

As simple as this, ETM took command of all keyboard and mouse actions, just at the touch of one switch, provided Automated Scan is on. One can dismiss Automated Scan in order to improve the speed, simply connecting three Serial Port pins and providing three buttons (sensors) to the user.

Microphone:

The microphone's role as sensor is to acquire any sound that indicates a command to be executed. When the software receives a noise signal picked up by the microphone, this signal is filtered and converted into a digital pulse, which is recognized by ETM indicating that an action should be executed.

Camera:

The camera feature was developed with the goal of acquiring the blink of the eye and turn it into an action. This sensor was not efficient due to calibration difficulties. This feature can be improved and was replaced by the optical sensor, which is more efficient.

Keyboard:

the computer keyboard may be used as a sensor. ETM disables all functions of the keys making them all work like a single button that can be pressed finding any limb of the body. For example, a keyboard can be set at a back of the wheelchair and be activated by the head. It is not recommended because this forbids the use of smart layouts, based on the so called “sendkeys” soft.

RESULTS E DISCUSSION

One of the advantages is the software allows the creation of layouts with customizable buttons. Unlike a computer user who uses conventional keyboards where the layout complies with industry standards such as the QWERTY arrangement, a user with special needs using virtual keyboard often want to change the key layout for better ergonomics, and even exclusion of some key strokes.

The software features a layout editor where you can create the layouts of keyboards, such as in Figure 2. For each layout it is assigned a name. It can be loaded when ETM is running, through a menu.

A layout is a set of buttons, not a set of keys as on a conventional keyboard, where each key has its function. Each layout button can receive some properties, besides its function. A sound can be associated to a button. An image can be associated to a button. Complex functions can be associated to a button.

Properties

(1) Image File: allows you to associate an image with a button. This property is useful when the user wants to create layouts with images representing the letter or text or action being sent / performed;

(2) Sound File: allows the user to associate sound files in Wave format. In this case, the sound will be played when a button is about to be chosen. This helps visual impaired users;

(3) Cursor File: allows you to associate the Windows cursor type files. In this case, the mouse cursor shape is changed to the shape of the cursor assigned to when the button is selected.

Text Key: its function is to simulate the character key. The main difference is that character keys are already preset for execution of the command in hand. For example, if one adds the letter A key on the layout, it is already configured to send the letter command. The text key allows, in a single action, send text or a series of commands, rather than sending a single character. This key has a property called Text, where the user can configure set phrases.

Shortcut for Layout: ETM allows user to add a button on the layout containing a name of another layout so that the user can switch between layouts. For example, if the user is using a layout with letters from A to Z and want to change to another containing only the mouse buttons. This allows small, efficient layouts. There will be a gain in the scan time, too.

Command Processor

ETM has a command processor system that allows sending a series of instructions for the operating system through the push of a single button. With this feature, the user can create a task automation processes facilitating his day-to-day. For example, he can create a button to load a program, perform tasks such as Ctrl + C (Shortcut to copy selected text), select text, save files in a text editor, among others. It is considered one of the main features of the software. It is implemented via the above mentioned Text Key. There is a SendKeys.txt file explaining its use. [6]

Keyboard Emulator

ETM works in the scanning mode; it can be automatic or manual. In the case of automatic scanning, highlighted with color lines will be skipping at a configurable time until the sensor is pressed.

Figure 3 illustrates ETM interface for the custom layout. The system works as follows: letters, numbers and arrows indicating mouse movement are scanned all the time, by label (line highlighted in yellow). When the marker is positioned on the desired line, the user triggers the sensor and the highlight is now on the characters of the selected line. To choose a character, the user presses the sensor to activate the letter or command he wants.

Mouse Emulator

The mouse emulation is similar to the keyboard emulation. The user selects a line, then selects the button that will perform the action he wants. The mouse emulator has several buttons for different purposes to ease the use of the mouse. For example, to prevent the user having to choose a button for clicking the mouse and then choose another button for moving the mouse, or "click and drag", there is a button that performs two actions at the same time. In fact, there are four keys to drag up, right, down, left. It is also possible to create drag keys 45 degrees or other directions. The mouse buttons were separated in colors for easy identification.

Figure 4 shows an ETM layout with buttons with all mouse options, plus keyboard arrows.

Sensor Example

The optical sensor is a special type of sensor. It consists of an electronic circuit specifically designed for ETM using infrared sensor capable of detecting certain types of movement such as, for example, opening and closing the eyelid. This type of sensor is used in more complex cases such as those with amyotrophic lateral sclerosis than in advanced stage can only perform this move. This sensor is extremely versatile and can be used in various ways. Although an electronic circuit, its construction is simple and low-cost, making it affordable.

Being a small and low-power circuit, it is generally fixed internally on the joystick. So, it is not necessary external energy source for operation because their power is supplied by the USB port. Figure 5 shows a pair of glasses adapted to the optical sensor.

ETM Voice

ETM Voice is a text-to-speech background program developed to use in addition to ETM. It talks whatever is put on Windows clipboard. This can be achieved by cleverly designing the layout through Command Processor. The purpose of this program is to enable the user to communicate with a person at any distance, provided a loudspeaker is suitably placed.

ETM Voice allows its use with any text editor. The user needs simply to select the text and press Ctrl-C! In doing so, the program starts talking the text automatically in Portuguese. Figure 6 illustrates ETM Voice Interface. This interface may be transparent to the user because it can be hidden.

Configuration Interface

The graphical interface configuration must be done by someone able to use the true mouse. It is not possible to configure ETM or edit layouts, using ETM. This is to prevent the user (a disabled person) to unwillingly interfere with the setting parameters. Delays, sizes, sensors choices, background and foreground colors are among settings (configurations) offered to customize the software.

Application

ETM is being used by several people and schools as a communication tool. It has been adopted by many people and institutions assisting people with disabilities in Brazil.

Due to its easy of customization and adaptation it is gaining new users every day. Although originally developed for people with cerebral palsy, the application extended to other types of disability, such as amyotrophic lateral sclerosis. The figure 7 shows an individual with cerebral palsy using ETM.

CONCLUSION

In this paper, we presented ETM system: a powerful configurable and customizable keyboard and mouse emulation software for AAC. It has been demonstrating efficiency as a solution for alternative communication for people with speech and motor difficulties.

The software can be downloaded for free through the project website [8]. All requests made on the project website [8] are analyzed and defined together with those responsible for the user. The challenge is to define which sensor best fits the need. ETM is constantly evolving and new studies have been conducted to add features such as artificial intelligence in predicting words to make it more versatile and agile.