BACKGROUND

Visuals such as maps, graphs, diagrams, and many others are common ways to communicate information to others and we are surrounded by them each day. However the visually impaired do not get to take advantage of such means of communication. A traditional solution is to use raised line drawings printed on specialized paper to convey the necessary information through the sense of touch. Unfortunately the production of physical tactile diagrams is a slow and expensive task.  They also do not allow for flexibility when exploring these diagrams, such as allowing zooming and decluttering/simplification (Rastogi and Pawluk, 2013).

Several groups have considered providing haptic/tactile computer interfaces to provide interactive access to virtual tactile diagrams.  One alternative is to use tactile displays in various forms (Metec AG, 2010; Petit et al., 2008; Owen et al., 2009) to provide information about the edges or textures underneath the device.  Unfortunately, large multi-pin devices are prohibitively expensive.  Small, moving multi-pin devices have been created but have difficulty tracking lines, which greatly slows access even when effective.  

Alternatively, haptic fixtures have been used (Abu Doush et al., 2009) to constrain exploration along data lines in graphs. This allows for quicker tracking of lines.  Unfortunately it is not clear whether this method will be effective for more complex diagrams, particularly ones using more complex representations (i.e., textures) rather than lines (but which are also more effective). In addition, existing systems tend to be expensive and needlessly 3D, and/or have a small workspace and provide low force levels.

In exploring physical tactile maps, two main paradigms have been observed (Magee and Kennedy, 1980; Symmons et al., 2005). One form of exploration is known as active exploration, where the user can freely explore the diagram and piece together the shape that is presented. It is thought to be hypothesis driven. Another method for approaching tactile diagrams is known as passive exploration. The user is guided along the raised lines on the diagram by a person that can clearly see the information presented. This guide moves the user’s hand with their own while the user focuses all cognitive function on identifying the subject of the diagram (Symmons, 2005; Vermeij, 1980).

The two methods have been examined under various conditions and it is not clear which one is more effective or whether allowing a mix of methods would be best.  This paper describes a low cost, “active mouse” which can allow both active movement by a user (and provide haptic feedback as to the diagram) and guidance of the passive user’s hand under the command of the “mouse”.  It is intended as an efficient method to access tactile diagrams, as well as provide a platform to understand the differences between active exploration and passive guidance.

DEVICE DESIGN

System Layout

The device created is a force feedback active computer “mouse” designed for access to 2-D virtual tactile diagrams. Its design requirements, in terms of allowing natural exploratory movements, facilitate the need of a responsive and dynamic system including sensors, actuators, and control modules.

The system is divided into two main components. The first is the software which handles the virtual tactile diagram and the control of the device’s many features. The second part is the actual device hardware.

The software is further broken down into two modules. The first module handles the virtual tactile diagrams. It is programmed in C# using Visual Studio and runs on any desktop PC. Shapes (i.e., objects and parts) can be drawn or interpreted in the virtual workspace. Currently, the software interprets them as virtual fixtures. These fixtures produce virtual forces in response to user input force and position. To control the end result, an admittance control model of the type V = k * F is devised where F is the user input force and V is the system output velocity. Alternatively, path planning can be used to “forceably” guide the user explicitly through a path, such as around the edge of a shape. The System output is fed into the second software module designed to control and manage the device hardware. This second module was programmed in Arduino’s C++ and runs on an Arduino Due. The Arduino software is responsible for collecting and relaying input from the sensors in the device to the PC software and calculating the response of each actuator in the device.

The final component of the system is the device hardware which consists of a 2D force sensor, RF position sensor and three servo motors as the actuators. As the user interacts with the device, the force sensor and position sensor collect the user’s intentions to move and relays the information to the software via the Arduino Due. Next the software determines a response and outputs a net velocity Vh,net through the use of the three servo motors.

Omni Drive Design

The device hardware is comprised of two main sensors and a drive platform. The force feedback component is a motorized drive platform consisting of three servo motors (HS-7955TG, Hitec), that were modified for continuous rotation, connected to omni-directional wheels. The wheels (custom made) come with smaller spinners placed around the radius of the main wheel frame and oriented orthogonally to the wheel’s axis of rotation. The wheels can achieve smooth 2D omni-directional movements on a given plane which is important for a natural exploratory feel for the device.

The three servo motors are concentrically positioned around the main vertical axis of the drive at 120° apart. This type of drive layout is known as a holonomic drive system. With the help of the omni wheels, this type of drive system allows for the control of all three degrees of freedom that the device has. The device can be linearly moved along the x and y axis and rotated around the z axis freely. Figure 2 below illustrates the design of both the holonomic drive system and the omni wheels respectively.

The device sensors consist of a 2D force sensor (MSI Model 462, Ultra-MSI) and a 2D RF position sensor that interfaces with a graphics tablet (Wacom Intuos Extra Large). Both sensors are housed in the device and are fastened on top of the drive platform. The force sensor specifically acts as a connector between the drive and the outer shell of the device. As the user applies a force to the outer shell, the force sensor will measure and relay it to the Arduino Due as an input force F for the admittance control model.

Guidance Virtual Fixtures

Virtual fixtures are forces and positions produced virtually that can be reapplied to an end manipulator for correcting or restricting movement. Guidance virtual fixtures (GVF) are a type of virtual fixtures that help to guide a human operator along a desired path. GVFs are categorized under two control models: impedance and admittance. An impedance model uses position and velocity as the input and outputs a force. This model is usually low in inertia and back drivable. An admittance model, in contrast, uses force and position as the input and outputs a velocity to the end effector.  This model type helps to constrain the user’s movements to a given path or region. High inertia and non-back drivable actuators are typically used to enforce the desired stiffness (Abbott et al., 2007).

For the current application a pseudo-admittance model is used that primarily constrains a user’s movements to a path but do allow the user to “break free” to freely explore other areas of the diagram. Such “soft” haptic fixtures have previously been described by (Bowyer et al., 2007). The method allows our device to seamlessly adapt to both passive and active tactile exploration methods used with traditional tactile maps.

The main goal of the admittance GVF model is to respond to user input by encouraging desired motions, such as to follow a curve, and limiting undesired motions, such as moving away from a curve. The position of the end manipulator Pe is taken and used to calculate the closest point on the virtual path (curve) to the end manipulator.

With the closest point achieved, we can calculate the desired d and undesired τ direction unit vectors which are dependent on the position of the end manipulator Pe (see Figure 3). The two unit vectors are the tangent and normal vectors of the path at the closest point respectively. The input force F from the hand is projected onto the desired and undesired directions to produce the desired Fd and undesired Fτ forces of the user.

$\stackrel{\rightharpoonup }{F_d}=F\ast \stackrel{\rightharpoonup }{u}$                                                          (1)

$\stackrel{\rightharpoonup }{F_{\tau }}=F\ast \stackrel{\rightharpoonup }{\tau }$                                                          (2)

From here the undesired force Fτ of the user is damped by an admittance value kτ and thanks to direction τ it is also redirected towards the path as well (like an attraction well to the path). The resulting product can be summed with the desired force Fd to produce the final output force Fvf used to adjust the user’s movements.

$\stackrel{\rightharpoonup }{F_{\mathrm{vf}}}=\stackrel{\rightharpoonup }{F_d}+K_{\tau }\ast \stackrel{\rightharpoonup }{F_{\tau }}$                                               (3)

The admittance model uses an admittance variable   to map the force to a final velocity (Bettini et al., 2004; Mihelj et al., 2012).

$\stackrel{\rightharpoonup }{V_{h,\mathrm{net}}}=\alpha \ast \stackrel{\rightharpoonup }{F_{\mathrm{vf}}}$                                                   (4)

Omni Drive Control Model

The omni drive of the device consists of three servo motors positioned concentrically 120° apart. To guide the user along a path on the virtual tactile diagram, the device’s drive module must take the commanded output velocity Vh,net (calculated through admittance control with virtual fixtures, as above) and decompose it into three distinct velocity components Vi = V1, V2, V3. More specifically the output for each servo is an angular velocity:$\stackrel{\cdot }{{\Theta }_{\iota }}=\stackrel{\cdot }{{\Theta }_1},\stackrel{\cdot }{{\Theta }_2},\stackrel{\cdot }{{\Theta }_3}$  (Tzafestas, 2014). Relating the wheel velocity components to the rotation of the motors:

$V_i=r\ast \stackrel{\cdot }{\Theta }$                                                           (5)

The radius of the drive’s wheels is r and Vi denotes the wheel’s linear velocity. The relationship of the wheel velocity to the omni wheel design is further revealed in Figure 4 below. The small free rolling spinners oriented at, run along the radius of the main wheel frame. The spinners induce their own velocity Vi,spinner. Together the wheel velocity Vi and spinner velocity Vi,spinner produce a net velocity Vh which contributes to the final Vh,net. Vh is given:

$V_h=\sqrt{{V_i}_{}^2}+{V_{i,\mathrm{roller}}}^2$                                            (6)

As Figure 4 shows above, the resulting velocity of the omni wheel is denoted as V_h and is at an angle δ. Thus V_h can also be written as.

$V_i=V_h\ast \cos \left({\delta }_i\right)$                                                 (7)

If we take into account the angle of V_h, shown as γ, in respect to the drive platform’s x axis and angle β which is the angle of the wheel’s linear velocity V_i in respect to the x axis, then we get δ = γ – β.

$V_i=V_h\ast \cos \left({\gamma }_i-{\beta }_i\right)$                                            (8)

$V_i=V_h\ast \left[\cos \left({\gamma }_i\right)\ast \cos \left({\beta }_i\right)+\sin \left({\gamma }_i\right)\ast \sin \left({\beta }_i\right)\right]$       (9)

If we distribute V_h then we can get the x and y components of V_h as:

$$V_{\mathrm{hx}}=V_h\ast \cos \left({\gamma }_i\right);V_{\mathrm{hy}}=V_h\ast \sin \left(\gamma \right)$$                     (10)

Thus substituting this into equation (9) and calculating for angular velocity as defined by equation (5), we get the following simplified system of equation for wheel 1, 2, and 3 respectively:

${\Theta }_1=\frac{V_1}{r}=\frac{1}{r}\left[V_{\mathrm{hx}}\right]$                                               (11)

${\Theta }_2=\frac{V_2}{r}=\frac{1}{r}\left[{-\frac{1}{2}V}_{\mathrm{hx}}+\frac{\sqrt{3}}{2}{V}_{\mathrm{hy}}\right]$                               (12)

${\Theta }_3=\frac{V_3}{r}=\frac{1}{r}\left[{-\frac{1}{2}V}_{\mathrm{hx}}-\frac{\sqrt{3}}{2}{V}_{\mathrm{hy}}\right]$                               (13)

SOFTWARE TESTING

The main testing software written in C# (Figure 6) contains a diagram editing panel capable of producing various simple paths such as line segments. The software successfully reinterprets the paths as virtual fixtures that map an input force F into a modified output velocity Vh,net.

OUTCOME AND DISCUSSION

We have successfully developed an omni directional drive prototype. The Arduino Due C++ code allows for an input net velocity to be decomposed into three distinct velocities for each wheel in real-time. The device’s drive is thus capable of producing smooth 2D translations of various magnitudes. Currently the most significant limitation is a loss of traction of the wheels on a smooth surface. Preliminary tests show the device capable of producing 0.36 pounds of force in any given direction before the plastic omni wheels lose traction with the surface underneath. Figure 7 below shows the drive prototype and wheel design. Traction is improved by operating on a matted rubber toolbox liner.

 

FUTURE WORK

The current prototype is developed in two separate components. The virtual fixtures software is separate from the actual device drive system. Future work includes the merging of the two components into a cohesive real-time system that is responsive to user inputs. We are also working towards a custom wheel design to improve traction and to miniaturize the entire design. End user testing and further device performance assessment is planned for the future.