A System for Providing Preview Capability to Blind People Navigating in Familiar Places
ABSTRACT
The Layout Change Detector is a prototype system for providing preview capability to blind persons in order to avoid accidents during navigation in familiar areas. The system has two major parts: the first uses computer vision for the detection of changes in the layout of a room between consecutive visits by the blind person and the second generates alert messages for describing those changes every time the person enters the room. This paper focuses on the message generation part where the codification of the messages and the medium of delivery are the most important issues. Speech was preferred over tactile displays for the delivery. A message codification scheme was developed based on the relative position of layout changes with respect to a room's landmarks. User evaluation endorsed the viability of such a system, although important improvements were suggested on how messages are structured and delivered.
KEYWORDS
Blindness, preview, perceptual anticipation, computer vision, mobility aids
BACKGROUND
When a person navigates through space, he continuously acquires perceptual information directly from that space. Timely provision of accurate and specific perceptual information allows for good perceptual anticipation and, therefore, good control of the mobility task, assuring safe travel almost without the help of memory (1). However, blind people have a severely reduced key perceptual system needed for mobility; they can perceive partially only a very small space around them that is not enough to allow for real perceptual anticipation. Even with aids like the long cane (with a limited range and restricted to detect spatial features on or near the floor), the perceptual anticipation is still very limited. Blind people can replace the missing perceptual anticipation with cognitive anticipation, which depends heavily on memory (1). This means that a memorial representation of a space has to be constructed using the perceptual information that can be acquired. With time and in familiar places, the memorial representation allows blind people to move fast and confidently without the use of mobility aids.
Even with excellent use of cognitive information, blind people are at a great disadvantage compared to sighted people when negotiating unexpected changes in a space. The reason is that memory inherently means past time. Sighted people can detect and negotiate any changes in a space that occurred since they last visited it; they always have a good preview of the space, regardless of the cognitive information they have about it. Indeed, in unfamiliar places this preview capability is key to comfortable travel. Blind people, on the other hand, lack this preview capability; in familiar places, they know how the space was the last time they travelled through it, but cannot detect the changes with anticipation.
RESEARCH QUESTION
The Layout Change Detector (LCD) system has the objective of exploring how to provide better preview capabilities for blind persons travelling in familiar spaces. The restriction to familiar places is important because it assumes that the person using the system already has good memorial representation of the space and uses cognitive anticipation to navigate in it. Because it aids in navigation, the LCD can be catalogued as a mobility aid, but it is not intended for general use nor to be a complete visual substitution system. Unlike traditional mobility aids, which are portable and intended to aid the blind person to navigate under any circumstances and in any kind of environment (2), the LCD system is intended to act as an environmental sensor fixed in a constrained environment with a pre-defined preview area. The main problem is how to convey preview information to the user in an efficient and usable way. As opposed to a sighted traveller who can quickly process preview information, a blind traveller may suffer an overload of perceptual information generated by the mobility aid and usually requires more time to react to that information.
METHOD
 |
The proposed physical structure of the LCD system is shown in Figure 1. The blind person sensor determines when he leaves the room and a static statistical model of the layout at that moment is generated. The same sensor determines when the blind person re-enters the room and at that moment the current layout is compared to the model and the changes are determined. The change detection algorithms are based on colour models that deal with camera noise and the effects of illumination changes (orientation, colour and intensity). A modified chromaticity model ( uv space) and a colour ratios model ( m space) were combined to achieve this (3). The detected changes are then described using speech to the blind person to make him aware of them. Speech was selected over tactile matrices as the preferred interface medium for three main reasons: it offers a more general solution since it requires no or almost no training, it is capable of delivering information from a distance without the user looking actively for it, and it is less expensive since it does not require special hardware.
 |
Since the change detection scheme does not include object recognition capability, layout changes were characterized through qualitative descriptions of their size and the relative position of their centroids with respect to fixed (or infrequently moved) reference points (landmarks) such as windows, sofas and tables (see Figure 2). Generation of alert (preview) messages was separated into two parts: analysis and message construction. The first analysed the area and position of each change, linking it with its two nearest landmarks. The second decided which size descriptor ( small , medium , big ) and position descriptor ( near , between ) were most suitable and constructed the appropriate message. The position descriptor was selected based on the ratio of the distances from a change's centroid to the nearest and second nearest landmarks. Thresholding this ratio segmented the scene into specific zones for all the possible near and between descriptions that can be produced with the defined set of landmarks. Up to three changes were reported at once using sentences either of the form “there is a {small, medium, big} change near <landmark name>” or “there is a {small, medium, big}change between <landmark name 1> and <landmark name 2>”. In the case of more than three changes, the message began “Multiple room changes have been detected. The biggest change is…”. The complete alert message was passed to a speech synthesiser.
User evaluation was conducted with three otherwise healthy blind people with good navigational skills. Key elements of evaluation were understandability and duration of alert messages, and the ability of the user to find the exact position of a layout change from the description conveyed in an alert message.
RESULTS
Users agreed that, with reliable change detection, the mere fact that a message was generated (regardless of its content) was useful because it warns the blind person to be cautious. Alert messages were considered well structured and understandable and all users were able to find the position of the described changes. An adequate definition of the landmarks proved to be critical: the nine landmark pattern in Figure 2 produced unambiguous descriptions. The average message duration of 4.1 seconds (over 23 tests) was considered too long for a real domestic application; the pause for hearing such a long message disrupts the natural flow of daily activities within the house. The longer of these messages described several changes. The users considered it necessary to establish a priority order for the changes depending on the risk they represent for the user, so that those with the highest priority are described first.
DISCUSSION
The LCD system appears capable of achieving its primary objective of helping blind persons to avoid accidents when things have changed in a room since his last visit. Although there was no problem producing clear, descriptive messages about the room's changes, trying to describe the whole situation of the room when the person enters into it proved to be too disruptive because of the time required to hear the lengthy message. To solve this problem in upcoming versions of the system it is proposed to divide the layout into high and low risk zones from which the changes can be ordered by priority and discard those that do not pose a risk to the user; this would also help to address the ambiguity problem. Moreover, instead of describing all the changes at once and overloading the user with information, it is also proposed to follow the blind user as he moves in the room and describe each change only once it becomes relevant to his current position. Going back to the preview concept, this means that the preview area would be reduced from the whole room to a pre-defined area positioned relative to the blind user. The system may continue to generate a general preview of the room using the static models approach, but provide only part of that information when it is needed in the current limited preview area linked to the moving user.
REFERENCES
- Foulke, E. (1996). The roles of perception and cognition in controlling the mobility task. International Symposium on Orientation and Mobility, Trondheim, Norway.
- Brabyn, J. A. (1982). New developments in mobility and orientation aids for the blind. IEEE Trans. Biomed. Eng., 29(4), 285-289.
- Martinez-Alarcon J.A. (2003). Layout Change Detector: a computer vision aid for the blind. M.Sc. dissertation, University of Dundee, Dundee, Scotland.
Author Contact Information:
Jorge A. Martinez-Alarcon, MSc,
Universidad Iberoamericana,
Prol. Paseo de la Reforma 880,
Lomas de Santa Fe, Mexico, D.F.,
14140, Mexico,
Office Phone +(52-55) 59504301
EMAIL: jorgea.martinez@uia.mx
