Development of Computer Vision-based Intelligent Electric Wheelchair System

Yutaka Satoh and Katsuhiko Sakaue

Information Technology Research Institute,
National Institute of Advanced Industrial Science and Technology (AIST)

Tsukuba, JAPAN

ABSTRACT

This paper describes the development of a support technology to enhance the independent mobility of disabled persons, featuring an electric wheelchair fitted with an innovative camera system " Stereo Omni-directional System (SOS) " . The SOS can capture omni-directional color images and distance information in real time, enabling automatic detection and avoidance of dangers in the wheelchair's traveling environment, and has a remote assistance function.

KEYWORDS

intelligent wheelchair, computer vision, wheeled mobility

BACKGROUND

In recent years, systems such as automatic speed adjustment and collision avoidance systems that use millimeter radar or stereo camera technology to sense the vehicle ahead have been applied to automobiles, and are becoming increasingly diffuse (1) . These functions are also of importance for electric wheelchairs. Although the market for electric wheelchairs is far smaller in scale than the automobile market, the need for these kinds of functions is greater. In particular, such technology is indispensable for providing disabled persons with a means to move around freely as they wish, safely and securely.

However, in contrast to cars, which can be assumed to move predictably in exclusive spaces, such as roads, electric wheelchairs have to share all kinds of living spaces, including indoor areas and crowded areas. For this reason, they require technology with more active sensing of the traveling environment. We thus installed the Stereo Omni-directional System (hereafter "SOS") capable of real-time capture of omni-directional, high-resolution color images and 3-D distance information, in order to provide traveling environment danger detection and avoidance, for preventing collision with obstacles and tipping over due to ground level irregularities, and also to enable transmission of omni-directional color images to a remote location for easy remote assistance.

SYSTEM CONFIGURATION

Figure 1: Overview . (Click image for larger view)

Figure 1 gives an outline of the system. The SOS (Fig ure 2) is a novel camera system capable of real-time, high-resolution capture of omni-directional color images and distance information, without any blind spots, in an extremely compact fist-sized design, 11.6 cm in diameter and 615 g in weight (2)(3) . The omni-directional color images and distance information acquired by the SOS are transmitted to a remote support person or a control PC (installed in the electric wheelchair) equipped with an automatic danger avoidance function.

In addition to being able to monitor the scene via a mobile phone or other means, the conveyed omni-directional images can be observed in an immersive display system to achieve a "telepresence" effect, which makes the remote support person feel present at the scene. The automatic danger avoidance system uses omni-directional distance information to detect obstacles and ground surface irregularities in real time, and then effects control to reduce speed or stop the wheelchair, as necessary. This kind of automatic danger avoidance function can serve to reduce the burden on the remote support person. For example, it makes conceivable a model in which one remote support person is able to support multiple electric wheelchair users.

Figure 3 provides an external view of the electric wheelchair prototype. The positioning of the SOS above and forward of the user's head offers the following advantages: (1) a wide observation range can be provided, without blind spots in the electric wheelchair's surroundings; (2) there is no hindrance to getting in and out of the wheelchair; and (3) the SOS is positioned at approximately body height, thereby providing clearance in normal living spaces.

TRAVELING ENVIRONMENT DANGER DETECTION

Here we describe the function for automatic detection and avoidance of dangers that tend to occur commonly and frequently, such as impacts with obstacles and tipping over due to level irregularities. In this system, danger factors in the traveling environment are detected from the omni-directional stereo information supplied from the SOS. The detection objects considered in this study are as follows: (1) travel obstacles: obstacles on the ground surface, level irregularities, etc. (2) collision objects: pedestrians, walls, desks, etc. (3) other: ropes or beams suspended in the air, etc.

Figure 4 shows an omni-directional distance image actually captured by means of the SOS. The higher the brightness of the pixels, the shorter the distance. Figure 5 is a histogram representing an orthogonal projection of the distance information onto the floor surface. This shows how objects that can potentially obstruct the movement of the electric wheelchair (center) can be detected. As the wheelchair is in actual movement, the histogram is used to narrow down the possible region of travel. Furthermore, in order to detect objects that are difficult to express in a histogram, such as a rope or beam suspended in the air, the system considers the 3-D position of edges that are near the wheelchair in the direction of travel. It can then activate controls accordingly. Note also that, in consideration of the user's intentions, the system will not, in principle, effect automatic avoidance behavior. That is, it will only reduce speed or stop the wheelchair.

USER CONDITION DETECTION

In this system, the SOS is positioned above and in front of the head of the user. Thus, while the environment surrounding the electric wheelchair is sensed, as described above, the user's riding posture and gestures can also be detected. Figure 6 shows an example of the system detecting an abnormality in riding posture (the user is slumped), and effecting an emergency stop of the wheelchair. Since during ordinary travel the upper body of the user remains essentially static, when the system detects any large change in posture it judges that an abnormality has arisen in the condition of the user, and thus activates an emergency stop. To detect the posture of the user, detailed position and shape information can be obtained, using the distance information obtainable from the SOS. From this, the concept can be expanded to the detection of finger-pointing gestures.

CONCLUSIONS

In this study we installed the SOS capable of real-time capture of omni-directional color images and distance information in an electric wheelchair, for the purpose of supporting independent mobility for disabled persons, and through the active collection of travel environment data, we developed a system to ensure user safety. We plan to expand on this research by conducting repeated experiments on real wheelchair travel, and to technologically enhance the automatic danger avoidance function. In addition, we intend to further investigate the transmission of images to an immersive display system and remote operation.

REFERENCES

1. H. H. Meinel, "Commercial Applications of Millimeterwaves: History, Present Status and Future Trends",IEEE Trans. Microwave Theory and Techniques, Vol. 43, No. 7, pp. 1639-53, 1995.
2. H. Tanahashi, D. Shimada, K. Yamamoto, and Y. Niwa, "Acquisition of Three-Dimensional Information in Real Environment By Using Stereo Omni-directional System(SOS)", Proc. IEEE International Conference on 3D Digital Imaging and Modeling (3DIM2001), pp.365-371, 2001.
3. S. Shimizu, K. Yamamoto, C. Wang, Y. Satoh, H. Tanahashi, Y. Niwa, "Moving object detection by mobile Stereo Omni-directional System (SOS) using spherical depth image", Pattern Analysis & Applications, 2005.

Author Contact Information:

Yutaka Satoh
National Institute of Advanced Industrial Science and Technology (AIST)
Central 2, Umezono 1-1-1,
Tsukuba, Ibaraki 305-8568, JAPAN,
Office Phone +81-29-861-2339
EMAIL: yu.satou@aist.go.jp