Tele-imaging System for Remote Assessment of Home Environment Accessibility

Jongbae Kim, PhD1 and Erik A. Porach BA2


University of Pittsburgh, Pittsburgh, PA 15203

ABSTRACT

While home modification has become recognized as an important intervention strategy for the quality of life of people with disabilities, the availability of skilled professionals with experience in home modifications for accessibility is limited. A Remote Accessibility Assessment System using Virtual Reality was developed by a University of Pittsburgh research team, which enables clinicians to assess the wheelchair accessibility of building environments from a remote location. The system was successful in assessing the accessibility without visiting the remote site. However, the research showed that it was difficult for a novice photographer to take good pictures for 3D reconstruction. In order to overcome this hurdle, we developed a Tele-imaging system using high-end IP cameras. We also improved 3D reconstruction protocol by determining how to merge partial3D models into a a model for the entire house. We constructed two whole 3D-models of a house by using the Tele-imaging system for a 3D model and a digital camera for another one. The result showed the Tele-imaging system can enhance the efficiency of our remote accessibility assessment system.  

KEYWORDS

Home Modification, Wheelchair User, Virtual Reality, Accessibility, 3D Reconstruction

BACKGROUND

Home modification has come to be recognized as an important intervention strategy to manage health care conditions, maintain or improve functionality, ensure safety, and reduce the wheelchair user’s dependency on others [1]. Effective home modification requires consultation with skilled professionals capable of assessing the home environment and identifying changes necessary to meet the wheelchair user’s needs. While there are many building and remodeling contractors able to perform the modifications, the availability of skilled professionals with experience in home modifications for accessibility is limited. Providing services in rural areas is particularly difficult. Such service requires lengthy travel times that increase cost and consume the limited time of skilled professionals.

Kim and Brienza [2] developed a remote accessibility assessment system (RAAS) through three-dimensional reconstruction technology. It enables clinicians to assess the wheelchair accessibility of users' built environments from a remote location. Although the previous version of RAAS showed its potential value through field trials, we found some limitations, e.g., difficulty for anovice photographer to take good pictures for 3D reconstruction and 3D models of only subparts of the whole home environment were created [3]. In this study, we improved the system by developing a Tele-imaging system and a whole house 3D-model reconstruction protocol in order to overcome these limitations.

RESEARCH QUESTION

The specific aims of this study were as follow:

  1. To develop a Tele-imaging system through which the 3D reconstruction technician can take pictures of remote home environment via a high-end IP camera.
  2. To test the feasibility with which the Tele-imaging system can improve the efficiency of the RAAS.

METHOD

Development of Tele-imaging System

Image shows client’s side where a part-time undergraduate student wears a cordless phone and headset and holds a remote of pan-tilt module under the camera. It also shows the technician’s side where a technician looks at the remote site of the client’s home at high frame rate via the video conferencing program provided by the camera company and an ftp program through which he can see the pictures, taken and transferred, at almost real time. Figure 1 : Web-based Tele-imaging System (Click for larger view)

We sent an undergraduate student with a digital camera and guidelines for successful picture-taking to the client’s home so that he could take pictures of client’s home environment in the previous version of RAAS. But it was not easy for a novice photographer to take 2D photos good enough to create 3D models. Therefore, we developed a remote imaging system through which a 3D reconstruction technician can take pictures of the client’s home environment from a distance.

In this updated protocol, An IP camera was sent with an undergraduate student to the client’s home at a remote location and we connected our research center and client’s home via a high-speed internet connection (Cable Modem or DSL). Then the IP camera showed us the remote home environment and a technician directed the student to move, pan, and tilt the camera by phone to where the angle of the camera was appropriate to take a good 2D snapshot from the remote research center. Figure 1 shows the client’s side where a part-time undergraduate student wears a cordless phone and headset and holds a remote of pan-tilt module under the camera.  It also shows the technician’s side where a technician looks at the remote site of the client’s home at high frame rate via the video conferencing program provided by the camera company and an ftp program through which he can see the pictures, taken and transferred, at almost real time.       

We can find many webcams and surveillance IP cameras on the market these days but most of them only provide a low resolution image snapshot function because they do not want to sacrifice their video frame rate. They want to see video clips with at least a frame every 3-5 seconds. Therefore we adapted a high-resolution IP camera, the IQeye 301 (IQinvision, Inc.), which provides better camera coverage, better picture detail and an overall superior IP video experience. This camera was also designed to record the high-resolution images on the local side, such as the memory card of the camera or the local computer connected to the camera via USB or Firewire, because current internet speeds do not allow the transfer of high-resolution images in real time (i.e. at least a frame per a second).  The viewer software allows pictures to be taken manually, but this function opens a file save dialog which requires a filename entry for every picture taken.  In addition, this function only saves the photos in the current low-resolution that is required for streaming. 

Image shows two jumpers connected between input and output ports on the backside of the cameraFigure 2: Jumpers on camera ports (Click for larger view)

Another feature of this camera is a trigger function which causes the camera to take pictures either when the camera sensor detects motion, periodically (every 10 seconds or every minute), or when it is triggered by an external device plugged into the input port.  The trigger function can also be used to control another device remotely through the output port using a button on the software viewing screen.  To get our photos to our remote location and in a high-resolution format, we connected a jump between output and input [Figure 2] which allowed us to remotely trigger the camera’s own automatic function.  This automatic function was set in the onboard memory of the camera to send a high-resolution image file with a time-stamped filename to an ftp server.  [Figure 3]  This ftp server could be the technician’s PC or a local ftp server.

Image shows a screen captured from trigger tap in the set-up menu of the IQeye video conferencing software. It shows that Trigger was set to act with FTP and  FTP was set with FTP hostname of ftp.pitt.edu , username, password, filename format, and file path.Figure 3: Trigger Settings page showing FTP options

This IQeye camera can take pictures in high-resolution as well as with wide fields of view because it provides various focal lengths; from 4.2 to 10mm. We can reduce the time and effort significantly in creating the 3D models because the technician who will create the 3D models can take appropriate pictures himself/herself as an expert while he/she can see the client’s home environment with streaming video.  Additionally, he/she will be using high-resolution and wide angle 2D photos compared to lower resolution, standard angle photos.

Feasibility Test

We took advantage of commercial photogrammetry software, Photomodeler Pro (EOS LTD) as a 3D reconstruction tool. When we developed the previous version of RAAS, we used Photomodeler 4.0, in which we could create a 3D model for each subpart of the home environment such as bathrooms, bedrooms, hallways, entrances, etc. but there was no function to incorporate subparts into an integrated 3D home model.

 In this study, we were able to create a 3D model of the client’s entire home by using the “Open Merged Project” function which was added in Photomodeler Pro 5.0.  By defining points within each room that are shared with the adjacent rooms, the software is able to merge the separate room models into one overall house model.

The use of high-resolution and wide angle images enabled us to create much better 3D models more easily and has provided a better view of the virtualized environment. We can compare two models: a whole home 3D model of a client’s house created by using a consumer level digital camera with standard lens and another 3D model of the same house by the IQeye IP camera. Through this feasibility test, we checked if several factors which were found to be critical in the previous imaging method were improved in the Tele-imaging system. We rated them with the Likert scale from 1 to 10, worst to best, to evaluate the two imaging methods.

RESULTS

The results of these feasibility tests are summarized in the Table 1.

Table1. Comparison of critical factors in between projects by old and new technologies
Projects Critical Factors New Project (wheelchair user’s home) Old Project (wheelchair user’s home)
Ease of taking good pictures
8
3
Ease of creating 3D models
8
5
Completion degree of virtual reality
9
4
Time to create 3D models
7
4
Communication among project members
9
3
Accuracy of measurement
9
9
Ease of understanding the architectural structure
9
7
Average
8.4
5.0

Looking at the Likert ratings in the table, we see that the accuracy of measurement was the same for both of the new projects as it was for the old project.  However, in all other catagories there is remarkable improvement in both of the updated RAAS projects over the older system.  Time consuming to reconstruct 3D models in the PhotoModeler still remains the most challenging aspect of the updated RAAS system, even though it has been improved.

DISCUSSION

Web-based remote accessibility assessment via2D-to-3D home environment reconstruction technology is a feasible and valid solution to meetremote consultation requirements, which serves people who live in rural areas and need home modifications because of mobility impairments. A study by Federal Communication Commission shows that high speed internet service is available in 99% of the residential areas in the United States. This makes our approach a favorable option for assessment over tele-imaging and online collaboration. However some residents don’t use high-speed internet even though it is available and some rural areas in which the high-speed internet is not available could be where this remote assessment system is most needed.

Besides indoor constructs, our approach could also support outdoor environment assessment. By adding wireless networking systems, it could take outdoor pictures of building areas such as the main entrance, porch, yard, playground, etc., which may require modifications (e.g., having ramps in place of staircases). Although we improved the ability to reconstruct 3D models from 2D photos by using the Tele-imaging system and updated features of Photomodeler Pro, it requires skillful, and time consuming manual efforts to make 3D models with Photomodeler.

REFERENCES

  1. Sanford JA, Pynoos J, Tejral A, Brown A. Development of comprehensive assessment for delivery of home modifications. Physical and Occupational Therapy in Geriatrics 2002; 20(2): 43-55
  2. Kim J. and Brienza D., "Development of a remote  accessibility assessment system through three-dimensional reconstruction technology", Journal of Rehabilitation and Development, Vol. 43, No. 2, Pg. 257-272, March/April 2006
  3. J. Kim, D.M. Brienza, R.D. Lynch, R. A. Cooper, M.L. Boninger. “Effectiveness Evaluation Of A Remote Accessibility Assessment System For Wheelchair Users Using Virtualized Reality”,  Archive of Physical Medicine & Rehabilitation,  2008, - To appear

ACKNOWLEDGMENTS

This work was funded through RERC on Telerehabilitation by the NIDRR. Opinions expressed are those of the authors and do not necessarily reflect those of the funding agencies.

Author Contact Information:

Jongbae Kim, PhD, University of Pittsburgh, 2310 Jane Street, Pittsburgh, PA, 15203 Office Phone (412) 586-6909  E-mail: jbkim@pitt.edu