Comparison of Web-based Videoconferencing Systems for Telerehabilitation Applications

Jongbae Kim, PhD1 and Ya’aqov Fuhrman2
1 University of Pittsburgh, Pittsburgh, PA
2 University of Michigan, Ann Arbor, MI

Abstract

Telerehabilitation (telerehab) is predicated on the use of remote communications. For this study, seven separate commercially available instant messenger (IM) systems were reviewed and evaluated for: ease of use, ease of set-up, video resolution and frame rate, audio clarity, video-audio synchronization, and connection reliability.  Each IM system was evaluated in conjunction with five separate connection modalities. Results were recorded using a five-point Likert scale. Of the seven systems evaluated, three demonstrated superior qualities. Results from this study and evaluation will be used to recommend the Skype, MSN, and ICQ instant messenger (IM) systems for telerehab purposes.

Keywords:

telerehabilitation, Internet, Instant Messenger, videoconferencing, web-based

Background

Telerehabilitation is “the application of telecommunication, remote sensing and operation technologies, and computing technologies to assist with the provision of the delivery of medical rehabilitation services at a distance.” [1] Telerehab services can be provided via phone, e-mail, and/or videoconferencing.  Videoconferencing (VC) is the most effective form of telerehab, due to the visual nature of the system and instant feedback.  In projects associated with the RERC on Telerehabilitation, VC can bridge the distance between rehabilitation centers located in major metropolitan areas and less accessible locations [2]. And a rehabilitation specialist can remotely view a client’s home and assess wheelchair accessibility.  Measurements made using the camera allows the specialist to make detailed recommendations [3].  Small, mobile web-based VC has advantages over large, more permanent systems, and can connect directly to a computer with reliable Internet access.  The webcams used are generally inexpensive, portable, and compatible with both MAC and PC computers.  Therefore, the RERC is researching the feasibility of web-based VC systems for telerehab purposes.

Ideally, a telerehab VC system will provide the users with clear, high-resolution images and clear audio, with consistent visual-audio synchronization.  In addition, the system should be inexpensive to purchase and operate, easy to set up in the client’s home, and be accessible by the clinician from a remote location. While the large and permanent VC systems were compared [4], we cannot find the applicability data for small web-based VC systems.

Methodology

Seven commonly available, no consumer cost, instant messenger (IM) systems were selected for testing: AIM®, Conference XP® , CU-See-Me® , ICQ® , MSN® , Skype®, and YAHOO®. Five internet connection methods were selected: Dial-up, Hot-Spot (public Internet access), home wireless, home cable, and LAN (local Internet network).  We attempted to download and use all systems using LAN.  CU-See-Me and Conference XP were not evaluated because they did not provide a secure connection and service for instant messaging (IM).  AIM, ICQ, MSN, Skype, and Yahoo were fully evaluated.  Dial-up and LAN were both tested using the University of Pittsburgh network.  Hot-Spot was tested at a local internet coffee shop. Home cable was Comcast service. A 802.11g and 802.11b compatible wireless router was used for home wireless. Tests were conducted to assess frame rate, resolution, audio clarity, synchronization, reliability, and ease of setup and usage.  We used two Logitech QuickCam Pro-5000® cameras on both ends.  We chose this camera because it was compatible with our computers, and contains a built-in microphone.  The two computers used were: (1) a Dell PC desktop with Intel Pentium 4 connected to the University of Pittsburgh LAN, and (2) a Sony Vaio PC laptop with Intel Pentium Centurino. Both operating systems were the Microsoft Windows XP.

Initially, the camera was installed and set up on each of the computers. Then the appropriate web-based VC system was downloaded. Each IM software was tested with the LAN connection within the institution for the baseline. A video conference was initiated for each IM system-Internet connection method combination. Each system’s functions were explored: frame rate, video resolution, audio clarity, synchronization, reliability, ease of use, and ease of set-up.  Reactions to using the systems were discussed between researchers and ranked using a Likert scale. A five point Likert scale was used, with one indicating low preference qualities and five indicating high preference qualities; a zero indicated a non-working function.  After all systems were evaluated, the mode and median function values were determined, and the system usability was assessed and ranked for each system-connection. We summarized using mode and median as the Likert scale provides ranked data.

A comparison of each IM system - connection method combinations was based on the mode and median values for each system function.  The mode determined initial ranking.  If more than one system had the same rank based on the mode, then function value medians were compared to determine the system rank.  If both the median and the mode for the system-connection combinations were the same, mean values could give some more idea to determine which one was more often preferred.

Results

As we used the mode and median to assess system ranking, ICQ, MSN, and Skype were determined to be the best systems for web-based VC overall.

Table 1: System ranks for each connection method
System Frame Rate Video Resolution Audio Synchronization Reliability Easy to Use Easy to set up Mode Median Best or Tie  Mean
LAN MSN
5
3
5
4
4
5
4
5
4
4.29
ICQ
5
3
4
4
4
5
3
4
4
4
AIM
4
2
5
5
3
4
4
4
4
3.86
YAHOO
1
2
2
2
3
2
4
2
2
2.29
SKYPE
5
4
5
4
5
5
5
5
5
X
4.71
Home Cable MSN
4
4
5
4
5
4
4
4
4
X
4.29
ICQ
5
2
4
4
5
4
3
4
4
X
3.86
AIM
4
4
4
4
4
2
2
4
4
X
3.43
YAHOO
2
2
5
3
4
3
2
2
3
3
SKYPE
4
4
4
4
5
5
5
4
4
X
4.43
Home Wireless MSN
4
4
4
4
3
5
4
4
4
X
4
ICQ
5
1
3
4
4
4
4
4
4
X
3.57
AIM
5
4
1
0
0
0
2
0
1
1.71
YAHOO
2
2
0
0
3
4
1
2
2
1.71
SKYPE
4
4
4
4
4
4
5
4
4
X
4.14
Hotspot MSN
1
5
5
2
3
2
5
5
3
X
3.29
ICQ
5
4
4
5
1
1
4
4
4
3.43
AIM
0
0
0
0
0
0
0
0
0
0
YAHOO
1
3
4
2
4
4
5
4
4
3.29
SKYPE
4
4
5
4
4
4
4
4
4
4.14
Dial-Up MSN
0
1
1
0
1
1
3
1
1
1
ICQ
2
2
1
2
3
2
3
2
2
2.14
AIM
2
2
0
0
0
0
0
0
0
0.57
YAHOO
1
0
0
0
0
0
0
0
0
0.14
SKYPE
3
4
4
3
3
3
3
3
3
X
3.29

For LAN connection, Skype got the highest mode and median but MSN, ICQ, and AIM also ranked high.

Skype, MSN, ICQ, and IM had the same ranks in aspects of mode and median at Home Cable connection.

Skype, MSN, and ICQ again had the same scores of mode and median at Home Wireless connection.

As for Hot-Spot connection, while MSN had the highest value of mode, it had lower median than ICQ, SKYPE, and YAHOO. AIM failed to connect the system at Hot-Spot and MSN and YAHOO showed score “1” for video frames as the video frames were very jerky.

Lastly, for Dial-up connection, YAHOO and AIM were not reliable at all, and MSN and ICQ were a little better but still not good at all for videoconferencing. Skype showed the best performance at Dial-up Connection but still had a question whether it could be enough for videoconferencing.

Overall, Skype had best score for four connections, MSN for three, and ICQ for two. But if we take into account of the inconsistency between mode, median and mean scores at Hot-Spot connection, Skype can be said to have the best score and MSN and ICQ had the next similar scores. YAHOO and AIM had the most connection problems.  For home wireless and dial-up, Yahoo would connect, but lose the connection after evaluation had begun.  For home wireless, dial-up and Hot-Spot, AIM required multiple connection attempts, or would not connect at all.

DISCUSSION

There are a wide variety of readily available VC systems that could be suitable for telerehab. This study identified three possible systems that can be used for telerahab: Skype, ICQ and MSN. The best systems were easily downloadable, reliable with multiple functions and user-friendly interfaces.  These will be key features for clinicians in the field since they will have limited accessibility to tech support and need a reliable system for communications.  Based on this study, Skype, ICQ and MSN are suitable web-based videoconferencing systems for telerehab purposes.

This study focused on video and audio quality, usability and accessibility for point-to-point videoconferencing between clinician and client. For medical applications, one of the most important requirements of VC system is network security. All of IM systems, Skype, AIM, ICQ, Yahoo, and MSN are not secure. In addition, large amounts of network traffic can affect videoconference quality [5]. Future research needs to focus on evaluating MAC compatibility and system security. 

Additionally, while initial explorations of whiteboard sharing, program sharing, file transfer, multi-casting, and snapshot fuctions were conducted, whiteboard sharing and program sharing worked well in only Skype; file transfer rate varied; multi-casting was not supported by any system; only Skype didn’t provide the snapshot function. These functions need to be investigated more comprehensively in the future studies.

References

  1. Cooper RA, Fitzgerald SG, Boninger ML, Brienza DM, Shapcott N, Cooper R, Flood K. Telerehabilitation: Expanding Access to Rehabilitation Expertise. Proceeding of the IEEE, 2001; 89 (8):1174-1193
  2. Schein R, Schmeler M, Kim J, Kim K, Regulski A, Fatula A. Proposed Protocol for Remote Wheelchair Consultation Using Telerehabilitation. Proceeding of RESNA 29th International Conference, 2006
  3. Kim J, Brienza DM. Development of a Virtual Reality Telerehabilitation System for Analyzing Accessibility of the Built Environment.  Journal of Rehabilitation Research and Development  2006; 43(2): 257–272
  4. JANET Video Technology Advisory Service, Evaluation Of ISDN/IP Videoconferencing Equipment, [Online] 2006, Available from: URL: http://www.video.ja.net/evaluation/index.html
  5. Patel V, et al. editors MEDINFO 2001, Chapter1, Crudele M, Cinque G, Neglia G, Razzicchia G, Venditti M. IP Videoconferencing using a Quality of Service Public Network. Amsterdam: IOS Press, 2001

Acknowledgements:

This work was funded through RERC on Telerehabilitation by the NIDRR and National Science Foundation (NSF) Research Experiences for Undergraduates. Opinions expressed are those of the authors and do not necessarily reflect those of the funding agencies.

Ya’aqov Fuhrman, a sophomore student of Mechanical Engineering Department of University of Michigan, worked for this study as an intern student at University of Pittsburgh 2006 summer.

Contact:

Jongbae Kim PhD
University of Pittsburgh
Department of Rehabilitation Science and Technology
2310 Jane St. Suite 1300
Pittsburgh, PA 15203
412-586-6905, jbkim@pitt.edu

Highlights

  • Source Ordered
  • No Tables
  • Very Compatible

Gargoyles

Disney produced a television show in the mid 1990s called Gargoyles. It's a great show and I'm a big fan. A few years ago Disney started to release the show on DVD. The last release was of season 2, volume 1. That was two years ago. Volume 2 has not been released. Why? Poor sales. So if you should find yourself wanting to support my work, instead I ask you pick up a copy of season 2, volume 1. It's a great show and you might find yourself enjoying it.