-->

 

Hestia – A Home Evaluation App: Usability Analysis

Rochelle J. Mendonca, PhD, OTR/L1, Suzanne Perea Burns, PhD, OTR2,3, & Noralyn Davel Pickens, OT, PhD3, & Roger O. Smith, PhD, OT4

Temple University1, Medical University of South Carolina2, Texas Woman’s University3, University of Wisconsin-Milwaukee4

Abstract

Mobile apps and evaluations are increasingly being adopted by service providers working with older adults and individuals with disabilities who desire to age-in-place. Service providers prefer using mobile apps for various reasons including storing reference information, supporting measurements, facilitating reasoning processes, and easing the burden of documentation (Burns & Pickens, 2016).  However, app development occurs substantially faster than vetting processes, compromising app quality and security (Cummings et al., 2013; Shen et al., 2015).  Further, many apps available in various marketplaces are not used by anticipated end-users (Dye, 2016) which may be attributed to the lack of consideration of end-user wants and needs (Doebbeling, Chou, Tierney, 2006). In an effort to obtain information from end users about their needs during the development process, we tested the usability of HESTIA, a home assessment app to support home evaluation and modification practice. Valuable feedback was obtained from health care professionals who perform home evaluations related to the content, structure and flow of the assessment and the app, which are currently being evaluated and integrated by the research team into the app.

introduction

An increasing number of individuals with disabilities and older adults are living longer and desire to remain in their homes (Cook, 2006, Wiles, Leibing, Guberman, Reeve, & Allen, 2012). This leads to a greater demand for home evaluations and modifications to ensure that their home environments support independent living. There are several stakeholders involved in this process and mobile- and app-based technologies are more frequently being integrated into the service provision of home evaluations and modifications. App-based technology has the capacity to meet the needs of home evaluators and address the limitations of implementing evidence-based practice in home safety evaluations. They also have the advantage of being used across different stakeholders to allow easier transfer of evaluation and modification recommendations.

Today, evidence-based app design involving formative research methods with targeted stakeholders must be considered to meet end-user wants and needs for content, features, design, and organization for data collection and report building. Furthermore, integrating technology to support reasoning processes among home evaluators meets the needs of home modification consumers (Burns, Pickens, Smith, 2017). HESTIA, the Home Evaluation with a Strategic Triangulating Integrative Approach, a new home safety assessment and reporting app, fully integrates a Person-Environment-Occupation (PEO) approach to home environmental evaluation. Formative research was completed during the development of the content and features of the HESTIA to meet stakeholder and end user needs. HESTIA is also continuously being tested for its reliability and validity. An additional crucial development step for app based evaluation systems is usability.

Usability is an attribute of every product – just like functionality. Usability refers to how people work with products and it is tested by making sure people can find and work with the functions of the product to meet their needs (Benton & Bove 2011; Pruitt & Adlin, 2006). Usability testing employs techniques to collect empirical data while observing representative end users use the product to perform representative tasks (Stanton, Salmon, Rafferty, Walker, Baber, & Jenkins, 2013). Usability testing is intended to assist with creation of products that are easy to learn and use, and provide utility and functionality that are highly valued by the target population (Benton & Bove 2011). This paper presents the results of the usability testing of the HESTIA with healthcare professionals who perform home evaluations.

METHODS

A convenience sample of 83 healthcare professionals were recruited to evaluate the usability of the HESTIA app. HESTIA has approximately 1500 questions that are divided into assessment basics (including personal information, co-habitators, caregiver information, animals, images of the home, and home evaluation team members), the person section (including home situation, health conditions, areas of concern and personal assistive technologies), the occupation section (including skills such as hand and arm skills, transfer skills, reaching skills, mobility skills, organizational skills; and occupational performance such as personal ADL’s, instrumental ADL’s, and falls), the environment section (including general safety and a details evaluation of each room in the house). The HESTIA also includes questions to evaluate the home owner’s quality of life and problem list by level of concern. This assessment is intended to be comprehensive, usable, and to have the capacity to generate reports for easy transfer and sharing of information.

Six detailed case studies of either older adults or individuals with disabilities who had different home independent living needs were created. The participants were required to work in pairs or groups of three to rate the HESTIA app using one case study. Upon completion of navigating through the app, each group answered 7 questions (1) What is your impression of the navigation of the app?, (2) What is missing from the app?, (3) What are drawbacks of using the app?, (4) How has the app increased your awareness of different aspects of the home safety assessment?, (5) When, where, and how would you use this app in your practice?, (6) What learning modules or FAW’s would you want when using the app?, and (7) Other comments or thoughts?

This study qualitatively analyzed the feedback from the participants which are being integrated into the HESTIA app for future testing and dissemination.

RESULTS

This figure shows a screenshot of the HESTIA app. On the left panel, you see the different sections of the app including personal information, information about the person, occupation, and environment, as well as a reporting option. The taxonomy of questions is displayed on the left, with the definition of each question and the response options on the right. The taxonomy can break down into further detail as needed and is shown in the figure. In this screenshot, the questions on display are related to the environment section. At the bottom of the screen, are options for make general notes and drawings, to go back, to go to next unscored question and to go to the next question.
Figure 1: Drop Down Options Available in HESTIA
Each of the questions were individually analyzed to obtain feedback related to the content and structure of the app. In response to the question related to the navigation of the app, some of the responses were ‘straightforward’, ‘easy to use’, ‘flexible’, ‘definitions great for the home evaluation team to convey specifics’, and ‘drop down menu is useful to ensure all areas are evaluated (shown in figure 1)’. A + sign next to each question indicates that the question has further detail that is available through drop down.

This figure shows a screenshot of the HESTIA app. On the left panel, you see the different sections of the app including personal information, information about the person, occupation, and environment, as well as a reporting option. The taxonomy of questions is displayed on the left, with the definition of each question and the response options on the right. The taxonomy can break down into further detail as needed and is shown in the figure. In this screenshot, the questions on display are related to the person section. The question highlighted is related to pain. The definition provided is physical suffering or discomfort and the response options are no concern, mild concern or major concern. At the bottom of the screen, are options for make general notes and drawings, to go back, to go to next unscored question and to go to the next question.
Figure 2: Detailed Questions for Level of Concern
For the question about what was missing from the app, some responses included adding a search box for diagnosis, adding an RV under type of home, including family members in assessment basics including their age and role, adding a fall risk assessment, adding questions related to CPAP machines and oxygen equipment, and adding prosthetics under assistive devices.

Selected feedback related to the drawbacks of using the app include clarify what ‘next unscored’ means, time consuming the complete the assessment, learning curve required, and include the level of concern instead of just problem or no problem, which was integrated into the app as shown in Figure 2.

         Participants mostly agreed that the app increased their awareness of different aspects of home safety assessment. They appreciated the step by step process which makes them aware of all components to consider and could potentially provide guidance for new therapists. They also pointed out that the app made them think of home safety aspects that they may not typically consider such as water temperature as shown in Figure 3.

This figure shows a screenshot of the HESTIA app. On the left panel, you see the different sections of the app including personal information, information about the person, occupation, and environment, as well as a reporting option. The taxonomy of questions is displayed on the left, with the definition of each question and the response options on the right. The taxonomy can break down into further detail as needed and is shown in the figure. In this screenshot, the questions on display are related to the environment section. The question highlighted is related to the shower in the bathroom. There are several questions related to shower which can be checked if they are concerns such as threshold height, doors, slick surfaces and grab bars. At the bottom of the screen, are options for make general notes and drawings, to go back, to go to next unscored question and to go to the next question.
Figure 3: Question about Water Temperature Included in HESTIA
Participants were then asked if they would use the app in practice and when. They identified that they would use the app for home assessment, family education and training, care planning, conferences, from an inpatient and outpatient perspective for the home consultations, for ensuring safety in the home, as an educational tool for health professional students and for communicating with administrators. They were also asked what learning modules they would like when using the app. They listed “how to use link, “list of local vendors”, “definitions of response sets”, “how to customize assessment”, and “link to pictures of equipment and how to purchase them”.

Participants were then asked if they would use the app in practice and when. They identified that they would use the app for home assessment, family education and training, care planning, conferences, from an inpatient and outpatient perspective for the home consultations, for ensuring safety in the home, as an educational tool for health professional students and for communicating with administrators. They were also asked what learning modules they would like when using the app. They listed “how to use link, “list of local vendors”, “definitions of response sets”, “how to customize assessment”, and “link to pictures of equipment and how to purchase them”.

Last participants were asked to provide any additional feedback. Some constructive feedback provided includes clarify what assistance for sleep means, identify when information is provided by caregiver, ability to have a drop down menu for credentials of home evaluators, an easier way to input measurements for bathroom or other rooms, personal measurements such as leg length and weight, need for precautions for hip replacements, cardiac diagnoses, lifting, seizure disorders, section on medication in environment section is unclear,  unable to locate level of assistance for transfers, and include prosthetics and orthotics.

Participants also indicated that they appreciated the camera feature, the comment box, ability to customize the app, dictation features, and links to abledata.com

Conclusion

Results from the qualitative analysis of the usability testing provided valuable feedback regarding the content and the structure of the app. Participants were also able to go through the app to identify areas that were confusing or missing. The feedback is currently being evaluated for integration in the app. Based on the feedback, we are also clarifying the areas that appeared to be confusing to the participants. Integrating usability analyses into the evaluation and app development process ensures the creation of products that best meet the needs of the end users and stakeholders.

References

Burns, S. P., & Pickens, N. D. (2016). Embedding technology into inter-professional best practices in home safety evaluation. Disability and Rehabilitation: Assistive Technology, 1-7.

Burns, S. P., Pickens, N. D., & Smith, R. O. (2017). Interprofessional Client-Centered Reasoning Processes in Home Modification Practice. Journal of Housing For the Elderly, 1-16.

Cook, D. J. (2006). Health monitoring and assistance to support aging in place. Journal of universal computer sciences, 12(1), 15-29.

Cummings, E., Borycki, E., Roehrer, E. (2013). Issues and considerations for healthcare consumers using mobile applications. Studies in Health Technology and Informatics, 183, 227-231.

Doebbeling, B.N., Chou, A.F., Tierney, W.M. (2006). Priorities and strategies for the implementation of integrated informatics and communications technology to improve evidence-based practice. Journal of General Internal Medicine, 21, 50–57.

Benton, S. A., & Bove, V.M. (2011). Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Indianapolis, IN: Wiley Publishing, Inc.

 Dye, J. (2016, March 4). 77 percent of users never us an app again 72 hours after installing. Retrieved from http://www.androidauthority.com/77-percent-users-dont-use-an-app-after-three-days-678107/

Pruitt, J., & Adlin, T. (2006). The persona lifecycle keeping people in mind throughout product design (Interactive Technologies). Amsterdam, Boston: Elsevier : Morgan Kaufmann , an imprint of Elsevier.

Shen, N., Levitan, M.J., Johnson, A., Bender, J.L., Hamilton-Page, M., Jadad, A.R., Wiljer, D. (2015). Finding a depression app: A review and content analysis of the depression app marketplace. Journal of Medical Internet Research Mhealth Uhealth,3(1), e16.

Stanton, N., Salmon, P., Rafferty, L., Walker, G., Baber, C., Jenkins, D. (2013). Human Factors Methods: A Practical Guide for Engineering and Design. London: CRC Press.

 Wiles, J. L., Leibing, A., Guberman, N., Reeve, J., Allen, R. E. S. (2012). The Meaning of “Aging in Place” to Older People. The Gerontologist, 52(3), 357–366, https://doi.org/10.1093/geront/gnr098

Acknowledgements

The HESTIA Project is supported in part by the U.S. Department of HHS/ACL, National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR), grant number 90IF0083-01-00. The opinions contained in this presentation do not necessarily represent the policy of the Department of HHS, and you should not assume endorsement by the Federal Government.