RESNA 27th International Annual Confence
Website Usability: Evaluation to Accommodate the Needs of Individuals with Traumatic Brain Injury
Due to cognitive and physical impairments, individuals with traumatic brain injury (TBI) may experience multiple difficulties accessing information on the internet. This study was a qualitative analysis of the accessibility and usability of the Brain Injury Association of Colorado (BIAC) website. Researchers videotaped sessions of people with and without TBI as they navigated the BIAC website and systematically categorized comments. They viewed tapes of people with TBI a second time to record observed difficulties navigating the website. Both audio comments and visual observation are especially helpful in the evaluation of websites designed to serve people with disabilities. Ultimately both types of analysis can help web designers incorporate assistive technology directly into the website.
Brain injury, assistive technology, website, computer, disability
According to the Centers for Disease Control and Prevention, an estimated 1.5 million people incur traumatic brain injury (TBI) each year. Approximately 80,000 – 90,000 individuals sustain TBI resulting in a long-term or lifelong disability (1). TBI presents cognitive problems for many people, including impaired memory and language (4) as well as reading difficulties (5). People with TBI may also experience physical problems such as locomotor difficulties (4) and difficulty using their hands to perform daily activities (5).
In order to find needed resources, many people with disabilities use the internet because of its quick and easy access through links to vast quantities of information (3). Several different protocols have been developed to evaluate website usability with a non-disabled population. According to a website by the Microsoft Corporation, usability testing may be performed through observation of direct use, interview, questionnaire, or a combination of any of the above (6). Zimmerman, 2003, outlines methods for usability testing similar to those used by Microsoft. He uses the term “verbal protocol analysis” (p.5) for a method in which participants browse the website and say their thoughts aloud. This method has been used to assess specialized websites for women, minority, and low-income populations (7).
Studies that have assessed people with disabilities in the use of computers focused on external assistive devices and specialized software that can be used with a person's home computer. One study assessed these assistive devices using a qualitative Q-sort method on university students with multiple disabilities. Two of the most common complaints about these external devices were that they were not easy to transport to different places and were associated with failures or crashes (2). Because of the inconvenience of external assistive technology, it is of critical importance that local disability resource centers, such as the Brain Injury Association of Colorado, make their websites specifically tailored to the population they serve. Furthermore, these websites should be accessible without relying on the use of external assistive devices.
In what ways could the Brain Injury Association of Colorado (BIAC) website be more useful and accessible for persons with TBI, family members, and others seeking information?
Colorado State University (CSU) researchers videotaped 28 participants navigating the BIAC website ( www.biacolorado.org ). Participants were given a warm-up activity of browsing the local weather website to encourage them to verbally share all opinions. The researchers then used an open-ended format in which participants entered the BIAC website and explored areas of interest to them. The sessions were videotaped at an angle so that only the computer screen and verbal comments were recorded. Participants were prompted by researchers as needed to continue expressing their thoughts about the website, and the sessions were terminated when participants felt they had thoroughly explored the site. According to internal review board regulations all participants gave informed consent after reading an information sheet.
| Type of Participant | Subtype | Gender | Age Group | Total |
|---|---|---|---|---|
| person with TBI | 6 male | 2 age 20-29 | 10 | |
| 4 female | 2 age 30-39 | |||
| 4 age 40-49 | ||||
| 2 age 50-59 | ||||
| family members | 3 female | 1 age 30-39 | 3 | |
| 1 age 40-49 | ||||
| 1 age 50-59 | ||||
| professionals | 2 OT's | 1 male | 4 age 20-29 | 6 |
| 2 PT's | 5 female | 2 age 30-39 | ||
| 1 special educator | ||||
| 1 COTA | ||||
| computer experts | 3 web designers | 3 males | 2 age 20-29 | 5 |
| 1 programmer | 2 females | 1 age 30-39 | ||
| 1 AT specialist | 1 age 40-49 | |||
| 1 age 50-59 | ||||
| general public | 2 males | 3 age 20-29 | 4 | |
| 2 females | 1 age 30-39 |
Each videotape was viewed and analyzed by three graduate students. Participant comments were sorted according to the section of the website to which they pertained, and emerging themes for each section were developed. Because the ten participants with TBI had verbal limitations and physical difficulties, their videotapes were viewed a second time. Occupational therapy students, educated in task analysis, looked for difficulties in the specific skills needed to navigate through a website.
Results and suggestions for improvements to the website were presented to BIAC and a statewide TBI Advisory Board, and a statewide committee was formed to implement website enhancements to begin January of 2004.
The participants expressed both positive and negative comments with particular areas of the website. Table 2 indicates general themes found in each section of the website. The number next to each theme indicates the number of participants who had the same general comment.
| WEBSITE SECTION | PARTICIPANT COMMENTS AND SUGGESTIONS | NUMBER OF PARTICIPANTS |
|---|---|---|
| Add a description to each title | 4 | |
| Add names | 2 | |
| THE BASICS | Add subheadings to index for easy scrolling down rather than scrolling through the whole page | 10 |
| Symptoms should become links to get further information on the symptom | 7 | |
| Define “compensatory” techniques | 4 | |
| Each paragraph should have a subheading | 3 | |
| Would like further information on Treatment and Causes | 3 | |
| SUPPORT GROUPS | Have a search engine, map, or table to just click on location of a group | 4 |
| What is “In House support?” – define this | 4 | |
| EVENTS | Would like to link to event site for more info | 2 |
| Would like to see social events | 2 | |
| Would like to see events on home page | 1 | |
| MEMBERSHIP | Online application | 5 |
| What are benefits for becoming a member? | 3 | |
| Where does membership money go? | 2 | |
| LINKS | Need to update links some not working or have new addresses | 6 |
| Give a brief description of each link; why go there | 4 | |
| Let user know that the link goes to a different browser window | 4 | |
| Links not working | 3 | |
| Would like to see links that address treatment | 1 | |
| PRIVACY | Possibly put as a small link on main screen | 3 |
| What are cookies? | 2 | |
| THANKS | Name misleading or confusing; possibly rename it “acknowledgements”, “supporters”, or “donors” | 4 |
| Content is way too long and overwhelming and maybe it's not necessary to have all names | 4 | |
| TBI TRUST FUND | Unclear of what is this- name is misleading or confusing | 6 |
| Have a top index to get to different meetings | 2 | |
| ABOUT US | Would like to see definition of brain injury moved to “The Basics” | 3 |
| To much text; sentences are to long | 3 | |
| Have each program and service be a link to further information; i.e. dates and when and where | 3 | |
| Have this information on the homepage rather than a separate page | 2 | |
| Have programs and services as a separate link on menu | 1 | |
| RESOURCE DIRECTORY | It is slow and always shows “No Services Found” | 19 |
| Only list options that have services | 7 | |
| Did not know to click on a category | 5 | |
| No results leads to a dead end would be nice to have “try another search” | 3 | |
| Wanted to see a list of options show up first | 3 | |
| Information sheets need to be updated | 3 | |
| Make transportation easier to find; some participants were looking for it and could not find it | 2 | |
| Did not know you could click on service name for more information | 1 | |
| GENERAL WEBSITE COMMENTS | Have a search engine within the site | 8 |
| Needs more pictures or eye catching graphics | 8 | |
| Would like a discussion board | 6 | |
| Typos throughout various sections | 4 | |
| More links and descriptions through out the site | 4 | |
| Would like to see statistics and causes | 3 | |
| Update and have update dates on the bottom of each page | 2 | |
| Would like to see transportation and socialization issued addressed | 2 | |
| Make home button within site more obvious | 1 |
Table 3 is an analysis of the ten people with TBI. The first column shows actual participant quotes describing navigational difficulties, while the second shows researcher observations of particular problems experienced by the participants with TBI.
| Perceived Difficulties | Specific Participant Comments | Researcher Observations |
|---|---|---|
| Reading Lengthy Text | "Some words are misleading or hard to understand ." | Slow to read paragraphs |
| "Would be easier to read in a book." | Liked text reader idea | |
| "Some people may not be able to read this." | Didn't spend time reading text or name lists | |
| "It's easier to listen to reading than use my eyes to read." | She read out loud as she scanned | |
| "I skip words when I read" | Perseverates on page | |
| "I prefer outlines and phrases rather than sentences and paragraphs." | Wants text information available on the internet | |
| . | "I can't hardly read at all." | |
| "This is too much reading." | ||
| "I usually enlarge text by copying and pasting." | ||
| "I can read small amounts of words." | ||
| Visually Focusing | "I have troubles with my glasses and vision." | Used finger to visually track list |
| It's easier to follow the more organized text with subheadings and bullets." | Participant had difficulty differentiating between links | |
| "I have no depth perception." | ||
| Staying with in the Website | Said "oops" when leaving the website early | Had trouble exiting out of links-prompted by researcher |
| Was easily distracted and went to different pages | ||
| Left the website to try an outside search | ||
| Went to irrelevant sites outside of BIAC | ||
| Exited website accidentally | ||
| Was outside the site for majority of time directed back by researcher | ||
| Concentration on Material | "So what are we doing now?" | Somewhat distracted by long paragraphs |
| Stared at screen | ||
| Had difficulty deciding which link to choose | ||
| Uncertain about where to go/what to click on | ||
| Perseverated on one topic the entire session | ||
| Manipulating scroll bars effectively | Was distracted before chance to move down | |
| Was confused by the two scroll bars with in the website | ||
| Was shaky and slow when using scroll bar | ||
| Clicks on bottom arrow instead of click and drag | ||
| Right clicked on mouse not sure how to move side bar | ||
| Manipulating Mouse | Needed a verbal prompt to click on something while in home page | |
| Repeatedly clicked on a category | ||
| Slowly moved mouse around page not clicking on anything | ||
| Returning to Previous Page | "I'm lost" (when outside of the site) | Had to be directed by researcher to get from an outside resource website back to BIAC |
| Did not know how to use the browser's "back" button | ||
| Understanding Information on the website | "I don't like acronyms. I don't know what they mean." | Knew needed to do more because nothing was happening |
| "What do they want me to do here?" | Didn't know how to get resources list | |
| "What should I go under?" | Seemed confused by some of the information | |
| "Some language is confusing" | Did not know to click on assistive tech. or submit button | |
| "What is 'In House' support" | ||
| Appearing over-stimulated | "Sometimes I have too much going at once." | Scrolled quickly through text. |
| Long lists and text created confusion; did not read much | ||
| Had difficulty sorting through all of the resource names | ||
| Appearing under-stimulated | "Nothing to it." (referring to home page) | Was hoping for pictures |
| "Too plain." (home page) | Stayed on only one link | |
| "It needs some more colors." | ||
| Other | "Some people don't know the word 'compensatory' " | Would like a discussion board |
| States "decline in coordination affects daily life." | Would like support groups during day | |
| "Needs to be more straight forward" | Wants more resources | |
| "Website is slow | Seemed interested in statistics | |
| "Site is more directed towards families." | Looking for causes | |
| "'No services found' is frustrating" | Wants therapy group for children | |
| "Membership should be explained more; who can be a member?" | Resource directory led to dead ends | |
| "I would like to have known about website at the onset of my TBI" | Would like to see a map for support groups and in resource directory | |
| Wanted more info on emotional aspect of TBI | ||
| Links need to be updated | ||
| Is used to searching within site to find what he needs within that site. |
The verbal protocol analysis method described by Zimmerman (2003) is limited in analyzing persons with disabilities. It relies on the participant to speak aloud what he or she is thinking while browsing an internet site. Therefore, for persons with physical or cognitive disabilities, it is recommended that a second analysis be performed using the visual portion of the videotape. To perform the visual analysis, carefully trained observers would watch attentively for functional difficulties within specific areas of the website. The combination of functional observations and verbal remarks can be useful in recommending assistive technology changes that would be advantageous for a particular website.
Because a majority of individuals with disabilities may not have access to external assistive technology devices in their homes or work places, assistive technology changes must be incorporated within the website to assure access to internet resource information. Some of these changes may include a built-in text reader button for each paragraph, a large “back” and “home” button located within the website, larger up/down scroll arrows, and simple graphics or shapes to enhance reading.
Further website evaluation should be performed with people who have various cognitive and physical disabilities. This would help researchers determine a comprehensive list of cognitive and motor difficulties to watch for as a standardized set of evaluation criteria.
This study was funded by the Center for Disease Control and Prevention.
Gina Ord, O.T.S.,
Colorado State University,
Department
of Occupational Therapy,
Fort Collins, Colorado 80523-1573
(970) 491-4892
EMAIL: gord@holly.colostate.edu