RESNA Annual Conference - 2010

BACKGROUND

Many advancements in vehicle technology have been developed in the past few years and are readily available as unregulated, after-market products (1).  While these devices are often marketed as promoting safety, questions have been raised about whether they enhance an older driver’s safe operation of a vehicle or detract from it (2).  Global positioning systems (GPS) are inexpensive devices that provide in-vehicle navigation.  Because of their increasing use by drivers of all ages, analysis of their usability is pertinent. 

One of the key goals of usability evaluations is to bridge the mental models between the end user and the designer through an optimal system image (interface) (3).  Within the field of human factors (HF), many methods exist to evaluate the usability of devices.  Hierarchical task analysis (HTA) is arguably the most widely used of all HF methods available (4).  An HTA breaks down a task to subtasks, from complex to constituent operational elements (operations) and presents possible plans for task and subtasks completions.  Another popular HF method is the state transition chart (STC) which is often used to describe interfaces, especially input syntax, at instantiations (5). 

The use of these methods to evaluate products like a GPS by civilians (as oppose to military personnel), especially seniors, is less common.  Recently, Fisk, Rogers, Charness, Czaja and Sharit (2009) used an HTA in combination with a human failure modes and effects analysis (HFMEA) to evaluate a self-checkout system in a retail environment (6).  As part of the input to the HFMEA, these authors suggests the use of a “checklist” to identify potential problems as seniors perform tasks and subtasks requiring perception, cognition and response execution.  Expanding on their work, an activity analysis (AA) based in occupational therapy, can be used to compliment a HTA.    An AA provides information about human knowledge, skills and abilities required to complete tasks and subtasks effectively (7).  The integration of task and activity analysis (TAA) provides a systematic way to study usability issues faced by seniors and their use of GPS devices.

RESEARCH QUESTION

Figure 1. Hierarchical task analysis (HTA) of selected subtasks for using a Garmin GPS (nüvi 250W)

The purpose of this paper is to describe use of the TAA approach in a usability study of seniors’ use of a GPS device and to present the preliminary results of a subset of the participants.  The study design and application of the TAA was informed by the HAAT model (8) which compels the researchers to focus on senior participants and their interaction with the GPS device (Garmin nüvi 250W), as they perform specific activities in a laboratory setting.  These activities include setting up and observing simulated navigational routes on the GPS device.  The work presented here is part of a larger project on understanding the cognitive load of GPS device use by older drivers.

METHOD

An observational study, supplemented by pre- and post- semi-structured interviews, was conducted on the usability of a GPS device by novice senior users, in a laboratory setting, in Elborn College at the University of Western Ontario.  Participants were recruited if they were over the age of 65, had a valid driver’s license, had minimal or no experience with a GPS device and did not have any physical, cognitive or sensory condition that would limit their ability to complete the required task.  Demographic information such as age, types of technology used for daily activities and number of kilometers driven per week were collected. 

To begin, each participant was given a brief orientation to the functions of the GPS device prior to their performance of the task of setting it up to obtain navigational information.  An HTA representation of this task is provided in Figure 1. 

Three subtasks were then completed with the device: 1) determine a route from the present (i.e. Elborn College) to a destination location saved in “favourites”, 2) determine a route from the present location to a destination address provided, and 3) find a “point of interest” (POI) enroute to a destination.  The participant was encouraged to think aloud as he or she completed each of the subtasks.  Performances of the subtasks were videotaped. At the same time, two researchers took notes of potential errors that were made by the participant. At the conclusion of the task, the participants completed a semi-structured interview where their actions and reasoning at points of apparent confusion or errors were probed. 

Figure 2. Sample integrated task and activity analysis

TAA approach:  An AA was completed for subtask operations in the HTA (see figure 2 for a sample).  Process (cognitive), motor and sensory requirements were identified. Specific assumptions were made about the conditions under which the device was used, for example the lighting, space and noise demands.  A further assumption was made that the vehicle was not in motion when these activities were completed.  Of relevance to the larger cognitive load project, the process skills identified included sustained attention to subtasks, temporal organization, adaptation or assimilation of information, accommodation of information and mapping (i.e. the ability to map the GPS graphics to real world objects).  This analysis identified both lower level skills in different areas (e.g. attention to a subtask in the cognitive area) and higher level skills (e.g. accommodation of information in the cognitive area).  Utilizing the knowledge gained through an integration of the task and activity analysis as a backdrop and a basis for a “checklist”, the first and third authors completed video analysis for five cases of seniors performing the specified subtasks with a standard template.  The analysis yielded results that were described rather than quantified.  For example we did not measure the length of sustained attention that was required in order to complete an operation.

RESULTS

A summary of the analysis of the video data collected with five seniors are presented.  Please refer to table 1 for basic demographics information.  All five seniors have used or are currently using everyday high technology.  Some examples include the computer, the internet, CD players, DVD players and ATM machines.

An inspection of the content described through the standard template used in the video analysis showed high inter-rater agreement, ranging between 82 to 100%.  The following were the key issues identified by the raters:

1. The most common issue in the successful use of the device was the tendency to use familiar strategies, even when these did not achieve the desired results. A difficulty that most participants encountered related to the process of entering an address. Most participants attempted to insert the entire street address, a strategy with which they were familiar. The device requested that the street number be entered first, prior to moving to a second screen where the street name was inserted. Once the incorrect information was entered, participants had difficulty recognizing they had made an error and did not know how to correct it.
2. Instances of a mismatch between the designer’s and the user’s vocabulary were apparent.  An example is the designer’s use of ‘via point’ to identify a location to which the user plans to detour once a trip had been programmed. Most participants did not understand the use of “via point” and selected other options in an attempt to complete the subtask.
3. Configuration and the colour scheme of some screens influenced the users to ignore key information.  For example, the request to input the name of the desired city was located at the top of the screen, using low contrast foreground/background. Most participants did not notice the instructions presented in this manner. Furthermore, for some participants, inefficient input resulted from the alphanumeric layout of the keyboard as opposed to the more familiar QWERTY.
4. Finally, there was some difficulty with error recognition and repair. This observation was most apparent when an error was made and the resulting screen did not yield the expected information. In many instances, participants did not recognize that they had made an error and consequently, didn’t know how to repair it. They tried to re-trace their steps or used a trial and error method in an attempt to reach a screen that would provide recognizable directions.

DISCUSSION

The HTA was a useful means for identifying subtasks and respective operations. The activity analysis provided a means to classify elements of the person’s function that were required to successfully complete the subtasks and operations as well as highlight elements that may pose difficulties for the participant.  Together the integrated TAA afforded an approach that is systematic in the identification and description of issues faced by seniors using a GPS system for the current and future research.  However, a couple of drawbacks to the approach should be noted.  First, during the video review of the observational phase of the study, there were some inconsistencies between raters as to whether some errors were of a visual or cognitive nature.  Did a participant miss a step because of a lack of visual cue, a lack of understanding of the sequence of necessary steps, or some combination of both?  In such a case, only the participant could identify the source of the problem.  Another drawback is in the rigid identification of operations through the HTA while omitting other screen elements, like on-screen text, colour scheme and icons, that may not be part of the plan(s) to complete a subtask but nevertheless influence to usability of the device.  The use of a STC in place of (or to supplement) a HTA along with an AA would help to make salient potential interactions between the user and the screen elements.  The use of a STC will also be relevant during route navigation in identifying other on-screen elements (e.g. color schemes for busy roads, highways, landmarks, etc.) 

Overall, based on the small sample presented of cases examined in this study, the GPS device (Garmin nüvi 250W) was simple and intuitive to use by the senior participants. The touch screen did not pose significant difficulties. A minimum number of steps are required to program the device for each of the subtasks. Navigation between the device screens was relatively user-friendly. When subtasks or elements of a subtask were repeated, most users seemed to learn the programming quite quickly.  However, instances of a mismatch between the designer’s mental model and that of the user’s through the user’s interaction with the system image were apparent (see results 1 and 2).  The main sources of difficulty resulted from cognitive aspects, specifically recognition of subtask sequencing and ability to recognize and apply new strategies, as well as confusion over the interpretation of words or phrases such as “via point”.  Visual elements were also important as participants routinely did not recognize instructions that were placed at the top of the screen with a low contrast display.  While these issues were found across some participants, caution should be taken in applying these results as users were new to the device and had limited practice sessions prior to attempting the observed subtasks.