RESNA Annual Conference - 2020

Am I Safe to Drive? A Digital Tool for Supporting Self-Assessment of Driving Ability

Surya S NetiBegin Superscript1End Superscript, James Ren Hou LeeBegin Superscript1End Superscript, Vanessa BachBegin Superscript2End Superscript, Jennifer BogerBegin Superscript1,3End Superscript

Begin Superscript1End SuperscriptSystems Design Engineering, University of Waterloo, Waterloo, ON;

Begin Superscript2End SuperscriptSchool of Public Health and Health Systems, University of Waterloo, Waterloo, ON;

Begin Superscript3End SuperscriptResearch Institute of Aging, Waterloo, ON


More than 75% of people older than 65 years of age hold a valid Canadian driver's license [1]. A recent estimate by the Alzheimer's Society suggests that 1 in 14 people above the age of 65 years are likely to have dementia [2]. For the 260,000 people in Canada alone that fall under both of these demographics, the decision about driving cessation becomes especially important. The current system includes evaluations by physicians, occupational therapists and the Ministry of Transportation at various stages of diagnosis [3], but due to the resources required for individual supervised testing, the process is often time consuming and tedious for the driver, expensive for the system and can even jeopardize the patient-physician relationship [3-5]. This is partially because there are no clear guidelines about when an older adult should hang up their keys [6]. Studies have shown that older adults who have stopped driving can experience depression and social isolation [7,8]; this is related to the independence and accessibility driving brings, especially in a North American context. Motivated by a need to balance road safety with independence, several applications that assess fitness to drive have been designed, however, these are predominantly from a clinical perspective with a focus on providing information for medical professionals and patient-evaluation [9]. This research aims to put the older adults at the center of the discussion regarding choices about their driving by creating SmartDrive, which is a digital self-assessment application that helps explore driving ability. SmartDrive uses validated cognitive tasks and is intended to promote reflection and safer driving decisions by providing appropriate recommendations. We used the Technology Acceptance Model [10] to evaluate acceptance of the SmartDrive application through the investigation of two research questions: 

  • How useful did the participants find SmartDrive? 
  • How usable did the participants find the interface of SmartDrive? 


Co-design of the prototype 

Our research questions focus on understanding the willingness of older adults to adopt an application that allows them to self-assess driving ability, and provides recommendations based on their performance in the cognitive tasks administered. Thus, SmartDrive was co-developed by the authors using bi-weekly discussions between a lead older adult as well as interactive meetings with a local seniors' group, Bits & Bytes. These interactions provide ongoing input regarding the language, layout and the style of feedback designed to maximize ease-of-use and comprehensibility for this demographic.

Our first stage of research was to conduct a pilot study where the first of the cognitive tasks, the Trail Making Test Part B (TMTb) [11-13], was digitally implemented on a tablet along with three different feedback presentation styles. The three feedback styles, Text-only, Visual Score Map and Text with driving images, were coupled with recommendations based on the completion time of the task (see Table 1). An average task completion time difference of 18 seconds between the digital and paper versions of the same task was established by administering both versions to 16 subjects, therefore the threshold times for the digital version were offset by this value compared to standardized paper scores.

Table 1. Feedback presented to the user based on their task completion time
TMTb time threshold Performance Label Recommendation
<57 seconds Average Repeat the task in two months 
>57 and < 255 seconds Below Average Visit physician for further assessment
>255 seconds Deficient Visit physician for further assessment

Study Protocol

This study was approved by the University of Waterloo Office of Research Ethics (ORE #41708). After informed consent was obtained, the participants were asked to complete two forms: (i) Demographics and Driving Questionnaire, (ii) Driver's 65 plus self-rating form. The former includes questions created for this research regarding age, driving experience and habits, and the latter comprises a 15-question self-rating driving assessment designed by the AAA Senior Driving [14], which allows senior drivers to examine driving performance. This was followed by an audio recorded semi-structured interview that discussed their ideas of such an application and concerns about using it. Participants were then asked to interact with the tool and perform the task, TMTb, while their interactions with the application were screen-captured. Different presentation styles of performance feedback were shown to the participants, and their preferences were discussed in an interview that followed. The task order of the TMTb (i.e., paper or digital) and the feedback presentation styles were randomized and balanced to avoid bias. Participants were also administered a Systems Usability Scale (SUS) to measure ease-of-use of the application and their overall scores were calculated.

Data Analysis

Audio recordings of the interviews and walkthroughs were transcribed verbatim. Preliminary deductive and inductive codes were collectively identified by two researchers and each transcript was coded following the defined coding scheme. This process was repeated until saturation. Each code was then discussed by three researchers and categorized into its respective overarching theme and finalized only after the authors reached a consensus. To analyze the potential of the digital task in estimating driving ability, the time taken by the participants to complete both the paper and digital versions of the TMTb were compared.


Participants were six older adults with a mean age of 73.83 yrs., all of whom possess valid full G drivers licenses and are actively driving (avg. of 90 kilometers in the past week). Demographics, scores from tests, and SUS are summarized in Table 2.

Table 2. Demographics, scores and performance results of participants
Participant Age (yrs.) TMTb Completion Time (seconds) AAA Senior Driving  (score dist. from threshold, direction) SmartDrive Feedback  (score dist. from threshold, direction)  SUS  Score  (max 100) 
Paper Computer
P1 73 74 42 GO (0) Average (15-)  94
P2 76 79 61 CAUTION (15+) Below Average (4+) 46
P3 77 73 60 GO (2-) Below Average (3+) 52
P4 70 51 68 CAUTION (2+) Below Average (11+)  96
P5 71 77 43 GO (0) Average (14-) 98
P6 76 179 163 CAUTION (8+) Below Average (106+) 65

Five participants were faster on the computer-based version of the TMTb (average time difference of 18.6 seconds), with four participants stating that they were more comfortable using the computer. One participant (P4) was faster on paper. Results of the AAA's self-rating results match that of the performance labels for five participants. However, the difference of the user's scores (P3) from the threshold are small, which could be the cause of the discrepancy between their SmartDrive and AAA scores.

To determine the acceptance of the application, the two research questions were addressed by employing the Technology Acceptance Model, Perceived Usefulness was analyzed through qualitative analysis of participant's responses and Perceived Ease-of-Use was measured by the usability scores obtained from the System Usability Scale (see Table 2). The themes that were identified through the qualitative thematic analysis were organized in their natural progression of participant's response to the feedback (see Table 3).

Table 3. Emerging themes (n = # of the six participants who mentioned the sub-theme)



Representative Quote


Acceptance of ability decline 

(n = 4)

Observation/comment by someone else (n = 2)

Wake-up calls (n = 4)

(q1) "people are getting older and we are not getting better at driving, you know let's face it"

(q2) "hopefully your family or spouse...a good friend told that you really shouldn't be driving"

(q3) "people start honking at you, getting tickets is another clue, starting to get lost...are indicators"

Motivations for use

Curiosity (n = 3)

Maintaining autonomy (n = 6)

Safety (n = 5)

Self-improvement (n = 6)

Scope for introspection (n = 2)

(q4) "You sort of investigate yourself, then you proceed and go [to the doctor]...who does further investigation"

(q5) "I would like to know, am I up to snuff?"

(q6) "how to correct a bad behavior, you know you need information for the person to learn"


Simple Language (n = 2)

Feedback style & colors (n = 6)

Availability of time & device (n = 6)

(q7) "I think it's very clean, I think that the colors are very good"

(q8) "colors are gentle...written in language that is easy to understand, and brief"


Applicability to driving (n = 5)

Score explainability (n = 6)

Feedback tone (n = 6)

Reliability (n = 6)

Appropriate suggestions (n = 5)

(q9) "subtle doesn't work…[need to] just say it"

(q10) "Deficient [as a performance label] have to be very delicate. It shouldn't sound like criticism"

(q11) "if you are close to even 75 [task completion time in seconds], I'd be really worried if you were driving. 255, forget it"

(q12) "one test one time doesn't really indicate that you are good or bad it just indicates that you are not familiar with the program"

(q13) "it was a part of the way you determine my reactions so yes, in that sense it was a suitable task"

(q14) "[a driving assessment] should put me behind a wheel...I don't know if there's a correlation between that [task] and how you would react in a car"

Action plan

Methods to improve driving (n = 6)

Seek medical assessment (n = 3)

Discuss with family/spouse (n = 5)

Change in driving habits (n = 3)

Alternative Transportation (n = 3) 

(q15) "The only thing I could do is be more observant...cut out all the small talk or minimize the diversions.

(q16) "it would not be a bad idea to ask my doctor and tell him that I am just a little over the average"



Results from this research suggest that participants were generally perceptive of their driving ability and understood that adjustments to their driving may be necessary to continue driving safely on the road [q1]. We discovered that this awareness was likely gained through external observations [q2] or through personal experience of driving mishaps [q3]. In addition to addressing these concerns and doubts about their driving safety, curiosity and the desire to self-improve were also noted by participants as factors that would motivate self-assessment through an application like SmartDrive [q4, q5, q6]. 

To encourage use of SmartDrive in this demographic, the feedback styles and layout were designed to be simple and intuitive. Participants generally agreed with the design choices [q7], favoring simple language, high contrast colors and low text density [q8]. While the significance of feedback tone was recognized by all participants, some expected wording that demonstrated a neutral or blunt phrasing [q9] while others preferred a sensitive one instead [q10]. An accurate yet diplomatic tone for wording will be explored with the co-design team for the next version. "Accessibility" also indicated that such a device should either be deployed in locations of public access (public libraries) or on digital media that are common in households. Two of the six participants owned tablets and regularly used a touch screen but preferred a desktop for its larger screen size. The remaining participants also suggested deploying such an application on a desktop with a mouse instead of a touchscreen-based tablet as they felt more comfortable with this mode of input. Since the task completion times would change based on the devices used, the next iteration of SmartDrive will remain tablet-based, however, a desktop version is recommended for future development.

"Trustworthiness" was influenced by the relevance of the task to driving ability, reliability in the scoring procedure and the validity of the suggestions presented [q11, q12]. Trust is hypothesized as a key factor in positive reception and continued use of the application. Reasons for trust were identified by analyzing those aspects in the design that were uniformly questioned by all participants and were stated to have affected their confidence in the application. Some participants viewed the task suitable for driving assessment [q13] while others questioned it [q14]. References in literature for task-validity and additional tasks that are more applicable to driving will be added to address these concerns.

Finally, to be useful and effective, participants insisted the application must provide actionable recommendations that would promote safe driving decisions. Several follow-up plans were suggested after the use of the application ranging from changing minor driving habits [q15] to seeking further assessment by medical professionals [q16]. Additionally, all six participants were interested in recommendations encouraging them to improve their ability which were not considered in the current design, motivating us to incorporate driving lessons and possible brain-training exercises as potential feedback for the next prototype.


This study presents results of user-testing the first prototype of SmartDrive, a driving self-assessment application for older adults designed to promote safe driving decisions by assisting them in exploring their driving ability. While the sample size was small (n=6), evaluation of the design revealed five overarching themes that affected perceived usefulness and usability. The first two themes identified ("Awareness" and "Motivations for Use") were consistent with those found in previous studies that evaluated driving assessment applications or strategies from a clinical perspective [15,16,17]. "Accessibility" and its sub-themes matched suggestions prescribed in previous usability frameworks designed for this demographic [18]. "Trustworthiness", its influencing factors and the different follow-up suggestions ("Action plan") recorded in this study were new and emergent themes that added to the existing rationale of user's willingness to adopt this self-assessment application and will be further explored. The themes identified in this study are a starting point for the development of self-assessment applications, including features that address the sub-themes mentioned can potentially encourage continued use by the target audience. All themes are being translated into design features in the next version of SmartDrive, which will be tested with a larger number of actively driving healthy older adults and those living with dementia through interviews and focus groups. It is hoped that applications such as SmartDrive can be used to foster critical thought about driving ability that may reduce crash risk by promoting safer driving decisions in older adults.


  1. Statistics Canada, Profile of seniors' transportation habits.
  2. Alzheimer Society of Canada. Prevalence and Monetary Costs of Dementia in Canada. Toronto. Alzheimer Society of Canada, 2016.
  3. Hopkins RW, Kilik L, Day DJ, Rows C, Tseng H. Driving and dementia in Ontario: a quantitative assessment of the problem. The Canadian Journal of Psychiatry. 2004 Jul;49(7):434-8.
  4. Carmody J, Traynor V, Iverson DC. Dementia and driving: an approach for general practice.
  5. Brown LB, Ott BR. Driving and dementia: a review of the literature. Journal of geriatric Psychiatry and Neurology. 2004 Dec;17(4):232-40.
  6. Piersma D, Fuermaier AB, De Waard D, Davidse RJ, De Groot J, Doumen MJ, Bredewoud RA, Claesen R, Lemstra AW, Scheltens P, Vermeeren A. Assessing fitness to drive in patients with different types of dementia. Alzheimer disease and associated disorders. 2018 Jan;32(1):70.
  7. Qin W, Xiang X, Taylor H. Driving Cessation and Social Isolation in Older Adults. Journal of aging and health. 2019 Aug 20:0898264319870400.
  8. Choi NG, DiNitto DM. Depressive symptoms among older adults who do not drive: association with mobility resources and perceived transportation barriers. The Gerontologist. 2016 Jun 1;56(3):432-43.
  9. Allan C, Coxon K, Bundy A, Peattie L, Keay L. DriveSafe and DriveAware assessment tools are a measure of driving-related function and predicts self-reported restriction for older drivers. Journal of applied gerontology. 2016 Jun;35(6):583-600.
  10. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly. 1989 Sep 1:319-40.
  11. Mathias JL, Lucas LK. Cognitive predictors of unsafe driving in older drivers: a meta-analysis. International psychogeriatrics. 2009 Aug;21(4):637-53.
  12. Stutts JC, Stewart JR, Martell C. Cognitive test performance and crash risk in an older driver population. Accident Analysis & Prevention. 1998 May 1;30(3):337-46.
  13. Duncanson H, Hollis AM, O'Connor MG. Errors versus speed on the trail making test: Relevance to driving performance. Accident Analysis & Prevention. 2018 Apr 1;113:125-30.
  14. AAA Foundation for Traffic Safety, Drivers 65 Plus: Check Your Performance,, August 2016.
  15. Betz ME, Jones J, Petroff E, Schwartz R. "I wish we could normalize driving health:" a qualitative study of clinician discussions with older drivers. Journal of general internal medicine. 2013 Dec 1;28(12):1573-80.
  16. Betz ME, Jones J, Genco E, Carr DB, DiGuiseppi C, Haukoos JS, Lowenstein SR, Schwartz R. Perspectives on tiered older driver assessment in primary care settings. The Gerontologist. 2016 Apr 1;56(2):272-81.
  17. Rapoport MJ, Sarracini CZ, Mulsant BM, Seitz DP, Molnar F, Naglie G, Herrmann N, Rozmovits L. A virtual second opinion: Acceptability of a computer-based decision tool to assess older drivers with dementia. Health informatics journal. 2019 Jun 18:1460458219852870.
  18. Becker SA, Webbe FM. Designing for older adult users of handheld technology. In2006 International Conference of the IEEE Engineering in Medicine and Biology Society 2006 Aug (pp. 3297-3300). IEEE.