Introduction

Compass software for computer access assessment includes eight skill tests that measure a user’s performance for mouse use, text entry, and switch use.  One of the most commonly used Compass tests is the Aim test, which examines target acquisition skill (i.e., the ability to click on an object).  This retrospective study calculated reference scores for Aim test results, based on data from 10 experienced mouse users without physical impairments, in order to help Compass users better interpret Aim test results.  The reference scores may eventually be useful for practitioners who must document their clients’ Functional Limitations using Medicare’s G-code modifiers.

BACKGROUND

The purpose of Compass software is to provide speed and accuracy data about a user’s ability to use various computer access options, such as input devices and display settings. This evidence helps an individual or practitioner determine which access solutions will best meet a user’s specific needs. Compass has undergone extensive research and usability testing, demonstrating its ease-of-use [1], measurement accuracy [2], and psychometric properties [3, 4].

The Compass Aim test presents a series of single targets on the screen, which the user selects by clicking on each target in turn.  Figure 1 shows a screenshot of Aim test.

Over the years, Compass users have occasionally asked for some reference scores reflecting the performance of computer users without impairments.  For the Compass typing tests, the words per minute measure is fairly straightforward to interpret.  But for tests like the Aim test, it can be hard to know if a trial time of, say, 2.3 seconds represents a fast time or a slow time relative to others. 

We’ve previously resisted computing reference scores because the most important comparisons in a Compass assessment are usually based solely on the client themselves, not on how the client compares to other people.  For example, to determine the pointing device that gives this particular person the best performance, the primary concern is this person’s own performance with each different candidate pointing device. 

However, recent changes in some practice settings have made reference scores more relevant.  As of July 2013, Medicare requires practitioners to rate and document Functional Limitations using G-code modifiers, in order to receive payment for services [5].  It is the practitioner’s responsibility to select one of 7 letter codes, each representing a different percentage of impairment.  With respect to computer skills, Compass scores could be used as the basis for determining the appropriate G-code modifier for an individual.

Research Goal

This research calculates reference scores for the Compass Aim test, in order to help Compass users better interpret the results they see in practice. 

Methods

Overview

We searched our previous Compass research studies to find Aim data for individuals who have no physical impairments and are experienced mouse users.  While the original purpose of the studies was not to gather normative data, the protocols and participants are similar enough to allow the results from the unimpaired participants to be grouped together, in order to produce reference scores that represent the unimpaired subject group.

Subjects

Across three similar studies, 10 unique individuals participated. All participants were experienced computer users, could see and interpret the test stimuli, and had no physical impairments related to their ability to use a pointing device. All individuals used a mouse when performing the Aim test.  

Data Set

Table 1 summarizes the data set used to calculate the reference scores, and Table 2 lists the subjects included in the data set and their basic characteristics.  Each of the 10 participants completed one Aim test, and those results comprise our data set. The Windows mouse settings of pointer speed and enhance pointer precision were at their default values in every test. The setup of the Aim test itself was very similar across the three studies, involving similar target sizes and distances. 

Table 1. Studies used to create data set.

Study (Year)	Aim Configuration	N
A (2005)	32 trials Small/toolbar Short/med/long	6
B (2008)	32 trials Small/toolbar/icon/large Short/long	1
C (2012)	8 trials Small/icon Short/long	3

Table 2. Subjects in the data set.

Study	Subject	Gender	Age
A	01	F	38
A	02	M	38
A	03	F	40
A	04	M	35
A	05	F	32
A	06	M	32
B	07	F	42
C	08	F	30
C	09	F	16
C	10	F	13

 

Data Analysis

The Aim test reports four main dependent variables, averaged across all trials presented during the test.  Trial Time is the total amount of time from when a target was presented until the user clicked in it.  Entries is the number of times the mouse cursor entered the target per trial.  Clicks is the number of clicks made per trial.  Error-free Trials is the percentage of trials successfully completed without any extra clicks.

 To calculate the reference scores, we computed basic descriptive statistics for each dependent variable: mean, standard deviation, and the 95% confidence interval of the mean. 

results

Individual Subject Data

Table 3 shows the Aim test data for each of the 10 subjects in the data set.  There was some individual variability in the data, particularly for Trial Time and Entries.  Trial Time ranged from 0.97 to 1.65 sec, while Entries ranged from 1.0 to 1.63.  Clicks and Error-free Trials are two measures for the same thing, as they both reflect the number of clicks made during target selection.  There is very low variability in these measures, as only a few inadvertent clicks were made by these subjects.

 

Table 3. Aim test data used to create reference scores.  EFT = Error-free Trials.

Subject	Time (s)	Entries	Clicks	EFT (%)
01	1.16	1.19	1.03	96.9
02	1.22	1.09	1	100
03	1	1.12	1	100
04	1.13	1.31	1.03	96.9
05	1.56	1.28	1.03	96.9
06	1.46	1.34	1	100
07	0.97	1	1	100
08	1.40	1.5	1	100
09	1.38	1.63	1	100
10	1.65	1.5	1	100

Aim Reference Scores

The descriptive statistics for each variable across the 10 subjects is shown in Table 4.  It is worth noting that these results are very similar for subsets of the data, such as Study A only, or only Studies B and C. 

Table 4. Descriptive statistics for each of the variables. SD = Standard Deviation. 95% CI- and CI+ = lower and upper bounds of 95% confidence interval of the mean.

	Time (s)	Entries	Clicks	EFT (%)
Average	1.29	1.29	1.01	99.1
SD	0.23	0.20	0.01	1.51
95% CI-	1.13	1.15	1.00	98.0
95% CI+	1.46	1.44	1.02	100.1

The averages serve as the basic reference scores, against which the Aim scores of other users can be compared.  The other statistics are presented to show the range in our estimate of the reference scores.  Overall, the confidence intervals are fairly narrow, which suggests that these reference scores may be reasonably valid, even with a fairly small subject pool. 

DISCUSSION

Using the Reference Scores

While these reference scores are not true normative values, they do provide general guidelines for interpreting Aim test scores.  Results suggest that the performance of experienced mouse users with no physical impairments is characterized by: a Trial Time between 1 and 1.5 seconds, Entries between 1 and 1.5 seconds, and roughly 100% Error-free Trials.

Typically, the Trial Time is the most relevant score, because that gives the bottom line of how long it took to successfully complete the trials.  Entries, Clicks, and Error-free Trial values are useful indicators of difficulty and also help distinguish the source of the difficulty.  Entries relates to successful control of the mouse cursor, while Clicks relates to successful control of the mouse button.

We are working to incorporate these reference scores into the Compass software itself, so that it will be readily available to users.  We also need to apply this to the G-code Modifier Scale and have provisionally mapped the G-code classifiers to particular ranges of Aim Trial Time measurements.  We’ll report on that work in the future when it is at a more advanced stage.

Other Pointing Devices

Note that these norms apply most directly to mice.  Our past studies only include two unimpaired subjects each for trackpad and trackball, so we cannot compute reliable reference scores for these devices at this time.  However, we report the data below in Table 5 just to provide a very rough indication of user performance. This small sample is consistent with our previous results showing better performance for the mouse relative to trackballs, trackpads, and other pointing devices [6]. 

Table 5. The minimal data available for other pointing devices.

Device	Subject	Time (s)	Entries	Clicks
Trackball	TB1	1.66	1.09	1.03
	TB2	1.50	1.09	1.03
	Average	1.58	1.09	1.03
Trackpad	TP1	1.93	1.09	1.03
	TP2	2.99	1.38	1.0
	Average	2.46	1.24	1.02

Limitations

As noted above, these reference scores are not true normative values.  The sample size is fairly small, and subjects do not represent a random sample.  There is also some inconsistency in the Aim configurations within the data set.  However, with 10 subjects and the relatively narrow confidence intervals, the reference scores do provide a decent guideline for the performance of experienced mouse users without physical impairments.

These reference scores only cover one Compass test, and only under single-click conditions.  Aim may be one of the most commonly used tests, but it would still be helpful to have reference scores for the other tests.

conclusions

Despite these limitations, these reference scores can be used to help interpret individuals’ Aim test scores.  If someone gets an Aim Trial Time below 1.5 seconds, their performance is comparable to an experienced mouse user without physical impairments.  We don’t want to overemphasize comparison to this reference score, as the comparison within an individual is still the most relevant one.  However, as more practice settings move toward rating individuals on a common scale, having valid bases for comparison may become more necessary.