INTRODUCTION

Augmentative and alternative communication (AAC) technologies are an effective and evidence-based intervention to improve communication and enhance meaningful participation in society for individuals who experience significant communication challenges [1]. In order to derive the benefits offered by AAC, the design of AAC technologies must be grounded in a theoretical understanding of basic visual cognitive processes [2].

Visual cognitive processing of AAC interfaces is a critical area for examination because communicating via AAC requires the use of an external display that is represented and accessed visually. Users must be able to visually attend to and extract information from the display [3]. A visual scene display (VSD) is one common type of AAC interface. VSDs depict people engaged in meaningful activities within an integrated scene such as a photograph, with language concepts embedded as hotspots. Wilkinson and Light [4] utilized eye-tracking technology to examine visual attention within natural scenes (i.e. VSDs) and found that humans exerted a powerful influence on visual attention from individuals with neurodevelopmental disorders.

However, AAC VSDs are often more complex than a single photograph. AAC technologies organize vocabulary across multiple displays in order provide with user with access to a larger lexicon. Navigation between displays is typically achieved through a menu that contains small thumbnails of possible displays. Therefore, it is critical to examine visual attention to more complex interfaces that include both a main display and a navigation menu. The spatial arrangement of the navigation menu is one feature that may influence visual attention patterns because location in the visual field has a powerful influence on visual cognitive processing [2].

The purpose of this investigation was to examine how the spatial arrangement of the navigation menu, relative to the VSD, influenced visual attention patterns by individuals with neurodevelopmental disabilities. This type of systematic empirical investigation will contribute to an evidence base from which clinicians, AAC manufacturers, and app developers can tailor the design of AAC technologies to the visual cognitive processing skills of users.

METHODS

Participants

The study included four groups of participants: three groups of individuals with significant language disabilities associated with autism spectrum disorder (ASD, n=13), Down syndrome (DS, n=13), or other intellectual disabilities (IDD, n=9); and one group of preschoolers with typical development (TD, n=20). Participants with disabilities were included in the study if their Peabody Picture Vocabulary Test (PPVT-4) [5] score was more than 2 standard deviations below the mean (i.e., <70). Table 1 presents demographic information for the participants in each group including PPVT-4 age equivalents, PPVT-4 standard scores, and chronological ages. A group-wise matching strategy was not employed in this study for several reasons. First, there are inherent limitations in group-wise matching [e.g., 6]. For example, matching based on receptive language skills would eliminate those participants with the most significant language challenges who are most likely to benefit from AAC. Second, due to challenges of obtaining valid data with individuals with disabilities, data loss can be significant, and the exclusion of participants based on language age could lead to small sample sizes and limit conclusions [7].

Table 1. Participant characteristics.

	TD	DS	ASD	IDD
Measure	mean (SD)	mean (SD)	mean (SD)	mean (SD)
PPVT standard score	114 (14)	46 (14)	29.5 (18)	40 (20)
PPVT age equivalent (years)	5.1 (1.5)	5.8 (2.0)	3.5 (1.9)	4.8 (1.8)
Chronological age (years)	4.1 (0.7)	16.5 (6.1)	15.7 (3.4)	15.5 (4.4)

Stimuli and experimental procedures

This investigation examined visual attention to stimuli that closely conform to the recommendations for the composition of VSDs. VSDs were constructed, each of which contained two children engaged in one of four meaningful shared activities; reading a book, eating snack, petting a dog, or swinging in a tire swing. In each stimulus, one of the photographs was featured in the center of the display, measuring about 17 by 17 cm; this was considered the main VSD for that stimulus. In addition to the main VSD, each stimulus also contained a smaller strip that simulated a navigation menu or bar (henceforth, “navigation bar”).  This navigation bar contained all thumbnail sized images of each of the four possible photographs (reading, snack, dog, swing). Figure 1 presents one example stimulus. The navigation bar appeared in one of 4 different locations: in some cases, it appeared at the top horizontally, in others on the bottom horizontally, on the left vertically, or on the right vertically. Each bar location was presented 4 times within the experimental task (i.e., 4 stimuli included the bar on the top, 4 included the bar on the left, etc.).

The stimuli were presented to participants on a TOBii T60 eye-tracker, which records point of eye gaze via a remote infrared camera that projects and detects light off of the participant’s pupils and cornea. A Dell laptop controlled stimulus presentation and data acquisition. Participants underwent two viewing conditions: free viewing and cued viewing. During the free viewing condition, the participant was given no instruction, and the image was presented on the monitor for 5 seconds. This free viewing was followed by an interstimulus-interval slide, during which time a pre-programmed spoken phrase instructed the participant to look at a particular target within the menu (i.e., “Look at the kids with the dog”). After this prompt, participants were presented with the same image a second time, for 5 seconds; this was the cued viewing phase. The focus of the analysis here was to examine the effects of navigation bar location on eye gaze fixation patterns during the free viewing phase.

Data preparation and dependent measures

A drawing function on TOBii studio software was used to create areas of interest (AOIs) on each stimulus. Each stimulus contained AOIs for the main VSD and the bar. Within each main VSD, there were AOIs for the children and the shared activity. The software matched the point of x-y gaze coordinates collected by the T60 monitor to the coordinates defined within each area of interest. This allowed for evaluation of visual attention to each area. 

The dependent measure was the percentage of each participant’s own total fixation time allocated to the areas of interest. For each participant, the amount of time spent fixated on each individual area of interest was divided by the time spent fixated anywhere on the image by that participant. This provided a proportion of time allocated to each area of interest based on each participant’s own total viewing time, rather than the total possible viewing time (i.e., 5s). This adjustment was made to account for individual differences in viewing time across participants.

Statistical analyses

Statistical analyses compare performance within groups, rather than across groups. One-way repeated measures ANOVAs, with bar location and as the within subjects independent variable, were conducted for each group. The alpha level for significance was set at .05 (unadjusted for multiple comparisons).

RESULTS

The first experimental question concerned how attention was allocated to the main VSD and the navigation menu across the four bar location conditions (top, bottom, left, and right). Figure 2 summarizes the data across the four bar locations within each group. The general pattern across groups and across bar locations was that participants allocated a greater proportion of their fixation time to the main VSD, while spending proportionally less time on the navigation menu. The main findings of the one-way repeated measures ANOVA (4 levels of bar location) varied across groups. For participants with DS and IDD, there were no significant differences in fixations to the VSD by bar location. For individuals with TD and those with ASD, the mean percentage of fixation time on the VSD differed significantly between the four bar locations, (TD (F(3, 57)=3.91, p= .013;  ASD (F(3, 36)=3.15, p=.037)). Post hoc pairwise comparisons revealed that in both groups, participants spent significantly more time fixated on the VSD when the bar was on the top relative to the bottom (TD: t(19)= 2.34, p=.030; ASD: t(12)= 2.64). For individuals with ASD, the greater time spent on the bar in the bar top condition also reached statistical significance relative to the bar left condition (t(12)=2.76, p=0.17),and the p-value approached but did not reach statistical significance relative to the right (t(12)=1.81, p=0.94); however, the effect size for this comparison is large (partial eta squared = .216). For those with TD, more time was spent on the VSD when the bar was on the left relative to the bottom (t(19)=2.50, p=.022) as well as the right (t(19)=2.21, p-.040).

The second experimental question concerned how attention was allocated to the children and shared activity within the main VSD across the four bar location conditions. Figure 3 summarizes the data across bar locations within each group. Notably, the figure illustrates that individuals with ASD spent approximately as much time on the children, overall, as did those with IDD, and close to the amount spent by those with DS or TD. Patterns of attention to the children and the activity varied systematically based on whether the bar was on the top (closest to the children) or on the bottom (closest to the activity). More time was spent on the children when the bar was proximal to that AOI on top than when the bar was on the bottom, for all groups but those with DS; this effect was of statistical significance for ASD (t(12)=3.76, p=.003) and approached statistical significance for TD (t(19)=2.06, p=.056). In contrast, more time was spent on the activity when the bar was proximal to that AOI on the bottom than when the bar was on the top; this effect was of statistical significance for all groups except those with IDD (TD: t(19)=5.90, p<.005; ASD: t(12)=3.72, p=.003, DS: T(12)=2.74, p=.018). This suggests that visual attention was driven to either the children or to the activity as a function of the proximity of the bar to each area of interest. Interestingly, in contrast to the pattern observed when the bar was on top (greatest attention to children, least attention to activity), the distribution of attention between the children and the activity was far more even when the bar appeared on the left, in all groups.

DISCUSSION

To date, very few investigations have examined eye gaze on displays that resemble actual VSDs—with a navigation menu, human figures, and a shared activity. Several important findings concerning general visual attention patterns, irrespective of menu location, warrant consideration. First, participants visually attended to both the main VSD and the navigation menu containing thumbnail sized images of VSDs. Given the visual complexity of these scenes, it was possible that participants may have only attended to one of these elements. For example, they could have become distracted by the small photographs in the menu and failed to attend to the VSD; however, visual attention was allocated to both of these important elements. A second overall consideration is that participants across groups attended to both the children and the shared activity within the VSD, while spending little time on the background elements in the VSD. This suggests that the presence of a natural scene may aid in the processing of the display [8]; however, it does not distract from the meaningful concepts. Attention to the human figures is especially notable for individuals with ASD, who have difficulty directing visual attention toward humans within actual social interactions [9]. Also, eye-tracking work suggests that individuals with ASD may spend less time fixating on humans within natural scenes compared to individuals with TD, particularly when the scene has high social demands (i.e., more than one person) [10]. However, the participants with ASD spent a similar proportion of fixation time on the human elements within the scene as individuals in other groups. This suggests that human figures and a shared activity should be included as key elements within AAC VSDs.

This study suggested that the location of the navigation menu influenced visual attention to the important major elements within the display (main VSD and menu) as well as to the elements within the main VSD (children and shared activity). Therefore, the spatial arrangement of the navigational features of AAC technologies is a design feature that warrants attention, as it may influence how individuals that use AAC attend to and extract information from the display. There are several clinical implications. Placing the menu at the top of an AAC display may be an optimal location to promote attention to human figures, particularly for individuals with ASD. For individuals with DS who show a strong preference for attending to people, we may promote attention to the shared activity by placing the menu on the bottom or left.

One limitation of this study is that small sample sizes limit generality. A second limitation is that this study only measures visual attention upon first exposure, and it does not examine patterns of visual attention over time. Future research should also aim to investigate patterns of eye gaze in more functional, ecologically valid tasks. For example, eye gaze fixation patterns could be evaluated within a task that requires a behavioral response of navigating to a separate page on an actual AAC display. Furthermore, it is important to evaluate viewing patterns within actual communicative interactions using VSDs. These studies should provide valuable information to improve the design of AAC technologies to align with the visual cognitive processing skills of users.