background

In the U.S. alone, the number of people with visual impairment (VI) increased 27% from 2000 to 2010, from 3.3 million to 4.2 million (NEI, n.d.; NEI, 2015). Approximately 8.5 million additional people have some form of color blindness (CBS News, 2014; U.S. Census Bureau, 2013). The number of people and proportion of the population with VI is predicted to continue increasing.

People with VI and color blindness can best discern differences in color based on color value. Color has three properties: hue, saturation, and value. Hue is commonly called color. Saturation (or intensity) is the richness of a hue compared to a gray that reflects the same amount of light. Value (or lightness) is the amount of light reflected back to the viewer’s eye. For people with VI or color blindness different hues can appear as the same color gray. Different values make objects visible, as in black-and-white photography or cinematography. Hence, creating a sign with at least a 70% color value contrast improves visual recognition for people with VI and color blindness.

To increase the accessibility of the built environment for people with VI and color blindness, a prior version of the Americans with Disabilities Act Accessibility Guidelines (ADAAG, 2002) referenced this 70% minimum contrast value for colors on signage and detectable warnings in pedestrian rights-of-way in an appendix. Due to the lack of a simple, low-cost test method, companies and governmental entities were unable to measure how well they met the contrast value guideline in ADAAG. In fact, this specification was removed and is not in the current ADAAG. Improving accessibility by increasing the ability to recognize important information in places such as grocery stores, gyms, and pedestrian rights-of-way would allow for people with VI and color blindness to complete more instrumental activities of daily living, such as shopping and using public transport or public rights-of-way to travel independently.  

A simple, low-cost method to assess color value contrast using common photographic light meters has been under development (Hilderbrand, Hurley, & Axelson, 2010). A quality photographic light meter can be obtained for less than $1000 and does not require extensive specialized training, unlike other technologies, such as spectrophotometers (approx. $5000). For testing, the light meter needs to measure the amount of light that falls on an object (illuminance) and the amount of light an object reflects (luminance). Illuminance is measured in lux or footcandles, and can be thought of as the brightness of the light in the room. Luminance is measured in candela per square meter (cd/m2), and can be thought of as the perceived brightness of an object. The color value contrast test method uses measurements of luminance to quickly and easily assess the color value contrast of two colors.

The ASTM F08.30 Fitness Products Committee assisted in the development of the test method, which is published in the ASTM F3022 Standard Test Method for Evaluating the Universal Design of Fitness Equipment for Inclusive Use by Persons with Functional Limitations and Impairments (ASTM, 2015). Hoogwout, Hilderbrand, Hurley, and Axelson (2013) completed a preliminary repeatability study, showing repeatability in the range of 1.4%–2.5% variation in color value contrast calculations for a single person performing the test method and an inter-operator variation in the range of 1.9%–3.9%. The only variable that changed in that study was the operator—light meter, location, and lighting all remained constant. Additionally, that study was done at light levels common in office areas (450–900 lux), based on the requirements in the draft of ASTM F3022 at that time. A further study was needed to test reproducibility with the revised lighting level in ASTM F3022 (100–400 lux), different light meters (of the same model), different locations, and different lighting. This current study was completed to fulfill that need for testing the reproducibility of the test method in ASTM F3022 for future use by fitness equipment manufacturers in determining and improving the color contrast on their fitness equipment. Test labs have also expressed interest for applications in other areas, such as accessibility assessments of signage and labeling.

Purpose

The purpose of this repeated measure study was to investigate the following research questions:

1. What is the repeatability precision (within laboratories) of the color value contrast method (as specified in ASTM F3022)?
2. What is the reproducibility precision (between laboratories) of the color value contrast method (as specified in ASTM F3022)?

Method

Operators

Eleven testers from ten laboratories performed the color value contrast measurement procedure. One tester chose to do testing at only the high light level; the other ten did testing at low light and high light levels.

Instrumentation and Instruction

Testers used a Sekonic L758CINE light meter and a standard 18% gray card and an X-Rite ColorChecker Classic color card. Testers were provided with background information and instruction regarding the technology and general test method, including a YouTube video demonstrating the procedure. The study protocol referred to the ASTM F3022-15 color contrast test methods.

Setup and Procedure

Table 1: Consistency statistics, h and k, 300–400 lux

Oi	Set A		Set B		Set C
	h	k	h	k	h	k
O1	-0.661	1.058	-1.435	1.219	0.191	0.896
O2	-0.074	1.009	0.973	0.928	-0.632	1.972
O3	0.419	0.000	1.192	0.000	-1.007	0.000
O4	0.419	0.000	1.103	0.611	-0.944	0.000
O5	-0.090	0.000	-0.300	0.000	0.779	0.000
O6	0.558	0.000	-1.425	1.724	2.013	1.345
O7	1.638	0.734	1.053	0.986	-0.164	0.493
O8	-1.926	2.385	-0.768	1.017	-0.894	2.063
O9	-0.105	1.193	0.396	0.761	-0.944	0.000
O10	-1.201	1.101	-0.579	1.277	0.779	0.000
O11	1.021	0.000	-0.211	1.044	0.822	0.000

Before tests were performed, testers were instructed to read through the protocol and practice using the equipment. The Test Coordinator was available for questions regarding the procedure throughout the testing period.

Each tester was requested to perform the protocol at two different light levels—at 100–150 lux and 300–400 lux. For each light level, testers were instructed to find a test environment that could maintain a constant level of light over one square foot of wall at the tester’s eye level. Following ASTM F3022-15, testers checked the light meter’s calibration, ensuring that there were no visible shadows on the gray card.

After affixing the color card to the wall in place of the gray card, testers performed the color value contrast measurement procedure on three different color pairs, testing each color pair three times, resulting in nine complete color value contrast calculations per tester, following the procedure in Hoogwout, et al. (2013). Using those measurements, the testers calculated the color value contrast of the color pair using the following contrast equation from the ADAAG (2002):

$$\mathrm{Contrast}:[\frac{(L_1-L_2)}{L_1}]\ast 100$$                                (1)

where

L₁ = the luminance of the lighter color

L₂ = the luminance of the darker color

The colors in each color pair are as follows: Set A, bluish green and black; Set B, yellow green and blue; Set C, white and yellow. These color pairs match those used in the preliminary repeatability study (Hoogwout, et al., 2013).

 

Table 2: Consistency statistics, h and k, 100–150 lux

Oi	Set A		Set B		Set C
	h	k	h	k	h	k
O1	0.528	0.000	-0.165	0.000	0.044	1.701
O2	-0.443	0.000	2.004	1.400	-1.896	0.000
O3	1.014	0.000	0.568	1.894	0.774	1.370
O4	1.430	0.000	0.736	0.000	0.600	0.000
O5	0.609	1.170	-0.671	0.000	1.283	0.000
O6	-0.789	0.000	0.605	0.000	-0.405	0.000
O7	-1.876	1.116	-1.154	1.482	-0.351	1.985
O8	0.239	1.637	-0.077	1.482	-1.303	0.000
O9	-0.917	1.827	-0.583	0.000	0.738	1.087
O10	0.205	1.170	-1.264	0.247	0.517	0.331

After all nine color value contrast tests were completed, the testers were requested to reaffix the gray card to the wall and repeat the illuminance and luminance readings to verify calibration and steady light levels.

Those testers who performed testing at both light levels then adjusted the light levels in the room or found another area with the required light level and performed the setup and test procedure again.

Data Analysis

ASTM E691-11 was used to analyze the data and calculate the repeatability and reproducibility of the color value contrast method (ASTM, 2011). The standard includes all equations and critical values.

REsults

For both light levels, the test results from each tester for each color set were statistically analyzed to see whether they were consistent at the 0.5% significance level. The value for h is the between-laboratory consistency statistic, determining whether or not the average of the tester’s results falls within the results that can be expected from different testers 99.5% of the time, using the test method on that color pair. If h is at or higher than the critical value of 2.34 for the high light level (Table 1) or 2.29 for the low light level (Table 2), the tester’s results exceed that 0.5% significance level and are considered a statistical outlier. No statistical outliers for h were found.

The value for k is the within-laboratory consistency statistic, examining the standard deviation of the results from that tester with that set of materials compared to the standard deviation expected between tests by the same tester 99.5% of the time. If k is at or higher than 2.13 for the high light level (Table 1) or 2.11 for the low light level (Table 2), then the result is a statistical outlier. One statistical outlier was found, which is shaded in Table 1 (O8 Set A k 2.385). The k value was not much over the critical value, and after further review of the tester’s adherence to the protocol, it was determined the test method was followed. Therefore, this data set was included in all analysis.

Table 3: Repeatability and reproducibility, 300–400 lux

	Set A	Set B	Set C
Average contrast (%)	85.49	74.51	34.44
sr (repeatability standard deviation)	0.63	0.99	1.29
r (repeatability limit)	1.76	2.79	3.61
sR (reproducibility standard deviation)	2.22	3.45	4.82
R (reproducibility limit)	6.22	9.65	13.49

The values in Tables 3 and 4 represent the average color value contrast calculated for each color set at that light level. Based on the statistical analysis run according to ASTM E691, there is a 99.5% chance that tests conducted by a single tester will fall within r (repeatability limit), with standard deviation sr, for each color set at that light level. Likewise, there is a 99.5% chance that testing conducted across testers will be within R (reproducibility limit), with standard deviation sR. The repeatability limit for each color set was lower (better) at the low light level; inversely, the reproducibility limit was higher (worse) for each color set at the low light level.

Discussion

The statistical outlier for Set A at the high light level was in the k value, or the comparative variability in the test results from that tester versus the variability in test results from other individual testers. Upon investigation, it appears the tester was blocking part of the light during the spot meter testing, although no visible shadows were seen. The test method in ASTM F3022 currently requires testers to check for a visible shadow when testing illuminance, but does not emphasize that the tester should ensure the path of light to the testing surface is not blocked during the spot meter testing. Hence, the tester followed the protocol. This suggests a clarifying note should be added to ASTM F3022.

This study was done at lower light levels than the preliminary repeatability study, which was done at light levels common in an office setting, based on a draft version of ASTM F3022 (Hoogwout, et al., 2013). Before the ASTM F3022 standard was published, additional preliminary data collected indicated that the average light levels in gyms was 100–400 lux, in contrast to the average office lighting of 450–900 lux. This study tested color value contrast levels within the range of ASTM F3022—average light levels in gym environments—and provides further insight into how lower levels of light affect color value contrast testing. Even at lower levels of light, this study showed that color value contrast readings taken by a single individual tend to be fairly consistent, but lower levels of light result in lower reproducibility. Measurements taken by different individuals in different settings vary by up to 15.8%, with less variation the higher the color value contrast and the brighter the illuminance. The luminance of lighter colors (e.g., white and yellow, Set C) vary the most based on illuminance levels, which helps explain the larger reproducibility limits for that color set—luminance readings for those colors are very sensitive to slight changes in illuminance.

Table 4: Repeatability and reproducibility, 100–150 lux

	Set A	Set B	Set C
Average contrast (%)	83.08	72.25	36.66
sr (repeatability standard deviation)	0.49	0.70	1.22
r (repeatability limit)	1.38	1.96	3.42
sR (reproducibility standard deviation)	2.91	4.58	5.66
R (reproducibility limit)	8.15	12.84	15.84

Overall, the repeatability and reproducibility limits found in this study show that the color value contrast method in ASTM F3022 is valid for use on fitness equipment and in other applications to confirm color value contrast for use in labeling and signage. Because of the reproducibility limits, manufacturers should be encouraged to choose color pairs whose color value contrast is close to 80% to ensure meeting the 70% minimum recommended contrast. Doing so will increase visual recognition and hence accessibility for people with VI and color blindness.

It should be noted that a total of seven different Sekonic L758CINE light meters were used by the testers for the high light level, and there were six different light meters used by testers for the low light level. At each light level, there were three sets of two labs that used the same meter due to the constraints of renting the light meters for labs that did not purchase them. Two of the testers were from the same laboratory and were counted as two separate testers for this study.

Conclusion

This study shows that the color value contrast test method allows fitness equipment manufacturers to increase the visibility of labeling, displays, moveable parts, and other key components on their equipment to increase access for people with VI and color blindness. It is expected that the repeatability and reproducibility limits found through this study will also enable other public and private sector entities to adapt this method to assess the color value contrast of current labeling, signage, and visual cues in the built environment and on products.

Future research will need to be conducted regarding the effects of several variables, including limits on which photographic light meters can be used reliably with this method. Currently, this method seems to work best with light meters calibrated at a tungsten setting versus a daylight setting. Additional work should be done to examine the effects of different lighting (especially LEDs) on color value contrast as tested by this method. In addition, field testing should be performed. The effect of a lighted sign in a dark environment, essentially switching the lighting source from the current method, should be studied. People with VI and color blindness have indicated that signs with extreme color value contrast (e.g., black and white) can also be difficult to read. The possible need for a maximum color value contrast should also be examined. Subject testing should be done to confirm that color value contrast pairs that pass the 70% minimum meet the needs for VI access.

The results of this study would support the inclusion of a 70% minimum contrast requirement in the body of the Americans with Disabilities Act Accessibility Guidelines (ADAAG). This would greatly increase accessibility for people with VI and color blindness, as entities required to follow ADAAG would improve the color value contrast of signage and other important visual cues. It is also anticipated that this method will be widely adapted to meet ADAAG in the built environment and for the testing and usability of consumer products.