The new color blind test had a busy first week. By 4 October, users had finished it more than 940,000 times. That is enough anonymous data to check the test against itself, so we did.
This post covers what the results look like, which of them held up when users took the test a second time, and what we are changing because of it.
What we looked at
Each finished test sends an anonymous summary: the result pattern, the three channel scores, how many plates were shown, how long it took, and a few facts about the screen. There is no name, age, or sex in it.
We read 946,159 finished tests. About 130,000 users took the test at least twice, usually within a few minutes, which gave us a pair of results to compare for each of them.
Almost everyone finishes
About 97 in 100 users who started the test reached a result. The typical run was 35 plates in just under three minutes, close to five seconds a plate.
What the results look like
How the 946,159 results split
Share of finished tests
The red-green scale is centered where it should be. A score of 100 is meant to be typical vision, and the median user scored 100 on both red-green channels.
The blue channel is not. The median user scored 88 there, which means our “typical” line for blue was set a little too strict.
Strong results hold up. Mild ones often do not.
The second attempts are the most useful part of the data.
What users got on a second try
Grouped by their first result. About 130,000 users took the test twice.
For a typical first result, "same" and "typical" are the same thing. Red-green rows count any red-green result as the same kind.
A typical result came back nine times in ten. A strong red-green result came back as red-green three times in four.
A mild result, a borderline result, or a blue-yellow result usually did not come back. Most of those users got a typical result the second time.
One caveat: users who retake the test are not a random sample. A surprising result makes a second try more likely, and people do a little better with practice, about 10% on every channel. Even so, the gap between strong results and mild ones is too wide to explain that way.
The night effect
True blue-yellow color vision differences are rare, far below 1 in 100 people. Our test gave that result to 4 in 100. Something else is going on.
The clock gives a hint.
Blue-yellow results rise at night
Share of finished tests with a blue-yellow result, by the hour on the user's clock
From late morning to evening, about 3.5% of results were blue-yellow. Between 3 and 6 in the morning, it was about 6%. On phones it was 4.2% across the day, on desktop computers 2.6%.
A phone late at night is often dimmed and often in a night mode that warms the screen and removes blue light. The test cannot see those settings, so we cannot prove the cause. The pattern fits it well: a screen that shows less blue makes the blue plates harder for everyone.
If you got a blue-yellow result, turn off night mode and any eye comfort or blue light filter, raise the brightness, and take the test again.
Speed matters
Faster answers, more red-green results
Share of tests ending in a red-green result, by average time per plate
Tests answered in under 2 seconds a plate are left out: almost half of those were inconclusive.
Users who averaged two to three seconds a plate got a red-green result 16% of the time. Users who took six to ten seconds got one 6% of the time.
This cuts both ways. Someone who cannot see a figure has nothing to study, so they answer quickly, and that alone makes fast tests lean toward red-green results. But the plates near your limit do need a proper look. A quick tap on a faint plate counts as a miss, and enough misses look like a color vision difference.
Screens differ
The same test reads differently on different screens
Median score by device. The dashed line is 100, the typical score.
Numbers on the right: red-green / blue. The scale runs from 70 to 120.
A few points up or down can come from the screen, not from your eyes. That is one reason a single mild result should not be read as final.
What we are changing
We built these changes from the data, and they are in final testing:
- Confirm before labeling. When a run is heading for any result other than typical, the test shows a few more plates on that channel before it gives the result. A typical eye reads these plates easily, so a result that was only a slip or two falls away.
- One slip counts for less. A single missed plate among many right answers is now treated as a slip, not as a sign of weak color vision.
- The blue channel is recentered, so the typical user scores 100 on it as on the other two.
- A note for hurried answers. When two plates in a row are missed in under a second and a half each, the test asks you to look a little longer.
In our simulations, the changes cut wrong results for users with typical color vision by about half, and users with a real color vision difference are found as often as before. The test’s changelog will show the date they go live.
How to read your own result
- A strong red-green result is likely real, more so if it comes back on a second try. An eye care professional can confirm it with a clinical test.
- A mild, borderline, or blue-yellow result is a reason to take the test again in good conditions, not a conclusion.
- Good conditions means full brightness, night mode off, no strong light on the screen, and a few seconds for each plate.
This remains a screening test. It does not diagnose. If you want to know more about how the plates work, see The New Color Blindness Test Scores Red, Green, and Blue Separately and Why Some Plates on the New Color Blindness Test Are So Hard.
Try the test
The color blind test takes about three minutes. If the result surprises you, check your screen settings and take it once more.
Comments could not be loaded. Check your connection and try again.