Product Research · 6 min read

Sixty-Seven Percent Is a Choice

The famous 67 percent pass mark for symbols is one committee's choice, not part of the test. Set your bar by risk, and count the opposite readings.

A team tests a new icon. Thirty-four of fifty people write the right meaning: 68 percent. Someone says the magic number, sixty-seven, and the icon ships. Nobody asks who set that number, or what the other sixteen people thought the icon meant. The second question is the one that matters.

That’s why a test needs more than a headline score: the wrong answers show where an icon fails and what that failure could cost.

Where sixty-seven comes from

The formal method for testing symbols is ISO 9186-1. Its second edition came out in 2014 and replaced the 2007 edition. You show people the symbol, tell them the context in words or a picture, and ask what it means. A trained person who did not design the symbol sorts the answers into correct, wrong and the rest. A 2023 guide to the test methods says to test with 50 people from the target group for each symbol variant in each country, and to compare up to three variants at once.

Here is the part most teams miss. The method does not set a pass mark. The 2007 edition says each variant is scored by its percentage of correct answers, and leaves it to the standards committee to "lay down the score" a symbol must reach. The same guide says it plainly: "The standard does not specify a minimum number of correct answers."

The 67 percent figure comes from the safety-sign side of ISO. A 2004 study by Mayhorn, Wogalter and Bell cites ISO 3864 as requiring 67 percent comprehension. A 2014 study by Duarte and colleagues calls 67 percent the criterion "defined by the ISO/TC 145 standard," the ISO committee for graphical symbols. In the United States, ANSI Z535.3 sets a higher bar for safety symbols: at least 85 percent correct, with no more than 5 percent critical confusion.

So 67 percent is a choice one committee made for one kind of sign. It is a reasonable choice. It also passes a sign one in three people get wrong.

The number that matters is the wrong answer

A critical confusion is an answer that reads the opposite of what the sign means. "Do not drink" read as "drinking water." "Stop" read as "go." The ANSI bar caps these at 5 percent because a wrong guess and an opposite guess are not the same failure. A wrong guess makes people pause. An opposite guess makes them act.

The research shows how often symbols fail both bars. Mayhorn, Wogalter and Bell tested 24 proposed homeland security symbols with 57 people. Nineteen of the 24 failed the 85 percent bar. Twelve failed the 67 percent bar. Sixteen had critical confusion above 5 percent. One symbol, a whistle, was understood by 12.3 percent of people, and 86.0 percent gave a critical confusion.

Standard symbols do not escape this. Zamanian and colleagues tested 10 workplace safety signs with 53 employees in Iran. Four of the 10 fell below the ISO and ANSI minimums. Duarte and colleagues tested 17 safety signs with 90 people in Lisbon and found that most were not well understood. The 2023 guide reports that of the 108 safety signs in ISO 7010:2016, only 9 meet all the test criteria, and 82 have no test results at all.

If the official signs on factory walls miss these bars, the icon your team drew last sprint has not earned the benefit of the doubt.

Set the bar before you test

My position: a pass mark is a design decision. You set it before the test, by the cost of a wrong reading, and you report the opposite readings next to the score.

A tiered bar is easy to write down:

**Low risk.** A wrong guess costs a click. A filter, a view toggle, a sort. A lower bar is fine, especially with a visible label.
**Medium risk.** A wrong guess costs time or work. Sync, share, archive. Use 67 percent as a floor, with few opposite readings.
**High risk.** A wrong guess loses data, money or safety. Delete, send, override, emergency stop. Use the 85 and 5 percent pair, and add a label even if the icon passes.

Write the bar and the reason into the icon's record before anyone sees results. A bar set after the results is not a bar. It is a description.

The strongest case against

The counterpoint is serious. High bars kill useful symbols. Many software icons are symbols in Peirce's sense: they mean by convention and must be learned. Almost no new convention reaches 85 percent on first sight. If every icon had to pass that bar cold, products would ship with almost none. Context also lifts scores. Zamanian's team found that text under a sign can change how well it is understood, and familiarity and training helped. A 67 percent icon with a label may work better in real use than an 85 percent icon alone.

All of that is true, and it supports the position. If the right bar depends on risk, labels and training, then the bar must be chosen on purpose, not borrowed from a safety-sign committee for every glyph.

Small samples lie quietly

One more trap. Most product teams test icons with ten people, not fifty. With ten people, each answer moves the score by ten points. Seven out of ten looks like a pass at 70 percent, but the honest 95 percent range for that result runs from about 40 to about 89 percent. You cannot tell a strong icon from a coin toss with that sample.

Ten people are still worth testing. Use them to find confusions, not to prove a score. One opposite reading out of ten on a high-risk icon tells you something a pass mark would hide. Fix it, then run the bigger test before you claim a number.

Icon lens

Many critical confusions are errors of mood, not of noun. People see the right object and read the wrong force: allowed instead of forbidden, safe instead of warning. Signage already splits the two: the frame carries the mood, the symbol carries the noun. Icon grammar makes the same split. The noun lives in the root glyph, and its force lives in a separate modifier, the Severity root (tivrata). It also gives the pass mark a home. The bar should follow Severity: a glyph that carries high Severity gets the strict bar and a mandatory label, while a quiet glyph can earn its meaning over time. Testing then asks two questions: did people get the noun, and did they get the force? Only the second failure makes people act wrong.

Try this

**Goal:** Decide pass marks by risk before you test, and see where your icons get read backward.

**Steps:**

1. Pick five icons from your product. For each, write the risk tier (low, medium or high), the pass mark you will accept, and what an opposite reading would be. 2. Show each icon at real size, on a plain screen, with one sentence of context ("You are editing a shared file"). Ask ten people who have never used your product what it means and what they would expect to happen. 3. Score every answer as correct, wrong, opposite, or no answer. Count opposites separately from the total. 4. Compare each icon to the bar you set in step 1, not to a bar you choose now. Any opposite reading on a high-risk icon is an automatic fail. 5. Redraw, relabel or move to a larger test every icon that missed its bar.

**Check:** You have a table with the bar set before the test, the score, the opposite count and a decision for each icon, and no bar was changed after you saw the results.

**Reference:** Appendix A.9, Accessibility and proof of comprehension.

Sources

[ISO 9186-1:2014 preview (second edition, replaces 2007)](https://webstore.ansi.org/preview-pages/ISO/preview_ISO+9186-1-2014.pdf)
[ISO 9186-1:2007 catalog entry and text (iTeh Standards)](https://standards.iteh.ai/catalog/standards/iso/bad7fc8c-ea57-459e-a95c-e57693354c22/iso-9186-1-2007)
[Safety signs test methods (bravecroc, 2023)](https://www.bravecroc.de/en/sicherheitszeichen-pruefmethoden/)
[Mayhorn, Wogalter and Bell, Homeland Security Safety Symbols: Are We Ready? (Ergonomics in Design, 2004)](https://www.safetyhumanfactors.org/wp-content/uploads/2020/07/244)Mayhorn,Wogalter,Bell(2004).pdf)
[Duarte et al., Safety sign comprehension by students, adult workers and disabled persons with cerebral palsy (Safety Science, 2014)](https://www.sciencedirect.com/science/article/abs/pii/S0925753513001859)
[Zamanian et al., Comprehension of workplace safety signs: a case study in Shiraz industrial park (2013)](https://www.researchgate.net/publication/320562669_Comprehension_of_workplace_safety_signs_A_case_study_in_Shiraz_industrial_park)
[IAEI Magazine, Safety Signs Part 2 (2008)](https://iaeimagazine.org/2008/september2008/safety-signs-part-2/)
[In Compliance, ANSI Z535.3: Safety Symbols in Focus](https://incompliancemag.com/ansi-z535-3-safety-symbols-in-focus/)