Why Our Vocabulary Test Stopped Using Synonyms and Antonyms

The AREALME English Vocabulary Size Test is one of our oldest tests. For more than ten years, each question showed an English word and asked for another word with the same meaning, or with the opposite meaning. Version 3 asks something simpler: what does this word mean?

This post explains why we changed the question format, what we kept from the old one, and what the change costs us.

A version 3 result. Each column is a group of words, from the most common to the rarest, and the green area adds up to the estimated vocabulary size.

What the Old Questions Looked Like

Two questions from the old test:

Question	Options	Answer
large: same meaning	tiny, faded, new, big	big
old: opposite meaning	tell, small, age, new	new

The format had real strengths. There were only about five words to read per question, the first questions were very easy, and writing a new question took little more than a word and four others. Many people with only a little English could take the test and enjoy it, and we wanted to keep that.

Most Words Have No Synonym or Antonym

A vocabulary size test works like an opinion poll. It can’t ask about every English word, so it asks about a random sample of words from each frequency group and scales the answers up to the whole group. That only works if any word can be drawn.

We checked every word on our list against WordNet, Princeton’s database of English. A synonym question is only fair when the answer is a more common word than the one being tested. Otherwise, you also need to know a harder word to get it right. By that rule:

Word group	Words with a fair one-word synonym or antonym
The 1,000 most common words	19%
Words 1,001 to 3,000	27%
Rarer groups	31% to 39%

Everyday words such as page, house, window, number and news have no clean one-word synonym or antonym, so a synonym test can never ask about them. It ends up measuring a special slice of English, mostly adjectives and verbs that come in neat pairs. The questions are no longer a random sample, and the word count drifts.

Every Question Tested Two Words

A synonym question checks two words at once: the word being tested and the answer. In the old test, 8 of the first 20 answers were rarer than the word they tested, for example:

Tested word	Expected answer
fly	soar
active	passive (opposite)
normal	extraordinary (opposite)
trash	junk

Someone who knew fly perfectly well but had never met soar was marked wrong, and the test counted it as not knowing fly.

Same or Opposite?

Mixing the two kinds of question also set a trap that has nothing to do with vocabulary. People who missed the switch picked the synonym when the question asked for the opposite. The old page colored synonym and antonym questions green and red to help, but those mistakes never fully went away.

What Version 3 Asks

Each question now shows a word, four short meanings and an “I don’t know” button:

Word	Options
page	part of a car, part of a house, part of a shoe, part of a book
number	like yes or no, like one or two, like red or blue, like rain or snow

This is the format of the Vocabulary Size Test used in language research (Beglar, 2010). Any word can be tested this way, so every question is a genuinely random draw from its group. The meanings are written in words that are easier than the tested word, so knowing the word is enough to answer.

For the 3,000 most common words, the meanings are one to four very common words and there is no example sentence, so the opening stays as quick as before. Rarer words also come with a short sentence that shows how the word is used.

Ranking Words by Everyday English

A random sample from a frequency group is only as good as the frequency list behind it. Web text alone ranks page as the third most common English word, and puts mortgage (719th) far ahead of milk (2,520th) and afraid (4,122nd). We now average the web counts with counts from 51 million words of film and TV subtitles, which follow everyday speech (Brysbaert & New, 2009). On the combined list, afraid is in the top 1,000 and mortgage has moved down to 1,718th. The first 20 questions of every test come from the 2,000 most common words on this list.

Guessing and “I Don’t Know”

With four options, a blind guess is right one time in four. Rather than hoping people won’t guess, the scoring works out how often each person guesses from their wrong answers and discounts lucky hits (Stoeckel, Bennett & McLean, 2016). Guessing does not raise your result, and the “I don’t know” button keeps the estimate precise.

What It Costs Us

Meaning questions take more work than synonym pairs. Each one needs four meanings that are clearly right or clearly wrong, written in simpler words than the tested word, with no option standing out by its length. The new bank has 774 questions. The first three frequency groups have 150 each, so people who take the test again rarely meet the same opening questions. We also publish the full calculation on the test page, including how many words on our list we could not count and why.

Try It

The English Vocabulary Size Test takes about ten minutes. It starts easy, adapts to your level in the last part, and gives an estimate of how many English words you know, with a 95% range, a level and a free certificate.

Sources

• Beglar, D. (2010). A Rasch-based validation of the Vocabulary Size Test. Language Testing, 27(1), 101-118. https://doi.org/10.1177/0265532209340194
• Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41(4), 977-990. https://doi.org/10.3758/BRM.41.4.977
• Miller, G. A. (1995). WordNet: A lexical database for English. Communications of the ACM, 38(11), 39-41. https://doi.org/10.1145/219717.219748
• Stoeckel, T., Bennett, P., & McLean, S. (2016). Is “I don’t know” a viable answer choice on the Vocabulary Size Test? TESOL Quarterly, 50(4), 965-975. https://doi.org/10.1002/tesq.325