How to: Cloze testing for UX writers

What is cloze testing?

Cloze testing is a common evaluation test to measure a text’s comprehension and readability, by omitting certain words.

•Comprehension is the ease with which a user can understand the intended meaning of a text and draw the correct conclusions from the it.

•Readability is (you guessed it) how easy a user can understand the words in a text.

The method

1. A user is provided with a sample of text, with every Nth word missing (usually N is 6, but a larger number can be used to make the test easier).

2. The user is asked to fill in the missing words to the best of their abilities. No time limit is required.

Calculating results

1. For each empty space, make a list of words entered by all user participants.

2. Then calculate the percentage of users who chose each word in each empty space.

3. The score can be expressed as a percentage. I.e. If 8 users out 12 entered ‘bill’ in the empty space, the word has a score of 8/12 = 66.6%. A score of 60% or above by most users indicates that the word is comprehended by most users and can be used.

4. In the scenario that no word used by user participants has a score of 60%<, we must continue testing until a word scores 60%<.

Test tips

•Include at least 10 users as participants.

•Ensure users know that spelling and punctuation errors don’t matter.

•Omit words that have a higher impact on users’ comprehension of the text. For example nouns and verbs like ‘bill’ or ‘discover’, rather than conjunctions or prepositions like ‘and’ or ‘with’.

•40–50 empty spaces is optimal for cloze testing (so a total word count after completion of 280–350 words). If working with microcopy, valuable insights can still be gleaned from a smaller number of empty spaces.

Open vs closed cloze tests

•In open cloze tests, users can choose the words they write in the empty spaces.

•In closed cloze tests, users choose from a shortlist which words they write in the empty spaces.

How can cloze testing help me as a UX writer?

Cloze testing helps us to align products with user expectations. This is a critical process for increasing overall usability, as well as KPIs such as conversion rate.

It also helps us to make tough decisions on which vocabulary components belong in our content design systems. As UX writers, a core responsibility is to write for users in their own style. Cloze testing is an invaluable tool for this.

‍