In short
A low word error rate shows that the recognized words match a human reference transcript closely. It does not show whether a viewer can read the subtitles in time.
The BBC Subtitle Guidelines allow editing when speech is too fast, and they set rules for reading time, line breaks, line length, and timing to the picture. Word error rate counts none of those. That makes it the wrong score for judging a subtitle file.
Some speech recognition vendors lead with one number: word error rate (WER). The number measures something real. What it measures is a transcript.
What does word error rate measure?
The OpenASR21 Challenge Evaluation Plan, version 1.3, published by the National Institute of Standards and Technology (NIST) on 31 August 2021, names word error rate as its primary metric. The plan adds up three kinds of error in the recognized text: deletions, insertions, and substitutions. It compares them with a human reference transcription and divides by the number of words in that reference.
In plain terms: a word the system missed, a word it added, and a word it got wrong all count against it. The reference is a human transcription of the same audio.
The plan's task is turning audio into written text. It does not mention subtitles, reading time, line length, or shot changes. That is not a flaw in the measure. The plan was written to score speech recognition, and that is what it scores.
Why are broadcast subtitles sometimes not word for word?
Broadcast guidelines prefer verbatim subtitles. They also expect editing when the speech moves too fast to read.
The BBC Subtitle Guidelines, version 1.2.5 (March 2026) say not to edit unnecessarily when there is time for verbatim speech. The aim is to give viewers as much of the soundtrack as possible within the limits of time, space, shot changes, and what is on screen (section 2.1). When a speaker talks very fast, the subtitles have to be edited as much as needed to meet the timing rules (section 5.3). For live news, current affairs, and most other genres, the BBC expects verbatim subtitles up to around 160 to 180 words per minute (WPM), and some editing above that speed (section 21.3).
The same guidelines rank the trade-offs. When editing, timing, and line breaks cannot all be satisfied, well-edited text and timing matter more than line breaks (section 3.10).
Ofcom's Guidelines on Providing Television and On-Demand Access Services take the same line. Subtitles should reflect speech verbatim as closely as possible. Some paraphrasing may be appropriate in limited cases, for example when dialogue is very fast. Filler words such as "umms" and "errs" can go when they carry no meaning (paragraph 3.6).
Key point
An edited subtitle differs from the transcript on purpose. Scored against a verbatim transcript, every word the editor removed counts as a deletion. Word error rate cannot tell a careful edit from a word the system lost.
The same gap sits behind our earlier article, a transcript is not a subtitle: the transcript follows the speaker, and the subtitle follows the viewer.
What can go wrong in a subtitle when every word matches?
A subtitle can match the transcript word for word and still fail the viewer. The BBC Subtitle Guidelines and Ofcom's Guidelines on Providing Television and On-Demand Access Services set the checks below. Each figure belongs to the document named next to it.
| What the viewer needs | Document and provision | What it says |
|---|---|---|
| Time to read | BBC Subtitle Guidelines, sections 4 and 4.1 | Recommended speed of 160 to 180 WPM. Timing is an editorial decision that depends on the speed of speech, the editing, and shot synchronisation. Judging a subtitle by the number of words on screen alone would be "an unacceptably crude approach". |
| Time to read | Ofcom guidelines, paragraph 3.6 | Subtitles stay on screen long enough to be read. Where it can be avoided, they do not stay up so long that they distract or run over shot changes. |
| Sensible line breaks | BBC Subtitle Guidelines, section 3.4 | Break subtitles and lines at logical points. Breaks inside a word are especially disruptive and should be avoided. |
| Sensible line breaks | Ofcom guidelines, paragraph 3.16 | Segment subtitles and lines at logical linguistic breaks. |
| Line length | BBC Subtitle Guidelines, section 3.1 | 37 fixed-width characters per line (CPL) on broadcast Teletext. Online, a line fits within 68 percent of the width of a 16:9 video, or 90 percent for 4:3, square, and vertical video. A file meant for both online and broadcast stays within 37 characters inside a region. |
| Timing to speech | BBC Subtitle Guidelines, section 5.1 | The subtitle appears when the speech starts and leaves roughly when that speech ends. |
| Timing to the shot | BBC Subtitle Guidelines, section 6.3 | Avoid a subtitle that starts in the middle of one shot and ends in the middle of the next. |
| No spoilers | Ofcom guidelines, paragraph 3.6 | Subtitles should not appear before key information is delivered, such as a punchline in comedy or a quiz. |
| Short live delay | Ofcom guidelines, paragraph 3.9 | Aim for a mean latency of no more than 4.5 seconds across live programming taken together. Latency is the delay between the speech and the live subtitle. |
| Sound that is not speech | Ofcom guidelines, paragraphs 3.1 and 3.11 | Subtitles represent speech and sound effects. Subtitles for people with hearing loss and d/Deaf people describe relevant sound effects, the presence and mood of music, and tone of speech. |
Word error rate, as the OpenASR21 Challenge Evaluation Plan defines it, records none of these rows. A file can score well and still break a line inside a word, sit across a cut, or leave out a sound effect the viewer needed.
How do you read a vendor page that leads with word error rate?
Read each score as that vendor's own claim about transcription. The three pages below publish or explain a word error rate. None of them states a subtitle reading speed, a line-length rule, or a shot-change rule.
| Vendor page | What the vendor states |
|---|---|
| AssemblyAI, "How to evaluate speech recognition models" (21 July 2026) | Its Universal-3.5 Pro model has a mean English word error rate of 5.6 percent (median 4.9 percent), measured across 26 public datasets, more than 80,000 files, and more than 250 hours of audio. Its Universal-3.5 Pro Realtime model has a pooled word error rate of 6.99 percent on Pipecat's open speech-to-text benchmark of voice-agent conversations. |
| Speechmatics, "Accuracy benchmarking" | Word error rate counts insertions, deletions, and substitutions against a human reference, as a percentage of the reference word count. Results depend a lot on the audio and the speaker. Punctuation, capitalization, and diarization are not measured by word error rate, and they matter for readability. |
| 3Play Media, The 2024 State of Automatic Speech Recognition Report | Word error rate is commonly used to judge the quality of automatic speech recognition. In 3Play Media's reading, a "99% accurate captions" label refers to word error rate, which means a word error rate of 1 percent. The measure only counts words, not capitalization, punctuation, or number formatting. |
Two of these vendors say plainly what the score leaves out. Speechmatics names punctuation, capitalization, and diarization. 3Play Media names capitalization, punctuation, and number formatting. A subtitle also has to meet rules for reading time, line length, and timing, and those sit outside any transcript score.
A word error rate is useful when you compare speech engines. In a vendor meeting, these questions tell you what the number can carry:
- What was the reference: a verbatim transcript, or edited subtitle text?
- What audio was it measured on, and how close is that to your shows?
- Was the score taken from a transcript or from the subtitle file you will receive?
- What does that subtitle file score on reading speed, line length, timing, and segmentation?
What should you ask for instead?
Ask for results on the file you will broadcast, measured against your own targets. The table above gives the reference points from the BBC and Ofcom. In practice, that means:
- reading speed for each subtitle
- line length for each line, against your delivery format
- timing to the speech and to shot changes
- segmentation at logical breaks
- sound effects and music, where your service needs them
- for live output, the mean delay between speech and subtitle
Then ask who checks what software cannot. The BBC Subtitle Guidelines say the BBC's own file validator cannot check every aspect of conformance, such as points that need editorial judgement (Appendix 5). Ofcom's Guidelines on Providing Television and On-Demand Access Services say good access services rest on informed, case-by-case judgements by trained people (paragraph 2.29). They also ask providers to review the quality of pre-recorded access services before transmission and again afterwards (paragraph 2.30).
Ofcom does say providers should consider quantitative models for aspects such as accuracy and latency. The guidelines do not name word error rate as that model.
Keep word error rate for what it does well: telling you how closely a speech engine hears the words. For the subtitles, measure what the viewer reads. That means reading speed, line length, timing, and segmentation, with a person who can fail the file.
Frequently asked questions
01
Does a low word error rate mean the subtitles are good?
No. Word error rate counts deletions, insertions, and substitutions in recognized text against a human reference transcription. That is how the National Institute of Standards and Technology defines it in the OpenASR21 Challenge Evaluation Plan, version 1.3. The BBC Subtitle Guidelines also judge subtitles on reading speed, line breaks, line length, and timing to the picture. The score records none of those.
02
Why can a well-edited subtitle get a higher word error rate?
Broadcast guidelines allow editing when speech is too fast to read. When a speaker is very fast, the BBC Subtitle Guidelines require the subtitles to be edited as much as needed to meet the timing rules (section 5.3). Ofcom's Guidelines on Providing Television and On-Demand Access Services allow limited paraphrasing when dialogue is very fast (paragraph 3.6). If you score that edited subtitle against a verbatim transcript, every word the editor removed counts as a deletion.
03
What does a "99% accurate captions" label mean?
In The 2024 State of Automatic Speech Recognition Report, 3Play Media reads that label as a word error rate of 1 percent. The same report says word error rate only counts words. It does not count capitalization, punctuation, or number formatting. That reading is 3Play Media's. Ask the vendor what its own number measures, and on which audio.
04
What should I ask a subtitle vendor instead of a word error rate?
Ask for results on the subtitle file you will deliver: reading speed, line length, timing to speech and to shot changes, segmentation, and sound that is not speech where your service needs it. Then ask who reviews the file. The BBC Subtitle Guidelines say the BBC's own file validator cannot check points that need editorial judgement (Appendix 5).
05
Is word error rate useless for subtitling?
No. It shows how closely a speech engine's words match a human reference transcription. That helps when you compare speech engines. It does not tell you whether a viewer can read the subtitles in time. Use it next to subtitle checks, not in their place.
Book a free demo
Want to see how Undertitle turns English shows and films into Dutch subtitles? We will walk through it with you.