What is automatic speech recognition (ASR)?
It is why a voice agent that sounded flawless in a demo starts mishearing addresses the day it takes real calls over a phone line.
Automatic speech recognition (ASR)
ASRspeech-to-textSTT
Automatic speech recognition is software that converts spoken audio into text. A voice agent, a transcription tool, or a dictation app all depend on it to turn what someone said into words a program can process.
An ASR model listens to an audio stream and outputs the most likely sequence of words. Accuracy is measured as word error rate (WER): the percentage of words the model inserted, deleted, or substituted wrong, compared to a human transcript.
Vendor demo numbers are usually measured on clean, high-sample-rate audio recorded close to a microphone. A phone call is neither. Standard telephone audio runs at 8kHz, roughly a third of what a laptop mic captures. It also arrives with dial tones, road noise, or a second voice in the room.
Why phone accuracy is a different number
A model can post a low word error rate on a benchmark and still struggle on your actual traffic. Accents, cross-talk, and hold music bleeding through the line all push the error rate up. So does a caller reading digits aloud. None of that shows up in a clean benchmark.
Streaming matters as much as accuracy. A voice agent cannot wait for a full sentence before it starts processing, or the reply lands late and the call feels broken. Streaming ASR returns partial text as the audio arrives. That lets a system start reasoning about a reply before the caller finishes talking.
We built the inbound voice agents for ZhoopZhoop, a multi-branch collision repair business. Callers there read out plate numbers and VIN strings over the phone. That kind of sequence, digits and letters spoken one after another, is exactly where a model tuned on ordinary conversation tends to slip. We tested transcription against the vocabulary those calls actually use, not a generic benchmark.
- Audio in8kHz phone line, not a studio mic.
- Streaming ASRPartial transcript builds as words arrive.
- Word error rateMeasured against a human transcript.
- Confidence checkLow-confidence spans flagged for confirmation.
- Text to modelDownstream logic acts on the transcript.
The confidence check is the step a demo build skips. Without it, a misheard VIN character goes straight into a booking with nobody catching it.
Common questions
01Is ASR the same as voice recognition?
No. Automatic speech recognition converts speech to text regardless of who is speaking. Voice recognition identifies who the speaker is, from their voice. A phone system can use one, both, or neither.
02Does higher ASR accuracy always mean a better voice agent?
Not on its own. A slightly higher error rate, paired with a fast confirmation step for a VIN or a policy number, can still be more reliable. Trusting every transcript blindly is the riskier design.
03Why does ASR accuracy vary between providers?
Providers train and tune models on different audio, and few publish word error rates measured on 8kHz phone audio with background noise. The number quoted in marketing is often measured on cleaner audio than the one your calls will produce.
