Excluding the agent's voice from recognition
Speech recognition company Soniox on Nov. 3 (late Nov. 3 Japan time) released a new speech recognition model, "STT v6." It comes in two versions, "stt-rt-v6," a real-time version that transcribes as people speak, and "stt-async-v6," which transcribes recorded files, and replaces the previous-generation v5. The company bills it as "the world's first speech recognition model" that knows what a voice agent is saying.
Introducing Soniox STT v6, the world's first speech recognition model that knows what your voice agent is saying and automatically filters out the agent's own voice.
Voice agents are systems in which AI interacts with users by voice over the phone or in apps. When the agent's voice from the speaker returns to the microphone, speech recognition can mistake it for the user's speech, causing the agent to stop mid-sentence or respond to its own voice. Soniox says such false interruptions and repeated requests to repeat will decrease.
Receiving the reply text, still listening for interruptions
The system consists of two features. One sends the reply text that the agent passes to speech synthesis into speech recognition as well, over the same channel as the user's audio. Because both sides of the conversation are known, the company said names, numbers, technical terms and short replies become easier to recognize.
The other uses that text as a cue to separate the agent's voice from the user's and exclude it from recognition. It keeps listening to the user's voice while the agent is speaking and picks up interruptions mid-utterance. Soniox says false interruptions occurring on speakerphones, laptops and in-car devices will decrease.
Both features are available only in the real-time version and work in all of the more than 60 supported languages. They do not replace conventional acoustic echo cancellation but are positioned to handle the voice left over after that processing, and the company asks users to keep conventional processing enabled. They require that the agent has the text of its own replies.
Real-time version costs $0.12 per hour
Pricing is $0.12 per hour of audio for the real-time version and $0.10 for the file version. Per token, the real-time version costs $2 per million tokens of audio input, $4 for text input and $4 for text output, and the file version $1.5, $3.5 and $3.5. One hour of audio corresponds to about 30,000 input tokens. The agent's reply text is billed as text input.
In its own comparison, Soniox says processing 1,000 hours a month costs $120, $900 less than the $1,020 for OpenAI's real-time transcription "gpt-realtime-whisper."
v5 will be removed on Dec. 15. The company said migration only requires changing the model name.
No accuracy figures for v6; previous generation fell short of the top
Soniox said v6's accuracy "significantly improved" over v5 across more than 60 languages, but it has not published error rate or latency figures. It said speaker identification, language detection and end-of-speech detection also improved.
As of Nov. 4, v6 was not listed on the speech recognition leaderboard of third-party evaluation site Artificial Analysis. The word error rate of the previous-generation file version, "Soniox v5 Async," was 3.8%, higher than 1.7% for Alibaba's "Fun-Realtime-ASR-preview," 2.0% for Microsoft's "MAI-Transcribe-2" and 2.2% for ElevenLabs' "Scribe v2." Its price of $1.66 per 1,000 minutes is on par with $1.67 for MAI-Transcribe-2 and xAI's "Grok Voice Transcribe 2.0," among the cheapest of the models compared.
Artificial Analysis's error rate is a weighted average over three kinds of audio: voice agent conversations, European Parliament speeches and earnings calls, weighted 50%, 25% and 25%.
Rivals detect interruptions by turn boundaries
In speech recognition for voice agents, a growing approach is to handle interruptions by detecting whether the user has started speaking. Deepgram's "Flux" builds detection of the start and end of speech into the model and stops the agent's audio when the user starts speaking. For the problem of the agent's voice coming back, it recommends echo cancellation on the device and, as a last resort, discarding transcripts by matching them against the agent's most recent utterance.
MarkTechPost published a comparison table listing model-based removal of the agent's voice as "not disclosed" for AssemblyAI, Deepgram and ElevenLabs. Voice model announcements continue: on Oct. 20, France's Mistral AI released "Voxtral 3" and ElevenLabs released "Eleven Realtime v1" for voice agents.
