From transcription to answering questions
NVIDIA on Oct. 28 (Oct. 29 Japan time) released the weights of "Nemotron-3.5-Transcribe-30B-A4B," an open model that converts speech to text, on Hugging Face. @aitrackerbot, an X account that tracks new model releases, was among the first to report it.
It is a "mixture of experts" model in the 30-billion-parameter class that uses only about 4 billion parameters for each token. It takes the form of a language model whose output can be changed by instructions, and a single model can transcribe 40 languages and translate between 31 languages and English. In 13 languages, it identifies up to 8 speakers and adds word-level timestamps. It also answers questions about the content of the audio in English, and can flag personal information and accept specified technical terms.
It can handle recordings of up to one hour but does not support real-time transcription while someone is speaking. It is released under a license that allows commercial use, and NVIDIA confirmed that it runs on a single NVIDIA H100.
Best English error rate among open models
On a Hugging Face leaderboard comparing English transcription error rates, it scored 5.33%, the best among open models. On Artificial Analysis' measure, it scored 2.81%, second among open models and behind proprietary models from Microsoft and Google.
Weaker in some languages
It does not support transcription in Norwegian, and its error rates exceed 20% in Greek and Hebrew. It was trained on 460,000 hours of audio in 48 languages.
Next text-to-speech version in development
Earlier the same week, on Oct. 27, @aitrackerbot reported that a public proposed change to "NeMo," NVIDIA's development software for speech AI, showed that a text-to-speech model, "NemotronTTS v2," is in development. It replaces the component that predicts fine details of the sound with a new design, and the engineer in charge has reported testing v2, according to the post. The proposal is awaiting review, the model has not been released, and the timing is unknown.
On Oct. 23, NVIDIA also released "Nemotron 3.5 Bolt E4B," a small model that runs on devices.
