A small model that runs in 4GB
NVIDIA on Oct. 23 (Oct. 24 Japan time) released "Nemotron 3.5 Bolt E4B," a small open model. It is meant to run inside PCs and small computers and to be used for voice assistants, game characters and robots. It was tuned from Google's Gemma 4 E4B.
The compressed version was built to fit in about 4GB of memory. In the company's measurements on a GeForce RTX 5070, it used 3.5GB for a short conversation and 4.3GB at a length of 32,000 tokens. On the embedded Jetson Orin Nano, it used 4.2 to 4.9GB, so it does not always fit in 4GB. The context window is up to 64,000 tokens.
Short replies for spoken language
It has functions for calling tools and for keeping a character's personality. It has a conversational mode that returns short replies that are easy to read aloud, and its reasoning function can be turned on or off as needed. Its average score is higher than the company's previous small model, but on some individual tests other companies' models score higher. The company also released its training recipe and an auxiliary model that speeds up generation.
The name of next-generation "Nemotron 4"
On Oct. 22, the day before the release, the unannounced model name "nemotron4-nano-preview" appeared in the public development records of the company's inference software "TensorRT-LLM." It appeared in a description of a system that hands the work of running new models to an AI agent, as the result of running this model on the latest GB300 graphics processor. @aitrackerbot, an X account that tracks the appearance of models, reported it.
According to @aitrackerbot, components for Nemotron 4's large "Ultra" and small "Nano" were also added to FlashInfer, an open library of components that speed up inference. Nano uses an architecture that switches among 512 expert components, and is configured to have a context window of 1 million tokens, it said. ORIENS has not been able to confirm this configuration, and NVIDIA has not announced Nemotron 4.
NVIDIA has been expanding its "Nemotron" series, for which it releases model weights and training data. In June it released the large "Nemotron 3 Ultra."
