NVIDIA developer submits vLLM support PR
A developer at NVIDIA on Nov. 17 (early Nov. 18 Japan time) submitted a PR (pull request, a proposed change) to the public repository of the inference software "vLLM," adding support for an unannounced model called "Nemotron Shortcut." vLLM is open-source software for running AI models on servers and serving them to users. The PR is titled "Add Nemotron Shortcut (text, MTP, vision-language)," and it adds the code for running the model in a new folder. No official announcement by NVIDIA has been confirmed, and the information is a leak seen in a public repository.
The GitHub account of the PR's author lists "NVIDIA" in the organization field of its public profile. The commits (records of changes) are also signed with an nvidia.com address, and six of the seven developers named as co-authors also have nvidia.com addresses.
The PR registers "nvidia/NemotronShortcut" as the model's location, with a setting indicating that it cannot yet be retrieved online (is_available_online=False). On Hugging Face's public API, no public model under that name could be confirmed as of the morning of Nov. 21.
Model type is "nemotron4"
In the PR, the model's configuration types are "nemotron4" and "nemotron4_text," and its architecture name is "Nemotron4ForConditionalGeneration." The PR page also showed, as a related item, a draft change titled "Port Nemotron4 frontend to 0.33" in the author's own repository.
The "Nemotron 4" name had also appeared on Oct. 22 as "nemotron4-nano-preview" in the development records of the company's inference software "TensorRT-LLM." The PR does not explain where Nemotron Shortcut fits within "Nemotron 4."
Text, image and video input, combining components of different kinds
According to the PR description, Nemotron Shortcut accepts text, image and video input. Images are read by a component built on vLLM's "Pixtral" image encoder, and video is read with multiple frames grouped into one unit.
The core of the model is a "hybrid" design that combines components of different kinds. The commit description says it combines a mixer (a component that mixes information across tokens) called "GDP (Gated DeltaProduct)," a sparse attention mechanism called "SDSA" and "shortcut MoE" blocks. In SDSA, an indexing component narrows down the items each token attends to to a fixed number of top candidates, and computes attention only over those.
The MoE (a design that runs only some of several smaller specialized networks) includes a technique named "shortcut." Because the experts that inputs are routed to read the block's input directly, routing to the experts can begin before the mixer finishes its computation, according to the PR.
The model also has an "MTP" component that predicts several upcoming tokens ahead to speed up generation. Weights are supported in 16-bit BF16, 8-bit MXFP8 and 4-bit NVFP4 formats, and the model also supports ways of splitting it across multiple GPUs (TP, PP and DP+EP). Holding the area that stores intermediate results (the KV cache) in NVFP4 awaits an update to the component library "FlashInfer." The vLLM implementation is for NVIDIA GPUs only, and the package description says "Only an NVIDIA implementation exists."
Tested on GB300, no performance figures given
The author writes that the model was tested on a single machine with four of NVIDIA's "GB300" graphics processors. In unit tests, 1,496 passed and one failed. The one failure also occurs on vLLM's main branch, according to the author.
The author says that with a 16-bit model, accuracy on "gsm8k," a test of math word problems, and the rate at which MTP's predictions are accepted were the same with vLLM's configuration and with the model's own, but gives no figures. One test of the NVFP4 experts component was skipped because FlashInfer lacked a needed feature. The PR does not give the parameter count, training data, license, release date or performance comparisons with other companies' models.
Merge appears close, prerequisite change already merged
The PR moved from draft to ready for review on the evening of Nov. 18 Japan time. After review, it was changed so that the model's configuration and image processing component are not kept in vLLM but are loaded from those distributed with the model. Two vLLM maintainers approved it on the night of Nov. 19 Japan time, and it was set to merge automatically.
PR #67941, which fixes the KV cache layout needed to run this model, was merged early on Nov. 20 Japan time. As of ORIENS's check at 8:55 a.m. on Nov. 21 Japan time, #68198 had not yet been merged.
- "nemotron4-nano-preview" appears in TensorRT-LLM development recordsLeak
- Prerequisite vLLM PR #67941 submittedLeak
- PR #68198 supporting Nemotron Shortcut submitted as a draftLeak
- PR #68198 marked ready for reviewLeak
- Two vLLM maintainers approve; set to merge automaticallyThird party
- PR #67941 mergedThird party






