Two models of 1.3 billion and 14 billion parameters, targeting 50 ms or less per frame

Apple's research team has released the weights (trained parameters) of FastFrames, an AI model that generates video from text, on Apple's organization page on Hugging Face. The release includes two models: the 1.3-billion-parameter FastFrames-S and the 14-billion-parameter FastFrames-L. According to the model cards, both are "autoregressive" video diffusion models that build each frame in sequence after the previous one, need only two denoising steps, and generate 480p video at "interactive latency."

On Nov. 17 (Nov. 18 Japan time), the team posted the paper "FastFrames: Video Generation at Interactive Latency" to arXiv. The seven authors are from Apple and Cornell University, and the paper says the work was done during the first author's internship at Apple. The project page sets the target for responding smoothly to user input at 50 milliseconds or less per frame.

According to Hugging Face's API, both repositories were created at 2:21 a.m. Nov. 17 Japan time. The time they were made public is not shown. The official GitHub repository linked from the model cards was created at 6:30 a.m. Nov. 18 Japan time.

The 1.3B version runs at 36 ms on one B200, the 14B version at 45 ms on eight

According to the paper's abstract, FastFrames-S runs at 36 milliseconds per frame, or 28 responses per second, on a single NVIDIA B200 GPU. Apple says it matches the quality of a state-of-the-art model that is nearly eight times slower. FastFrames-L runs at 45 milliseconds per frame, or 22 responses per second, on a node of eight B200s, and Apple says it delivers significantly better visual quality and realism than FastFrames-S.

In an example given on the model card, FastFrames-S took a total of 35.4 milliseconds to generate one 480p frame on one GPU. That breaks down into 16.1 and 16.2 milliseconds for two passes of the main video network (DiT), 2.2 milliseconds for updating its memory and 0.9 milliseconds for converting to pixels. The sample generation produces an 832 x 480 video of 81 frames at 16 frames per second.

Decoding to images and memory updates took over half the delay

According to the paper, the team profiled inference in existing models and found that two steps usually treated as fixed parts account for more than half of total latency. They are decoding the compressed representation back into images (VAE decoding) and updating the "KV cache," which stores information from earlier frames. FastFrames replaces both with lightweight alternatives and also speeds up the main video network's computation with implementation optimizations.

FastFrames-L was built by distilling the 14-billion-parameter version of the video generation model Wan2.1 into a model that runs in two steps, following the recipe of an existing method, Causal Forcing++. Apple also released the intermediate training checkpoints along with the final version. FastFrames-S keeps the two-step Causal Forcing++ generator as is and trains only the components around it. As a result, FastFrames-S's Hugging Face repository holds only the added components and takes up about 1.8 gigabytes, compared with about 174 gigabytes for FastFrames-L.

Use limited to research, measured on the latest B200

The license for the weights, the "Apple Machine Learning Research Model License," limits use to research purposes. It defines research purposes as non-commercial scientific research and academic development intended to advance scientific knowledge, excluding commercial exploitation, product development and use in commercial products or services. Derivative models built from the weights are subject to the same restriction.

The FastFrames code is released under the "Apple Sample Code License." The LingBot-World v2 code and weights included in the repository follow the upstream CC BY-NC-SA 4.0 license. Running the models requires Linux (x86-64), an NVIDIA GPU and a driver compatible with CUDA 12.8.

The paper lists the use of B200s for its measurements as one of its limitations and describes running the models on-device as a future possibility. In measurements in the appendix, NVIDIA's H100 GPU was roughly 25% to 50% slower depending on the configuration. The models also assume 480p; at 720p, FastFrames-S takes 127 milliseconds on one B200 and FastFrames-L 116 milliseconds on eight B200s, above the 50-millisecond target.

On Hugging Face, users outside Apple have published three apps (Spaces) that use FastFrames-L. As of Nov. 21, FastFrames-L had 488 likes and FastFrames-S had 111.

Applied to existing models, LingBot camera control runs 16 times faster

The team also applied the FastFrames techniques to two existing models. They are LingBot, a video model whose camera can be moved with key presses, and RealWonder, which moves dragged objects in line with physics. For the 1.3-billion-parameter LingBot, latency fell from 842 milliseconds with the fastest official configuration (two GPUs) to 52 milliseconds on eight GPUs, 16 times faster. RealWonder fell from 1,141 milliseconds (official, one GPU) to 51 milliseconds (eight GPUs), and the 14-billion-parameter LingBot from 1,373 milliseconds to 233 milliseconds (both on eight GPUs).

The paper says LingBot and RealWonder remain slower than FastFrames' text-to-video models because they need more denoising steps, generate multiple frames per step and require extra computation to process the control inputs. On Hugging Face, Apple also released components for the 1.3-billion-parameter LingBot accelerated with the FastFrames techniques. All measurements were made on B200s at 832 x 480.

In Apple's own comparison, faster but behind some models on "Semantic"

Table 1 of the paper compares generation time and scores on the video benchmark VBench, using the same prompts and seeds at roughly 480p. Higher VBench scores are better, and the table shows three scores: Total, Quality and Semantic. Measurements are on one B200 unless noted, and rows using eight GPUs are marked in the table.

FastFrames-S generated in 36 milliseconds, shorter than the 272 milliseconds of Causal Forcing++, also a 1.3-billion-parameter model, and Apple says it is nearly eight times faster. Its Total score of 84.23 topped Causal Forcing++'s 84.11, but its Semantic score of 79.92 was below Causal Forcing++'s 80.20 and Self-Forcing's 81.13. FastFrames-L (158 milliseconds on one GPU) was faster than Krea-Realtime (1,675 milliseconds), a model of the same size, and its Total score of 84.44 edged Krea-Realtime's 84.42. Its Semantic score of 80.29, however, trailed Krea-Realtime's 81.81 and Wan2.2's 81.60, and its Total score was below Wan2.2's 84.65.

Chart from Apple showing the relationship between VBench scores and generation latency. Higher scores and lower latency are better, and the shaded region marks the target of 50 milliseconds or less
Chart from Apple showing the relationship between VBench scores and generation latency. Higher scores and lower latency are better, and the shaded region marks the target of 50 milliseconds or less (Source: FastFrames: Video Generation at Interactive Latency (project page, Figure 1))
Generation time and VBench scores, as measured by Apple and listed in Table 1 of the paper (one NVIDIA B200 unless noted, roughly 480p). Types follow the paper's classification
ModelTypeSize (billion parameters)Generation time (ms)VBench TotalQualitySemantic
Wan2.1Bidirectional1.330,44483.8084.5680.73
Wan2.2Bidirectional14.3117,73484.6585.4181.60
MiniMax-H3Bidirectional33.138,45484.4385.1781.47
HunyuanVideo1.5Bidirectional8.31,555,50883.1884.1479.35
LTX2.5Bidirectional19.011,26483.6284.7878.96
MAGI1.1 DistilledAutoregressive23.919,73682.0483.4076.60
Helios DistilledAutoregressive14.32,54483.4384.5379.03
Krea-RealtimeAutoregressive14.31,67584.4285.0881.81
LongLive2.0Autoregressive5.050583.6584.5480.09
CausVidAutoregressive1.366283.1884.1579.31
Self-ForcingAutoregressive1.371784.1084.8481.13
Causal Forcing++Autoregressive1.327284.1185.0980.20
FastFrames-SAutoregressive1.33684.2385.3079.92
FastFrames-S (8 GPUs)Autoregressive1.31484.2385.3079.92
FastFrames-LAutoregressive14.315884.4485.4880.29
FastFrames-L (8 GPUs)Autoregressive14.34584.4485.4880.29

On Self-Forcing's generation time, the paper notes that the figure reported for that model is an average that includes cheaper early frames before the KV cache fills, while Apple measured with a full cache. No third-party evaluation under the same conditions has been confirmed.