A framework for letting general-purpose models operate robots
A research team from China's Tsinghua University and the University of California, Berkeley, on Nov. 1 (the same day Japan time) published a paper on "uni-robot-agent," a framework that lets general-purpose AI models operate robots. On Nov. 3 (Nov. 4 Japan time), Yuxin Zuo, who leads the project, announced the results on X. The code is released free of charge on GitHub. He said that using "Qwen3.8-Flash-Next," a model whose weights Alibaba has released, the framework surpassed published results using OpenAI's "GPT-6 Astra" on a robot manipulation benchmark.
‼️ Can Qwen directly control the robot as well as GPT-6 Astra? 🚀 Introducing uni-robot-agent: a unified harness for Robot-Use Agents (RUAs), a first step toward RUA training and local deployment. We tested uni-robot-agent with Qwen3.8-Flash-Next running locally: 1️⃣ On RoboDojo, its score rose from 1.90 to 31.39, surpassing the RoboDojo Team's score with GPT-6 Astra. 2️⃣ On real-world tasks, it outscored fine-tuned π0.5 by 5.75 points and zero-shot Dream-1 by 10 points.
In robot-use agents, a general-purpose AI model that handles text and images carries out work by calling tools such as "move arm" and "grasp" in sequence while watching camera footage. Such models have been shown to handle objects without additional training, but until now they relied on models from U.S. companies whose weights are not released. uni-robot-agent leaves model weights untouched and serves as a foundation (harness) that standardizes how observations are passed, the form of tools and how the results of actions are reported. The research team describes it as a framework that brings open models to competitive performance and can also be used for further training.
Beats dedicated robot models in simulation
RoboDojo, the benchmark used for evaluation, measures accuracy in handling objects and the ability to complete long procedures across 42 tasks in a simulator, on a 100-point scale. Qwen3.8-Flash-Next scored 1.90 with the previous framework, "RoboProbe," but 31.39 with uni-robot-agent. It fully completed 25.00% of the tasks. It surpassed dedicated models trained for robot manipulation: Physical Intelligence's π0.5 scored 11.44 and China's Xiaomi's Xiaomi-Robotics-1 scored 20.07.
However, the 28.97 for GPT-6 Astra that the post said was "surpassed" is a published value measured with RoboProbe. GPT-6 Astra run on the same uni-robot-agent scored 51.90, more than 20 points above Qwen's 31.39. China's DeepSeek's open model DeepSeek-V4.1-Flash scored 27.11. Chinese-language explainer site AI Post Hub pointed out that judging the outcome only by comparison with the 28.97 measured under different conditions would be a mistake.
With depth information (distance to objects), Qwen3.8-Flash-Next scores 35.89 and GPT-6 Astra 56.76. With both depth and camera calibration information removed, Qwen drops to 9.13.
Large gap with Astra on 19 real-robot tasks
The research team also tested 19 tasks on an ARX dual-arm robot, such as standing up test tubes and solving the Tower of Hanoi. According to the project blog and Zuo's post, Qwen3.8-Flash-Next scored 53.50 out of 100. It outscored π0.5, fine-tuned with 10 hours of data per task, at 47.75, and Dream-1 without additional training at 43.50. GPT-6 Astra run on the same framework scored 80.50.
The first version of the paper, published Nov. 1, gave lower values than the blog and post: 49.50 for Qwen, 75.50 for Astra and 35.50 for Dream-1. Conditions also differed in that the agents used depth information from a camera that can measure distance, while π0.5 and Dream-1 used only color video.
Estimated cost per task $1.09, versus $21.29 for Astra
The research team estimated the API cost per RoboDojo task from each company's published pricing. Qwen3.8-Flash-Next cost $1.09, about 1/20 of GPT-6 Astra's $21.29. DeepSeek-V4.1-Flash cost $0.94. The per-million-token prices used in the estimate were $10 for input and $50 for output for GPT-6 Astra, and 0.80 yuan for input and 2.70 yuan for output for Alibaba's qwen3.8-flash (Beijing region, as of Oct. 22-23), converted at 6.6729 yuan to the dollar.
On the other hand, Qwen used 37.67 million tokens per task, more than twice Astra's 16.78 million. Reaching a score of 25 also took Qwen 95.5 minutes, compared with 42.6 minutes for Astra. The post says the model was "running locally," but the costs were calculated by applying Alibaba's API pricing.
Robot manipulation ability emerged with the Qwen3.8 generation
The paper also tested the same framework on models of different generations and sizes to examine when robot manipulation ability emerges. The Flash, Plus and other models from Qwen3.5 through 3.7 tested in the paper did not fully complete a single task on RoboDojo. In Qwen3.8, the 27-billion-parameter Qwen3.8-27B completed 8.06% of the tasks and Qwen3.8-Flash-Next 25.00%. China's Zhipu's GLM-5.3 Flash completed 0%. Among OpenAI's models, GPT-5.6 Sol completed 19.17% and GPT-6.1 Sol 37.50%.
The research team said robot manipulation performance was linked to each of four general abilities and concluded that future training should improve them together. In older Qwen generations, errors in calculating coordinates and rotations accounted for 47.3% to 81.8% of failures.
Code under MIT license, real-robot connections not yet released
The uni-robot-agent code is released on GitHub under the MIT license. Running the agent requires OpenAI's Codex or Anthropic's Claude Code. The first version works only with three simulators including RoboDojo, and connections to ARX One and Franka real robots are listed as "coming soon."
Qwen3.8-Flash-Next has a structure that uses 6 billion of its 125 billion total parameters per token, and its weights are released. News site TechCity Authority pointed out that the 19 real-robot tasks were measured on the research team's equipment and cannot be reproduced externally with the released code alone. All the figures are reported by the research team, and no third-party reproduction results have been published.
The research team wrote in the paper that precise manipulation and reliably completing long procedures remain challenges. On RoboDojo's precise manipulation category, Qwen scored only 10.21, below Astra's 28.75. Because of cost constraints, Astra was evaluated with only one random seed and layout.
