Built on a U.S.-made open model
Cognition, the U.S. company behind AI programmer "Devin," announced a new AI model for software development, "SWE-2 Flash," on Nov. 4 (Nov. 5 Japan time). The company positions it as a fully American coding model and claims it outperforms all U.S. open-weight models (models with released weights) on leading evaluations. It is available from the same day in the development tool "Devin Desktop" and the command-line version. The weights will not be released publicly, and the company is building a program to share them securely only with partners and researchers.
Introducing SWE-2 Flash, our new fast & efficient coding model. On leading evals, it outperforms all American open-weight models, even ones 3x larger. To balance transparency and cyber risks, we are building a program to securely share the weights with partners & researchers.
The base is "Inkling-Small," whose weights Thinking Machines Lab released on July 30. It has 276 billion total parameters, with only 12 billion used in each processing step. Cognition said it added mid-training on the scale of several trillion tokens to this model and further strengthened its software development capabilities through reinforcement learning.
The higher-end "SWE-2," released on Sept. 10, is based on "Kimi K3," a 2.8-trillion-parameter model from China's Moonshot AI. With this model, the company switched to a U.S.-made base.
One exception to the "beats U.S. rivals" claim
The table in Cognition's blog lists eight evaluations, with three U.S. open models: Inkling-Small, Reflection AI's "Beam 1.0" and Meta's "Muse Spark Open." Among them, SWE-2 Flash scored 24.0% on FrontierCode 2.0, far above Muse Spark Open's 6.2%. FrontierCode grades whether real code changes are of a quality that would be accepted, and was created by Cognition itself.
In the same table, however, it scored 50.0 on SWE-Atlas QA, below Muse Spark Open's 54.0. On DeepSWE 1.1, it was 60.1 versus 60.0, a margin of only 0.1. The image attached to the X post is narrowed to six evaluations and does not include SWE-Atlas QA.
The company did not name the "3x larger" models, but the largest U.S. open model in the table is Muse Spark Open with 816 billion parameters, about three times the size of SWE-2 Flash. Thinking Machines' higher-end model "Inkling" (975 billion parameters) is not in the table. The conditions under which the evaluations were run could not be confirmed in the text of the announcement either.
Trails Chinese models on four of eight benchmarks
Widening the comparison to Chinese open models, the results are mixed. It lost to DeepSeek's "V4.1 Flash 1017" and Zhipu's "GLM 5.3 Flash" on four of eight benchmarks each. The gap is most striking on Terminal-Bench 4.0, which measures work in the terminal: SWE-2 Flash scored 20.0%, against 42.3% for DeepSeek and 41.8% for Zhipu.
Compared with models whose weights are not released, SWE-2 Flash outperformed OpenAI's "GPT-6 Luna" on six of eight benchmarks. It fell short of Anthropic's "Claude Haiku 5.5" on seven benchmarks, all except DeepSWE 1.1. The only benchmark on which it outperformed the higher-end SWE-2 was FrontierCode 2.0, at 24.0% versus 23.8%.
Results on all eight benchmarks
All are figures published by Cognition, and no results measured by a third party under the same conditions have been confirmed.
| Benchmark | SWE-2 Flash | Inkling-Small | Beam 1.0 | Muse Spark Open | GLM 5.3 Flash | DeepSeek V4.1 Flash 1017 | GPT-6 Luna | Claude Haiku 5.5 | SWE-2 |
|---|---|---|---|---|---|---|---|---|---|
| FrontierCode 2.0 | 24.0 | 3.2 | 2.9 | 6.2 | 14.2 | 19.3 | 21.9 | 27.7 | 23.8 |
| FrontierCode 1.1 Main | 45.5 | 21.5 | 22.8 | 24.5 | 31.8 | 43.9 | 42.4 | 46.4 | 50.0 |
| DeepSWE 1.1 | 60.1 | 17.4 | 43.3 | 60.0 | 63.4 | 77.0 | 66.6 | 59.3 | 73.0 |
| Terminal-Bench 4.0 | 20.0 | 0.9 | 4.0 | 8.0 | 41.8 | 42.3 | 16.4 | 39.2 | 27.3 |
| Terminal-Bench 2.1 | 89.2 | 64.7 | 80.1 | 76.4 | 84.3 | 91.0 | 82.0 | 94.4 | 92.8 |
| SWE-Atlas QA | 50.0 | 27.4 | 33.9 | 54.0 | 56.5 | 71.0 | 45.2 | 56.5 | 56.3 |
| SWE-Bench Pro v2 Hard | 89.9 | 56.5 | 76.5 | 82.4 | 88.2 | 86.3 | 80.4 | 90.2 | 90.6 |
| ExploitBench | 34.8 | 11.3 | 19.5 | — | 36.0 | 31.1 | 44.7 | 38.0 | 36.1 |
Three times faster at one-fifth the price is the company's claim
Cognition says SWE-2 Flash is three times as fast as SWE-2 at one-fifth the price. However, it gave no dollar amounts, and input and output prices per 1 million tokens could not be confirmed. Devin's pricing page also did not list SWE-2 Flash as of Nov. 5.
The basis for the speed claim is a comparison that simulated a replay of the run log for a single Terminal-Bench 4.0 task at settings of 200 tokens per second and 60 tokens per second. The company said the logs were matched to the same length. The 200 tokens per second is a setting for the simulated replay, and speeds measured in actual use have not been published.
Weights not released, limited to partners and researchers
To support defenders, Cognition trained the model specifically for cybersecurity work. On the cyber evaluation ExploitBench, it said the score rose from 11.3% for the base Inkling-Small to 34.8%. Saying the same capabilities could also be used by malicious attackers, the company decided against a public release of the weights.
The company said it will disclose details of the sharing program in the coming weeks, and the eligibility conditions are not yet known. It explained that companies and governments need environments in which they can run and monitor the model on infrastructure that includes their own facilities. The weights of the base Inkling-Small were already released by Thinking Machines in July.
Free through Nov. 30, offered within Devin
The official blog shows the model as free through Nov. 30. RuntimeWire reported that this applies to paid Devin users, but this could not be confirmed in the text of the official blog. Devin's pricing is $20 a month for the individual Pro plan and $200 a month for Max. Pricing from December has not been disclosed.
Cognition raised more than $2 billion on Sept. 8, at a valuation of $48 billion. On Sept. 25, it announced that its annualized revenue had exceeded $1 billion. In announcing SWE-2 as well, the company highlighted low cost alongside performance.
