🔍 Read the full analysis: How Close Open-Weight Models Are To IBM’s On-Prem AI Vision on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
IBM CEO Arvind Krishna says open-weight models can deliver about 90% of frontier-model performance at a fraction of the token cost, supporting a case for enterprise on-prem AI. An October 26 Artificial Analysis snapshot puts the best downloadable model at 80% of the leading proprietary model; a higher-scoring model’s weights have not yet shipped. Hardware utilization, staffing, licensing and security requirements all affect whether running models in-house saves money.
IBM CEO Arvind Krishna said on October 21 that open-weight models can cost as little as 10% of frontier models while being “90% as effective,” making them a potential fit for enterprises running AI on their own infrastructure. But an Artificial Analysis benchmark snapshot dated October 26 puts its top downloadable model at 46.3, or about 80% of the leading proprietary model’s 57.6; a model that would narrow the gap has not yet released its weights.
Krishna made the comparison during the Q&A on IBM’s third-quarter earnings call, describing open models as costing “25% to 10% of the token cost of the frontier models.” He said large enterprises could begin with two or three racks, and cited routine tasks such as summarizing emails, examining contracts and answering customer calls as work that may not require the most capable available model. These are his estimates and examples, not results from a named benchmark or a deployment study.
Artificial Analysis, which applies the same ten evaluations across models and identifies whether weights can be downloaded, showed MiMo-V2.6-Pro at 46.3 as the strongest downloadable model in its October 26 snapshot. Proprietary Claude Opus 5.5 scored 57.6. That puts the open model at about 80% of the leader on this index. The source’s historical comparison puts the best open model at 51% of the best proprietary model at the end of 2024 and 74% at the end of 2025. The percentages compare scores on that index; they do not establish equal performance on every enterprise task.
One possible exception is Kimi K3.1 Pro, scored at 55.1, about 96% of the proprietary leader. Moonshot had promised to release its weights by November 4, according to the source material. Until the weights are available, it does not qualify as a downloadable open-weight model for this comparison. The source also says no licence had been published and describes its 2.8-trillion-parameter scale as a substantial data-centre requirement.
IBM's 90% claim, checked
Open weights reach 80% of the leader
Krishna says open-weight models are "90% as effective". On an independent index the best model you can download today reaches 80% of the best closed one, up from about half two years ago.
Best open model as a share of the best closed
Artificial Analysis Intelligence Index, current version, best downloadable vs. best proprietary model at each date.
What Krishna said
Illustrations from the earnings call, not a named benchmark.
- Late 2024"70" vs. "20 or 30"Krishna, Q3 call, 21 Oct · no named benchmark
- Late 2025"70 to 50"Krishna, Q3 call, 21 Oct · no named benchmark
- Now"90% as effective"Krishna, Q3 call, 21 Oct · no named benchmark
Top downloadable models today
Intelligence Index score and licence type. Read the licence before the leaderboard.
| Model | Index | Licence |
|---|---|---|
| MiMo-V2.6-Pro Xiaomi | 46.3 | MIT |
| DeepSeek V4.1 Flash 1017 DeepSeek | 45.1 | MIT |
| GLM-5.3 Z AI | 44.8 | own commercial licence |
| Step 5 Preview StepFun | 43.7 | own commercial licence |
| Muse Spark Open Meta | 38.9 | licence with conditions |
In points the gap did not shrink: 7.5 points at the end of 2024, 11.3 today. Closed models moved too.
Sources: Artificial Analysis, Intelligence Index v4.3.2 (read 26 Oct 2026), IBM Q3 2026 earnings call transcript (21 Oct 2026), Artificial Analysis on Kimi K3.1 Pro (23 Oct 2026).
Powered by Thorsten Meyer AI
The Economics of Running Models In-House
IBM’s pitch matters to companies weighing recurring model charges against the capital and operating costs of private infrastructure. Owning or controlling the hardware may help organizations keep data and logs within their environment, an issue Krishna highlighted for banks and traders. Open weights can provide the permission to run a model locally or in a private cloud, but the source notes that licences vary. Downloadable weights do not automatically mean unrestricted commercial use.
The source’s price examples show why utilization matters. An eight-GPU H200 server is listed from $172,643, while renting the same GPU class on AWS is quoted at $63.30 an hour. A simple comparison in the source estimates that purchase cost would be matched after about 114 days at full use, but about 15 months at 25% utilization. Those estimates exclude staff, cooling and networking, and do not account for every difference between purchased and rented services. For an enterprise with uneven demand, idle capacity can change the calculation substantially.
Krishna also acknowledged a deployment constraint: organizations need people who can work on kernels and underlying networks. He said that skills gap remains for all but the most sophisticated enterprises. That qualification complicates any claim that lower token prices alone make on-prem AI cheaper or easier. IBM has a commercial interest in the shift: it sells on-prem infrastructure and expects demand to support its Spyre cards. Its earnings-call comments should be read as an interested company’s assessment, not as an independent cost audit.
Products Worth Considering
Amazon Kindle Scribe Colorsoft 64GB (1st Generation)
Apple iPad Air 13-inch (M4), 128GB, Wi-Fi 7, Space Gray
Amazon Kindle Paperwhite Signature Edition (12th Generation, 2024), 32 GB, Metallic Black
As an affiliate, we earn on qualifying purchases.
From Model Scores to Rack Costs
Krishna described the performance gap as narrowing, recalling broad comparisons in which an open model scored 20 or 30 against a proprietary model at 70 in late 2024, then 50 against 70 in late 2025. Those figures were illustrations in his remarks, not scores tied to a named evaluation. The Artificial Analysis series offers a separate comparison: its source data show open models moving from 51% of the leading proprietary score at the end of 2024 to 80% on October 26, 2026. The current absolute score gap is 11.3 points, compared with 7.5 points in the source’s end-of-2024 figures; relative shares and raw point gaps tell different stories.
Cost comparisons also depend on how a model is accessed. The source reports that hosted DeepSeek V4.1 Flash 1017, whose weights it says are MIT-licensed, is priced at 3% to 6% of Claude Opus 5.5’s list price. But it also cites closed GPT-6 Luna at $0.10 per million input tokens and $0.50 per million output tokens, illustrating that proprietary services can be inexpensive for some workloads. IBM separately claimed its llm-d project can run large models at five to ten times lower cost; the source says it could not independently check that figure.
The practical question is not simply open versus closed. It is whether a particular model performs well enough for a task, whether its licence permits the intended use, and whether an organization can run it efficiently. The source also describes an unrelated enthusiast development: an open-source Linux module called gpu-unlock reportedly raises a tensor-core speed limit on some GeForce cards. Its reported gains vary by workload, and Nvidia had not commented. That development is separate from IBM’s enterprise case and does not establish that consumer graphics cards are suitable replacements for data-centre systems.
The Gaps Behind the Benchmark
The benchmark comparison is a snapshot, not a guarantee of performance across real enterprise workloads. Artificial Analysis provides a common index, but the source does not establish how its scores map to a given company’s accuracy, latency, privacy or reliability requirements. Krishna’s 90% figure was not tied to that index, so the two measures should not be treated as a direct confirmation or refutation of one another.
Kimi K3.1 Pro’s weights and licence were still pending in the supplied material, and its promised November 4 release had not occurred as of the October 26 benchmark reading. Model prices can also change, and the cited server-versus-cloud comparison leaves out operational costs and differences in service. The source does not provide an independently verified total-cost analysis of IBM’s proposed deployments or its llm-d cost claim.
The November Weights Test
The next near-term test is whether Moonshot releases Kimi K3.1 Pro’s weights by its stated November 4 date and publishes a licence that permits the intended uses. If it does, the model’s reported 55.1 score would make it a much closer downloadable competitor to Claude Opus 5.5 on the cited index; until then, it remains a promised release rather than an available open-weight option.
For enterprise buyers, the more practical next step is to evaluate candidate models on their own tasks and compare full deployment costs at realistic utilization. That includes hardware, staffing, networking, cooling, licence terms and data-handling requirements. IBM’s subsequent sales and deployment disclosures could help show how much demand its on-prem AI pitch attracts, but the source material does not identify a scheduled company milestone that would settle the performance or cost questions.
Key Questions
Are open-weight models already as capable as the leading proprietary models?
Not according to the October 26 Artificial Analysis snapshot cited here. Its best downloadable model, MiMo-V2.6-Pro, scored 46.3 against Claude Opus 5.5’s 57.6, or about 80% of the leader on that index. The result does not predict performance on every task.
What did IBM’s CEO mean by “90% as effective”?
Krishna used the phrase to argue that a less expensive open-weight model may be sufficient for routine enterprise tasks. The source says he did not tie the 90% figure to a named benchmark, so it is best treated as his broad comparison, not a standardized measurement.
Would running a model on company hardware necessarily cost less?
No. The economics depend on how heavily the hardware is used and include costs beyond the purchase price, such as staff, cooling and networking. The supplied estimate says an H200 server’s purchase cost matches renting the same GPU class after about 114 days at full load or about 15 months at 25% load, before those additional costs.
Is Kimi K3.1 Pro an open-weight model now?
Not in the October 26 snapshot described in the source. Moonshot had promised its weights by November 4, but they had not shipped by the benchmark reading. The source also says a licence had not been published.
Source: ThorstenMeyerAI.com
