🔍 Read the full analysis: Kimi K3.1 Pro’s Near-Frontier Score Raises Questions About Its Weights on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Artificial Analysis scored Moonshot AI’s Kimi K3.1 Pro at 55.1 on its Intelligence Index, placing it fourth among models listed on October 24, 2026, and close to leading closed models. The model is available through Moonshot’s services, but its weights are only promised by November 4 and no licence has been published. Moonshot’s headline claims about capabilities and a 65% hallucination reduction come from its own tests, while independent measurements differ.
Moonshot AI’s Kimi K3.1 Pro scored 55.1 on Artificial Analysis’s Intelligence Index, ranking fourth in the October 24 table and just below the leading proprietary models. The model is available through Moonshot’s apps and API, but its weights are not yet public: the company has promised to release them by November 4, without publishing a licence for the release.
Artificial Analysis lists K3.1 Pro as a mixture-of-experts model with 2.8 trillion total parameters and 104 billion active parameters, a one-million-token context window and text and image input. It went live on October 23 in the Kimi apps, Kimi Work, Kimi Code and the Kimi API. The listed API price is $2 per million input tokens and $10 per million output tokens.
In the independent evaluator’s October 24 comparison, K3.1 Pro’s 55.1 score put it fourth, behind Claude Opus 5.5 at 57.6, Claude Sonnet 5.5 at 56.0 and Muse Wonder Full at 55.8. Artificial Analysis estimated its cost at $2.11 per Intelligence Index task, compared with $5.98 for Opus 5.5 at its highest reasoning setting. These are evaluator-specific figures, not a guarantee that every user or workload will see the same cost or performance.
Artificial Analysis also measured a large gain in agentic coding tasks: K3.1 Pro scored 67% on Terminal-Bench 4.0, versus 13% for the earlier Kimi K3. On the knowledge-work benchmark GDPval-AA, it scored 1,846 Elo, behind Opus 5.5 at 1,866 and Muse Wonder Full at 1,857. Its score on Humanity’s Last Exam was 52%, compared with 47% for K3 and 61% for Opus 5.5. These results indicate uneven improvements across task types, rather than a lead on every benchmark.
Infographic 01 · Claimed vs. measured
Moonshot says “matches the strongest.” Independent tests say: close.
The KPIs at the top are independent measurements. Vendor numbers and independent numbers come from different harnesses, so they are shown side by side, never mixed.
Moonshot reports
vendor claims own harness, partly internal tests
| Benchmark | K3.1 Pro | Comparison |
|---|---|---|
| Terminal-Bench 4.0 | 65.7% | vs. Claude Opus 5.5: 66.4% |
| BenchCAD with tools | 92.7% | vs. Claude Opus 5.5: 96.2% |
| OfficeQA Pro v2 | 64.4 | vs. Claude Opus 5.5: 62.2 |
| Kimi Hallu Bench Hard internal | 8.3% | vs. Kimi K3: 23.4% (about −65%) |
Artificial Analysis measured
independent highest reasoning setting per model
| Measurement | K3.1 Pro | Comparison |
|---|---|---|
| Terminal-Bench 4.0 | 67% | vs. Claude Opus 5.5 (max): 60% |
| GDPval-AA | 1,846 Elo | vs. Claude Opus 5.5: 1,866 |
| Humanity’s Last Exam | 52% | vs. Claude Opus 5.5: 61% |
| AA-Omniscience hallucination rate | 44% | vs. Kimi K3: 53% · Gemini 4 Argon: 15% |
The 65% hallucination drop holds on Moonshot’s own stress test. On an independent knowledge test, the rate fell from 53% to 44%.Moonshot: the stress test rates “do not represent overall error rates in daily use.”
Sources: Moonshot AI, Kimi K3.1 Pro tech blog (23 Oct 2026) (left card, vendor charts); Artificial Analysis launch article (23 Oct 2026) and Artificial Analysis model page (data as of 24 Oct 2026) (KPIs and right card).
Powered by Thorsten Meyer AI
Why the Weights Matter
The score matters because open-weight models have generally trailed leading proprietary systems in the comparison cited by Artificial Analysis. The previous top open-weight entry in its table, Xiaomi’s MiMo-V2.6-Pro, scored 46.3; K3.1 Pro’s 55.1 would put it much closer to the top closed models if the promised weights are released. Artificial Analysis describes it conditionally as likely to lead open-weight models once its weights are published.
Public weights could give organisations more control over deployment, testing and data handling than an API-only service. That possibility is not yet the same as a practical self-hosted option. A model with 2.8 trillion total parameters presents substantial hardware and operating demands, and the licence will determine how people may use, modify or redistribute it. Until both the files and their terms are available, companies cannot judge the full deployment and legal implications from the benchmark score alone.
The cost comparison is also relevant to teams using paid model APIs. Artificial Analysis’s estimate puts K3.1 Pro below Opus 5.5 in cost per Intelligence Index task, but at roughly the same listed input and output token prices as Claude Sonnet 5.5. The task-cost estimate depends on the evaluator’s benchmark and settings; it should not be read as a universal price advantage for every application.
Products Worth Considering
Apple MacBook Air 15-inch Laptop with M5 Chip, 16GB Memory, 512GB SSD, Midnight
Apple MacBook Air 13-inch with M5 Chip, 16GB Memory, 512GB SSD, Silver
Bose QuietComfort Wireless Bluetooth Headphones – Active Over‑Ear Noise Cancelling with Mic, USB‑C Charging, Deep Bass, up to 24 h Playtime,
As an affiliate, we earn on qualifying purchases.
From Kimi K3 to K3.1 Pro
Moonshot describes K3.1 Pro as a more heavily post-trained version of Kimi K3. The earlier model scored 43.6 on the Artificial Analysis table, compared with 55.1 for K3.1 Pro. On that evaluator’s Terminal-Bench 4.0 run, the scores were 13% and 67%, respectively. The evaluator’s separate measurements also show a smaller rise on Humanity’s Last Exam, from 47% to 52%, making the scale of the reported improvement dependent on the task.
Moonshot’s launch materials describe work in game engines, parametric 3D modelling, finance and long-running tasks. The company says the model can build projects using Unity, Unreal Engine and Godot, and shows examples it says were created end to end. It also reports a 92.7% result on BenchCAD with tools, below the cited results for Claude Opus 5.5 at 96.2% and GPT-6 Astra at 95.9%. These examples and figures are vendor-reported; they are not independent verification of the demonstrations.
One prominent company claim is that K3.1 Pro cut claim-level hallucinations by about 65% on Moonshot’s internal Kimi Hallu Bench Hard, from 23.4% for K3 to 8.3% for K3.1 Pro. Moonshot says the test was deliberately designed to provoke errors and does not represent everyday error rates. In a separate Artificial Analysis evaluation, the rate on AA-Omniscience fell from 53% for K3 to 44% for K3.1 Pro. The two results use different tests and should not be treated as equivalent measurements.
“The test is deliberately crafted to provoke errors.”
— Moonshot AI
Licence and Claims Still Unsettled
The key open question is whether Moonshot will publish the weights by its stated November 4 target. As of October 24, they were not available, and the company had not published a K3.1 Pro licence. Until those details appear, the model cannot be described as an available open-weight release, and users cannot establish what permissions or restrictions would govern deployment.
It is also not clear how the model will perform outside the reported benchmarks, demos and Moonshot’s own tests. Artificial Analysis provides independent measurements for selected tasks, but those scores do not verify every claim about complete game projects, CAD workflows or finance work. The company’s 65% hallucination-reduction figure is specific to its internal stress test; Artificial Analysis reported a different, smaller change on its own measure. Neither number establishes a general error rate across routine use.
Self-hosting remains another practical unknown. The listed parameter count signals a model that could require major computing resources, but the supplied material does not give tested hardware requirements, deployment costs or performance for private installations. The available API pricing does not answer what running the eventual weights would cost.
The November 4 Release Test
The next milestone is Moonshot’s stated plan to publish K3.1 Pro’s weights by November 4. A release would allow users and researchers to inspect the files, test local deployment and check the licence terms. If the date passes without a release or an updated timetable, the model’s status as a prospective open-weight contender will remain unresolved.
Readers should watch for the licence, the actual weight files and independent evaluations after release. Those would clarify whether the benchmark performance can be reproduced outside Moonshot’s hosted services, what hardware is needed and whether the model’s practical costs match the API comparison. Until then, the strongest confirmed conclusion is narrower: K3.1 Pro has a high score on a third-party index, while its availability as a self-hosted model remains pending.
Key Questions
What did Kimi K3.1 Pro score?
Artificial Analysis scored it at 55.1 on its Intelligence Index in figures listed on October 24, 2026. It ranked fourth in that comparison, behind Claude Opus 5.5, Claude Sonnet 5.5 and Muse Wonder Full.
Can users download K3.1 Pro weights now?
No. The model is available through Moonshot’s apps and API, but its weights had not been published as of October 24. Moonshot has promised them by November 4.
Has Moonshot published a licence for the weights?
No K3.1 Pro licence was published in the supplied information. The licence terms will matter for people considering modifying, distributing or deploying the model.
Is the 65% hallucination reduction independently verified?
No. The 65% figure comes from Moonshot’s internal Kimi Hallu Bench Hard. Artificial Analysis measured a decrease from 53% to 44% on its separate AA-Omniscience evaluation, which is a different test and result.
Does the benchmark score mean K3.1 Pro is easy to run locally?
No. The benchmark does not establish local hardware needs or operating costs. Artificial Analysis lists 2.8 trillion total parameters, and the supplied material includes no tested self-hosting requirements.
Source: ThorstenMeyerAI.com
