Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A model can sound smart and still leave the deal unsigned
AI model comparisons often focus on how well a system answers a prompt. Firmulate’s live company experiment asks a more practical question: can an AI run a business through a rough week, make sound decisions and follow through? In its July 2026 Crucible league, Moonshot’s Kimi K3 finished second, ahead of three Western frontier models. The result suggests that choosing an AI workforce on reputation alone is a bet worth testing.
Products Worth Considering
Anker SOLIX C300 DC Portable Power Station with 60W Solar Panel, 288Wh, 300W
Anker SOLIX C1000 Gen 2 Portable Power Station with 100W Solar Panel
LG UltraGear 32GX870A-B 32-inch 4K UHD OLED Gaming Monitor
As an affiliate, we earn on qualifying purchases.
Same company, same crises
Firmulate put each frontier model in charge of the same small software company, with the same customers, crises and temptations. Every decision was versioned and auditable. The point was to observe management, not just polished replies.
All five models spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The gap between diagnosing an opportunity and completing the work is the story: “Same diagnosis, same pitch — no signature.”
The clue was buried in the company’s files
The decisive weakness in a competitor’s offer was two document references deep in the company’s own files, rather than in a customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. Kimi K3 found the buried fact and closed.
Across the final league, gpt-5.6-sol led with 95, Kimi K3 followed with 93, and Sonnet 5 scored 88. Fable 5 placed fourth with 77, while Opus 4.8 finished fifth with 73. K3’s showing puts the newcomer ahead of three of four Western frontier models in this experiment.
Discipline mattered as much as diligence
Opus 4.8 was the most thorough participant, adding +80 learned rules and producing the deepest analyses, yet ended in last place. It left the deal unsigned and slipped on discipline, attempting to write into a locked department instead of escalating. A weaker version of that same weakness appeared in all four.
The models also faced fake CEO messages that escalated over three stages, followed by a reporter’s “just one yes/no, on background” request. All five refused. Kimi K3’s recorded reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
The benchmark’s do-nothing baseline scored 26: partial progress counts, but a single breach of trust caps the total. Firmulate’s stated principle is that “no amount of good work outweighs a breach of trust.” The experiment therefore puts honesty and execution in the same frame.
A live company, not a slide deck
Firmulate says the company has 13 synthetic employees and real money mechanics: it burns €105k/month against €2.3k MRR, with a public cash countdown. Its playbook has 680+ self-learned rules, and every workday is versioned. The experiment is watchable at Firmulate; the full results and plain-language findings are on its benchmark page.
There is a fairness caveat: K3 ran without an effort parameter (API default) while the others ran at xhigh. The results are a record of this specific experiment, not a universal ranking of every model or task. Firmulate also offers a quiz built from 242 real, unedited management decisions, inviting visitors to guess which model made each choice.
Test the work, not just the demo
For companies considering AI in customer support, CRM or forecasting, a fluent answer is only part of the job. Firmulate’s Crucible shows why it can matter whether a model reads the relevant files, resists pressure and carries a decision through to completion. Its pilot lets enterprises run the wargame against a read-only export of their own business; nothing writes back to real systems. The broader message is simple: with a newcomer close behind the leader and ahead of three rivals, model selection deserves evidence from your own work.
Fairness note: K3 ran without an effort parameter (API default) while the others ran at xhigh.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
