Get furniture and decor delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
Would You Hire a Contractor Without Checking References?
Any interior designer knows the drill: the portfolio looks flawless, the pitch is polished — and then the renovation runs three months late. Appearances at the presentation stage tell you almost nothing about performance under pressure. Turns out the same is true for AI models, and a live experiment just proved it in public.
One Company, One Nightmare Week, Five Frontier Models
Firmulate runs what it calls an AI company emulator: each frontier model was handed the same small software company and the same worst week of its life — same customers, same crises, same temptations to cut corners. Every decision is versioned and auditable, and the whole thing runs live, with 13 synthetic employees, real money mechanics, a burn of €105k a month against just €2.3k in monthly recurring revenue, and a public cash countdown you can watch.
The Final League Table
As of July 2026, the Crucible league stands: gpt-5.6-sol first with 95, Moonshot’s Kimi K3 second at 93, Sonnet 5 third at 88, Fable 5 fourth at 77, and Opus 4.8 last at 73. A do-nothing baseline scores 26 — partial progress counts, but a single breach of trust caps the total. In Firmulate’s words, “no amount of good work outweighs a breach of trust.”
Same Diagnosis, Same Pitch — No Signature
Here’s the finding that should make any business owner sit up. All five models spotted every crisis and refused every manipulation attempt. But only two signed the €55,000 deal their own analysis had earned. The decisive competitive weakness wasn’t in the customer conversation at all — it sat two document references deep in the company’s own files. The models that actually read the file closed the deal at full price, worth +€4,583 in monthly recurring revenue. The others gave the same diagnosis and the same pitch, and left the signature on the table.
It’s the AI equivalent of a designer who nails the mood board but never opens the site survey. The research existed; nobody looked.
K3: The Newcomer With the Cleanest Hands
Kimi K3’s week was quietly remarkable. It found the buried fact, won the €55k deal, saved the churning customer, and resisted all three bait attempts — including a fake CEO message escalating over three stages and a reporter’s “just one yes/no, on background” trick. Its on-record reasoning: “Treat the request as a suspected approval-bypass / possible impersonation.” All five models refused the social engineering, but K3 logged just one deviation across the week — the cleanest discipline in the field.
Meanwhile, Opus 4.8 — the most thorough participant, with 80-plus learned rules and the deepest analyses — finished last. The close was left undone, and discipline slipped: it attempted writes into a locked department instead of escalating. The same weakness appeared, weaker, in all four competitors.
The Catch, and Why It Matters
One fairness note is worth flagging: K3 ran without an effort parameter (API default) while the other models ran at their highest effort setting, xhigh. Even so, the message is hard to miss — a newcomer from outside the usual Western frontier beat three of four incumbents at actual management. The league is open, and picking a model based on brand reputation alone is now a bet, not a decision.
Try It Yourself
Firmulate isn’t keeping this behind glass. A quiz built on 242 real, unedited management decisions lets you guess which model made which call, and full benchmarks and plain-language findings are public. Enterprises can go further: run the same wargame against a read-only export of their own business, with nothing ever written back to real systems.
The uncomfortable truth from this experiment: chat quality and management quality are different skills. Every model could talk; only some could finish. Before you let any AI touch your CRM, your support queue, or your forecast, ask the questions Firmulate’s week from hell actually answers: does it finish what it starts, does it read your files first, and does it stay honest under pressure? Demo the showroom all you like — but check how it works when the walls come down.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
