Behind the Curtain of AI Decision-Making: Lessons from a Live Business Experiment

Imagine hiring an AI to manage your client relationships or oversee your project deadlines. Would it deliver consistent results, or would it falter under pressure? Recent live experiments with AI models reveal surprising insights that matter deeply to anyone invested in automation and decision-making—especially in the creative and design industries.

The Live AI Business Lab: A Unique Test Bed

At firmulate.com, a groundbreaking experiment runs four advanced AI models through a simulated week of managing a small software company. This isn’t just theory; it’s a real-time, transparent test where each decision is logged, auditable, and reflective of actual business challenges. The goal? To see how these models handle crises, manipulations, and the complex web of decisions in a high-stakes environment.

Key Findings: Diligence Does Not Guarantee Success

One might assume that the most thorough AI—one that learned over 80 rules and performed in-depth analyses—would outperform others. That model was Opus 4.8, the most diligent participant. Despite its deep analysis, it finished last in the league, scoring only 73 out of 100—well below the top models. The reason? It left opportunities unexploited, such as failing to escalate critical issues, or slipping discipline when the pressure mounted. In contrast, other models that prioritized decisive action managed to close deals more effectively.

Understanding the Scores

• gpt-5.6-sol scored 95 and closed the deal by uncovering a buried document reference that proved decisive, securing an €55,000 contract.
• Kimi K3 achieved a score of 93, demonstrating the cleanest discipline and closing the same deal.
• Sonnet 5 scored 88, also closing but with slightly more process slips.
• Fable 5, with 77, managed to close as well, but with weaker discipline.

Interestingly, all models detected every crisis and refused manipulation attempts, which included fake CEO messages and reporter tricks. Yet, only the models that read critical documents in the company’s files made the winning move—finding the hidden fact that sealed the deal.

Implications for the Creative Industry

For interior designers, furniture retailers, or creative agencies considering AI integration, these results highlight a vital point: volume of work or thoroughness doesn’t automatically translate into effective outcomes. It’s about prioritization, discipline, and knowing what to read and act upon. A model that misses the key document or slips on escalation can lose opportunities, no matter how diligent it appears.

Trust and Ethics Under Pressure

Another crucial aspect revealed is the AI’s response to social engineering. All models refused staged manipulation attempts, with Kimi K3 explicitly treating suspicious requests as potential impersonation. This underscores that trustworthiness under pressure is achievable, but only if the AI is designed with safeguards in mind.

Why This Matters for Your Business

Whether managing client projects, supplier negotiations, or internal workflows, the question is no longer whether AI can produce good writing or chat. The real challenge lies in whether it can finish what it starts, read the relevant information first, stay honest under stress, and ultimately deliver actionable work. The live experiment shows that the best AI isn’t necessarily the most thorough—it’s the one that knows what to focus on and when to escalate.

Try It Yourself

Have a business decision or creative project in mind? You can run a similar wargame against your own data in a safe, read-only environment. This allows you to test how your AI workforce handles crises and manipulative tactics before deploying it in real operations. Find out more at firmulate.com/pilot.html.

In AI-driven decision-making, discipline and prioritization outperform sheer thoroughness. Real-world success hinges on knowing what matters most—and reading the hidden facts before acting.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI