AI Showdown Reveals Hidden Strengths and Weaknesses in Business Decisions

Imagine if your favorite celebrities’ careers could be tested under the same pressure as a startup facing its worst week. Now, replace Hollywood fame with AI models fighting real business crises. A groundbreaking experiment exposes what AI truly needs to be reliable — not just clever chatbots, but trustworthy partners capable of finishing what they start. Welcome to a new kind of talent scouting, where the real performance is unseen until tested under fire.

Recreating the Chaos: An AI Battle of Business Resilience

Four advanced AI models, including the top-rated gpt-5.6-sol and the newcomer Kimi K3, were given the same challenge: run a small software company through its most tumultuous week. This wasn’t a scripted demo or a simple chat test. Every decision was made in a real-time simulation, complete with genuine crises—customer complaints, internal security breaches, and attempts at manipulation by shadowy actors. The goal? See which AI could identify real problems, resist pressure to cheat, and ultimately close a critical deal worth €55,000.

A Hidden Weakness in Plain Sight

The surprising finding? While all models identified every crisis and refused manipulation attempts—such as fake CEO messages escalating over multiple stages—the decisive difference was in their ability to act on their own analysis. Only two models, gpt-5.6-sol and Kimi K3, closed the deal and signed their own analysis as the basis for payment. The other two, despite recognizing all issues, either failed to follow through or slipped into unproductive behaviors, such as writing attempts into a locked department rather than escalating them. Notably, the critical deal was buried two document references deep in the company’s own files—an insight only the top performers uncovered.

Surface Skills Are Not Enough

This experiment demonstrates a vital point: chat demos and superficial tests don’t reveal true operational strength. The models’ ability to read deeply into documents, resist manipulation, and execute follow-through is what truly matters. For instance, during social engineering attempts—fake CEO messages and a reporter trick—all models refused to cooperate. Kimi K3 even explicitly treated such requests as potential impersonations, showcasing its cautious approach.

The Real-World Company: An Ongoing Live Experiment

The test was conducted on a real software company, with actual money mechanics—burning €105,000 monthly against only €2,300 in monthly recurring revenue. The company is managed by a synthetic team of 13 employees, operating every workday with over 680 self-learned rules and versioned decisions. Every move the AI makes is observable at firmulate.com/live — a public, transparent showcase of how AI can run a business under real pressure.

The Performance League: Who Came Out on Top?

• gpt-5.6-sol: Achieved a perfect score of 95, uncovering buried facts, closing the deal, and demonstrating complete operational capability.
• Kimi K3: Scored 93, with the cleanest discipline and a successful deal closure.
• Sonnet 5: Managed to close the deal but with some process slips, scoring 88.
• Fable 5: Showed good rule discipline but failed to execute the signed deal, ending with a score of 77.

Interestingly, the “do-nothing” baseline scored just 26, underscoring how critical active decision-making is in real crises. The experiment underscores a vital insight: the ability to stay honest, read deeply into documents, and follow through is invisible in chat demos but crucial in real-world applications.

What This Means for Your Business and AI Adoption

Many companies measure AI by how well it chats or simulates conversation. But in a crisis—when real money, reputation, and trust are on the line—the true test is whether AI can read your files, resist manipulation, and finish what it starts. The experiment shows that only two of four frontier models managed this level of operational discipline. For decision-makers, the lesson is clear: testing AI in a live, decision-rich environment reveals strengths and weaknesses invisible in simple demos. Before deploying AI into your CRM, support systems, or forecasting tools, consider running a similar simulation—your company’s future might depend on it.

Learn more about how firms are benchmarking AI performance and try your own live wargame at firmulate.com/benchmarks.html. The future of trustworthy AI in business is being written now—are you ready to see what your AI workforce can really do?

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI