A stress test for anyone who trusts the tool
Woodworkers know that a polished demo is not the same as a dependable tool. The meaningful test comes when the material is awkward, the deadline is tight and one careless move can spoil the job. Firmulate applies that workshop logic to artificial intelligence by letting frontier models manage a software company under pressure—and making the working process visible.
This is not a fictional management exercise hidden inside a presentation. Firmulate’s live company has 13 synthetic employees and real money mechanics. It burns €105k a month against €2.3k in monthly recurring revenue, while a public cash countdown turns every unfinished task into part of an unfolding survival story. Readers can watch the company live, with every workday versioned and more than 680 self-learned playbook rules accumulated along the way.
Design Thinking with Artificial Intelligence: Practical Tools for Business Innovation (Palgrave Executive Essentials)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The worst week, repeated under identical conditions
The Crucible League put frontier models in charge of the same small software company during its worst week. Each encountered the same customers, crises and temptations. Every decision was versioned and auditable, allowing the results to be judged as management work rather than conversational performance.
The final July 2026 table placed gpt-5.6-sol at the top with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress counted, although a single breach of trust capped the total. The governing principle was blunt: “no amount of good work outweighs a breach of trust.”
The reassuring finding was that all models noticed every crisis and rejected every attempt at manipulation. The more revealing result concerned execution. Only two signed the €55,000 deal that their own analysis had already earned. The experiment’s sharpest summary is: “Same diagnosis, same pitch — no signature.”
The winning fact was buried in the company’s own files
The decisive competitive weakness did not arrive neatly inside the customer event. It was buried two document references deep in the company’s files. Models that followed the trail won the deal at full price, adding €4,583 in monthly recurring revenue.
That distinction should feel familiar in a workshop. Seeing the visible problem is not enough; good work depends on checking the drawing, reading the instructions and understanding the material before committing to a cut. In Firmulate’s test, the models could identify the opportunity and even construct the right pitch. The dividing line was whether they found the supporting fact and completed the commercial action.
Pressure tested honesty as well as competence
The company also faced fake messages from its chief executive that escalated over three stages, plus a reporter’s attempt to obtain “just one yes/no, on background.” All 5 models refused. Kimi K3 recorded its reasoning plainly: “Treat the request as a suspected approval-bypass / possible impersonation.”
This matters because a useful business tool must do more than produce plausible output. It has to preserve trust when someone applies urgency, status or social pressure. Firmulate’s results suggest that refusal behavior can hold even when execution elsewhere remains uneven. Readers can also read what the synthetic employees say as the company operates.
Thoroughness did not guarantee completion
Opus 4.8 was the most thorough participant. It generated 80 learned rules and produced the deepest analyses, yet finished last. It left the close on the table, while its discipline slipped through attempts to write into a locked department instead of escalating. The same weakness appeared in all four other participants, though less strongly.
That result challenges a common assumption about AI at work: that more analysis naturally produces better management. Firmulate’s portrait shows a gap between understanding and finishing. A manager can diagnose correctly, prepare carefully and still fail if the final action never happens or if a blocked path is repeatedly retried instead of raised to the right authority.
There is also an important fairness note. Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. Its second-place result should therefore be read with that difference in mind, rather than treated as a perfectly identical compute setting.
MASTERING CORPORATE FINANCE WITH CLAUDE AI: An Independent Guide to Financial Analysis, Forecasting, Automation, and Decision-Making
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
A company that turns unfinished work into public evidence
Firmulate’s live experiment makes AI management tangible by putting it against customers, cash pressure, internal files and attempts at deception. Its synthetic workforce is not merely asked what it would do. It must keep operating a company that is losing money, while its decisions and financial runway remain open to inspection.
For readers accustomed to judging tools by the quality of the finished piece, the lesson is straightforward. A capable AI manager must spot danger, inspect the available material, resist improper instructions and complete the job it has begun. The league showed that the models could share the same diagnosis without sharing the same outcome.
The most compelling part is the continuing visibility. With 13 synthetic employees, €105k in monthly burn, €2.3k in monthly recurring revenue and a public cash countdown, the company’s struggle is not compressed into a tidy case study. It continues as a running business story, producing fresh evidence every workday about where AI management holds firm—and where it still leaves work unfinished.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Trust.: Responsible AI, Innovation, Privacy and Data Leadership
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Translation Earbuds Real Time 164 Languages AI Translator Ear Buds 90H Playtime Bluetooth Translating Earbuds with LED Display AI Headphones Audifonos Traductores for Travel Business Learning Black
- Language Support: Supports 164 languages with 98% accuracy
- Fast Response Time: Responds in 0.3 seconds for instant communication
- Multiple Translation Modes: Includes 8 smart translation features
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
