Imagine if your business tools could read deeper into your company files — not just surface-level data, but the hidden details that make or break deals. In a recent live experiment, AI models were tested in a simulated crisis week with real money mechanics and customer dilemmas. The surprising part? The models that looked two steps deeper into internal files consistently outperformed their peers, closing deals that others simply missed.
Listen free for 30 days with Audible
Thousands of audiobooks and originals — cancel anytime.
As an affiliate, we earn on qualifying purchases.
The Power of Reading Between the Lines
In today’s fast-paced corporate environment, AI tools are often judged by how well they generate responses or handle customer inquiries. But a groundbreaking live experiment by Firmulate reveals a more critical skill: reading and understanding your internal files before responding. This focus on multi-hop reasoning — that is, the ability to connect information buried deep in internal documents — proved decisive in a simulated week of crises for a small software company.
The Experiment Setup
The experiment involved four frontier AI models, each running the same scenario: a small software company facing its worst week. They encountered identical crises, temptations to cheat, and customer demands. Every decision was recorded and auditable, ensuring transparency in performance. The models were tasked with diagnosing issues, making strategic decisions, and ultimately closing a significant €55,000 deal.
Key Findings: Deep Reading Wins
All four models successfully identified every crisis and refused manipulation attempts, such as fake CEO messages and reporter tricks. However, only two models managed to close the deal — and crucially, those were the ones that read past superficial data into the company’s internal files. The decisive weakness of competitors was buried two document references deep inside their own files, not in the obvious customer interactions.
The models that uncovered the hidden fact and acted on it secured the full €4,583 monthly recurring revenue (MRR) deal. Conversely, those that missed the buried information left the opportunity on the table, despite their accurate diagnoses and pitches.
As an affiliate, we earn on qualifying purchases.
The Human and AI Lessons
This experiment underscores a vital lesson for businesses considering AI integration: it’s not enough for an AI to produce convincing responses. The AI must read and reason across internal documents before making decisions — a property that is increasingly measurable and directly impacts business outcomes.
Social Engineering Tests
When subjected to social engineering — fake CEO messages escalating over three stages plus a reporter trick — all five models refused to manipulate or approve suspicious requests. Kimi K3 highlighted the importance of treating such requests as potential impersonation or approval bypass attempts, demonstrating a cautious and ethical decision-making process.
The Live Company Simulation
Behind the scenes, the live company simulation involved 13 synthetic employees operating with real money mechanics — burning €105,000 monthly against a mere €2,300 MRR, with a public cash countdown and over 680 self-learned playbook rules. Every decision and process was versioned daily, providing a transparent window into AI behavior under pressure.
As an affiliate, we earn on qualifying purchases.
The Implications for Business Decisions
While the models performed similarly in diagnosis and resisting manipulation, the difference in deal closure highlights a pivotal factor: thorough internal reading. As firms look to deploy AI in customer support, CRM, or forecasting, the question is no longer just about language fluency. It’s about whether the AI can finish what it starts, read your files deeply, and stay honest under pressure.
What Does This Mean for Your Business?
In a world where AI can make or break critical deals, understanding its capabilities beyond chat is essential. The experiment shows that the most effective AI agents are those that can locate and interpret buried information — a skill that directly correlates with business success.
multi-hop reasoning AI solutions
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Benchmark Performance Summary
- gpt-5.6-sol scored 95: found the buried fact and closed the deal — the full performance.
- Kimi K3 scored 93: also closed the deal, with the cleanest discipline of the field.
- Sonnet 88: closed the deal, but with some process slips.
- Sonnet 77: closed the deal, but with more process slips and discipline lapses.
- Baseline: 26 — partial progress, with breaches of trust capping performance.
This experiment is a clear signal: AI’s ability to read and reason across internal documents isn’t just a technical curiosity — it’s a business-critical skill that can be measured and optimized.
As an affiliate, we earn on qualifying purchases.
Watch It Live and Try It Yourself
Interested companies can run their own AI wargame against a read-only export of their business data, testing how well AI agents perform under realistic pressures. This approach ensures no real systems are affected, allowing a safe, practical evaluation of AI readiness for your own critical decisions.
Learn more and watch live demos at firmulate.com and see how AI models are tested, verified, and improved for real-world enterprise performance.
The future of AI in business isn’t just about chat quality. It’s about whether AI can read your files deeply, reason across complex data, and stay honest under pressure. Those skills are measurable and could determine your next big deal.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Labor Day sales Picks
labor day deals
As an affiliate, we earn on qualifying purchases.
