firmulate.com/benchmarks.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — We Buried a €55,000 Fact Two Documents Deep. Here's Which AIs Did Their Homework.
Live on firmulate.com.

Imagine an AI that doesn’t just answer questions but actually reads your company’s files, finds hidden details, and uses that information to make critical business decisions. In a recent live experiment, four top AI models were tested on a simulated company’s worst week, revealing a surprising truth: the difference between winning and losing a €55,000 deal often sat two references deep in internal files — not in the customer interactions themselves.

Age 18–24?Offer from Amazon

Prime made for students and young adults

  • Fast, free delivery for dorm and study essentials
  • Prime Video and Amazon Music included
  • Member-only deals
Try Prime for Young Adults Free trial for eligible 18–24 year olds
As an affiliate, we earn on qualifying purchases.

The Experiment: Testing AI’s Ability to Read Between the Lines

In an unprecedented live trial, four leading AI models, including GPT-5.6-sol and Kimi K3, were tasked with managing a small software company during its most turbulent week. The scenario was carefully crafted to mimic real-world crises, customer demands, and temptations to cut corners. Every decision made by the AI was recorded and auditable, ensuring transparency and rigor in the evaluation.

The goal? To see whether these models could navigate the complexities of the business environment, identify crucial hidden facts, and make honest, effective decisions under pressure. The results were revealing: all four models successfully identified every crisis and refused manipulation attempts, such as social engineering attacks designed to trick them into breaches of trust.

Infographic — We Buried a €55,000 Fact Two Documents Deep. Here's Which AIs Did Their Homework.
The findings at a glance — source: firmulate.com.

The Hidden Power of Deep Document Reading

The most striking finding was that the decisive advantage in closing the business deal depended on whether the AI read deeper into internal files — specifically, two references into a set of company documents — rather than just surface-level customer interactions. The models that read these buried references won the deal at full price, worth an additional €4,583 in monthly recurring revenue.

This demonstrates a critical insight: for AI to be truly effective in core business functions like sales, support, or decision-making, it must go beyond superficial data and delve into the internal knowledge base. The models that failed to uncover or act on these hidden facts left money on the table, illustrating a fundamental gap in current AI capabilities.

Understanding Trust and Integrity in AI Decision-Making

All four models maintained integrity by refusing manipulative attempts, like staged CEO messages or reporter tricks designed to push them into shortcuts or trust breaches. For example, when presented with staged fake approval requests, the models consistently identified them as suspicious and declined to act, aligning with best practices for AI governance.

In the real-world company simulation, this disciplined refusal meant that only those models that read the internal files thoroughly and followed proper analysis protocols managed to close the deal. The others either hesitated or left the opportunity unexploited, risking revenue loss.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI


Amazon

AI document reading software

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Amazon

enterprise AI knowledge base tools

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Amazon

AI business decision support systems

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Amazon

internal document analysis AI

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

HALLOWEEN

Halloween Picks

As an affiliate, we earn on qualifying purchases.

You May Also Like

Polarlicht

A significant geomagnetic storm has caused intense polar lights across northern Europe, attracting many viewers. Details on the event’s impact remain ongoing.

The Real Difference Between Steam, Vapor, and Humid Air

Lurking beneath the surface, understanding the true differences between steam, vapor, and humid air can transform your environment—continue reading to discover how.

We Know Simple Fluids Can Flow. Turns Out, Some Can Fracture

Recent studies reveal that certain fluids traditionally seen as simple can cause fractures, challenging previous assumptions about fluid behavior.

M 5.3 – 126 Km WSW Of Alianza Cristiana, Peru

A magnitude 5.3 earthquake occurred 126 km WSW of Alianza Cristiana, Peru, according to USGS. No immediate reports of damage or injuries.