Nobody Really Knows How You’re Using AI — And That Affects You
AI-generated content. Written by Yuniax's automated editorial system. It does not undergo article-by-article human review; editorial responsibility is ours. Spotted an error? Email hola@yuniax.ai.
Regulation (EU) 2024/1689 (AI Act), Art. 50(4).
The most widely used AI in the world — and nobody really knows how anyone uses it
There’s something striking about the AI ecosystem that most people accept without questioning: nearly all the data on how AI is actually being used in the world comes from the very companies selling that AI.
That’s exactly what MIT Technology Review laid out in its analysis published on August 18, 2026. The argument is straightforward: companies like Anthropic and OpenAI regularly publish reports on how products like Claude or ChatGPT are being used — but they only release the data they choose to share. As Anka Reuel, a doctoral researcher at Stanford’s Trustworthy AI Research group, put it: *»There is no independent source to corroborate it.»*
For anyone reading this from the front lines of business — with processes to optimize, teams to scale, and technology investment decisions on the table — this isn’t academic philosophy. It’s a concrete operational problem.
The publication bias nobody talks about
When a car manufacturer tells you their model is the safest on the market, you ask for third-party crash test data. When a software vendor presents their retention rate, you ask for an independent audit. With AI, however, we’ve somehow normalized accepting vendor-supplied data without that filter.
The usage reports published by major AI platforms carry an inevitable structural bias: they highlight what reinforces their commercial narrative. Successful use cases, the verticals where the model performs best, the most satisfied users. What you won’t find in those reports is the real dropout rate in critical workflows, the processes where automation fails silently, or the sectors where ROI hasn’t lived up to the promises.
This isn’t an accusation of bad faith. It’s simply business logic: you publish what helps you grow. But for you — the person deciding where and how to implement AI in your operations — that commercial logic can create costly blind spots.
What this means if you’re scaling with automation
Imagine you’re evaluating whether to automate an entire department — customer service, lead qualification, post-sales support — using AI agents. To make that decision well, you need to understand how that technology actually behaves in environments similar to yours. And that’s where the problem becomes tangible:
Most existing benchmarks are synthetic or lab-controlled. The evaluations published by AI labs are conducted under conditions designed to maximize model performance. They don’t reflect the noise, ambiguity, and irregularity of real business data built up over years of operations.
The success stories in circulation are curated. When a vendor presents you with a successful implementation case, you’re seeing the output of a selection process — they publish the ones that worked. The failures, and the mediocre results, don’t show up in any report.
There is no neutral reference source. Unlike other technology markets where independent analysts have access to aggregated, anonymized data, in generative AI the access to real usage data sits entirely in the hands of the providers.
What does that mean in practice? If you build your automation strategy exclusively on what vendors tell you, you’re building on sand.
The data point that matters most to you isn’t in any external report
There’s one genuinely useful conclusion to draw from all of this, and it may be the most valuable one: the only AI usage data that truly matters for your company is your own.
Not the average across ChatGPT users. Not the financial-sector success story in Anthropic’s report. Yours. How the model performs on your data, with your specific scenarios, inside your workflows, under your operational constraints.
That has a direct implication for how any serious implementation should be structured:
First, instrument before you scale. Before deploying an agent into full production, you need a real instrumentation phase where you measure — with your own systems — what happens when that agent operates inside your business. Resolution rate, cases escalated to humans, average response time, interpretation errors. Your business defines those metrics, not the vendor.
Second, define success on your own terms. An automation process isn’t successful because the model scores well on a lab benchmark. It’s successful if it reduces friction in your operations, frees your team to focus on higher-value work, and if the total cost of that automation makes sense compared to the alternative. Only you can measure that.
Third, be skeptical of external aggregate data for specific decisions. Aggregated usage reports can be useful for spotting broad trends, but they’re a poor foundation for concrete implementation decisions. The fact that «sector X mostly uses AI for Y» tells you nothing about whether that application makes sense within your own process architecture.
The transparency that’s missing — and how to compensate for it
What MIT Technology Review is pointing to has no easy short-term fix. Genuine independent transparency on real AI usage would require providers to hand over access to data they currently treat as a competitive asset. That’s not going to happen overnight.
But that doesn’t leave you without options. There are several things you can do today to navigate that opacity more confidently:
Demand contractual performance metrics. When you work with a vendor or implementation partner, the contract should include measurable KPIs tied to system performance in your specific environment. Not vague promises — metrics with precise definitions, compliance thresholds, and consequences if they aren’t met.
Build your own evaluation layer. If you have — or are building — significant automated workflows, you need your own logs, your own dashboards, and your own evaluation criteria. Total dependence on vendor dashboards to understand whether your automation is working is a blind spot you can eliminate.
Diversify your sources of insight. Vendor reports aren’t useless, but they need to be complemented by independent academic analysis, practitioner communities, and above all, the direct experience of other operations leaders implementing in similar contexts.
Pilot rigorously before committing. The best way to understand how AI actually performs in your business is to run a controlled pilot with a clear measurement methodology from day one. Not a pilot to «see what happens» — one designed to answer specific questions about operational viability and return on investment.
What this changes about how you read the market
There’s a side effect of this opacity worth naming: it distorts competition. If companies make adoption decisions based on partial data that systematically favors larger providers — those with bigger communications budgets and more capacity to publish success stories — the market doesn’t necessarily reward the technology that works best. It rewards the technology that’s told best.
For a company that genuinely wants to scale with automation, that means the competitive edge isn’t in copying what the reports say. It’s in building the internal capability to evaluate, measure, and adjust using your own judgment.
The MIT Technology Review piece is uncomfortable because it exposes a collective naivety: we’ve assumed that AI usage data was reasonably objective. It isn’t. It’s marketing data with technical methodology. Recognizing that isn’t a reason to freeze any automation strategy — but it is a reason to execute one with your eyes wide open.
The knowledge of how AI actually performs inside your operations isn’t going to come from any external report. You have to build it yourself.
What this means for your business
At Yuniax we build the system that automates the operations keeping you from growing: content, lead capture and back-office working together so your business scales without adding headcount.
Sources
Frequently asked questions
Why aren’t OpenAI’s or Anthropic’s AI usage reports enough?
Because they only publish the data they choose to share — there is no independent source to verify it, as Stanford researcher Anka Reuel notes in MIT Technology Review.
How does the opacity of AI usage data affect business decisions?
If benchmarks and usage metrics come exclusively from the vendor, your adoption or scaling decisions are based on partial information — which raises strategic risk significantly.
What can a company do today to reduce that risk?
Instrument its own workflows: measure the performance, cost, and outcome of each automated process internally, without relying solely on vendor dashboards.
Does this mean companies shouldn’t bet on AI in their business processes?
No — it means doing so with your own metrics and clear contracts, evaluating real results in your own operations rather than the success stories the vendor chooses to publish.
Turn your content into customers, without it depending on your time
At Yuniax we build the system that attracts, qualifies and nurtures your customers automatically: content, funnels and automation working together so your business grows without you being in every step.