Why an AI pilot never reaches production: the four blockers that are not about the model

Eighty-eight per cent of large Brazilian companies already have some artificial intelligence initiative. Forty-eight per cent are still in pilot.

Both numbers come from the survey Antes de TI, a Estratégia 2026, run by IT Forum with 243 large companies, and together they describe a specific situation: half the market has proved the thing works and has not managed to put it into production. That is not scepticism. It is a stall.

The explanation heard at events is that maturity is missing, or data, or people. Our experience points somewhere less comfortable: what stalls a pilot almost never has to do with the model. It has to do with four boring subjects nobody wants to raise in a meeting about AI.

In one sentence — a pilot happens in an environment without consequence. Production is the same work with consequence — and it is the consequence, not the technology, that demands what is missing.

What changes between pilot and production

It is worth naming the difference before listing the blockers, because it explains all of them.

In a pilot, a person is watching. They pick the cases, check the output, fix whatever came out crooked and tell the story in the presentation. The system does not need to know what to do when it does not know — the person knows.

In production nobody is watching. A correct answer becomes a posting; a wrong answer becomes a posting too; and the discovery happens weeks later, in reconciliation. Everything the person did without noticing has to become design.

It is the same reason autonomy inherits the environment: granting autonomy removes the last human sensor from an environment that may have no other. Each of the four blockers below is a sensor that has to be built in its place.

the four blockers that keep an AI pilot from reaching production: which identity the process acts with, what happens when the same message arrives twice, where the case it cannot solve goes, and who explains the decision six months later
None of the four is about the model. All four stop the move to production.

Blocker 1 — which identity does it act with?

This is the question that ends more AI projects than any technical limitation, and it tends to show up late: when the process runs on its own, which user signs the posting?

In the pilot it runs with the account of whoever is demonstrating — usually broad access, because that is who built it. In production that does not hold: an automated process with a person’s permissions is an audit finding waiting to happen.

And then the conversation stops being about AI and becomes role design. Which activity does this process need? Display only, or change as well? Over which data, for which companies? Who answers for what it does?

It is known work, but it is other work — and it is where most pilots discover nobody budgeted time for it. Worth understanding first how catalog, space and activity divide a role, because all three layers show up here.

Blocker 2 — what happens when the same thing arrives twice?

A pilot processes a list. Production receives events, and events repeat: the timeout fired on the other side, the queue resent, somebody reprocessed a message that had already gone through.

A deterministic process repeated produces a duplicate, which is bad enough. A process with AI repeated can produce a different duplicate — because there is no guarantee the second pass decides like the first. That turns a known problem into one nobody knows how to investigate.

The fix is the same as in any serious integration — a key that identifies the operation and an “already processed” answer — and it has to be decided beforehand, not afterwards. The mechanism is detailed in order billed twice: why the same message arrives twice.

Blocker 3 — where does what it cannot solve go?

Every pilot has an accuracy rate and it is always good, because the cases were chosen. In production the rest arrives: the illegible document, the new supplier, the case that was not in the sample.

The question that decides whether it goes live is not “what is the accuracy rate”, it is “what happens to the ones it gets wrong”. There has to be an exception queue with an owner, a deadline and a way back — and somebody has to accept that this queue exists.

Projects that promised full automation tend to stall right here, because accepting the exception queue feels like accepting failure. It is not: it is the difference between an honest system and one that hides the problem until reconciliation.

Blocker 4 — who explains the decision six months from now?

An audit does not ask whether the model is good. It asks why that document was classified that way, in March.

If the answer depends on running it again and hoping for the same result, there is no answer. Production requires recording the decision together with what supported it: which input, which version, which rule, which alternative was discarded. That is logging, not AI — but it is an obligation for anyone putting an automated process on top of tax or accounting data.

The one-sentence test — before approving the move to production, ask: if this gets it wrong on a Thursday night, who finds out, how, and how fast? If the answer is “we would notice eventually”, the pilot is not yet production — regardless of the accuracy rate.

What goes wrong, by name

Choosing the case by technical convenience. The first pilot tends to be the one that was easiest to demonstrate, not the one that hurt most. It works, it impresses, and it creates no pressure to solve the four blockers — because nobody misses it in production.

Measuring the pilot by accuracy rate. That is the laboratory metric. The production metrics are cost per transaction, time to detect an error, and the size of the exception queue.

Leaving security for last. When the process identity comes up for discussion after the pilot is finished, the result is usually redoing the design — or going live with too much permission, which is worse.

Promising there will be no exceptions. It buys approval and loses credibility in the first month.

Treating bad data as an AI problem. Inconsistent master data breaks an automated process with or without a model. If the pilot shone, there is a good chance it ran on a clean sample that does not represent the base.

The honest limit

None of this is an argument against AI — it is the list of what separates a good experiment from a system. The four questions are the same ones asked of any automation; the difference is that with AI they are skipped more often, because the demonstration impresses and the conversation drifts to the model.

It is also honest to say that solving all four does not guarantee a return. An automated process built well on top of a bad business process delivers the same bad result, faster. The question that comes before all of them — should this process exist this way at all? — remains a human one.

And there is a scheduling consequence that changes planning: the work on the four blockers is usually larger than the pilot itself. Planning the move to production as “another two weeks after the proof of concept” is planning it wrong.

If the starting point is understanding the environment before automating anything inside it, begin with the IT diagnostic. To see AI applied to a real process, with the limit stated plainly, see read the XML and post inside SAP. And for the architecture overview, enterprise AI: the challenge is not the model, it is the architecture.

Frequently asked questions

Why do so many AI pilots never reach production?

Because a pilot happens in an environment without consequence — a person picks the cases, checks the output and fixes whatever comes out crooked. Production is the same work with consequence, and everything that person did without noticing has to become design.

What are the four blockers?

Identity (which user the process signs the posting with), repetition (what happens if the same message arrives twice), exception (where the case it cannot solve goes) and audit (who explains that decision six months later). None of them is about the model.

Is accuracy rate a good metric for deciding to go live?

No. It is a laboratory metric, and it always looks good because the pilot cases were chosen. The production metrics are cost per transaction, time to detect an error and the size of the exception queue.

How long does the work after the pilot take?

Usually longer than the pilot itself. Planning the move to production as “another two weeks after the proof of concept” is the most common planning mistake in this kind of project.