Agentic AI in the supply chain: why 95 % accuracy is a risk, not a benchmark

The hire you would never make

Picture a new employee. Brilliant, works around the clock, optimises complex freight routes in seconds. But in five cases out of a hundred, he books hazardous goods through a tunnel where that is forbidden, or promises a customer stock that does not exist.

Would that person get write access to the ERP system on day one? Of course not.

That is precisely what is happening under the label of agentic AI. Benchmarks of “95 % accuracy” are celebrated, and a basic industrial truth gets lost along the way: in a supply chain, 95 % is not a good grade. It is a disaster.

One part in twenty scrapped? The line would stop.

The misunderstanding: deterministic against probabilistic

To use AI safely in an industrial setting, leaders need one distinction. It is not technical. It is logical.

1. The old world, ERP and EDI, is deterministic. Input A produces output B. Always. An error is a bug, and the system usually stops. Fail-safe.
2. The new world, AI agents, is probabilistic. The model computes likelihoods. It guesses at a very high level. When it is wrong, it does not stop. It produces a plausible, incorrect answer and carries on. Fail-silent.

We are deploying systems with the power of a sports car and the judgement of a toddler. What they need is adult supervision, in the literal sense: a duty of oversight for autonomous systems.

What autonomy actually costs

The discussion usually circles API pricing and licences. The real cost sits in what an unguarded agent can do.

Two cases, costed out:

• The small logistics error. An agent misreads “urgent” in an email and books express freight for C-parts. Around 3,300 euros, freight surcharge plus manual reversal.
• The reputational one. A support agent confirms a delivery to a key account to be helpful, with no stock behind it. Over 12,000 euros, contractual penalty plus the part you cannot invoice: trust.

The danger is not that AI never works. The danger is that it mostly works, and that we therefore stop looking.

The engineering answer: FMEA for probabilistic systems

The method already exists, on the factory floor rather than in IT. Manufacturing has used FMEA, failure mode and effects analysis, for decades.

The classic risk priority number:

RPN = S × O × D

• O, occurrence: how often does the failure happen? One to ten.
• D, detection: how likely are we to catch it before it ships? One to ten, where one means certainly caught and ten means never.

Why the classic formula breaks on AI

The problem sits in D.

In a manual process, a clerk checks the order, and detection is high. An autonomous agent acts in milliseconds. Once the error has happened, once the wrong email has gone out and the order is posted, it is too late.

With agentic AI, detection tends towards zero and D goes to ten.

The extended formula

The missing factor is leverage, the degree to which autonomy multiplies a single mistake:

Risk = (O × L) × (S × D)

• O, occurrence in the model: how often does it hallucinate? One for a large model with grounding, ten for a small one without context.
• L, leverage through autonomy: what can the agent reach? One for a read-only chatbot, ten for write access to the ERP or the authority to release a payment.
• S, severity: what does the error cost, financially or legally? One for internal irritation, ten for a stopped line or a regulatory breach.
• D, detection by the system: how effective are the technical guardrails, not the people? One where a deterministic rule blocks the error reliably, ten where nothing checks automatically.

The chain of agents

Modern systems are rarely one agent. They are a chain of n agents in a workflow.

Three agents, for example: one reads the demand, one calculates the quantity, one places the order.

This is where error propagation begins. If the first agent hallucinates, the second accepts that fiction as fact. The third acts on it.

Total risk = Risk₁ × Risk₂ × … × Riskₙ

The risk does not add up. It multiplies, because every downstream agent legitimises the decision of the one before it. Three agents without guardrails do not carry three times the risk. They carry a failure that travels three steps deeper into the process before anyone notices.

Defence in depth: five layers

Lowering that number takes more than prompt engineering. It takes architecture.

1. Deterministic guardrails, in hard code. Before the model thinks, rigid rules define the boundaries. No purchase order above a threshold without release. Here the old world beats the new one.
2. Synthetic validation. A second, specialised critic agent reviews the work of the first. One drafts the message, the other checks it against compliance rules.
3. Human in the loop, for the last mile. On critical decisions the agent only prepares. Draft mode. A person presses the button.
4. Process isolation. Agents never write into the live system. They write into a staging area, and the data is posted only after validation.
5. The stop button. Logic that takes the agent off the network the moment its error rate climbs.

Leadership, not prohibition

Keeping AI out of the supply chain is not an option. The advantage in speed and analysis is too large to forgo.

But the treatment has to change. Not magic. A junior consultant:

1. No mandate for million-euro decisions on day one.
2. Results reviewed, with a method that already exists.
3. Boundaries defined before the first run, not after the first incident.

That is what AI-first leadership means in practice. Establish that supervision and the horsepower reaches the road. Skip it and the car ends up in the ditch, at speed.

---

#AgenticAI #AI #SupplyChain