Order 88214 was billed twice. Same quantity, same customer, same date — two documents. Nobody keyed it in twice, nobody clicked the wrong screen. The carrier integration answered twice, and the system believed both answers.

This is the most tiresome defect in a distribution operation, because it breaks nothing. The process moves on, the truck leaves, the invoice is issued. The error shows up later — in reconciliation, in negative stock, or in the phone call from a customer asking why they were charged twice.

Why the message arrives twice

This is not an exotic defect. It is the normal behaviour of any transport that cannot guarantee exactly-once delivery.

The timeout that is not an ending. The carrier sends the confirmation, your API is slow to answer, their timeout fires. They resend. But the first one arrived and was processed — only the reply was lost on the way back.

The automatic retry. Every serious queue retries on failure. If the failure is in the reply and not in the processing, the retry becomes a duplicate.

The manual reprocess. Somebody sees the message in error, fixes it and sends it again — not knowing the original had already gone through.

The two channels. The same information arriving by file and by API, each on its own schedule, because the migration from one to the other was never finished.

The fix has a name, and it is simpler than it sounds

It is called an idempotency key: an identifier the sender generates and repeats on every resend of the same operation. The receiver stores the key, and when it comes back, answers “already processed” instead of processing again.

The hard part is not the concept. It is agreeing with the partner on which field is the key — and that conversation almost never happens.

One check that usually pays off the next day, straight against the database:

-- same delivery, same partner, different messages
SELECT a.docnum, b.docnum, a.mestyp, a.rcvprn,
       a.credat, a.cretim, b.credat, b.cretim
  FROM edidc AS a
  JOIN edidc AS b ON  b.mestyp = a.mestyp
                  AND b.rcvprn = a.rcvprn
                  AND b.docnum > a.docnum
 WHERE a.status = '53'
   AND b.status = '53'
   AND a.credat >= '20260801'
 ORDER BY a.credat, a.cretim;

The result is almost never empty, and a good share of it is legitimate — the same message type for the same partner happens all the time. What matters are the pairs seconds or minutes apart, because a retry duplicate has a time signature: it arrives glued to its twin.

What goes wrong, by name

Deduplicating on the wrong field. Using the order number as the key fails when the same order legitimately has two partial deliveries. The key has to identify the operation, not the business document.

Deduplicating only over a short window. Keeping the keys from the last two hours solves the immediate retry and does nothing for the manual reprocess, which happens the next day. The window has to cover human behaviour, not just the queue.

Treating it as an error instead of a state. A repeated message is not a failure — it is expected. If it lands in the error queue, somebody will reprocess it and create the third copy.

Finding out from the customer complaint. The most expensive path and the most common one. The duplicate has been sitting there for weeks, and the first person to notice is outside the company.

Where AI genuinely helps here

Deduplication itself is a deterministic rule and must stay that way — nobody wants a model deciding whether two orders are the same. But three expensive parts of the work are reading and comparing.

Finding the suspect pairs in history. Sweeping months of messages and proposing which pairs carry the signature of a duplicate is comparison at volume, against observable criteria.

Reading the partner layout and pointing at the key candidate. Integration documentation is long and the part that matters is two fields. It is the same reasoning we apply to watching the banking cycle before automating it.

Explaining why that specific message failed. Cross-referencing the error with the partner history shortens the investigation.

Where it does not belong: cancelling a document, reversing an invoice and deciding what to do with a load that has already left. Those three stay with people.

The honest limit

Idempotency done well eliminates the technical duplicate. It does not eliminate the business duplicate — the customer who really did order twice, the rep who entered the same order in two channels. Those cases are identical in the eyes of the system and different in the eyes of the operation, and no automatic rule settles them alone.

And there is a boundary limit: if the partner does not generate a stable key, you have nothing to store. You can work around it with a content fingerprint, but that is an approximation — two legitimately identical deliveries do exist, and the approximation will block one of them sooner or later.

What changes is where the problem is found. With the cycle observed, the duplicate shows up in minutes, on a dashboard, before it becomes an invoice. Without it, it shows up in the monthly reconciliation — or on the phone.

If the starting point is understanding what exists today before touching anything, begin with the IT diagnostic. For the sector overview, see IT for services and distribution. And if the volume of inbound documents is the immediate pain, the path is reading the XML and posting inside SAP.

Frequently asked questions