Every conversion to S/4HANA reaches a moment when the project stops and turns to look at master data. Not because anyone planned it — but because the business partner conversion rejects records with an invalid tax ID, the material load fails on a classification that does not exist, and the error list arrives with thousands of lines carrying messages that only make sense to whoever wrote the program.

At that point the project usually takes the wrong decision: fix during the load. It works in the first cycle, creates permanent debt and leaves the source exactly as dirty as it was. Master data cleansing is not a migration stage — it is work that has to happen before, and that migration merely exposes.

What “dirty” means in practice

Bad master data is almost never missing data. It is data that is plausible and wrong, data that passed every validation in the old system because the old system did not validate.

• Duplication. The same supplier registered three times, spelled differently, with similar addresses and sometimes different tax IDs because someone mistyped one once.
• Mandatory in the target, optional in the source. S/4 demands what ECC tolerated empty — and the list of new demands is long.
• Inconsistent format. Tax IDs with and without punctuation, addresses with the number in the street field, telephone numbers in three decades’ worth of formats.
• Wrong or missing classification. A material with no material group, an out-of-date tariff code, a unit of measure with no conversion set up.
• Dead record. A customer with no activity for eight years, a discontinued material, a plant that no longer exists — all active, all being migrated.
• Broken relationship. The record points to a condition, a bank or a plant that has already been deleted.

In an ECC-to-S/4 conversion in a RISE environment that we supported, using a Bluefield approach — selective, where you choose what crosses over —, that list was precisely what defined the scope. Bluefield only delivers on its promise if someone decides what does not go; and to decide, you first have to see.

See before you fix

The first deliverable is not a correction: it is a quality indicator per object. How many customers, suppliers and materials exist; how many are active; how many have had activity in the past twenty-four months; how many fail each rule.

This dashboard changes the project conversation, because it turns “the data is bad” into a number. And it almost always reveals the most useful finding of all: most of the master data is not used. A record with no activity does not need to be corrected — it needs to be blocked and left behind.

It is the same scope logic we described in data conversion with AI: the question that saves the most is what does not go.

The design: n8n orchestrates, the ERP stays the owner

We run n8n on the client’s own infrastructure as the conductor of the cycle: it extracts the master data, applies the rules, assembles the context, calls the model for whatever demands judgement, presents the proposal and returns the correction to the ERP through the standard path. The master data never leaves home and the correction comes in through the front door, with an audit trail.

The rules are code. A valid tax ID, a mandatory field filled in, a normalised format, an existing relationship — all of that is deterministic checking, it has to give the same answer every time and it should not pass through any model.

AI comes in at three points, all of them where the exact rule is weak.

Deduplication by similarity. “Comercial São Jorge Ltda”, “Com. Sao Jorge LTDA” and “SÃO JORGE COMERCIAL” are the same supplier to any person and three different keys to an exact comparison. The model groups them by similarity of company name, address and tax ID, assigns a score and proposes the group — with the record that should survive and why. The merge is confirmed by a person, because merging the wrong records does real damage to history.

Classifying what is missing. A material with no material group, no tariff code or no unit: the model reads the description, compares it with the company’s own already-classified items and proposes one. The internal history is the teacher — and a better teacher than any generic table.

Translating the error. The rejection message from the load is usually cryptic and sometimes points to the wrong place. With the record and the rule to hand, the assistant explains what actually failed and what to correct.

The cycle, and why it repeats

Cleansing is not a single pass. It is a loop that runs several times, and each turn is cheaper than the one before.

1. Measure. A quality indicator per object and per rule, with an exportable list of the records that failed.
2. Discard. Block whatever has no use and no retention obligation. It is the step that cuts the most effort and the one that meets the most resistance.
3. Merge. Deduplicate what is left, with the approval on record.
4. Complete. Fill in what the target requires, with a proposal from the AI and a review by the owner of the master data.
5. Reload and re-measure. The same indicator, with the new number beside the old one.

In the conversion we supported, this loop ran alongside the load test cycles — two cycles of load only, then one with integrations and three full integrated cycles. Each cycle returned a list of failures that became cleansing tasks at the source, and the next cycle measured whether the list had shrunk. A cleansed client (SAP) in a separate environment is what makes this possible without touching production.

Who decides what is true

The part no tool solves: master data needs an owner. Not the IT team, which does not know whether two customers belong to the same corporate group; and not the project, which comes to an end. Someone in the business area that uses the master data has to answer for the decisions — and carry on answering after go-live.

Without that, the data goes dirty again within months, because the cause was never technical: it was the absence of criteria and of an owner. Automation shortens the correction work; it does not replace governance.

What remains after the migration

The immediate gain is a load that passes and a schedule that does not slip. Three other things last.

The rules become permanent checks. What failed during migration now runs periodically over the new master data. The database starts life with a defined quality standard instead of degrading until the next migration.

The criteria are written down. What counts as a duplicate, what justifies a block, how a new material is classified — decisions that lived in the heads of the people doing the data entry become a document anyone can consult.

Traceability answers the audit. For every record merged or blocked there is a record of who approved it, when and on what basis.

The honest limit: cleansing does not fix the master data creation process. If anyone can still create a supplier without a check, the database goes dirty again — more slowly, but it goes. Cleansing is the start; governance is what sustains it. It is the same discipline as data and integration, applied to master data instead of the flow.

If your conversion is still being designed, it is worth first reading the e-book on implementation at Inove Academy, which shows why most project problems are not technical — and the guide to brownfield, greenfield and bluefield, so you can choose the approach knowing what each one demands of your data.