Data quality assurance on a product catalogue is a rule library, not a dashboard. The dashboard is what you show the board. The rules are what stop a broken SKU reaching a channel. Most teams we meet have built the dashboard and never written the rules, so the number moves and nothing changes. This piece is the rule library. Four families of checks, how to write them, where to run them, and what to do when they fail.
What data quality assurance means for a product catalogue
DAMA UK’s working group published six primary dimensions for data quality assessment: completeness, uniqueness, timeliness, validity, accuracy and consistency. That framework is sound, and it was written for records in general. Catalogues need something narrower and more executable.
A dimension is a category of concern. A check is a line of logic that returns pass or fail on one value, on one SKU, at one moment. You cannot run a dimension. You can run a check.
We group catalogue checks into four families:
- Completeness. Is the value there at all?
- Conformance. Is it in the right shape?
- Consistency. Does it agree with the other values around it?
- Plausibility. Could it be true?
Every rule you write belongs to one of those four. If you cannot place it, the rule is not specific enough yet. Our product data services work almost always starts by rebuilding this library. The client arrives with a quality score and no idea which rule produced it.
Stage 1: Write the rule before you write the check
A rule card has five fields. Write them in a spreadsheet before anyone touches code.
| Field | Example |
|---|---|
| Rule statement | Every SKU in Cable Glands carries Thread Size |
| Scope | Node: Cable Glands. All markets. |
| Test | Thread Size is populated and is a member of the Thread Size list |
| Severity | Blocking at channel export, warning in PIM |
| Owner and fix path | Category manager, electrical. Fix by supplier request. |
The fifth field is the one teams skip, and it is the reason libraries die. A check with no named owner produces a queue nobody works.
Write cards for your twenty highest-revenue nodes first. Not the whole catalogue. Twenty nodes will give you eighty to two hundred rules, which is a quarter of real work.
Stage 2: Completeness checks
Completeness is the easiest family to build and the easiest to build badly.
Mandatory by node, not by catalogue. A catalogue-wide mandatory list forces you to the lowest common denominator. Thread Size is mandatory for cable glands and meaningless for workwear. Mandatory flags belong on the node, inherited down the tree.
Conditional mandatory. If Hazardous equals true, then UN Number is required. If Sold By equals Length, then Unit Length is required. These conditional rules catch the gaps that flat mandatory lists never see.
Media completeness. At least one image at your minimum resolution. Three images for A-class lines. A datasheet for anything with a technical specification.
The sentinel problem. This is where most completeness checks quietly fail. A rule that tests only for null will pass “N/A”, “TBC”, “-”, “0”, “Various”, “See website” and “XXX”. We find those strings in every catalogue we audit. Maintain a banned-value list per data type and test against it in the same rule. Otherwise your completeness score is measuring typing, not enrichment.
Stage 3: Conformance checks
Conformance asks whether a populated value is in the shape the rule demands.
Identifiers. A GTIN is 8, 12, 13 or 14 digits and passes the GS1 modulo-10 check digit calculation. Length alone is not enough. Run the check digit.
List membership. Any attribute typed as a select must contain a value from its list. Free text creeping into a select field is the most common conformance failure we see. It is almost always a spreadsheet import bypassing validation.
Units separated from values. “240v”, “240 V”, “240 volts” and “240” are four values to a filter and one value to a human. Store 240 as a number with a unit code. This one rule does more for faceted search than any amount of copywriting, and it matters most in technical specification data.
Patterns. Manufacturer part numbers usually follow a brand-specific pattern. Encode the pattern per brand as a regular expression. It will catch transposed characters that no human review picks up.
Format and limits. Dates in ISO 8601. Decimal precision fixed per attribute. Character limits per channel, checked before export rather than after rejection.
Stage 4: Consistency checks
Consistency compares values to each other. These rules take longer to write and catch the errors that embarrass you publicly.
Uniqueness. One GTIN maps to one SKU. Duplicate GTINs across two live SKUs is a marketplace suspension waiting to happen.
Parent and variant agreement. Every variant of a parent shares the parent’s brand, node and non-varying attributes. If two variants of the same product disagree on material, one of them is wrong.
Dimensional arithmetic. Net weight cannot exceed gross weight. Packed dimensions cannot be smaller than product dimensions. These are two lines of logic and they catch hundreds of rows.
Cross-attribute logic. If IP Rating is IP67 or above, Enclosure Material must be populated. If Voltage is populated, Frequency should be too. Write these per node, with the category manager sitting next to you.
Locale consistency. If a SKU is published to a market, every field required in that market’s locale exists in that locale. Half-translated products are worse than untranslated ones.
Stage 5: Plausibility checks
Plausibility is the family generic data quality frameworks leave out, and it is the one that catches the errors customers actually notice.
A value can be present, correctly formatted and internally consistent, and still be nonsense. A 2,400 kg internal door. A screw with a length of 30,000 mm. A pallet of screws priced at £0.02.
Range guards per node. Set a plausible minimum and maximum for every numeric attribute on every node. A screw is between 3 mm and 300 mm long. Anything outside that is flagged, not blocked.
Statistical outliers. For each numeric attribute within a node, flag values more than three median absolute deviations from the node median. This finds unit mix-ups automatically, without anyone writing a range by hand.
Duplicate content. Flag any description string that appears on more than five SKUs. Copy-paste from one variant to a whole family is endemic, and search engines treat it exactly as you would expect.
Price sanity. Flag any price more than ten times or less than one tenth of the node median. This is a data check, not a pricing decision, and it belongs to the same library.
Plausibility rules should almost never block. They produce a review queue. The value is in the queue being short and specific.
Stage 6: Scoring, thresholds and what blocks
Score at three levels: the SKU, the node, and the channel. A SKU score is the weighted pass rate across the rules in scope for it. Weight by severity, not equally.
Use three severities and be strict about what earns the top one:
- Blocking. Fails stop the record publishing to the affected channel.
- Warning. Fails appear in the owner’s queue and do not stop publication.
- Informational. Fails are counted and reported, nothing more.
Start with fewer than ten blocking rules. Teams that set everything to blocking on day one take the catalogue offline, get overruled by commercial, and lose the programme. Add blocking rules one at a time, once the warning queue for that rule is already empty.
Set the publish gate per channel, because channels disagree. A trade website will accept a product that Amazon rejects. Measuring the difference is also how you connect quality work to product content performance rather than to an internal score. We routinely take blocking failures at export from a double-digit percentage into low single digits within a quarter.
Stage 7: Where the checks run
The same rule runs in four places. One library, four executions.
- At supplier ingest. Reject or quarantine at the door. This is the cheapest place to catch anything, and the only place where you can push the cost back to the sender.
- On save in the PIM. Immediate feedback to the person typing, while they still have the datasheet open.
- At channel export. The last gate before a rejection costs you money.
- Nightly full sweep. Because rules change, catalogues drift, and records that passed in March may fail in July.
If your rules only run at export, you have built an alarm, not an assurance programme.
Stage 8: Routing failures to whoever can fix them
Every failure has a cause, and the cause determines the queue:
- Supplier data. Goes to the supplier scorecard and back through supplier data onboarding, not to your enrichment team.
- Enrichment gap. Goes to the content queue, prioritised by revenue.
- Model problem. The rule is wrong, the list is missing a value, or the node needs splitting. Goes to the data owner.
- System fault. An integration is truncating or transforming values. Goes to IT with the failing rows attached.
Reporting one undifferentiated number of failures guarantees nobody moves. Reporting four queues with four owners gets the work done.
Why data quality assurance programmes stall
Four reasons, in the order we see them.
Rules written by people who never see the failures. Bring the category managers into the rule-writing session, or you will write rules that are technically valid and commercially pointless.
Everything blocking on day one. Covered above. It is the fastest way to lose executive support.
No retirement. Rules accumulate. A rule that has not failed in twelve months is either perfectly enforced upstream or no longer relevant. Review the library twice a year and delete.
Scores without owners. If the quality score has no name against it, it is a weather report.
A data quality assurance rule library you can build this quarter
- Weeks 1 and 2. Pick the twenty highest-revenue nodes. Write rule cards with the category managers. Expect eighty to two hundred rules.
- Weeks 3 and 4. Build completeness and conformance checks. These are mechanical and give you an early baseline.
- Weeks 5 to 8. Build consistency and plausibility checks. Slower, and where the interesting failures live.
- Weeks 9 and 10. Scoring, severities and the first three blocking rules.
- Weeks 11 and 12. Routing, queues and the supplier scorecard.
Twelve weeks, twenty nodes, one library that runs in four places. Then repeat for the next twenty nodes.
Key takeaways
- Data quality assurance is a library of executable rules, not a score on a dashboard.
- Every catalogue rule belongs to one of four families: completeness, conformance, consistency, plausibility.
- Completeness checks that only test for null will pass “N/A” and “TBC”. Maintain a banned-value list.
- Plausibility is the family most frameworks skip and the one customers notice.
- Run the same rules at supplier ingest, on save, at export and nightly.
- Start with fewer than ten blocking rules and add them only when the warning queue is empty.
- Route failures by cause to a named owner, or nothing gets fixed.
We build these libraries as part of our product data services. The taxonomy and attribute work that makes the rules possible usually runs alongside it. If you have a quality score and no rules behind it, book a thirty-minute call and we will walk through your top twenty nodes.