A day in the life of a…..Data Quality Consultant

As part of our ‘A day in the life of a….’ series we chat with one of our data quality specialists, we find out just how fundamental the quality of your data is for the success of a PIM solution implementation project.

Many consultants we’ve spoken to have remarked on how important data quality is in a PIM or MDM project.

Of course. In recent years, there’s been greater attention from clients given to data quality. They’ve seen the consequences of not doing so – reputational damage, losing sales, compliance issues, and so on. So, it’s a positive trend that over the last 10 to15 years, awareness has been increasing.

Data quality is tied up with the entire lifecycle of the data – from creation to destruction. It’s also linked to governance and access controls. It needs to be validated before it goes to the master data system and gets used throughout the organisation.

What are the key dimensions related to data quality?

Firstly, uniqueness. The best example is customer information – CRMs, billing information and so on. We have to be sure that, where data is concerned, we are using the single source of truth. You should define that source and which system it comes from. There are tools with pretty advanced logic which use probabilistic logic to identify possible duplicates.

How do you know which versions to eliminate?

To get a grip of who they are and what they do, the quickest you can do is a couple of weeks. It might be a small piece of work where you carry out an expert analysis on a particular project and how they can make it better. But a longer initial engagement would be around six weeks, if it’s a big project which hits lots of different parts of the company and you really need to get under the bonnet to examine its constituent parts.

How do the tools help?

Well, there are sorts of dictionaries and thesauruses. For example, for personal details, I might need to know that ‘Jonathan’ is the same as ‘Johnny’.

What other dimensions are there?

Consistency, which is sometimes called integrity. That means consistency across your entire data landscape. Again, your customer information and product information must be similar or the same. Fixing data inconsistencies is pretty challenging in itself because it needs to be at a granular level. If you have multiple systems, you have integration processes which are doing conversions and transformations, so they’re changing data from one API into another…and you don’t have any control over what’s going on with the data sometimes.

At which stage of a PIM Project is a data quality specialist involved? All the way through?

It depends on the size of the project, but usually, I’m involved once we receive completed RFPs, because even at that stage, with the right information I can get a pretty good idea of where data quality ‘hotspots’ might exist. I’m also there during the discovery phase, in the early stages of an active project.

…and presumably, you work closely with the data migration specialists?

Indeed. I’m heavily involved during the data migration phase. When we start the migration process, we should have a separate item on the meeting agenda to speak about data quality so that I can understand the problems and come up with suggestions and solutions.

So, once data migration is complete, is that the end of the quality specialist’s role?

No it isn’t. When you go live with the new PIM, you should have some sort of archival process, to dump your ‘intermediate artifacts’ like raw data. This is for audit purposes. That’s the end for the data migration specialist. For the data specialist, there should ideally be some additional initiatives. For instance, to monitor the quality of the data, we should come up with a dashboard or some kind of alerts. In order to do so, you need to define your matrices for KPIs. For example, what does it mean if we say data is ‘accurate’? A measure could be that 90% of your customers need the email address field validly populated. Then you can onboard tools which check the quality based on the metrics you’ve set.

So there’s a kind of data quality minimum threshold…

Right, and the alert function can tell you if you’ve onboarded a number of email addresses which fall below the quality threshold and which could therefore have a potential impact on your business processes.

Is there a typical day for a data quality specialist?

I’m often flagging quality issues and I may be helping with finding solutions, like enrichment processes and so on. And in between times, I’m talking to the client stakeholders and decision-makers to keep them updated.

You know, at the end of the day, it’s not hard to put yourself in the shoes of the customer or purchaser who has to suffer with a company’s bad data. Nowadays, there’s no excuse for it and we are all becoming unforgiving customers if we’re frustrated during our ‘journey’. As a specialist, when you drill down into the root causes of these kinds of problems, you’ll often find that data quality has at least something to do with them!

If you would like to find out more about how our product data management consultants can create value for your business, we’d love to hear from you – Ben Adams, CEO Start with Data

We’re always looking for talented people! Read more about our culture, the experience required and our current roles available. We’d love to hear from you – Joanna Hall, Head of Talent, Resourcing & People Operations