A day in the life of a….Data Migration Consultant

In a PIM transformation project, the data migration specialist is responsible for the movement of data across platforms, channels, and even entire networks. As part of our series ‘A day in the life of a….’ we speak to one of our experts about their key role in a PIM project.

Can you tell us a bit about your background.

I studied informatics at university quite a few years ago. In fact, I ended up getting consultancy work before I finished my course, which made it pretty complicated to finish my master’s thesis! Still, it was all good experience!

In a nutshell, what is data migration?

As I see it, it’s purely migration from one system to another. So usually, it’s from a legacy system to a new system. It’s a pretty complex process and I’d say it frequently isn’t given the attention it deserves from stakeholders.

Why’s that?

Well, they may be planning to implement a PIM or MDM or CMR solution, and they only look at it in detail from a process and usability perspective. Then, they say “OK, we’ve sorted those details. Let’s just shift the data from the old system to new one.” Unfortunately, it’s never as simple as that.

What kinds of tasks do you carry out during a typical day’s work?

I’m both leading and supporting multiple tasks on an ongoing basis. So I’m setting the expectations for the data migration process, helping to identify the sources, ensuring that there is a clear understanding from the business itself, and that documents are being signed off. I’m also constantly monitoring that the delivery is following the rules which we agreed on.

How complex is data migration?

Well, like most other data-related activities, migration needs to start with analysis, by creating a system inventory document. Basically, that means asking yourself whether what you have in the legacy system is the only source of data you need to migrate – or does it need to be enriched, blended with other sources, and so on. This complexity often isn’t taken into account at the beginning.

Obviously, you have to define the scope, which itself has two dimensions. Firstly, which data elements do you need to migrate? Secondly, do you need to extract all the historical data and dump it wholesale in the new system?  

Like saving data just for the sake of saving it?

Yes, right. In fact, there are good reasons for not doing so. For instance if data is out of use, there’s no point in polluting the new system with useless data, because that useless data will just slow it down.

There is another key dimension of migration, which is who owns the data? It’s not a great idea to keep data if the person who owned them left the company last year!

So, the scoping exercise has to be carried out with the stakeholders?

Absolutely. No IT person can decide on this. Sure, they can do an inventory, but it’s the business which decides what should or should not be in the new system.

You said stakeholders don’t really appreciate the importance of good data migration execution. Can you elaborate?

Of course. You have to work with the product manager to emphasise its importance. To be honest, stakeholder buy-in – from strategic to user levels – is crucial. It must be embedded in all transformation projects. So, in most projects, you have discovery workshops where different teams present different parts of the solutions to be built – user experience, integration, migration and so on. Data migration needs to have its own section of the ‘discovery agenda’. What comes out of discovery should be a migration plan, or strategy, which needs to be signed off before taking any action.

…and presumably, the time needed depends on the size and type of company, and its business requirements…

Yes. I mean, you can do generic preparation like preparing different templates, different procedures for uploading to a new system. These processes are easier than before, because there are a lot of tools on the market which help you to assess data quality. Then there’s the ‘education’ aspect – preparing local stakeholders to migrate the data. Many really need some guidance and hand-holding.

How useful are the tools?

They’ll definitely help, but only after you’ve done your homework and established the scope. The next step is to look at the quality of the data – again, the tools help with this very important phase. But data quality can be potentially very problematic.

Why?

Sometimes, clients tend to think “Our data’s fine! Nothing to worry about here.” That’s never true! I always try to emphasise the need for extensive profiling of the data to really assess the quality.

Do you get client pushback on this issue?

Sometimes. I remember one project, where we were carrying out an MDM project and migrating data from a legacy system. The product owner didn’t foresee a need for data profiling – he just said “It’s OK, we don’t have the time for it. Just take the data, do some transformations and load it.” After several escalations, I obtained approval for profiling…and it turned out there were huge problems and half the data couldn’t be loaded at all. It seems like the attitude is “Let’s park this little issue, load everything into the new system, and …we’ll sort it all out later” – But in my experience, it never gets sorted.

…and I guess if you’re migrating bad data into a new system…

…you’re risking that the new system will operate at a sub-optimal level. The user interface looks great, lovely icons…but half the products aren’t visible.

Apart from lack of awareness, what else causes issues with data quality?

Ownership. There’s often a lack of ownership of the data. So, it’s hard to find an owner who has the authority to decide what to do with certain quality issues. We can fix errors in the source system, or park the less serious problems, or maybe use automated corrective processes. But at the end of the day, it’s not my data. I’m not going to be the one using it in the new PIM system. So it really is in their interests to have an owner – someone who is ultimately accountable for the data and who can make decisions about its quality, based on clearly established criteria.

So, you need someone to step up and own it for quality reasons?

Not only quality. One important thing is to understand is how the data from the old system are mapped to the new one. If you have a bunch of product information, categories, attributes and so on, you need to do the markings between them. This is a joint task. No-one from IT should do this alone. You may work with the CDO, data architects, whoever, but there must be meaningful involvement from the client side.

…and data mapping is important, right?

Yes. We need to understand how the data from the old system are mapped to the new one. So, you have some product information – categories, attributes and so on – you need to do the mapping between the two systems. This is a joint task. You must have someone from the business and not just from IT. It could be the CDO, whoever, but there must be input from the client side.

What’s the most satisfying part of being a data migration specialist?

Strangely enough, if you do the job properly, no-one notices because everything goes so smoothly! So in that sense, ‘no news is good news’. On the other hand, if there are any problems, you’ll be getting a call for sure!

If you would like to find out more about how our product data management consultants can create value for your business, we’d love to hear from you – Ben Adams, CEO Start with Data

We’re always looking for talented people! Read more about our culture, the experience required and our current roles available. We’d love to hear from you – Joanna Hall, Head of Talent, Resourcing & People Operations