Data Quality
Good product data management enables data quality.
Data is of high quality, if the data is fit for the intended purpose of use and correctly represents the real-world construct the data describes. Start with Data helps clients increase the accuracy, completeness and relevance of their product data through better processes, data governance and technology.
What is Data Quality?
Data has quality in relation to
- the decisions which can be enhanced by their use
- the actionable initiatives which ensue
- the business insight, intelligence and knowledge gained
Any measure of quality needs rules and standards and when you agree on data quality rules, they should factor in the degree of value that data provides to an organisation. If data in a certain context has an extremely high value, this suggests that an extremely rigorous set of data quality rules are required for any data used in that context.
Why data quality is important to an organisation?
The adage of ‘garbage in, garbage out’ still holds firm when talking about data quality. If it is poor, that impacts on many levels of the business.
What are the real costs to businesses arising from poor data quality?
- Extra costs incurred by completing a work task with flawed data – and then having to do the task again
- Damage to brand reputation – the impact of poor-quality data use will become more severe as the eCommerce world spawns new competitors and customers (individual and corporate alike) demand excellent data as the norm. Complaints will rise, returns will proliferate, and customers will turn to more reliable and trouble-free alternatives
- Impact to eCommerce sales – If your eCommerce site cannot present sufficient product data to support a self-service buying decision, this is due to incomplete product data within your databases and how product data is syndicated (or not!) between trading partners
- Damage to industry reputation among partners like suppliers and marketplaces
- Poor decision making – Bad quality data means bad information, making any actionable insights untrustworthy – its inaccuracy or incompleteness could have business outcomes which put the company at risk
- Reduction in operational efficiency – The opportunity costs drain productivity and create avoidable manual work. This leads to endemic ‘data wrangling’, ‘data munging’ and ‘data janitor’ work within organisations – a situation where, according to expert estimates, staff spend half their working life bogged down in collecting and preparing wayward digital data
Data Quality Dimensions
Data quality meets the following six dimensions:
Completeness
how much of a data set is populated rather than left blank
Timeliness
how useful or relevant the data is based on its age
Validity
ensuring the data is a true reflection of the type of information you originally recorded
Uniqueness
whether a data entry is duplicated, uniqueness is guaranteed by one recorded version of a data entry
Accuracy
defining whether the information you hold is correct or not (not the same as validity, which is a measure of the correctness of the type of data)
Consistency
how far you can safely compare data across different storage areas, data sets and media and whether they have been recorded in the same way
Common data quality issues
Data can be degraded, corrupted and rendered unusable in a variety of ways;
Inaccurate data may contain incorrect personal details such as email addresses, phone numbers. It may also have missing information about product dimensions and measurements leading to problems with the size of delivery vehicles being used. These inaccuracies can be down to something as simple as a field inadvertently left blank or misspellings.
Non-compliant data doesn’t comply with regulatory or legal requirements.
Uncontrolled data becomes polluted if it is left without constant monitoring.
Unsecured data with no security protocols can be open to access and attack by hackers.
Static data is never updated, causing it to become obsolete.
Dormant: data left in a repository loses its value – unshared, unused and essentially useless.
Understanding what data quality looks for your organisation is an important step in overcoming data quality issues. Overcoming data quality issues is possible through better product data management processes, data governance and technology.
Data Quality Management
As information sources rapidly increase and legal and regulatory compliance becomes more complex, organisations need to access, ingest and reuse data from widely differing origins in a consistent and reliable way. Best practice for data quality management means taking a proactive approach, rather than a reactive approach. This involves checking and measuring the degree of data quality before it enters a company’s core systems.
Data Quality Maturity Model
Using the 5-stage data quality maturity model helps to evaluate the degree of data quality maturity which exists in a given organisation. A scale from worst to best practices would identifies;
- A purely reactive approach – no plan, no standards, essentially firefighting.
- Siloed management – each department or area do what they like without reference to the larger picture.
- Coordination – there is a common acceptance and understanding of data problems, and steps are being taken to remedy them.
- Governed procedures and policies – there is a framework of stewardship, and policies in place.
- Proactive data quality management – procedures, policies and AI-driven functionalities are in place to act on data quality problems before they occur
Data governance
Best practices in data quality management usually refer to a unified platform under a data governance framework. This means a set of initiatives which are also applied to PIM projects:
By using unified procedures for monitoring and scoring data quality directly within a governance framework, organizations can implement quality controls to ensure end-to-end data quality from source to destination.
Data Quality Tools
Data quality tools are designed to enhance the accuracy, completeness, relevance and consistency of a company’s data. Most tools fall into four general categories:
- Data Cleansing (or scrubbing)
- Data Auditing
- Data Migration
- Data integration
Some will focus on one category, but as data analytics technology matures, cross-functional solutions are becoming more common. Data quality informs many cross-functional elements of an organisation including master data management and meta data management, and the best tools for your task are determined by your case’s specific needs. you choose a data quality solution, you will need to understand which of these areas you need to focus on.
Of the many software providers, most offer specific capacities;
– data integrity and data cleansing tools
– drag-and-drop graphical interface
– near real-time synchronisation of data
– merging of duplicate records
– deduplicating import files
– measurement of performance against internal or external metrics and standards
With advances in AI capabilities, based on semantic technology data quality tools make use of semantic discovery and built-in pattern recognition and can automate repeatable tasks on a scheduled basis.
Data Quality Consulting Services
Our Data Quality consultancy services help clients overcome data quality issues through better product data management processes, data governance and technology. Our three phase Product Data Quality Assessment process will identify poor quality data, estimate the impact on the business, and to advise on actions to improve your data quality.
Three Phase Product Data Quality Assessment
A targeted data quality assessment to guide your organistaion on a prioritised roadmap for process, data governance and technology improvements for product information management
Discovery
Phase 1
- Define the role & benefits of data quality for product data as part of your strategy
- Recommend a high level data quality framework for product data
- Validate scope of assessment (data sources, processes etc)
Evaluation
Phase 2
- Perform initial data quality assessment to include process mapping, data source analysis & technology assessment
- Identify pain points and data quality issues through process
- Identify existing rules & patterns
- Categorise opportunities into data governance, process and technology backlog
- Assign benefits to each opportunity
Play Back
Phase 3
- Prioritise opportunities
- Create high level prioritised roadmap
- Show examples of issues and route causes
- Plot on high level data quality maturity model
- Executive summary of roadmap
We pride ourselves on our flexibility and adaptability to clients’ requirements and it may be the case that your organisation will need to access certain parts of our Data Quality consultancy services, rather than the full wraparound offering. No problem! Our mission is to serve your needs with expertise so that the outcome aligns totally with your brief.
Find out more
If you would like to find out more about how product data management, PIM and MDM can create value for your business, we’d love to hear from you – Ben Adams, CEO Start with Data