Data Onboarding

A formalised process of ingesting product data from a diverse range of sources and processing those data so they can be standardised, stored, and used by internal teams.

As competition in eCommerce intensifies, the faster and simpler the data onboarding process is, the more competitive the company. At Start with Data, we recognise that successful and efficient product data onboarding is the springboard for optimised new product introduction (NPI) and internal processing. It enables high-speed preparation and enrichment of these data for distribution to multiple sales channels in a high-quality, compliant, and consistent ‘golden record’.

What is Product Data Onboarding?

As a retailer, distributor, or manufacturer, you receive many products from vendors and suppliers daily. Your primary concern is to ingest those data smoothly, quickly and with minimal complications. The current market of providers can respond to these requirements by selling tools deploying AI-powered functionalities.

Providers are now offering the capacity to define required information attributes and catalog structures so that the suppliers can input the product information directly using an onboarding tool. The retailer, distributor or manufacturer can then review and accept the information. This optimized process greatly enhances productivity by eliminating the need to convert or reformat product data manually. Configurable data quality management (DQM) ensures a standardised quality of supplier data, allowing the retailer or distributor to drastically reduce (or even eliminate) any incoming data backlog and onboard new seller and vendor feeds easily.

What is vendor, supplier or seller onboarding?

Supplier (or vendor) onboarding

Suppliers come in all types, as do the protocols they apply for exporting product data applied to individual SKUs. Quality and reliability can vary greatly, so key onboarding requirements are high performance product data imports, especially when dealing with very high data volumes. The market trend from providers in responding to increased volumes of heterogeneous supplier data is towards;

  • configurable transformation
  • validation and adjustment rules
  • comprehensive data quality management

Seller onboarding

Sellers may be providing catalogs of products and what onboarder teams need to manage are multiple file structures, including ETIM, eCl@ss, and more. An onboarding tool can apply relevant edits and changes in bulk, thus adapting existing catalogs into the required format.

  • Segmenting data and feeds for brand-specific strategies
  • Handling special promotions data and feeds
  • Managing and integrating local inventory data and feeds
  • Adapting when brands change to new systems, attributes, and naming conventions

What you want from a data onboarding tool?

The destination organisation requires an onboarding system offering the capacity to;

  • Define data requirements for supplier products
  • enable suppliers to send product information directly
  • Streamline workflows when approving incoming product data
  • rapidly review the acceptability and compliance of product information
  • accept and approve new products from trusted suppliers in bulk
  • easily scale onboarding capacity to support diverse supplier sources

What functionalities can data onboarding tools offer?

The functionalities offered by data onboarding tool providers are expanding in responding to issues around reliability, speed, and ease of use. Among attributes they offer are:

Data mapping

From supplier/vendor to destination, complete oversight of the product data flow is possible

Data merging

To consolidate data sets in multiple formats

Bulk editing

Allows for the mass-treatment of data concerning variants, media assets and unstructured data

Integration with PIM

Many onboarding tools integrate smoothly with existing technology stacks, leading to improvements in operational efficiency and reduced time-to-market

Delta uploads

Delta files make it unnecessary to copy the entire contents of a file and only update the parts which change, saving bandwidth and time - especially useful for cloud-based SaaS onboarding services

Data Quality Analysis

Channel readiness reports give accurate assessments and current quality standard status, including elements like missing information

Benefits of a modern data onboarding system

To convince customers to invest in data onboarding tools, software solution providers must make the benefits demonstrable. Fortunately, it is no longer a challenge to provide companies with a straightforward and intuitive way to onboard product data. Much of the ‘grunt work’ is AI-powered. The software (and data importer) does the work for the end users by using advanced validation logic – the onboarding tool can work out what exactly the data is, where it is from and where it needs to go. The capabilities involved can;

  • Automatically match columns and corresponding data to your software’s actual data fields. Users can confirm the validity of the match before permitting it to enter the system
  • Create collaborative data workspaces where, when a team member adds new data key base information is captured:
  • Authorised data owners and stewards can invite specific collaborators to add data to the workspaces, but can also create an approval process, so they can;
      • Request specific data sets from team members.
      • Limit the workspaces they can access to import data.
      • Review all data submitted before approving it for import into the platform.

 

Thus, the right data ends up where it should be. The added value is that the control over the data flow means they will be cleaner, more accurate and more usable from the outset. This improves efficiency, speeds processes and eases implementation of streamlined workflows.

Mistakes to avoid

Manual Input

Product data is often compiled from fragmented sources from disparate supplier systems, which results in convoluted and confusing spreadsheets. Onboarding has historically involved manually collating and organising these data before internal processing and enrichment can be carried out. The potential problems of human error and data mishandling ultimately lead to inconsistent and unreliable data reaching customers. Even with a purchased data onboarding tool, the risks or reverting to manual data onboarding still exist;

  • Misconfiguration of the new tool
  • Using an ‘off-the-shelf’ solution when a more tailored onboarder would be better
  • Lack of business user knowledge due to lack of training
  • Lack of awareness of the range of functionalities available

Supplier Portals

Many retailers still rely on use of supplier portals. However, the criteria suppliers use to set up their portals do not necessarily match your standards or integrate with your onboarding portal. Multiply that scenario by thousands and unless the onboarding tool is configured to your needs and specifications, the receiving internal team still needs to manually normalise the data to get it into the required form.

Mismatch in requirements

Slow product onboarding has many causes such as different formats and delivery methods of product data from manufacturers and suppliers that must be unified to fit the retailer’s catalog or commerce system. Conversely, the manufacturer may need to handle a wide range of requirements for different formats from the retailer. Depending on circumstances, the retailer or the manufacturer mandates the format. Whatever the case, there is often a mismatch in requirements, which causes delays in the onboarding process.

4 common onboarding scenarios for product data

1. Onboarding product data for resell products

 This scenario applies to retailers, distributors, and other merchants. Manufactures may also have product ranges which are not produced in house, but which are key add-ons when selling own-produced products. The end game is to create the optimal set of product information required to facilitate a purchasing decision by the end customer. The whole process must start robustly by obtaining the completest possible set of product information that the supplier can provide. Nowadays, as the eCommerce environment expands exponentially, much of the customer journey is driven by low self-service. Therefore, the demand for product information has increased dramatically during this journey – no longer is it touchpoint-based.

2. Onboarding product data for raw materials and packaging

This scenario exists for product manufacturers. What is required from incoming data is product information for quality assurance, or for organic production, how to apply the right blend to produce an entirely consistent final product.

Another key factor is the growing demand for sustainability in manufacturing. That is driving the trend for a larger volume of information on the provenance and traceability of finished products and how they are packaged. This includes the sourcing of ingredients and under what conditions these components were manufactured, grown, or produced.

3. Onboarding product data for parts used in MRO

When it comes to manufacturers who run production facilities, the issue of product data for parts and components in Maintenance, Repair and Operation (MRO) is a separate scenario. Most manufacturers use facility management around their logistic facilities, offices, and other locations where products for MRO are required. The fast-emerging ‘Internet of Things’ (IoT) means that these kinds of products are fast becoming more and more ‘intelligent’, as well as being automatically operated. As such, the near future is likely to see an explosion in the volume of product data for this category of component or part.

4. Onboarding product data for indirect products

Most companies use products and services like office supplies, furniture, travel services and more. These purchases also need to be onboarded and while their product data is minuscule in comparison to the previous scenarios, the trend towards AI in procurement operations will naturally increase the need for product data onboarding in this area.

Questions to Ask a Product Data Onboarding Provider

How can your onboarding tool make sure we onboard data which adheres to a certain level of quality standard?

It is feasible to streamline the product data verification and standardisation processes as long as both sides understand the requirements and functionalities of the platform used. At the end of the day, it will save both sides time, make life easier from a resources perspective and enhance the relationship between partners. Of course, the key to success is a high degree of collaboration among all stakeholders.

I have a frequent ‘headache’ with product data bottlenecks. We seem to spend far too long manually correcting and reformatting when supplier onboarding. Will your tool solve that problem definitively?

Using dedicated data onboarding software, this ‘bottleneck’ problem is significantly alleviated as you have comprehensive oversight of the product data flow from source to destination. The ‘clearly-established sets of standards’ mentioned above apply in part to effectively mapping incoming product data;

  • Determining normalised and standardised definitions for data taxonomy and clearly defined attribute labelling/categories
  • Maintaining ongoing checks to audit these taxonomies and attribute definitions
  • Categorising products with similar attributes and highlighting these connected SKUs for eradication of copied data

 

The key is to leverage pre-build connectors, templates, and mapping tools for adopting & mapping data standards into a PIM or MDM platform. The creation, management, and onboarding of all file types (for example, BMEcat) can easily be simplified and parts of it can be automated. The most important initial move is to do a deep dive into the requirements of your system infrastructure so that the solution you choose can be integrated with all your existing tools (like PIM and ERP).

How can I work with my suppliers and vendors so that the onboarding process is easier and better for all concerned?

If the distributor or retailer has an integrated relationship with supplier onboarding feeds, they are receiving product data updates constantly. This assumes such a relationship has been built up, and probably works best for larger distributors with an established product data governance framework and a limited number of suppliers. Even so, it is eminently possible to onboard product information files and integrate them with a variety of sources, including PIMs, ERPs, or Data Asset Management (DAM) systems.

Product Data Onboarding Consultancy Services

Our Product Data Onboarding Consultancy Services help clients overcome data onboarding issues through better product data management processes, data governance and technology.  

Discovery & Design

Implementation

Support

We pride ourselves on our flexibility and adaptability to clients’ requirements and it may be the case that your organisation will need to access certain parts of our Product Data Onboarding consultancy services, rather than the full wraparound offering. No problem! Our mission is to serve your needs with expertise so that the outcome aligns totally with your brief.

Find out more

If you would like to find out more about how product data management, PIM and MDM can create value for your business, we’d love to hear from you – Ben Adams, CEO Start with Data

Case Study

“Start with Data are helping transform product data management, laying scalable technology and data governance foundations”