Some would argue that getting structured product data right in eCommerce is as critical as getting pricing correct. Product data tells the customer what they are buying and entices them to spend more money on upgrades or accessories. When product data is wrong, customers might overlook a product or request a refund when the product isn’t what you said it was. This is opportunity lost and a customer service issue. Both ding profit.
Our experience is with more complicated products that have multiple layers of components, but these processes exist in any company that has a large number of products. For simplicity sake, we will frame the discussion around the following categories of data and where the breakdowns occur in getting the structured content out to your eCommerce site:
Manufacturing data – manufacturing data is used to identify parts from an internal perspective. This data is technical in nature and typically errors in formatting or grammar are acceptable because the product data is for an internal audience.
Marketing data –manufacturing data is used as a baseline for marketing content to ensure the technical representation of the data is accurate, but in many cases the data is re-written with a marketing bent.
Channel data – your channel partners may require data to be presented in a specific way or described differently than marketing data.
Other audiences – of course there are other internal audiences that need the data in various business functions such as accounting and inventory management, but we are restricting the discussion to eCommerce here.
In most organizations, since different types of data have different audiences, they often reside in different systems. This means the data needs to be transferred from one system to the next, enhanced, then marches further upstream until it reaches the desired audience.
Breakdowns occur when transferring data from manufacturing to marketing. Manufacturing data is usually stored in a PLM system where the products are defined. Marketing data is typically authored in an eCommerce system or even CRM. Many times there is an ERP system in between each system and the data is structured differently. This means a translation must occur from one system to the next which could be done in an MDM system or an equivalent data aggregation and distribution tool. If all the data was perfect, this is still a lot of moving parts to coordinate to keep data integrity. Now, throw in the fact that a lot of times liberties are taken in defining the data such as certain fields are overloaded with text meant for a downstream system. It makes it almost impossible to trace data back to its source when issues occur, so many times downstream authors decide to re-write the content rather than fix the upstream issues.
Re-authoring content for marketing is a commonly used to solving data issues. Marketing folks are re-writing data regardless because the audience is different for their content consumers. But often, re-authoring is used to fix data rather than re-audience it. This is a bad practice, but exacerbating the problem is the fact that the systems typically mirror organizational structures as well and the different authors operate in silos. The organizational impedance is too great to overcome and authors will just re-write it for their own context. This means that the data is still wrong for the original author’s audience and if the data was ever changed in the original source, then none of those changes would be reflected downstream.
Codifying rules for data manipulation is the third area where problems occur. As mentioned, upstream users may employ tactics to send signals to downstream systems like overloading text fields or appending text to the front of names. One of our customers used a price of $99999 to signify that the part was disabled. There were no input rules around it, so untrained users thought that any number over $99999 would work or that $999999 would work which wasn’t the case. The MDM system had been coded to recognize exactly $99999 and nothing else. Hundreds of such rules can exist in a large scale deployment.
All of these complications lead to errors on the eCommerce site and sometimes these errors are costly. One customer we worked with left a component off the description of their product which lead the pricer to discount the product and then they shipped 50 free optical drives for free! In another case, an inferior product was labeled with high end components but priced compatible with the lower end product. Customers purchased the product and received the inferior product. Customer service reps dealt with the issue for the next two weeks costing the company money for each return.
There’s no silver bullet for solving these issues. The inconvenient truth about data production is that it is complicated when you are doing it on a large scale. The lynchpin is twofold: solid processes for addressing data issues and secondly a holistic MDM system that is capable of pulling in data from many different sources and then using business rules to transform the data into many different formats needed by the downstream systems. Without these two things you’re going to be chasing your tail tracking down various flat files and reports, importing them into too many independent systems. So what do you need to reduce errors in the process?
A solid MDM system that is capable of pulling in data from many different sources and then transforming it to the format you need for your eCommerce system
The next thing that’s needed is a hierarchical representation of data in the eCommerce system set up so that the manufacturing data is at the lowest level, then eCommerce, then Channel. This ensures that when new technical data comes in, it doesn’t overwrite what you have and you can use the data if you want to.
A good process for identifying errors in data and then getting it corrected in any of the failure points. This cannot be underestimated and no system is going to do it for you.
We’ve been able to do this with a number of customers, but like I said, it’s not easy. Only blood, sweat and tears will get you through it. But once you’ve corrected the failure points, you’ll see harmony descend on the process and your data quality will go up resulting in improved profits.
Some would argue that getting structured product data right in eCommerce is as critical as getting pricing correct. Product data tells the customer what they are buying and entices them to spend more money on upgrades or accessories. When product data is wrong, customers might overlook a product or request a refund when the product isn’t what you said it was. This is opportunity lost and a customer service issue. Both ding profit.
Our experience is with more complicated products that have multiple layers of components, but these processes exist in any company that has a large number of products. For simplicity sake, we will frame the discussion around the following categories of data and where the breakdowns occur in getting the structured content out to your eCommerce site:
In most organizations, since different types of data have different audiences, they often reside in different systems. This means the data needs to be transferred from one system to the next, enhanced, then marches further upstream until it reaches the desired audience.
Breakdowns occur when transferring data from manufacturing to marketing. Manufacturing data is usually stored in a PLM system where the products are defined. Marketing data is typically authored in an eCommerce system or even CRM. Many times there is an ERP system in between each system and the data is structured differently. This means a translation must occur from one system to the next which could be done in an MDM system or an equivalent data aggregation and distribution tool. If all the data was perfect, this is still a lot of moving parts to coordinate to keep data integrity. Now, throw in the fact that a lot of times liberties are taken in defining the data such as certain fields are overloaded with text meant for a downstream system. It makes it almost impossible to trace data back to its source when issues occur, so many times downstream authors decide to re-write the content rather than fix the upstream issues.
Re-authoring content for marketing is a commonly used to solving data issues. Marketing folks are re-writing data regardless because the audience is different for their content consumers. But often, re-authoring is used to fix data rather than re-audience it. This is a bad practice, but exacerbating the problem is the fact that the systems typically mirror organizational structures as well and the different authors operate in silos. The organizational impedance is too great to overcome and authors will just re-write it for their own context. This means that the data is still wrong for the original author’s audience and if the data was ever changed in the original source, then none of those changes would be reflected downstream.
Codifying rules for data manipulation is the third area where problems occur. As mentioned, upstream users may employ tactics to send signals to downstream systems like overloading text fields or appending text to the front of names. One of our customers used a price of $99999 to signify that the part was disabled. There were no input rules around it, so untrained users thought that any number over $99999 would work or that $999999 would work which wasn’t the case. The MDM system had been coded to recognize exactly $99999 and nothing else. Hundreds of such rules can exist in a large scale deployment.
All of these complications lead to errors on the eCommerce site and sometimes these errors are costly. One customer we worked with left a component off the description of their product which lead the pricer to discount the product and then they shipped 50 free optical drives for free! In another case, an inferior product was labeled with high end components but priced compatible with the lower end product. Customers purchased the product and received the inferior product. Customer service reps dealt with the issue for the next two weeks costing the company money for each return.
There’s no silver bullet for solving these issues. The inconvenient truth about data production is that it is complicated when you are doing it on a large scale. The lynchpin is twofold: solid processes for addressing data issues and secondly a holistic MDM system that is capable of pulling in data from many different sources and then using business rules to transform the data into many different formats needed by the downstream systems. Without these two things you’re going to be chasing your tail tracking down various flat files and reports, importing them into too many independent systems. So what do you need to reduce errors in the process?
We’ve been able to do this with a number of customers, but like I said, it’s not easy. Only blood, sweat and tears will get you through it. But once you’ve corrected the failure points, you’ll see harmony descend on the process and your data quality will go up resulting in improved profits.