ERP Blog | Videos | Podcasts

EAG's Recommendations for Data Hygiene Best Practices

Written by Shaun Orthmann | Mar 11, 2020 7:55:21 PM
You don’t have to be a rocket scientist to understand that having a wrong phone number for a person can keep you from reaching that person.

On a personal level, everyone has experienced the hassle of needing to reach someone, only to find out that they've changed their number. On the macro level, bad data contaminates business records so rapidly that it can be difficult to keep up. Every year thousands of people change jobs, get phone numbers, get married or divorced, move to a new city — and their lives take all kinds of different directions. Data hygiene best practices must be followed to contain and quarantine bad data.

What if it takes longer than expected for a B2B to reach out to a hot prospect? 30% of people change jobs every year, and now your point of contact is working for a different company. B2C models contend with the data churn of 43% of consumers changing their phone numbers in a year, while 25% to 33% of email addresses are outdated within 12 months.

All of this has a significant impact on your delivery and response rates, not to mention how much time you will waste trying to trace a prospect.

As you can see, B2B data deteriorates at a very fast pace, and if you fail to keep abreast of these changes, you’ll find yourself among the 62% of organizations who are infected with inaccurate customer data. Proactive and preventative data scrubbing can result in more accurate customer data.

What is Data Hygiene?

Data hygiene refers to the collective processes conducted to ensure the cleanliness of data. Data is considered clean if it is relatively error-free. Dirty data is a term used to describe inaccurate, incomplete, and inconsistent data. Following data hygiene cleaning steps can counteract data churn and make it easier to embark on a data migration project.

Common Data Hygiene Errors

Bad data can be caused by any number of factors. Here are some of the most common errors:

  • Missing data/missing fields
  • Inconsistent mapping
  • Data that is not validated at the time of entry
  • Separate overlapping systems with duplicate or conflicting data
  • Generalized, badly defined values
  • Confusing naming schema
  • Out of date, invalid, inaccurate data
  • Sloppy formatting/invalid syntax that could break the new system

The Trillion-Dollar Cost of Bad Data Hygiene

According to Forbes, violation of data hygiene best practices costs the U.S. economy approximately $3.1 trillion annually. But where does that expense come from?

Let’s look at the 1-10-100 Quality rule, as formulated by George Labovitz and Yu Sang Chang. This states that it costs $1 to verify a data record as it’s entered, or at the point of data capture (the prevention cost); it costs $10 to cleanse and deduplicate the record (the correction cost); and it costs $100 continuing to work with a record that’s never cleansed (the failure cost).

While this is expressed in US dollars in the above description, it can be understood to mean any number of ‘units’, measured in financial terms. We can also use 1-10-100 to measure ‘cost’ in terms of resources or time.

What this means in practical terms is that if you don’t invest in data hygiene services, you can end up losing a lot of money.

That’s not to say that all companies are being short-sighted about B2B data hygiene. As described above, data can go bad rapidly. B2B data decays at a rate of 70% per year, and the average company loses 12% of its revenue as a result. When you look at these percentages, the $3.1 trillion lost in the US economy in a year comes into clearer focus.

Data Hygiene Best Practices

Make your ERP software selection a robust system that includes functions which make it easy to mandate certain fields be filled in. Build in some validation functionality, and do some logic checks. Establish standardization rules and add constraints. As much as possible, make the system validate the data at the point it is entered.

There are ERP tools and integrations for keeping records updated in real-time. As well, there are ERP tools that automate data cleansing.

If you’re infected with bad data, it is going to take a dedicated project devoted to parsing, standardizing, cleansing and validating the data to implement data hygiene best practices. ERP Advisors Group can provide guidance in designing such a project.

Once you’re disinfected, you can put in systems and personnel to maintain scrubbed data and ensure data cleanliness.

Be watchful for indicators that dirty data has started to impact your business. Failing to follow data hygiene best practices can be an underlying reason why your performance as a business is struggling. Dirty data infestation might be an underlying driver as to why you’re not closing deals, or sales deals are taking longer, or accounting isn't collecting faster, or manufacturing costs are going up, or engineering is buying too much product — there are endless examples.

Here at ERP Advisors Group, we specialize in providing expert guidance on data hygiene and data migration. We help our clients assess the quality of the data, whether it is good or bad, and how much effort is required to clean it. We can provide direction as to what ERP tools to select for data cleansing and for keeping data updated, as well as guidance in designing projects to disinfect dirty data.