When Bad Data Becomes Acceptable Data

Sometimes you need to weigh the options and determine if all your effort to fix bad data is actually worth it.

Yes, bad data costs the company money, added expenses and so much more.

Yes bad data may critically impact decision making at your organization.

Yes, it will take effort to get rid of bad data.

However, when you start your efforts to clean the bad data think of the following decisions you must make when you take the mountainous task to scrub the data clean.

1) Is the ROI worth it?


a. If you have two bad records that are wrong, do you spent the same amount of time working on a $3 error vs. a $30,000 error. I think it’s obvious, but some of us will burn the midnight oil to get ride of the $3 error just as vehemently as the $30,000 error.


b. BUT remember any error that makes it to a customer’s bill will become a big topic for the customer. Your evil over-billing practices may become the twitter topic of the day. And nobody wants that now do they!

2) How long before the error disappears?


a. With archiving and deletion jobs, that record may be gone before you know it. Saving you some valuable time to concentrate on more pressing matters.

3) What’s the impact of keeping it?


a. Yes, what is the impact of keeping that bad, bad data. BUT ask yourself who is using the data and what type of decision is being made with it?

4) Can you communicate the bad data to the users?


a. This may sound strange to you, but if you have to get the data out and you know the exact problem, duplicate records, over-billing, etc., etc. then make the knowledge workers aware of what they are looking at and reporting.


b. Sometimes just letting the knowledge workers in on the problem will prevent a disastrous decision from being made. They can identify the situation in the foot notes, or remove it all together from their reports and inform the decision maker that the issue exists.

On a personal note, I say clean it, scrub it, polish it, make the data shine, like it has never shined before. BUT, we don’t live in an ideal world and you may have to keep the rotten data there. It may be primarily for budgetary reasons or the size of the impact may be very minimal. So when this is the case ask yourself what’s the impact, what’s the ROI, how long will it be there and can you communicate the situation to the users. When this happens the data is still bad data but becomes acceptable data.



Leaving bad data in there may be viewed as the lazy-man's solution, but when you're swamped with 13 other data quality issues to tackle you need some methodology to identify the key situations and those that can be pushed aside.