Garbage In: What Makes Bad Data
Important decisions based on bad data are likely to have bad consequences. What makes data bad? In this program, we look at five aspects of bad data: bad constructs, bad measurement, sample bias, too little variance, and too much "story" (which leads to manipulation of numbers). It's crucial to know the signs of bad data so you can recognize and correct the problem.
Introduction: "Garbage In": What Makes Bad Data
Oh data, how do I make thee bad? Let me count the ways... Bad Constructs, bad measurement, sample bias, too little variance, too much story... There are so many ways to get this wrong. Knowing this can help us avoid making important decisions based on bad data.
Lots of things that interest us are hard to measure directly, so we use "constructs" to assess them. But sometimes we create constructs that don't mean what we think they do. Picking a bad construct is most fundamental mistake in data analysis. When we work with a bad construct, our data is bad a...
Even when you measure the right thing, your data will be bad if you measure it in the wrong way. "Accuracy" is a good word that describes what we get from good measurement. But when our measures are not accurate, our data will mislead us.
If your sample is biased, it doesn't matter very much what you do with it. With selection bias, sample bias, and more, it's a lot easier to get a biased sample than get a representative one.
Problems with Variance
If every employee gets a positive evaluation, then what is the purpose of the evaluation in the first place? Without variance in data, we can't really draw any conclusions about what data means. Yet often in the workplace we value consistency so much that we drive variance out of key measures. Th...
Too Much Story
The desire to tell a certain story through our data can lead us to manipulate results and create a misleading picture. Don't let the pressure to succeed overwhelm your commitment to understanding the world as it is.
Andrew Hill Bio
Before co-founding BurnBright, Andrew was a professor at the U.S. Army War College, where he taught senior national security professionals. As the War College's first Chair of Strategic Leadership, Andrew's research focused on connecting people, strategy, and innovation. He has a doctorate from H...