The discovery of big data has reached nearly epiphany scale. Such hype around a new idea makes many people blind for the traps hidden behind the great potential.
Harvard Business Review has lately published many insights regarding big data in their blog network. The latest take on this hot topic by Kate Crawford discusses the hidden biases in big data. Crawford describes examples where big data originated from consumer mobile devices with social media features contains hidden biases caused by social and ethnographical factors. In such cases the generalization of results evidently leads to misinterpretations even if the data first seems comprehensive.
Hidden biases are not only an issue with big social media data but the same principles apply also to carefully collected corporate data. No matter how lavish your data is, it never makes the detailed understanding of its nature and shape redundant. In fact, quite the opposite. The more excessive your data is, the more accurately tailored your supportive tools need to be.
“As soon as the data was deeply understood, the model producing important predictions was created accordingly and the adaptive tool started to pay off…”
Our team worked with an industrial company that had recently deployed nice mathematical tools to refine their abundant data into demand predictions. These predictive analytics formed the foundation for operational management and decisions. The only issue was that the forecasts were far from accurate which led to poor managerial decisions affecting operational efficiency in over 30 countries.
The data was complete and the tools were based on rigor statistical methods so how could the derived predictions be wrong? Situation was frustrating as those predictions were the only meaningful information regarding successful decision-making.
There was a hidden bias in fundamental assumptions regarding distribution of the data. False assumptions made predictions inaccurate which directly led to uninformed decisions. As soon as the data was deeply understood, the model producing important predictions was created accordingly and the adaptive tool started to pay off. Decisions were supported by accurate analyses.
Specialized business analysts are able to tell you how reliable your data is for different predictive analyses. Once the biases in data have been detected, multi-method approaches are often needed to interpret and analyze the data accurately.
Don’t waste the excessive potential that lies in your business data. Take actions to first understand it thoroughly and build your predictive analytical tools on that. It will pay off.