The dangers of pursuing a data-led strategy led by imperfect people.
It might be time to rethink just how data-led your decision making process should be.
The last decade has seen a drastic shift towards data based decision making. Yet many ‘decision makers’ hail from a time before "big data". The mythical big data revolution was expected to make decision making easier; to signal the end of subjective, fallible opinions. Data analysis would provide the “right” answer, objective and completely devoid of bias. But what should we do if the data doesn’t provide an answer at all?
In the past, you could ask for opinions, weigh them on their merit and decide which path you believed to be best. Sure, this system regularly led to sub-optimal answers, but it always led to answers. Analysis is not usually clear cut, yet we pressure analysts to provide a recommendation. If you push for an answer that isn't there, your likely to get the wrong one. But now this wrong answer is 'supported' by the data and treated as 'fact' in a way that opinions never were.
'Facts' can be dangerous. Data is far from devoid of bias; in most cases the interpretation of data comes from people. Big data has not removed the human element from decision making, it has only shifted the power. Typically shifting responsibility down the chain, to less invested individuals.
The people making key decisions, are usually disconnected from those analysing the data. This is risky given that data can be made to lie. If you give a half decent analyst a big enough database they can prove anything. If I wanted to prove that player engagement is improving, I could. If I wanted to prove that player engagement is falling, I could prove that too, with the exact same set of data.
The ideal of 'data-led decisions' is certainly worth striving for, much like science as a whole it is faultless in theory. But I can almost guarantee that your current set up is far from faultless. Unless you work for a global giant like Google, data corruption is happening at your company, . It may not be biased, or dishonest analysis but, mistakes, data gaps, misinterpretations, communication breakdowns and forced conclusions all lead to bad decisions.
For example, let’s imagine your mobile registration journey is impossible on certain devices. Some customers may decide to register later on a desktop, others will abandon registration altogether. You may ask an analyst to look at how to optimise acquisition. Very few companies have the cross-device tracking required to identify the problem. So based on data alone your analyst may suggest focusing on desktop; as it's the best performing channel. That would be a data-led decision, based on sound data analysis. But you won't find many experienced marketers who thinks it's the right one!
The problem with 'big data' is that it's still relatively new, and in most cases, represents a hypothetical concept. In theory, big data provides a full view of all available information. But it's almost impossible to process all that data without sophisticated machine learning, or Google level resources. Yet, big data is consistently and successfully sold as the answer to all your problems.
It has become a buzz word used to confound and impress, not satisfy. Many "Big Data Specialists" rely on their colleagues lack of understanding, and hide behind complicated sounding phrases like "ETL Process Optimisation" and "Transformation Logic Complexities" to mask their own big data ignorance.
In the meantime, we rely on these imperfect data analysts, to translate imperfect questions, manipulate imperfect data, interpret inconclusive outputs and somehow make perfect recommendations. Data analysis is the best tool at our disposal, but it's far from perfect. Decisions based on data are usually more successful than those based on "feel". But data analysis is a tool and tools can be misused, never forget that.
In the meantime find out how you can improve your data led decision making process HERE.