Why Big Data is not the (basis of) solution?

Why Big Data is not the (basis of) solution?
Every company’s marketing department is looking for the ways to better target and profile it’s customers and prospects. In recent years so many of initiatives have launched to collect, store and analyze Big Data to achieve higher targeted marketing objectives, yet it seems most of these projects and efforts do not live up to the expectations. Why is it so? What is missing here if the data is readily available?


Well, I believe the answer is relatively simple, but the actual solution not so much. Having data is one, important, element of driving a targeted marketing approach, but knowing how to use data properly is the real key to nailing this concept. Such know-how on data usage can be broken down into 2 domains:

  1. Analytics – how to properly analyze and present data
  2. Connection to the core business – knowing how to use all that comes out of Analytics

The cost of data collection and storage have rapidly decreased in the last decade. This presents an opportunity to gather information about customers and markets which was not readily available before. Nevertheless, this revolution is driven by technology and not business. Until these technological advancements are embedded right into core activities, it remains technology for the sake of technology. A concept which has difficulties driving the bottom line.

I want to drill down a bit into what are some key elements and concepts driving the above mentioned 2 pillars, so that Big Data (or actually any sort of data) can drive revenue.

First of all, with or without Big Data technologies at your fingertips you, or someone on your team needs to be able to do proper analytics. The whole point here is that it does not matter how much data you have, if you are unable to derive (and present) real insights from it, it is all no use.

  • Do not forget the rules of applied statistics! As a blog post on Simply Statistics blog states well, there are some procedure and issues statisticians face for quite some time, from before Big Data era. This includes controlling for outliers, externalies goes all the way to reproducibility.
  • analytics,talent gapHave managers who are able to challenge analysts. Analytics is a (painfully) iterative process and I have seen so many times that we just believe the analyst without questioning the results. Sloppy analytics in many cases is the result of having no challenger of the results. And yet again, it does not matter what (or how much) data we are talking about. Also it is noteworthy that the biggest talent gap concerning analytics won’t be the analysts themselves, but the managers who can interpret and use these results.

The other burning issue is the ability to connect analytics and business. Ask yourself: how many useless reports, data visualizations and analytics studies have you seen? I bet most of you know the pain. Doing analytics and big data for the sake of producing amazing models is a fun, but costly hobby, so you need to see to a few points to get benefit out of them.

  1. Analytics should challenge the status quo. If you are doing the project only to cement in the already existing routine then it is simply a waste of money. Measurements and reporting is important, but analytics need to produce recommendations for change. Without that you will only have some fancy reports and no business values.
  2. Do the analysis only if you are willing to change. A test or a new predictive model is only as good as much as you can use it. You need to make sure at the beginning of an analytical project that you are willing to change behavior or methods according the the results or don’t even start. You may decide that you want to know your customers’ behavior better, but how good is that if you are not adapting your marketing campaigns (or entire communication approach) to the findings of the study?
  3. Plan out change management. It is awesome that you are already challenging ways of working and producing analytics with the intention of actually applying the results, but no one in an organization lives on an isolated island. You need to have a game plan and get buy in from other departments. Reorganizing processes, organizational structures and policies are only parts of this game, but what really matters is that you need others to believe in the results of the study. You need to go out to challenge, convince and sell like with any other proposal for change.
  4. Have the analytics department report to business. One definite pitfall is having IT govern analytics. IT’s contribution is crucial, but again you are not doing analytics and Big Data to have technology, but to do business. Recent trends also show analytics as a totally separate function, putting it on the board, but I actually disagree with this approach. Unless your company’s sole business is data and your actual products are analytics all these departments/analysts need to serve a business object, thus the responsible person for delivering this objective should have the final say. There is an ongoing discussion on how to insert analytics into the organizational structure, but what is important that they answer to the person in charge of delivering the business.

This list might not be comprehensive, but I think it gets my point across: you need more than technology or you might even not need much of technology to thrive in the data driven economy. I believe it is more important to get the above points right than to have the latest Hadoop or NoSQL servers in place.

I would wish to hear your take on the above, so please do comment!


Leave a Reply