Expert Column: Big Data Hype Yields Real Results with Advanced Analytics

There’s been plenty of buzz about big data: the analysis of information that is too large to store on a single computer. MIT’s Sloan Management Review calls big data analytics a “mandate.” Five years ago a McKinsey article referred to big data as “the next frontier for innovation and competition,” and Kroger Chairman David Dillon referred to data analytics as the retailer's “secret weapon.” Big data has been hailed as a revolution, yet 70 percent of organizations are not using big data at all. What gives?

PG readers with long memories may recall another so-called revolution: the arrival of a technological breakthrough that promised to transform the shopping experience for consumers, provide retailers and suppliers with invaluable insights to optimize the supply chain, and inform collaborative planning. It promised to save you money and make you money. Like big data, this revolution languished for years and was labeled a “failure.” The revolution? Grocery store scanners. The time? The 1970s! With all the big data hype, it’s easy to forget that grocery experienced a big data revolution 40 years ago. A second revolution is on its way.

There’s a reason for the chatter. The motivation behind big data is simple: bringing more data to bear on an analytical problem leads to better insights and better decisions. Big data analytics has already transformed several industries: web search (Google), shipping logistics (UPS) and social media (Facebook). At this point, nearly any online interaction is driven in some way by big data analytics, whether it’s searching, purchasing or interacting.

So why is big data stuck in what Gartner calls the “Trough of Disappointment?" It’s too complicated for the rest of us. After all, Google, UPS and Facebook are tough acts to follow. They’re enterprises with high degrees of software and analytics acumen, and their successes are the result of years (sometimes decades) of sustained investment. This expertise has been critical because of the relative immaturity of big data tools combined with the need for well-organized data. Deploying and maintaining big data platforms such as Hadoop is complicated, and in the end, Hadoop is very far removed from the actual business problems retailers and vendors seek to solve. (If you don’t know what Hadoop is, you’ve made my point.)

Online businesses have distinct advantages because by definition they need to have well-developed technology capabilities. For these reasons, many “fast followers” have struggled with big data adoption. We haven’t heard much about these struggles until recently because as with many would-be revolutions, those shouting into the megaphone haven’t necessarily been the on the front lines of the struggle. Big data’s most prominent boosters have been enterprise software companies, consultants without retail experience, and technology bloggers. They’re “staggered” by Uber’s estimated $10 billion annual revenue but seem to be unaware that U.S. grocery revenue is 60 times greater.

Nevertheless, big data brings clear opportunities for retailers and their suppliers. In the world of retail, big data frequently means moving from store-level data to transactional data such as receipts. Loyalty programs are a classic example. Understanding purchasing behavior at an individual level helps grocers understand how to provide a more relevant, personalized experience to shoppers by having the right products on shelves, offer compelling products that shoppers want, and become more responsive to change. Big data technology can complement a people-centered approach to grocery retailing by focusing on individuals rather than segments or divisions.

The good news is that past disappointments are being addressed. Big data and analytics tools are maturing in their breadth and ease-of-use. Apache Spark, MicroStrategy and Tableau are but three examples. Organizations are increasingly treating data as a valued asset, better positioning them to ask and answer relevant business questions. Finally, we’re learning that big data is not the point, insight is. Data-driven insight is sometimes driven by big data, and other times by “small” data such as store sales, weather or surveys. The common thread is advanced analytics of the predictive (forward-looking), or prescriptive (decision-based) varieties.

For example, sales forecasting that accounts for the realities of retail and supply chain can be used to inform purchasing decisions, plan promotions, reduce out-of-stock and spoilage, and enable true collaboration between suppliers and retailers. The key is to structure internal and external data with predictive models, packaged in a way that addresses real business problems. Data can run the gamut from planograms, receipts, loyalty data, weather, deliveries and online reviews. Analytics can come in the form of time series analysis, text mining, forecasting, optimization or even artificial intelligence. Whatever the specifics may be, big data platforms are better equipped than ever to help marshal data and analytics to solve real business problems, creating real opportunity for most industries, including grocery. Hype is slowly giving way to real results.