Apocryphal Now - Don't let the facts get in the way of a good story

Back in the 1980's when I was working in a busy contact lens practice, every few months the phones would ring hot with patients concerned about an article that had appeared in a trade union newsletter. The article told a story about a worker at the Boston shipyards who suffered an arc welding flash while wearing contact lenses resulting in the lenses fusing to his eyes. We all knew that this couldn't happen and calmed the waters each time but despite our best efforts, the article just kept re-appearing quite literally for years. The only thing that changed was the date of the alleged incident.

A few weeks ago I wrote about Artificial Intelligence and cited a couple of examples that will probably end up becoming the apocryphal "poster-child" stories for that industry. Perhaps not unlike the well-known "beer and diaper" story which purports to demonstrate the power of data mining (before it had a name change to big data). The story goes: A retailer found that beer and diaper sales were strongly correlated, thinking that the husbands were picking the diapers up on their way home from work and at the same time grabbing a case of beer. The clever retailer made sure that diapers and beer were co-located and bingo, sales of beer went up.

Unfortunately, big data seems to be going down the same apocryphal track as the Boston shipyard contact lens story.

That doesn't necessarily mean that big data is inherently flawed. Mathematicians have been using data analysis for hundreds of years to explore the relationships between variables, interpret time series and conduct scientific trials. The major change in recent years is not with the statistical interpretation of the data but with the digitising of more and more events so they can be analysed.

However, with more data comes more headaches and higher risk of drawing the wrong conclusion. It is only natural when you see a correlation to think cause and effect. Google Hormone Replacement Therapy (HRT) flawed studies and you'll find pages of problematic analysis and conclusions. Handling data is not child's play and can have serious ramifications when not handled well.

My advice for the average retailer, (now there's a nice statistical term) before embarking on the search for the holy grail, is to focus on the things that matter most; customer service, products, stock, staff training, pricing strategy, the supply chain, the store fit-out and so on. When it comes to analysis, make sure you have a good grip on the basics, Moving Annual Turnover, stock levels, stock turn, GMROI, freshness etc. Identify what works best for your business and plan it accordingly.

Finally, keep big data in perspective. The winning retailers in Australia aren't doing it with big data they are doing it with good old fashioned 4 P's.

We don't need it to tell us that sales are inversely proportional to the number of dead flies in the window or staff not greeting customers.