Big Data in the New Age of Real-time Retail

Guest Post by Patrick Carter, Founder and President, Digital Folio, Inc.

Digital Folio provides a next-generation retail competitive intelligence network that leverages the power of the crowd to deliver near real-time retail product pricing and availability data to help businesses analyze and solve product pricing and positioning issues.  For more information visit:  www.retailapi.com

The Biggest Retail Big Data opportunity exists OUTSIDE your own retail operations – and it’s NOT Social.

Most Retail Big Data articles address the data opportunity within a retailer’s own sales and supply chain or via leveraging social data or the like. At Digital Folio we believe an even greater, fundamental Big Data opportunity to drive significant ROI lies in having a very granular, real-time picture of what the retail market and your competitors are doing. Acquiring this knowledge, however, requires a re-invention of how you gather market intelligence.

Retail is real-time and volatility is increasing dramatically.

As the web and mobile have grown, retailers have made information more transparent in return for far greater consumer reach and consideration to drive sales. Ironically consumers were the first to embrace this new transparency through comparison shopping applications.

As a result, consumers are shopping and making decisions in real time – not based on week-old prices or circulars – and they are never going back. In an effort to keep up, retailers are deploying increasingly complex re-pricing platforms that allow them to change prices up to every six seconds. The result: dynamic, intraday price changes for some items as often as seven times in four hours. This new age of real-time retail with such a vast amount of constantly changing market data has been impossible for most to manage.

Moreover, online information has gotten more and more detailed as its expanded to include not only price but store availability, shipping intervals, etc. And the permutations have gotten more complex – including the growth of zip/regional pricing (Websites Vary Prices, Deals Based on Users' Information) and even pricing based on the browser or platform a shopper is using

Reinventing Market Intelligence

All of this represents an incredible amount of retail Big Data that, if effectively harnessed, would allow retailers to know the patterns around when a competitor changes a price, when a competitor goes out of stock on a ‘must-win’ item, and how long it typically takes a competitor to replenish. All this and more is now exposed on the web – and, increasingly, at a store/zip level – as retailers strive to inform shoppers of what is near them, at what price, and whether it is available in-store/pick-up, ship-to-store, or for same-day delivery.

The problem is that the rate of pricing/product change and numbers of data/channel permutations have completely outstripped current approaches to try and simply “crawl” the web and get a static “sample” of market prices.

Digital Folio: A Level II Trading System for Retail – Powered by Microsoft Windows Azure

Unlike the stock market, there is no real-time data resource for the retail marketplace. Retailers effectively execute dynamic pricing platforms for a large part in the blind, with most pricing decisions based on historical market snapshots or isolated business rules.

Stock brokerage houses don’t play the market with last week’s newspaper – they use a real-time Level II Trading System that helps gauge each market opportunity and drive yield from it. Digital Folio is the first Level II Trading System for Retail – empowering retailers and consumers alike.

The industry’s first next-generation retail competitive intelligence network, Digital Folio delivers the retail market's most current and complete product pricing and availability data to help retailers solve product pricing and positioning issues. At the heart of the Digital Folio platform is a proprietary, crowd-sourced price/product accuracy architecture that can leverage the power of thousands of shoppers across the web to automatically detect price fluctuations. The entire platform runs in real time on Windows Azure and provides the unprecedented capability to update 10,000 products a second as well as manage a database of over 5 billion data points. As a result, Digital Folio offers unmatched, real-time data accuracy, recency, and breadth – including the unique ability to detect zip code-level and device/browser-level retail pricing variations.
Detecting, tracking, normalizing, and maintaining real-time retail changes on such a massive scale required taking an entirely different approach to seeing and solving the problem as well as resulted in several patent-pending Digital Folio innovations.

Digital Folio is part of Microsoft’s BizSpark One program for high-potential start-ups and has worked closely with Microsoft and the Microsoft Windows Azure team in developing a platform with significant scalability, extensibility, and high-performance as part of a real-time 24/7 operation.

Success Lies in Embracing the Opportunity vs. Fearing the Transparency

Chaos breeds opportunity. The new real-time retail age provides an unprecedented retail marketplace Big Data opportunity to succeed for those who embrace it. Just as digging beneath the surface of dynamic pricing parking meters moves one past the fear of higher-priced parking and into innovations that can reduce traffic and emissions and deliver a more informed, faster, and better parking experience through integrated parking guidance applications, true innovation in retail will result from truly examining data and acting on the opportunities it presents on a regular basis. All in real time, of course.