X-Factor + Twitter = predictable odds?

I’m not ashamed to admit that a couple of weeks ago I spent my Saturday evening with my girlfriend, nachos, a comfortable sofa and X-factor (equivalent to ‘American Idol’ in the U.S.). 

I am a person who gets very excited every time he realises that life has been changed by technology – Saturday night was one of those times!  Whilst I have spent many a Saturday night in the past watching one of X-factor or one of the other incarnations of the original Pop Idle TV show, this year is different.  Not I only was I having the usual debate with my girlfriend about who could and could not sing, but we were also both monitoring my Twitter and Facebook feeds and discussing what our friends (and some strangers) thought about each act.

Every time I posted a status up on Facebook account I had a response within seconds, and several responses within a few minutes, each with a different reaction and opinion from one of my friends.  Whilst I would not advocate replacing a night in with friends in person with Facebook conversations, it is a great way to share your thoughts with more friends than you can fit in your living room (which is not many in my case).

My set of Twitter friends/followers is more focussed on work, so there was not too much X-factor chatter going on within my network.  However, the power of Twitter comes from monitoring and engaging with a set of strangers who have a common interest (X-Factor being the common interest in this case).  I set up my TweetDeck Twitter client to monitor the #xfactor hash tag, which resulted in a literally hundreds of comments per minutes (more than the client could handle in fact) from people publically expressing their thoughts on the acts before they had even finished the song.  It was strangely entertaining to hear strangers thoughts on the acts, although I personally limited my posts to Facebook.

X-Factor is for the most part a competition for which the outcome is driven by a public vote.  Each week the public phone/text their votes for the act they wish to ‘save’, which results in a final 2 who have a ‘sing off’ competition on which the judges base their decision on who to send home.  Assuming the show works the same as in previous years, the final winner will be decided purely on a public vote.

So whilst the judges make a decision on the final 2 contestants each week, the public vote is a key factor in deciding who makes it through the process without having to sing for their survival.

The guys who make the odds

This made me wonder about betting companies and how they are impacted by Facebook, Twitter and other real time data sources.  A quick search on Bing shows numerous companies who offer betting odds on all of the x-factor contestants…

Whilst I am not overly familiar with the processes these companies use to maintain the list of odds for each contestant, I have now doubt that it is based on a complicated set of formulas which consume data from experts, TV watching patterns, phone voting patterns and many more factors.  But this year, perhaps for the first time, Twitter and Facebook (and perhaps other) feeds are (or at least should be) an essential part of the analysis when calculating the odds.

Whilst Facebook status updates are often only available within a connected private network of friends, Twitter updates are public and available for everyone to read and subscribe to.  Twitter have a public API which allows anyone to extract data from the service which can be used to ‘monitor’ real time information about a specific topic.  The API does have a usage cap for standard users, although many companies also have special agreements which allows them to pull data at a faster rate.

‘Listening’ to social networks

There are plenty of examples of services available which can ‘read’ text, extract sentiment and produce actionable summaries of user intent from large quantities of user generated text.  I have seen algorithms which can tell you that 78% of X factor tweets over the past week have been discussing ‘Danyl’, 76% of those tweets before this weeks results were ‘positive’ and ‘excited’, whereas 89% of the ‘Danyl’ tweets after the result were ‘angry’ or ‘shocked’.  OK, I made those numbers up, however technology is absolutely capable of generating those types of summaries with enough data, and a good set of seed tweets.

I have no doubt that the betting companies have teams full of people who develop incredibly complex mathematical models to generate betting odds, I wonder how they are making use of Twitter data?  I also wonder whether this Twitter have realised that and made it part of their revenue strategy!?  A few sets of example correlation data and some fancy graphs would make the Twitter sales team’s job pretty easy,when they head in to the bookies’ offices to get them to buy in to advanced integration with the Twitter API.  I would not be surprised if there was a top secret secure access version of Twitter which provided customised analysis of selected data sets for companies who will pay enough (and the government).

Anyone can do it

Another reason which makes it absolutely essential for the betting companies to use Twitter data in their calculations is the fact that it is all available publically.  It is relatively easy for reasonably smart people to create a custom feed of public data, store it, analyse it and use it to make statistically sound decisions.  With so much power in the hands of consumers, the guys who are coming up with the betting odds can not be complacent and continue to innovate and stay ahead.

And it’s getting easier!  We are getting to the stage where the people placing the bets can perform a reasonably decent analysis of the data without the need for sentiment analysis, statistical models or even downloading the data.  The recent addition of real time Twitter data to Bing marks a new evolution in real time search functionality.  Bing allows users to search the public Twitter feed, view trending topics, group tweets by topic and view a live feed based on a specified search string.  In the run up to the X-factor results show, the Bing Twitter trending topics provided a clear indicator that ‘Danyl’ was a popular discussion topic…

A quick search on the Bing Twitter site also shows some insights in to what the ‘popular’ topics/links are in relation to ‘Danyl’…

In fact, even the reactions to the results may serve as insights to how the following week’s vote will go.  This week, the two acts who ended up in the bottom two were a surprise and were certainly not the two worst acts (in my opinion).  Danyl had to sing in the final show down, but was voted to stay by the judges.  The responses on Twitter suggest that the public will not assume he is safe next week, so it is unlikely (in my opinion) that he will end up in the final 2 again soon, unless he performs really badly of course…

Conclusion

Over the past 3 or 4 years, the evolution of wide spread global social networking means that bookies, TV shows any more other companies need to keep up with technology and monitor online user behaviour in a way which was not necessary before. 

A 12 year old script kiddie in his bedroom, with a basic set of web programming skills, has the power to produce real time analysis of current affairs, the type of capability only associated with secret big brother government agencies less than 10 years ago (or at least those depicted in Will Smith movies).  I can’t help but wonder whether betting agencies have their own secret bunkers (ok perhaps not), with ex-hackers monitoring the social networks and feeding key data in to the guys coming up with the odds for X-factor.  I also wonder how long it will be before Will Smith is asked to appear in a film about it.

Is your business listening to social networks enough?  Are some of your customers more aware of the chatter about your products on social networks than you are?  And what about your competitors?. . .