Twitter – Crystal Ball Into the Future?

What if you had a crystal ball that could predict the rise and fall of the stock market? Or that predicted which candidate would win the election and thus if your taxes would rise or fall? It’s hard to predict both the vagaries of financial markets and the winner of a tight election race. Hundreds of thousands of people determine the result in both case, and statisticians try to measure the direction by taking sample polls. Now analysts have a far better tool, a crystal ball really, to predict how masses of humanity will behave within the near future. It’s called Twitter.

People like to express their emotions on Twitter, and decisions that are made by large numbers of people are more often driven by emotions than deep analysis. Emotions by nature are often irrational making them difficult to predict. But people’s tweets expose the temperature of their emotions, allowing analysts to make an assessment on the trend in aggregate. This is exactly what researchers Johan Bollen and Huina Mao at the Indiana University-Bloomington did to predict movements in the Dow Jones Industrial Average (DJIA). They discovered that when people’s emotions were calm (as identified by a set of words that express calmness), the stock market remained fairly stable for the next week. The power of their predictive algorithm trained on 9.8 tweets by 2.7 million people from February and December 2009: a staggering 87.7% in accurately describing the state of the DJIA for the next six days. Suffice it to say that since Bollen and Mao announced this result last week many hedge funds and banks are running to create algorithms that will mine Twitter data.

Twitter has been getting a great deal of attention lately in its ability to chart trends, including an HP study that predicted which movies would be blockbusters by analyzing tweets. The study’s algorithm was able to predict the opening weekend box-office performance of 24 films with 97% accuracy. Another study conducted by Tweetminister predicted which candidates in the UK elections would win their seats based on how many times the candidate was mentioned on Twitter. They showed that national predictions based on their twitter algorithm were 90.5% accurate.

The potential for mining Twitter data is unlimited: books, videos, music, to name just a few. Their marketers will all be interested in the crystal ball of Twitter to predict irrational human emotions.

Ayesha and Parag Khanna explore human-technology co-evolution and its implications for society, business and politics at The Hybrid Reality Institute.