The other day I read a great article in the UK version of Wired Magazine about how the role of the data scientist is increasing as our species continues to produce more and more data. The article talked about a website called Kaggle that offers cash prizes to people who can come up with solutions to data mining problems. As the website states, "Kaggle is a platform for data prediction competitions that allows organizations to post their data and have it scrutinized by the world's best data scientists. In exchange for a prize, winning competitors provide the algorithms that beat all other methods of solving a data crunching problem…"
Such competitions are becoming more and more prevalent. Netflix used one to create a more accurate way for recommending movies to people and currently there are some hospitals that are searching for a better way to predict which patients are likely to be readmitted. As we continue to use more and more data the need for these types of competitions is likely to continue. And with good reason. According to IBM's website, "Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone."
Obviously I don't have tens of thousands of dollars lying around to offer someone to build an algorithm for me but if I did I would want it to be one that searches for and tracks the best new ideas. This would make it easier for government officials, entrepreneurs, and even regular people like me to find out about what's new and exciting and potentially disruptive. Why does this matter? Well if you work for the government and you could have known about the Arab Spring months before it started wouldn't you have wanted to? If you're a venture capitalist and you could have known about Facebook before anyone else wouldn't you have wanted to?
The way that this algorithm would work is that it would combine data pulled from a variety of sources including blogs, newspapers, magazines, and social media sites like Twitter. The more times a particular topic is written about, linked to, shared, tweeted, liked, etc. the higher it's rating would be. A value could also be assigned to the sources that are discussing a particular topic so that a cover story on Time Magazine holds more weight than a Facebook status update.
It shouldn't be too hard to create this algorithm because Twitter already uses something similar to populate their trending topics list. The only difference between that algorithm and this new ideas finder is that it wouldn't just be limited to one source and it wouldn't just be limited to a particular moment in time. It's not about what's trending right now. It's about finding things that are slowly gaining in popularity over a long period of time.
For example, I previously wrote about an off shore business incubator that will be based on a cruise ship in the Pacific Ocean in an attempt to speed up innovation (http://greatestideaever.wordpress.com/2012/05/15/23-off-shore-incubator/). The other day I saw that somebody else had tweeted about it several weeks after I had written about it. This to me means that this topic is becoming more popular. As more and more people continue to talk about it and link to it it's going to become even more mainstream. Eventually it will get a cover story on Time Magazine. Wouldn't it be great then if there was a way to track these type of ideas as they gain traction? To know about the next big thing before it becomes the next big thing?
Hey cartoon man, finding the best ideas would be a lot easier with the help of an ideas algorithm.
Like this:
Be the first to like this.
No comments:
Post a Comment