Crowd sourcing (Credits: CoolTown Studios)
What if you could tap the untold reams of data you have floating around your organization to solve a major problem – and you could do it in a fun, low-cost way? If you follow the lead of NetFlix and Kaggle, you can do just that. In an intersection of three of the biggest catch phrases of the moment, companies are using crowdsourcing to analyze big data, generating solutions to their biggest problems via online competitions. By offering a prize for the best solution to a particular challenge, these inventive companies are tapping into some of the best minds the Internet has to offer, without having to hire one person.
As best we can tell, the idea came from the consumer space. In 2009 NetFlix made waves by offering a $1 million prize to whomever could create the best algorithm based on massive amounts of NetFlix subscriber data for recommending movies based on past preferences. Spectators tracked the top teams on the NetFlix leaderboard, with progress shown in real time and participants racing to out-do each other. The competition was fierce right up to the moment that BellKor’s Pragmatic Chaos was awarded the ultimate prize.
Any competition with that kind of prize is likely to generate a lot of buzz and a high number of participants. But most companies could not offer that kind of payout (or, not without going bankrupt). The good news: Based on the experience of Kaggle, your online competition need not offer a six-figure prize in order to get a lot of meaningful participation.
Kaggle is a Silicon Valley startup (tagline: “We’re making data science a sport”) brings together PhDs, data analytics gurus, data scientists, and computer scientists who use analytics to solve complex problems for businesses, governments and basically any other organization that has tons of data. Kaggle invites anyone to go against its elite community of experts to solve a variety of problems, ranging from identifying patients that will be admitted to the hospital in the next year using historical claims data to predicting the online sales of a consumer product based on data about product features.
Unlike the NetFlix prize, these competitions carry more modest awards, with many starting at just $1,000 or so (and some even lower than that). Kaggle takes a portion of the prize money in return for hosting and promoting the competitions. The investment required to launch a competition can be very low, and a plethora of talent (including a lot of recent grads who can’t find jobs) is yours for nothing up front. And you are helping your participants by giving them the potential for resume-building publicity.
If you start small, it’s hard to imagine much of a risk, as long as you pick the right problem and share only data that is not confidential or proprietary. (The latter is a topic for another blog.)
Kaggle says it now has over 50,000 experienced data scientists competing for these prizes. The payoff to the sponsoring company is clear. Every competition run by Kaggle has outperformed the existing best-of-breed predictive model with upwards of a 340% improvement against the benchmark. The ROI includes reduced churn, higher customer satisfaction, increased revenue and the cool factor, in spades.
Big data is one of the biggest problems facing marketers today. It can be notoriously difficult to predict the demands of consumers. There’s no lack of information but they struggle with what to do with it. It is often unclear how to use data to improve their business.
Do you have big data and unsolved problems? If you’re lucky enough to have access to a room full of “quants” to help you explore, visualize and predict the future based on historical data, there are numerous tools that can help you mine mountains of data including SAP Predictive Analysis. If not, you should consider putting crowdsourcing plus gamification plus analytics to work for you. The potential return is without limit. Ask the world to help you solve your thorniest problem – and give them the data to do it – and you just might be surprised at the results that the community open entirely new revenue streams. Think of what you could do if you could remove the barriers that are holding back your business.
Well said!