Big Data The key to happiness lies in managing expectations. I have found this to be true in personal and professional endeavors. From that perspective, choosing a product to buy personally does not differ much from corporate technology or project selection. In both cases we identify the needs or requirements that we perceive to be important. Depending on the economic impact of our decision, the process could be more or less structured. Even impulse selection is based on some expectation of output, though we may not bother to verbalize it. Imagine grabbing a candy at a store’s cash register and experiencing a bitter taste upon unwrapping and putting it in your mouth. The word “candy” created an expectation of sweetness, but the bitter taste created experience of disappointment.

Returning to the world of corporate management, I find it amazing how many professionals do not believe in miracles yet continue to shop for silver bullets. I am referring to shopping for “tools” before the “job” is understood.

Don Peppers wrote a very good summary “Big Data: What Does it Really Mean?” that generated a lot of comments. Some of commentary was whether “big data” is a buzzword or a real “tool”. To all the people who use the term “tool” in the context of “big data” – it is a buzzword, and all they want is to sell and to buy another silver bullet. “Big data” is a concept or strategy.

“Big data refers to things one can do at a large scale that cannot be done at a smaller one.”

Viktor Mayer-Schonberger and Kenneth Cukier

If you purchase a “big data” technology platform or tool without having an existing, or at least formalized, business process that this purchase would enforce, you likely will experience disappointment. Technology and the economics of computing infrastructure make it possible today to correlate multiple sources of data for finding patterns that can provide evidence in support of management decisions. However, once again, technology seems to outpace the ability of organizations to fully adopt results they can generate into their practices. Humans are more comfortable relying on beliefs rather than seriously considering inconvenient evidence.

Ask yourself: “What decision would I make if I knew XYZ?” and “Would I have enough confidence in the information to make a decision if it does not agree with a common perception?”

There are very few established business processes that are structured to take advantage of emerging evidence. The velocity of data outpaces our ability to adapt to change so much that the “big data” buzz will probably go out of fashion before serious management practice changes will take place.