“Big data” analytics is hot. Read any IT publication or website and you’ll see business intelligence (BI) vendors and their systems integration partners pitching products and services to help organizations implement and manage big data analytics systems. The ads and the big data analytics press releases and case studies that vendors are rushing out might make you think it’s easy -- that all you need for a successful deployment is a particular technology.
If only it were that simple. While BI vendors are happy to tell you about their customers who are successfully leveraging big data for analytics uses, they’re not so quick to discuss those who have failed. There are many potential reasons why big data analytics projects fall short of their goals and expectations. You can find lots of advice on big data analytics best practices; below are some worst practices for big data analytics programs so you know what to avoid.
“If we build, it they will come.” This repeats the classic mistake made when organizations develop their first data warehousing or BI system. Too often, IT as well as BI and analytics program managers get sold on the technology hype and forget that business value is their first priority; data analysis technology is simply a tool used to generate that value. Instead of blindly adopting and deploying something, big data analytics proponents first need to determine the business purposes that would be served by the technology in order to establish a business case -- and then choose and implement the right analytics tools for the job at hand. Without a solid understanding of business requirements, the danger is that project teams will end up creating a big data disk farm that really isn’t worth anything to the organization, earning them an unwanted spot in the “data doghouse.”
More on managing big data analytics programs
Find out how big data analytics can help organizations fulfill the promise of predictive BI
Learn about the differences between big data analytics and big data storage
Get tips from consultants and vendors in our collection of big data analytics videos
Assuming that the software will have all the answers. Building an analytics system, especially one involving big data, can be complex and resource-intensive. As a result, many organizations hope the software they deploy will be a silver bullet that magically does it all for them. People should know better, of course -- but still they have hope. Software does help, sometimes dramatically. But big data analytics is only as good as the data being analyzed and the analytical skills of those using the tools.
Not understanding that you need to think differently. Often, people keep trying what has worked for them in the past, even when confronted with a different situation. In the case of big data, some organizations assume that “big” just means more transactions and large data volumes. It may, but many big data analytics initiatives involve unstructured and semi-structured information that needs to be managed and analyzed in fundamentally different ways than is the case with the structured data in enterprise applications and data warehouses. As a result, new methods and tools might be required to capture, cleanse, store, integrate and access at least some of your big data.
Forgetting all the lessons of the past. Sometimes enterprises go to the other extreme and think that everything is different with big data and they have to start from scratch. This mistake can be even more fatal to a big data analytics project’s success than thinking that nothing is different. Just because the data you’re looking to analyze is structured differently doesn’t mean the fundamental laws of data management have been rewritten.
Not having the requisite business and analytical expertise. A corollary to the misconception that the technology can do it all is the belief that all you need are IT staffers to implement big data analytics software. First, in keeping with the theme of generating business value mentioned above, an effective big data analytics program has to incorporate extensive business and industry knowledge into both the system design stage and ongoing operations. Second, many organizations underestimate the extent of analytical skills that are needed. If big data analysis is only about building reports and dashboards, enterprises likely can leverage their existing BI expertise. However, big data analytics typically involves more advanced processes, such as data mining and predictive analytics. That requires analytics professionals with statistical, actuarial and other sophisticated skills, which might mean new hiring for organizations that are making their first forays into advanced analytics.
Big data analytics is only as good as the data being analyzed and the analytical skills of those using the tools.
Treating the project like it’s a science experiment. Too often, companies measure the success of big data analytics programs merely by the fact that data is being collected and then analyzed. In reality, collecting and analyzing the data is just the beginning. Analytics only produces business value if it is incorporated into business processes, enabling business managers and users to act upon the findings to improve organizational performance and results. To be truly effective, an analytics program also needs to include a feedback loop for communicating the success of actions taken as a result of analytical findings, followed by a refinement of the analytical models based on the business results.
Promising and trying to do too much. Many big data analytics projects fall into a big trap: Proponents oversell how fast they can deploy the systems and how significant the business benefits will be. Over-promising and under-delivering is the surest way to get the business to walk away from any technology, and it often sets back the use of the particular technology within an organization for a long time -- even if many other enterprises are achieving success. In addition, when you set expectations that the benefits will come easily and quickly, business executives have a tendency to underestimate the required level of involvement and commitment. And when a sufficient resource commitment isn’t there, the expected benefits usually don’t come easily or quickly -- and the project is labeled as a failure.
Big data analytics can produce significant business value for an organization, but it also can go horribly wrong if you aren’t careful and don’t learn from the mistakes made by other companies. Don’t be the next poster child for how not to manage a big data analytics deployment.
ABOUT THE AUTHOR
Rick Sherman is the founder of Athena IT Solutions, which provides consulting, training and vendor services on business intelligence, data integration and data warehousing. Sherman has written more than 100 articles and spoken at dozens of events and webinars; he also is an adjunct faculty member at Northeastern University’s Graduate School of Engineering. He blogs at The Data Doghouse and can be reached at email@example.com.
This was first published in March 2012