Big Data, Big Problems

Brian Wood Blog

Caron Carlson of FierceCIO noted the article below.

Clearly there is a big opportunity for companies to get their arms around big data.

Emphasis in red added by me.

Brian Wood, VP Marketing


Study: Most big data projects are never completed

Scope of big data initiatives is often underestimated

Big data is often a big hassle, and new research shows that most of the projects that are initiated are never completedIn many cases, the scope of a big data project is underestimated and too few resources are allocated, reports Dennis McCafferty at CIO Insight.

Adding to the trouble, according to a study by Infochimps, the people actually implementing big data projects aren’t always consulted early on. A lack of cooperation among departments involved in the project is a major impediment to completing it, said 39 percent of the IT professionals surveyed by Infochimps. Technical stumbling blocks were cited as a big problem by 41 percent of the participants, and a failure to accurately anticipate the project’s scope was cited by 58 percent.

The hardest part of a big data project is the processing of data, according to 43 percent of the survey respondents. But managing the data and analyzing it were considered the most difficult part by 42 percent and 41 percent, respectively.

Four-fifths of the IT pros said that their companies have a hard time finding the right expertise, and 76 percent said they have trouble finding the right tools.


Big Problems for Big Data

Memo to CIOs: Your IT teams really want to help you tackle big data.

After all, such projects serve an essential purpose in supporting organizational goals.

And, yes, knocking this kind of undertaking out of the park looks great on your resume. But, unfortunately, most of these big data efforts never get completed, thanks to a myriad of unresolved issues.

For starters, companies too often fail to accurately determine the project’s scope, and leave CIOs and IT departments with inadequate resources and staffing to get the job done.

Often, those charged with implementation are the last consulted,” says Jim Kaskade, CEO of Infochimps, which helps companies deploy big data environments in public, private and virtual private clouds.

“CIOs need more insight into the too-often overlooked views of those charged with the heavy lifting, and companies need to start with the business problem first to properly scope their projects.”

More than 300 IT department staffers–most of whom have a big data project currently underway–took part in the research.

To call up an infographic on the survey, click here