To manage big data, go beyond IT

Big Data

Chipping away at big data takes special talents -- and they're not always in the IT department.

If you have a big data program to manage, you need a talented IT person, right?

Not necessarily. Big data brings its own challenges and calls for some very specific skills, according to members of a panel discussion at a Dec. 3 hosted by Nextgov.

Although data has always been around, the amount seen today is what makes management so challenging, said Micheline Casey, principal at CDO, LLC and former chief data officer for the State of Colorado.

“The other thing that’s critical,” she said, “is that we have this nexus of what’s happening with the interconnected systems. The dynamic pace of change of innovation that’s just really requiring people to think proactively and not just reactively about data and how manage data assets.”

Big data is just not about the volume of information, said Jeff Butler, director of research databases at IRS Research, Analysis and Statistics, but the many variations of it that add to the challenge. These new types of data, whether unstructured or textual, are forcing a reevaluation of skills in analyzing those variations, he said.

Also, Butler said, organizations traditionally used to manage data in monthly, quarterly or annual cycles. Today, that tends to be done in real-time, or close to it. The question then for agencies on the federal level, he said, becomes how to adapt to those real-time analytics models?

Today “data just happens” and is generated without much thought, said Michael Rappa, director of the Institute for Advanced Analytics and professor at North Carolina State University. Thirty years ago, organizations collected data with purpose and intent. The process required time, money and energy that needn’t be spent if there was no identified need for the data, he said.

Today, Rappa said, the challenges for those who manage and analyze all this data lies in how to do it in a predictive way and very quickly. “That’s the talent pool that’s not really in place,” he said. “That’s the career track that really doesn’t exist in most agencies just quite yet .”

The right kind of person to take on the challenge is often not an IT person, Rappa stressed. Agencies are now starting to work on figuring out what the operational role looks like, and the “data scientist” job title gets used a lot. However, the term is misleading because the person who handles the big data challenge “is not a scientist and they shouldn’t be totally fixated on data,” Rappa said.

Agencies need to be more data-centric and think of data upfront, not just as an afterthought, Casey said.

“Building a culture around data is very different from the culture virtually all state and federal agencies and commercial sector business have today, quite frankly,” she said. “It doesn’t matter if you get the best data scientists in a room; if no one across the organization knows what to do with the data, the insight they come up with doesn’t matter. Data is a business issue, not an IT issue.”

Agencies also need to create an organizational capacity to take in, consume and distribute data across their ecosystem to make use of the insights data scientists provide, Casey suggested.

“Otherwise it’s not doing anyone any good -- having a room full of data scientists and Hadoop clusters,” she said.

About the Author

Camille Tuutti is a former FCW staff writer who covered federal oversight and the workforce.


    sensor network (agsandrew/

    Are agencies really ready for EIS?

    The telecom contract has the potential to reinvent IT infrastructure, but finding the bandwidth to take full advantage could prove difficult.

  • People
    Dave Powner, GAO

    Dave Powner audits the state of federal IT

    The GAO director of information technology issues is leaving government after 16 years. On his way out the door, Dave Powner details how far govtech has come in the past two decades and flags the most critical issues he sees facing federal IT leaders.

  • FCW Illustration.  Original Images: Shutterstock, Airbnb

    Should federal contracting be more like Airbnb?

    Steve Kelman believes a lighter touch and a bit more trust could transform today's compliance culture.

Stay Connected

FCW Update

Sign up for our newsletter.

I agree to this site's Privacy Policy.