Big data presents unique challenges to agencies

Big data may be on everyone’s lips but as agencies begin to think about how to manage, leverage and store large volumes of information, few have given much thought to actual challenges that may arise

Speaking at a June 21 event on big data held in Washington, Aaron Bird, general manager at Adobe Insight, said agencies grapple with dealing with large amounts of data that are beneficial and interesting when merged into a single platform. The management, sharing and storage of that unsiloed data present the largest roadblocks, he said.

Bird highlighted how the government and industry have different challenges around big data, saying it’s potentially easier to solve big-data problems for commercial-sector customers than federal clients primarily because of access to data.

Federal agencies have to consider security and privacy to a much higher degree, he said.  However, the primary challenge for agencies just launching efforts into big data is how to actually get the data.  Agencies may know from which sources they want to extract data but to actually pull it together often poses another roadblock, Bird said.

Another obstacle is getting multiple agencies to coordinate and communicate with each other, a seemingly tall order, Bird stressed.

“The infrastructure and technology may exist but [agencies] have a different set of challenges in fighting through processes and policies to bring that data together,” he said. “That’s where everyone seems to start, and few agencies have actually successfully gotten past that."

The June 21 panel discussion was part of Federal Computer Week’s Executive Briefings event series.


About the Author

Camille Tuutti is a former FCW staff writer who covered federal oversight and the workforce.

FCW in Print

In the latest issue: Looking back on three decades of big stories in federal IT.


  • Anne Rung -- Commerce Department Photo

    Exit interview with Anne Rung

    The government's departing top acquisition official said she leaves behind a solid foundation on which to build more effective and efficient federal IT.

  • Charles Phalen

    Administration appoints first head of NBIB

    The National Background Investigations Bureau announced the appointment of its first director as the agency prepares to take over processing government background checks.

  • Sen. James Lankford (R-Okla.)

    Senator: Rigid hiring process pushes millennials from federal work

    Sen. James Lankford (R-Okla.) said agencies are missing out on younger workers because of the government's rigidity, particularly its protracted hiring process.

  • FCW @ 30 GPS

    FCW @ 30

    Since 1987, FCW has covered it all -- the major contracts, the disruptive technologies, the picayune scandals and the many, many people who make federal IT function. Here's a look back at six of the most significant stories.

  • Shutterstock image.

    A 'minibus' appropriations package could be in the cards

    A short-term funding bill is expected by Sept. 30 to keep the federal government operating through early December, but after that the options get more complicated.

  • Defense Secretary Ash Carter speaks at the TechCrunch Disrupt conference in San Francisco

    DOD launches new tech hub in Austin

    The DOD is opening a new Defense Innovation Unit Experimental office in Austin, Texas, while Congress debates legislation that could defund DIUx.

Reader comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group