Does your agency need big data? Maybe not.

Big Data word graphic

Although it seems to be everywhere, big data is not something that every agency needs to worry about. What makes the difference?

If you don't have the tools to turn big data into useful information, you need them, right? Maybe not. While big data tools have proven to be useful in applications such as weather forecasting and continuous monitoring of government financial transactions, smaller agencies would be wise to evaluate their missions before jumping on board the big data bandwagon, according to Bill Franks, chief analytics officer for Teradata's global alliance programs.

At a recent Washington conference, Franks delivered an argument called "Cutting Through the Hype: What you Really Need to Know About Big Data," in which he explained big data is not necessarily about the velocity, variety, veracity or volume of data – it's about uncovering previously unknown value in that information.

"It's not just about more data, it's not even that big data is fast, often it's new information you didn't have before," said Franks, who also authored a book on big data in 2012 called "Taming the Big Data Tidal Wave."

The definition of big data? "Who cares? It's what you're doing with it," Franks said.

Frank said big data is all around us – from the ads that promulgate our Internet experiences to the logs that document our movements on body-controlled video games – but stressed to FCW that federal executives "should let their missions drive" whether they pursue big data solutions or not.

"Large agencies that produce vast amounts of information obviously have to consider big data, but smaller agencies that deal with much smaller amounts of information, what would the real value be there?" Franks said. "Is it going to uncover something they didn't have before that brings value?"

Big data is a trending technology, said David Powner, director of IT management issues at the Government Accountability Office, but agencies shouldn't use technology for its sake alone.

"Technology is a means to advance mission," Powner said. "Agencies aren't using cloud because it is the in thing, they're using it to improve their missions. With big data, the question should be, 'What are you doing to do with it, does it fit your mission, and are you mashing it together as an extension of your mission.'"

Down the line, though, insights from piles of unstructured and structured data at most or all agencies might well drive federal policy, performance management and budgeting.

Mark Forman, former administrator for e-government and IT at the Office of Management and Budget, said that in today's reality, big data probably is not needed by every agency. But as the technology continues to improve and more potentially valuable insights emerge, Forman said the question will change.

"The issue then is not do you need big data or not, it is whether or not key decision-makers are going to use information or insights in making decisions," Forman said. "Those of us ... who believe in good policy analysis do prescribe using better and better data to make decisions, but there continues to be a group of people who base policy analysis on anecdotes and political will – they simply don't use data to make informed decisions."

All the data in the world, he added, does not fix that.

Note: This article was updated to clarify a description of Bill Frank's statement on the true value of big data.

About the Author

Frank Konkel is a former staff writer for FCW.

FCW in Print

In the latest issue: Looking back on three decades of big stories in federal IT.


  • Anne Rung -- Commerce Department Photo

    Exit interview with Anne Rung

    The government's departing top acquisition official said she leaves behind a solid foundation on which to build more effective and efficient federal IT.

  • Charles Phalen

    Administration appoints first head of NBIB

    The National Background Investigations Bureau announced the appointment of its first director as the agency prepares to take over processing government background checks.

  • Sen. James Lankford (R-Okla.)

    Senator: Rigid hiring process pushes millennials from federal work

    Sen. James Lankford (R-Okla.) said agencies are missing out on younger workers because of the government's rigidity, particularly its protracted hiring process.

  • FCW @ 30 GPS

    FCW @ 30

    Since 1987, FCW has covered it all -- the major contracts, the disruptive technologies, the picayune scandals and the many, many people who make federal IT function. Here's a look back at six of the most significant stories.

  • Shutterstock image.

    A 'minibus' appropriations package could be in the cards

    A short-term funding bill is expected by Sept. 30 to keep the federal government operating through early December, but after that the options get more complicated.

  • Defense Secretary Ash Carter speaks at the TechCrunch Disrupt conference in San Francisco

    DOD launches new tech hub in Austin

    The DOD is opening a new Defense Innovation Unit Experimental office in Austin, Texas, while Congress debates legislation that could defund DIUx.

Reader comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group