Big data's big hurdle: federal policy

adrift in a sea of data

Big data has tremendous potential to help agencies glean insights from a sea of information, but sometimes policy stands in the way. (Stock image)

Big data has all the promise in the world to help agencies glean otherwise impossible insights from massive repositories of data, but some believe federal policies thus far have blunted big data’s potential in the public sector.

"Policy questions are always a potential derailer," said Josh Sullivan, a data scientist and vice president at Booz Allen Hamilton.

"We’ve worked with a number of big agencies on practical pilots and nationally enterprised adoptions," Sullivan told FCW. "Inevitably, security policy, data protection policies and sharing policies were all antibodies to all the technology."

Sullivan said that some intelligence agencies have exceled out of necessity at finding innovative solutions in big data that navigate policy, developing mature big data models that combine IT efficiencies, "smart" data and cloud analytics to create a portfolio on which data-driven decisions are made.

But most other federal agencies, Sullivan said, are content to reach the point at which they’ve jumped into the cloud and virtualized some of their IT infrastructure.

They are reacting to the big data buzz with outdated, old-hat polices rather than proactively embracing potentially huge financial gains and insights through policy that outlines questions that big data can actually answer.

"Until people start saying big analytics instead of big data, I don’t know if we’ll see the government realizing all the latent patterns there for us to discover," Sullivan said. Agency executives can start framing outlines by "writing down 10 game-changing questions you cannot answer today that would substantially move the needle on your mission," he added.

Technology changes fast, said Mark Forman, former administrator for e-government and IT at the Office of Management and Budget, but said broader federal polices aren’t changing with them.

"Laws relating to privacy haven’t kept up with the changes in data as a result of social media and the Internet," said Forman, who believes decision makers will one day base major decisions on insights from structured and unstructured big data.

Forman said the Privacy Act, enacted in 1974 to keep the government from using personal citizen data for reasons not in the public’s best interest, could stand a 21st century refresh.

Some, he said, thought the Bush-era Patriot Act might be the catalyst for such a change, but the controversy over the counterterrorism measure eventually quieted.

Yet big data, and particularly the ability to collect huge bits of unstructured data, has changed how the government gathers information. What used to be a linear approach to information gathering is now a "topsy-turvy" approach where mined data can infringe on existing privacy laws.

"It’s hard to scrape off privacy-related information," Forman said. "There is a lot of research going on right now in understanding how we modify privacy laws and paperwork burden laws in a totally different era, and not just in the United States. The question is, what is the catalyst for this?"

Aubrey Vaughan, managing director of the public sector at Oversight Systems, a big data analytics software provider, said unfortunately no "silver bullet exists" to solve the policy dilemma short of a group of feds championing the effort.

"I think we are in the infancy stages of big data, a lot of agencies are being told a lot of different things about big data, and it is a buzzword," Vaughan said. "Some agencies are putting the emphasis now on analytics, and efficiencies are driving it more to the surface and creating more groundswell in the government to do more now."

About the Author

Frank Konkel is a former staff writer for FCW.

The Fed 100

Save the date for 28th annual Federal 100 Awards Gala.


  • computer network

    How Einstein changes the way government does business

    The Department of Commerce is revising its confidentiality agreement for statistical data survey respondents to reflect the fact that the Department of Homeland Security could see some of that data if it is captured by the Einstein system.

  • Defense Secretary Jim Mattis. Army photo by Monica King. Jan. 26, 2017.

    Mattis mulls consolidation in IT, cyber

    In a Feb. 17 memo, Defense Secretary Jim Mattis told senior leadership to establish teams to look for duplication across the armed services in business operations, including in IT and cybersecurity.

  • Image from Shutterstock.com

    DHS vague on rules for election aid, say states

    State election officials had more questions than answers after a Department of Homeland Security presentation on the designation of election systems as critical U.S. infrastructure.

  • Org Chart Stock Art - Shutterstock

    How the hiring freeze targets millennials

    The government desperately needs younger talent to replace an aging workforce, and experts say that a freeze on hiring doesn't help.

  • Shutterstock image: healthcare digital interface.

    VA moves ahead with homegrown scheduling IT

    The Department of Veterans Affairs will test an internally developed scheduling module at primary care sites nationwide to see if it's ready to service the entire agency.

  • Shutterstock images (honglouwawa & 0beron): Bitcoin image overlay replaced with a dollar sign on a hardware circuit.

    MGT Act poised for a comeback

    After missing in the last Congress, drafters of a bill to encourage cloud adoption are looking for a new plan.

Reader comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group