Big data

Pro tips for using big data

Placeholder Image for Article Template

As federal agencies begin incorporating substantial big-data capabilities into their organizations, they're grappling with some of the nitty-gritty details of how to do it. A couple of pro tips: Focus on the mission, and mix and match information from different sources to find innovative ways to use data.

Shawn Kingsberry, CIO at the Recovery Accountability and Transparency Board, advised federal agencies to "ignore the buzzword soup" of technology and focus on what they want to achieve with big-data applications.

"Technology can divert attention from the business needs," he said during a big-data conference in Washington on Feb. 25. "At the end of the day, you know the problem you're trying to solve, but sometimes we can't focus on that because we're worried about the latest buzzword."

A tight focus can lead to innovative thinking that yields useful big-data solutions without spending money on new technology. For instance, Kingsberry said his agency combined the Justice Department’s fraud indictment information with audits of big recipients of federal assistance to find data that indicated possible criminal activity.

Similarly, mixing and matching big databases helped the Social Security Administration develop datasets for verifying disability claims, said Herb Strauss, SSA's deputy CIO.

He said the agency combines its deep pool of information with outside databases such as LexisNexis to match property ownership records against the information supplied by claimants.

Agencies can learn a great deal by sharing information with one another and mining external data sources, Strauss added.

He said SSA has been working in a big-data environment since its origins in the 1930s, although the technological capabilities were quite different when the agency was first tasked with assigning and maintaining accounts for every American. That responsibility eventually expanded to include tracking survivor and disability benefits and other duties that increased the amounts of data SSA monitored.

Today, applying big-data technology and techniques is an ongoing process that requires continued attention. "It can't be like a cat fight, with 10 seconds of intense activity followed by a five-year pause," Strauss said.

Although SSA has been sifting data since the Great Depression, Kingsberry's agency has been at work only since the Great Recession.

He detailed the construction of the site that tracks data on spending under the American Recovery and Reinvestment Act of 2009, which led audience members to ask about how to track unstructured data. Such information can present problems because some systems cannot process it uniformly. That sparked further discussion about the difficulties of sharing data, unstructured or otherwise, across agencies.

Kingsberry said his agency took responsibility for data input from the beginning to ensure that it was presented in a uniform manner.

He added that when he works with other agencies to access their data, he sets up memoranda of understanding that explicitly state what each agency expects and what their responsibilities are.

About the Author

Mark Rockwell is a staff writer at FCW.

Before joining FCW, Rockwell was Washington correspondent for Government Security News, where he covered all aspects of homeland security from IT to detection dogs and border security. Over the last 25 years in Washington as a reporter, editor and correspondent, he has covered an increasingly wide array of high-tech issues for publications like Communications Week, Internet Week, Fiber Optics News, magazine and Wireless Week.

Rockwell received a Jesse H. Neal Award for his work covering telecommunications issues, and is a graduate of James Madison University.

Click here for previous articles by Rockwell. Contact him at or follow him on Twitter at @MRockwell4.

The Fed 100

Save the date for 28th annual Federal 100 Awards Gala.


  • computer network

    How Einstein changes the way government does business

    The Department of Commerce is revising its confidentiality agreement for statistical data survey respondents to reflect the fact that the Department of Homeland Security could see some of that data if it is captured by the Einstein system.

  • Defense Secretary Jim Mattis. Army photo by Monica King. Jan. 26, 2017.

    Mattis mulls consolidation in IT, cyber

    In a Feb. 17 memo, Defense Secretary Jim Mattis told senior leadership to establish teams to look for duplication across the armed services in business operations, including in IT and cybersecurity.

  • Image from

    DHS vague on rules for election aid, say states

    State election officials had more questions than answers after a Department of Homeland Security presentation on the designation of election systems as critical U.S. infrastructure.

  • Org Chart Stock Art - Shutterstock

    How the hiring freeze targets millennials

    The government desperately needs younger talent to replace an aging workforce, and experts say that a freeze on hiring doesn't help.

  • Shutterstock image: healthcare digital interface.

    VA moves ahead with homegrown scheduling IT

    The Department of Veterans Affairs will test an internally developed scheduling module at primary care sites nationwide to see if it's ready to service the entire agency.

  • Shutterstock images (honglouwawa & 0beron): Bitcoin image overlay replaced with a dollar sign on a hardware circuit.

    MGT Act poised for a comeback

    After missing in the last Congress, drafters of a bill to encourage cloud adoption are looking for a new plan.

Reader comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group