Big Data

Analytics guru Nate Silver offers advice for agencies

Nate Silver

Analytics expert Nate Silver addresses the Adobe Government Assembly. (FCW photo by Camille Tuutti)

There is no shortage of hype about big data in the federal IT space, and separating the buzz from tangible benefits has thus far proven difficult.

So says Nate Silver, who recently authored a book about predictive analytics and successfully predicted the presidential election outcomes in all 50 states in his Five Thirty Eight blog for the New York Times.

Silver, speaking Feb. 12 at the Adobe Government Assembly in Washington, D.C., cautioned the audience that big data is "not a panacea" and won't single-handedly "solve all our problems."

Big data may well become one of the most potent tools in the government's IT arsenal and lead to countless data-driven solutions in the near future, but Silver stressed that it will still take good, old-fashioned brain power to engineer and makes sense of them.

"The people in this room are still pretty important," Silver said. "Don't become too enamored with computers."

In fact, he predicted, imaginative individuals will play a critical role in determining the future of big data in the federal sector.

Silver recounted several "cautionary tales" in big data drawn from his book, "The Signal and the Noise: Why so Many Predictions Fail, But Some Don't." And he offered three suggestions to federal agencies looking to thrive in the era of big data:

Think probabilistically, embrace uncertainty

Silver, who once earned six figures as a poker player, used an analogy from his favorite game to describe how agencies can employ big data. Playing the percentages, he said, can be beneficial, and routinely making the right play generally leads to foreseeable outcomes. But embracing uncertainty -- throwing a change-up now or then -- can lead a player to unexpected profit on the poker table, and could lead agencies to unexpected successes that boost their cost-benefit margins.

Sweat the small stuff when you face big problems

As a tool, big data can unlock all kinds of insights from massive amounts of data, but small changes in the way we approach humongous data sets can drastically change outcomes. Seemingly minor details should never be overlooked, Silver said.

Question your assumptions; question everyone else's assumptions, too

Making sense of the noise can be challenging, but finding signals in the noise does not always mean they are connected, Silver said. It's also important to be cognizant of what people are saying, and why.

"Beware of people who tell you (big data) will solve all your problems," Silver said.

Big data is a challenging concept for any user, but Silver noted one more challenge facing the public sector that perked the ears of the agency-heavy audience.

"Bureaucracy is the enemy of imagination," he said, as the crowd erupted into laughs, smiles and knowing glances across the table.

About the Author

Frank Konkel is a former staff writer for FCW.

The 2015 Federal 100

Meet 100 women and men who are doing great things in federal IT.

Featured

  • Shutterstock image (by venimo): e-learning concept image, digital content and online webinar icons.

    Can MOOCs make the grade for federal training?

    Massive open online courses can offer specialized IT instruction on a flexible schedule and on the cheap. That may not always mesh with government's preference for structure and certification, however.

  • Shutterstock image (by edel): graduation cap and diploma.

    Cybersecurity: 6 schools with the right stuff

    The federal government craves more cybersecurity professionals. These six schools are helping meet that demand.

  • Rick Holgate

    Holgate to depart ATF

    Former ACT president will take a job with Gartner, follow his spouse to Vienna, Austria.

  • Are VA techies slacking off on Yammer?

    A new IG report cites security and productivity concerns associated with employees' use of the popular online collaboration tool.

  • Shutterstock image: digital fingerprint, cyber crime.

    Exclusive: The OPM breach details you haven't seen

    An official timeline of the Office of Personnel Management breach obtained by FCW pinpoints the hackers’ calibrated extraction of data, and the government's step-by-step response.

  • Stephen Warren

    Deputy CIO Warren exits VA

    The onetime acting CIO at Veterans Affairs will be taking over CIO duties at the Office of the Comptroller of the Currency.

  • Shutterstock image: monitoring factors of healthcare.

    DOD awards massive health records contract

    Leidos, Accenture and Cerner pull off an unexpected win of the multi-billion-dollar Defense Healthcare Management System Modernization contract, beating out the presumptive health-records leader.

  • Sweating the OPM data breach -- Illustration by Dragutin Cvijanovic

    Sweating the stolen data

    Millions of background-check records were compromised, OPM now says. Here's the jaw-dropping range of personal data that was exposed.

  • FCW magazine

    Let's talk about Alliant 2

    The General Services Administration is going to great lengths to gather feedback on its IT services GWAC. Will it make for a better acquisition vehicle?

Reader comments

Wed, Feb 13, 2013

I like the comment :

"As a tool, big data can unlock all kinds of insights from massive amounts of data, but small changes in the way we approach humongous data sets can drastically change outcomes. Seemingly minor details should never be overlooked, Silver said"

As any expert in charts can tell you, "The way you look at data all depends on what you want the data to say!". (Yes, I know that was not his intent, but it is true none the less.)

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above