Big Data

Analytics guru Nate Silver offers advice for agencies

Nate Silver

Analytics expert Nate Silver addresses the Adobe Government Assembly. (FCW photo by Camille Tuutti)

There is no shortage of hype about big data in the federal IT space, and separating the buzz from tangible benefits has thus far proven difficult.

So says Nate Silver, who recently authored a book about predictive analytics and successfully predicted the presidential election outcomes in all 50 states in his Five Thirty Eight blog for the New York Times.

Silver, speaking Feb. 12 at the Adobe Government Assembly in Washington, D.C., cautioned the audience that big data is "not a panacea" and won't single-handedly "solve all our problems."

Big data may well become one of the most potent tools in the government's IT arsenal and lead to countless data-driven solutions in the near future, but Silver stressed that it will still take good, old-fashioned brain power to engineer and makes sense of them.

"The people in this room are still pretty important," Silver said. "Don't become too enamored with computers."

In fact, he predicted, imaginative individuals will play a critical role in determining the future of big data in the federal sector.

Silver recounted several "cautionary tales" in big data drawn from his book, "The Signal and the Noise: Why so Many Predictions Fail, But Some Don't." And he offered three suggestions to federal agencies looking to thrive in the era of big data:

Think probabilistically, embrace uncertainty

Silver, who once earned six figures as a poker player, used an analogy from his favorite game to describe how agencies can employ big data. Playing the percentages, he said, can be beneficial, and routinely making the right play generally leads to foreseeable outcomes. But embracing uncertainty -- throwing a change-up now or then -- can lead a player to unexpected profit on the poker table, and could lead agencies to unexpected successes that boost their cost-benefit margins.

Sweat the small stuff when you face big problems

As a tool, big data can unlock all kinds of insights from massive amounts of data, but small changes in the way we approach humongous data sets can drastically change outcomes. Seemingly minor details should never be overlooked, Silver said.

Question your assumptions; question everyone else's assumptions, too

Making sense of the noise can be challenging, but finding signals in the noise does not always mean they are connected, Silver said. It's also important to be cognizant of what people are saying, and why.

"Beware of people who tell you (big data) will solve all your problems," Silver said.

Big data is a challenging concept for any user, but Silver noted one more challenge facing the public sector that perked the ears of the agency-heavy audience.

"Bureaucracy is the enemy of imagination," he said, as the crowd erupted into laughs, smiles and knowing glances across the table.

About the Author

Frank Konkel is a former staff writer for FCW.

The Fed 100

Save the date for 28th annual Federal 100 Awards Gala.

Featured

  • computer network

    How Einstein changes the way government does business

    The Department of Commerce is revising its confidentiality agreement for statistical data survey respondents to reflect the fact that the Department of Homeland Security could see some of that data if it is captured by the Einstein system.

  • Defense Secretary Jim Mattis. Army photo by Monica King. Jan. 26, 2017.

    Mattis mulls consolidation in IT, cyber

    In a Feb. 17 memo, Defense Secretary Jim Mattis told senior leadership to establish teams to look for duplication across the armed services in business operations, including in IT and cybersecurity.

  • Image from Shutterstock.com

    DHS vague on rules for election aid, say states

    State election officials had more questions than answers after a Department of Homeland Security presentation on the designation of election systems as critical U.S. infrastructure.

  • Org Chart Stock Art - Shutterstock

    How the hiring freeze targets millennials

    The government desperately needs younger talent to replace an aging workforce, and experts say that a freeze on hiring doesn't help.

  • Shutterstock image: healthcare digital interface.

    VA moves ahead with homegrown scheduling IT

    The Department of Veterans Affairs will test an internally developed scheduling module at primary care sites nationwide to see if it's ready to service the entire agency.

  • Shutterstock images (honglouwawa & 0beron): Bitcoin image overlay replaced with a dollar sign on a hardware circuit.

    MGT Act poised for a comeback

    After missing in the last Congress, drafters of a bill to encourage cloud adoption are looking for a new plan.

Reader comments

Wed, Feb 13, 2013

I like the comment :

"As a tool, big data can unlock all kinds of insights from massive amounts of data, but small changes in the way we approach humongous data sets can drastically change outcomes. Seemingly minor details should never be overlooked, Silver said"

As any expert in charts can tell you, "The way you look at data all depends on what you want the data to say!". (Yes, I know that was not his intent, but it is true none the less.)

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group