Big Data

Exploiting the data revolution

abstract head representing big data

 

Digital technologies have irrevocably changed business in recent years, and life in the era of big data inherently signifies that business decisions may never again lack for data.

But this big-data age, where crunching petabytes isn’t uncommon and compute can be purchased on-demand, is just "the warm-up act," according to Andrew McAfee, co-author of "The Second Machine Age: Work, Progress and Prosperity in the Age of Machines.”" 

"We’re nowhere near done with innovation and transformation, and I firmly believe during our lifetimes, we’ll witness the biggest transformation since the Industrial Revolution,” said McAfee, speaking at VMWare’s Government Innovation Summit on March 6. McAfee is a principal research scientist at the MIT Sloan School of Management.

"The Industrial Revolution helped us overcome the limitations of our muscles. We are now overcoming the limitations of our minds," McAfee said. "Over the next 50 years economies will go through the biggest change since the Industrial Revolution."

Having ridden in Google’s autonomous car and hung out with Baxter, a robot billed to have behavior-based common sense, McAfee said advances in digital technology, robotics and automation are beginning to change the world.

The changes have been rapid, limited only it seems by Moore's Law, money and imagination.

Jeopardy champion Ken Jennings was defeated by IBM's Watson less than two decades after former chess champion Garry Kasparov was defeated by IBM's Deep Blue computer, highlighting an important point: humans can now build machines that are better at tasks outside of arenas like number-crunching. Machines have always owned number-crunching. Now  they’re getting better than humans at other activities.

Machines can produce and sift through increasingly deep seas of data as fast as people can blink, leading to all sorts of "weird performance changes" that apply across the private and public sectors.

But the biggest changes occur not from machines or people, McAfee said, but from the combination of people and technology. McAfee used freestyle chess as an example. While the fastest chess-playing supercomputer can defeat the best human player, that supercomputer is dominated by a team of chess players and a slower computer.  

"[Machines and people] don’t have to be head to head. When we bring people and technology together correctly, the results are amazing," McAfee said.

The most obvious example in IT of teamwork between humans and machines is the use of computer algorithms to make predictions and data-driven decisions.

McAfee referenced Nate Silver, formerly of the New York Times, whose data-driven model predicted the presidential race correctly in all 50 states in 2012. Machines are great at producing data, compiling it and storing it, but there still is no substitute for human ingenuity in finding what best to do with data. And increasingly, organizations are releasing their big datasets to "the geeks of the world" for help. Allstate, for example, ran a 12-week contest for data nerds in hopes of improving its liability predictive prowess. In three months, the data teams had improved Allstate’s predictive baseline by 340 percent.

"In the era of big data that we're in, data is the lifeblood of science. It is how you get smarter about the world," McAfee said.

Because of this influx of technology and data, organizations and federal agencies face enormous opportunities. At the same time, they face daunting challenges handling the kind of disruption that’s only going to make the IT world seem more chaotic. Government sometimes struggles with innovation because of its traditionally slow-moving procurement process, while technology seems to evolve every few months.

"How we react to [the disruption] will be the defining issue for businesses, organizations, the economy and society for the rest of our lives," McAfee said. "Let's not screw this up, shall we?"

 

About the Author

Frank Konkel is a former staff writer for FCW.

FCW in Print

In the latest issue: Looking back on three decades of big stories in federal IT.

Featured

  • FCW @ 30 GPS

    FCW @ 30

    Since 1986, FCW has covered it all -- the major contracts, the disruptive technologies, the picayune scandals and the many, many people who make federal IT function. Here's a look back at six of the most significant stories.

  • Shutterstock image.

    A 'minibus' appropriations package could be in the cards

    A short-term funding bill is expected by Sept. 30 to keep the federal government operating through early December, but after that the options get more complicated.

  • Defense Secretary Ash Carter speaks at the TechCrunch Disrupt conference in San Francisco

    DOD launches new tech hub in Austin

    The DOD is opening a new Defense Innovation Unit Experimental office in Austin, Texas, while Congress debates legislation that could defund DIUx.

  • Shutterstock image.

    Merged IT modernization bill punts on funding

    A House panel approved a new IT modernization bill that appears poised to pass, but key funding questions are left for appropriators.

  • General Frost

    Army wants cyber capability everywhere

    The Army's cyber director said cyber, electronic warfare and information operations must be integrated into warfighters' doctrine and training.

  • Rising Star 2013

    Meet the 2016 Rising Stars

    FCW honors 30 early-career leaders in federal IT.

Reader comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group