Big Data

Big ideas about big data could win prizes

Government agencies have been using competitions with some success to harness the collective brainpower of the public in solving thorny technology problems. Now three agencies – NASA, the Energy Department and the National Science Foundation -- are using the model to approach big-data conundrums. Separately, the NSF and the National Institutes of Health are funding additional research into big data through a more conventional grants process.

The NASA-NSF-DOE project is only the first of a planned series of contests, run through the crowdsourcing site

“Big Data is characterized not only by the enormous volume or the velocity of its generation but also by the heterogeneity, diversity and complexity of the data,” said Suzi Iacono, co-chair of the interagency Big Data Senior Steering Group, in a statement posted to TopCoder announcing the challenge. “There are enormous opportunities to extract knowledge from these large-scale diverse data sets, and to provide powerful new approaches to drive discovery and decision-making, and to make increasingly accurate predictions. We’re excited to see what this competition will yield and how it will guide us in funding the next round of big data science and engineering.”

Before the data can be used, however, it has to be homogenized. As the information flows in from a variety of data sources, it does not easily blend. So the question that entrants in the first contest must answer, as reported on GigaOm, is: “How can we make heterogeneous (dissimilar and incompatible) data sets homogeneous (uniformly accessible, compatible, able to be grouped and/or matched) so usable information can be extracted? How can information then be converted into real knowledge that can inform critical decisions and solve societal challenges?”

NASA logoThose interested in tackling that question must register by Oct. 13 and submit their entry by Oct. 19. Three $500 prizes will be awarded. According to GigaOm’s report, future contests will concern more specific applications such as energy, health care and earth science.

Steve Kelman, Weatherhead Professor of Public Management at Harvard University, and an FCW Columnist, has long advocated the use of competitions to find innovative solutions.

"Contests now seem to be spreading beyond developing solutions for very specific, isolated problems to the kind of challenge NASA has created here, which is for working on a major conceptual challenge faced by many organizations, in both government and industry," he said. "It’s a real experiment to see whether the challenge format can successfully work in such less-concrete areas, but I am very pleased that NASA is giving this a try."

However, he suggested, the prize amount might be too low for an effort that could require a lot of work.

Jason Cursan, director of the NASA Tournament Lab, said contests are effective ways to get innovative ideas, even for relatively low prize amounts, because the winning competitors get some prestige and an accomplishment for their resumes, leading to career advancement. "We get unsolicited ideas all the time," he said, quoted in FCW’s sister publication GCN. "It doesn’t require a high incentive to get high-quality ideas."
Meanwhile, the NSF announced it has awarded grants totaling nearly $15 million for new fundamental research projects in big data. The awards are intended to fund efforts to develop better tools and methods for turning massive amounts of data into useful information.

NSF made the awards to eight research institutions at universities, on behalf of NIH. "To get the most value from the massive biological data sets we are now able to collect, we need better ways of managing and analyzing the information they contain," said NIH Director Francis Collins in a written statement. "The new awards that NIH is funding will help address these technological challenges--and ultimately help accelerate research to improve health--by developing methods for extracting important, biomedically relevant information from large amounts of complex data."

About the Author

Technology journalist Michael Hardy is a former FCW editor.

Rising Stars

Meet 21 early-career leaders who are doing great things in federal IT.


  • SEC Chairman Jay Clayton

    SEC owns up to 2016 breach

    A key database of financial information was breached in 2016, possibly in support of insider trading, said the Securities and Exchange Commission.

  • Image from

    DOD looks to get aggressive about cloud adoption

    Defense leaders and Congress are looking to encourage more aggressive cloud policies and prod reluctant agencies to embrace experimentation and risk-taking.

  • Shutterstock / Pictofigo

    The next big thing in IT procurement

    Steve Kelman talks to the agencies that have embraced tech demos in their acquisition efforts -- and urges others in government to give it a try.

  • broken lock

    DHS bans Kaspersky from federal systems

    The Department of Homeland Security banned the Russian cybersecurity company Kaspersky Lab’s products from federal agencies in a new binding operational directive.

  • man planning layoffs

    USDA looks to cut CIOs as part of reorg

    The Department of Agriculture is looking to cut down on the number of agency CIOs in the name of efficiency and better communication across mission areas.

  • What's next for agency cyber efforts?

    Ninety days after the Trump administration's executive order, FCW sat down with agency cyber leaders to discuss what’s changing.

Reader comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group