Intelligence agencies seek crystal ball

The Intelligence Advanced Research Projects Activity is seeking technology that can help intelligence analysts predict global events such as mass migrations, disease outbreaks, economic instability and natural disasters.

“The [Open Source Indicators Program’s] methods, if proven successful, could provide early warnings of emerging events around the world,” said Jason Matheny, OSI program manager at IARPA, in a press release. IARPA is part of the Office of the Director of National Intelligence.

Other unusual events IARPA would like the technology to anticipate include political upheavals, humanitarian crises, mass violence and resource shortages.

Currently, there are few methods for fusing and analyzing publicly available data, such as Web search queries, blogs, Internet traffic, financial market activity, traffic webcams and Wikipedia edits, according to IARPA’s solicitation.

The OSI program aims to use new statistical methods that continuously and automatically analyze public data to alert analysts to potential global upheavals. The technology must extract, process and correlate data to reveal patterns preceding unforeseen events and generate warnings.

IARPA wants “data extraction techniques that focus on volume, rather than depth, by identifying shallow features of data that correlate with events,” the solicitation states. In addition to helping identify patterns that precede events, officials want the system to develop probabilities of specific events happening.

The technology dovetails with a similar project that uses crowdsourcing technology to predict potential future events and their probability of occurring. On July 15, IARPA launched a beta version of the Aggregative Contingent Estimation system (ACES) website, called Forecasting ACE, which aggregates respondents’ answers to an online survey, GCN reported.

IARPA stressed that the program will not include information that identifies, geolocates or tracks individuals.

The idea of using public data as a predictor has been catching on with the rise of social media and other crowdsourcing efforts. Monitoring social media and search trends has proved to be effective in predicting the stock market, the box office performance of movies and even the spread of the flu, GCN reported in October 2010.

About the Author

Kathleen Hickey is a freelance writer for GCN.


  • 2018 Fed 100

    The 2018 Federal 100

    This year's Fed 100 winners show just how much committed and talented individuals can accomplish in federal IT. Read their profiles to learn more!

  • Census
    How tech can save money for 2020 census

    Trump campaign taps census question as a fund-raising tool

    A fundraising email for the Trump-Pence reelection campaign is trying to get supporters behind a controversial change to the census -- asking respondents whether or not they are U.S. citizens.

  • Cloud
    DOD cloud

    DOD's latest cloud moves leave plenty of questions

    Speculation is still swirling about the implications of the draft solicitation for JEDI -- and about why a separate agreement for cloud-migration services was scaled back so dramatically.

Stay Connected

FCW Update

Sign up for our newsletter.

I agree to this site's Privacy Policy.