Crowdsourcing the price of everything

Shutterstock image: digital charts and graphs.

For decades, the Labor Department's Bureau of Labor Statistics has focused on prices of consumer goods, sending employees out to inspect prices in stores to generate the data for the Consumer Price Index. But the bureau may be getting close to adopting a model that focuses a bit more on the purchasers instead.

Since the summer of 2015, BLS has been in talks with Premise, a price data crowdsourcing firm, said Jim Thomas, BLS' index methods, analysis and evaluation chief.

The work is "experimental" but "accelerating," he said.

Thomas was one of many economists and statisticians speaking at the World Bank's "The Future of Price Statistics" event on March 30, where Premise announced a pilot project with the World Bank.

In more than a dozen developing countries from Brazil to Vietnam, Premise's data collectors (i.e. locals armed with smart phones) generate 1.5 million "observations" every month, Premise CTO Joe Reisinger said. The observations typically take the form of a picture of a store shelf, but can also include the input of, say, a quote for monthly rent.

The ability to gather granular data, observe it as it comes in and tweak collectors' data requests as the process progresses all combine for a powerful tool, said World Bank statistician Nada Hamadeh.

BLS' Thomas said his bureau is experimenting with web scraping to capture precisely that online data, as well as exploring the sourcing of data from companies and other sources to get a fuller price picture.

"We need alternative data sources, and they need official statistics," Thomas said.

But multiple economists suggested that Premise's approach could prove useful for capturing regional price differences. American Southerners, for example, can face radically different prices than their compatriots to the north or west, but so long as the U.S. CPI is calculated by a relative handful of BLS employees, regional differences will remain understated.

Harvard economist and former Treasury Secretary Larry Summers predicted a convergence between developing- and developed-world methods as price data capture technology continues to evolve and capitalize on the combination of easily captured images and analysis that smartphones afford. (Summers serves on Premise's board of directors.)

Summers noted how fundamental price data is to economic analysis – "No economic data are of any real meaning at all without prices," he said – and said that in the history of science, most advancements haven't come from the testing of brilliant hypotheses.

They've come, as they did with the telescope and microscope, merely from getting a better look at what's going on, Summers said. "It's ultimately observation and common sense, rather than genius, that drives scientific progress."

Summers is betting that crowdsourcing via smartphone will prove to be price statistics' telescope.

About the Author

Zach Noble is a former FCW staff writer.


  • Defense
    Soldiers from the Old Guard test the second iteration of the Integrated Visual Augmentation System (IVAS) capability set during an exercise at Fort Belvoir, VA in Fall 2019. Photo by Courtney Bacon

    IVAS and the future of defense acquisition

    The Army’s Integrated Visual Augmentation System has been in the works for years, but the potentially multibillion deal could mark a paradigm shift in how the Defense Department buys and leverages technology.

  • Cybersecurity
    Deputy Secretary of Homeland Security Alejandro Mayorkas  (U.S. Coast Guard photo by Petty Officer 3rd Class Lora Ratliff)

    Mayorkas announces cyber 'sprints' on ransomware, ICS, workforce

    The Homeland Security secretary announced a series of focused efforts to address issues around ransomware, critical infrastructure and the agency's workforce that will all be launched in the coming weeks.

Stay Connected