End-to-end census test took place without 2020 tech

The Census Bureau's plan to save money on the 2020 population count needs work, according to an agency watchdog, and the results of recent tests may not help as intended.

The 2018 end-to-end test – intended as a dress rehearsal for the 2020 decennial census – has limited utility, according to a Feb. 6 Department of Commerce Inspector General report.

Specifically, the bureau ran into trouble testing the quality control aspects of its unified tracking system of data collected by listers – temporary employees who canvas block-to-block to confirm address data -- because "it could not generate [quality control] reports."

Additionally, the test did not use the devices that enumerators will use to get address data and spatial information designed to optimize the efforts of listers. This technology is billed as a key money saver because it is intended to maximize the effectiveness and reduce labor costs.

Right now, the IG doesn't know if they will work as intended.

"The actual mobile devices that listers will use during in-field address canvassing for the 2020 Census were not available to assess device performance," the report states.

Another issue is the success rate of in-office canvassing, the process which some blocks are categorized as having the same number of dwellings as in the previous census – called a "passive" block. Blocks where the number of dwellings have gone up or down are characterized as "active." In-office canvassing is accomplished by using aerial imagery.

Active blocks are canvassed by listers in the field as a run-up to the final population count, while passive blocks are not. According to the IG report, the Bureau estimated that the use of aerial imagery will lower the workload for field personnel by 70 percent.

However, analysis by the IG found that "in-office address canvassing is yielding incorrect results or, at the very least, results that are inconsistent with in-field address canvassing."

Inaccuracy in designating blocks creates two risks – that some households may not receive Census forms and that ultimately field staff may have to take on more work, jeopardizing cost savings.

In response to those findings Ron Jarmin, then serving as acting director, countered that the watchdog "overstated error levels," and estimated only two percent of addresses "represented potential coverage errors."

IG also found issues with the operational control system used by supervisors to make sure enumerators are fulfilling their responsibilities. Auditors also observed in on-site visits some listers "did not comply with in-field procedures" in ways that could affect the count's accuracy.

Census's revamped address canvassing operation has already had issues with its tech.

In 2017, the bureau abandoned its active block resolution program, a way of validating address data without the use of human canvasser. As a result, the bureau estimated the number of in-person household visits in its 2018 dress rehearsal to nearly double.

Over the summer, Al Fontenot, associate director of Decennial Census Programs Directorate, downplayed some of the tech issues the bureau found in the end-to-end test.

The OIG wants the bureau look at how incorrectly categorized blocks will affect data accuracy and certain demographic groups, and remedy errors causing the incorrect categorization of blocks. The watchdog also recommends the speedy resolution and prioritization of in-field alerts pertaining to data quality.