Oversight

Census dinged by IG for canvassing error rate

Census Bureau headquarters in Suitland, Md. 

Census Bureau headquarters in Suitland, Md.

A recent Department of Commerce inspector general report raises questions about the quality control procedures and data quality related to how the Census Bureau uses satellite imagery to count its blocks.

As part of its larger turn to more modern tech solutions to cut costs, Census opted to reduce field operations by leaning on satellite imagery to survey America's approximately 11 million blocks and identify any changes made from the office, rather than traversing each block on foot.

In 2010, Census spent $444 million to hire almost 149,000 temporary workers to walk all 11 million blocks and identify every household where someone could live. By comparison, the new process, known interactive review, involved hiring 136 clerks at an estimated cost of $12.9 million.

The OIG report noted that the bureau did not complete a formal cost estimate for the interactive review operations and that its estimate "is not transparent and discrepancies between estimated and actual costs, if found, cannot be researched and identified."

The bureau confirmed to the OIG that, as of June 8, 2017, all 11 million or so blocks had been reviewed at least once.

Auditors, however, found that Census's quality assurance plan for the interactive review operation, which was drafted but never finalized, does not document the methodology used to calculate error rates and their potential impact on data quality.

To make sure the process is conducted correctly, interactive review entails a double-blind review -- meaning that after the initial review, a second clerk reviews the same block. If the two agree, no additional reviews are performed. If they don't, a third reviewer, called the adjudicator, breaks the tie and can charge errors to either clerk.

According to Census policy, if a clerk has an error rate below 5 percent after completing 500 blocks, only a sample of his or her future reviews will go through a double-blind check. But if a clerk's error rate exceeds 5 percent during or after the baseline period, he or she is supposed to have all future reviews checked.

However, auditors found, in practice, "every single clerk qualified for sampling after his or her first 500 blocks," calling into question the accuracy of the collected data.

Additionally, auditors found that the reason Census performed fewer quality assurance reviews was because bureau policy allows adjudicators to forgive errors when imagery data is "highly ambiguous."

Because these forgiven errors are not factored into error rates, auditors reported that this practice "interferes with business processes established to ensure" low error rates.

OIG recommends Census finalize the quality assurance plan, complete with clear explanations of how errors rates are calculated and descriptions of internal controls. The auditors also recommend ensuring current practices for error forgiveness are addressed when developing business rules to meet the 5 percent undetected error rate.

Census said it had no fundamental disagreements with the findings or recommendations and will prepare a formal action plan, clearly define all internal controls and revise its business rules.

About the Author

Chase Gunter is a staff writer covering civilian agencies, workforce issues, health IT, open data and innovation.

Prior to joining FCW, Gunter reported for the C-Ville Weekly in Charlottesville, Va., and served as a college sports beat writer for the South Boston (Va.) News and Record. He started at FCW as an editorial fellow before joining the team full-time as a reporter.

Gunter is a graduate of the University of Virginia, where his emphases were English, history and media studies.

Click here for previous articles by Gunter, or connect with him on Twitter: @WChaseGunter

Featured

  • IT Modernization
    Eisenhower Executive Office Building (Image: Wikimedia Commons)

    OMB's user guide to the MGT Act

    The Office of Management and Budget is working on a rules-of-the-road document to cover how agencies can seek and use funds under the MGT Act.

  • global network (Pushish Images/Shutterstock.com)

    As others see us -- a few surprises

    A recent dinner with civil servants from Asia delivered some interesting insights, Steve Kelman writes.

  • FCW Perspectives
    cloud (Singkham/Shutterstock.com)

    A smarter approach to cloud

    Advances in cloud technology are shifting the focus toward choosing the right tool for the job and crafting solutions that truly modernize systems.

Stay Connected

FCW Update

Sign up for our newsletter.

I agree to this site's Privacy Policy.