Oversight

Census dinged by IG for canvassing error rate

Census Bureau headquarters in Suitland, Md. 

Census Bureau headquarters in Suitland, Md.

A recent Department of Commerce inspector general report raises questions about the quality control procedures and data quality related to how the Census Bureau uses satellite imagery to count its blocks.

As part of its larger turn to more modern tech solutions to cut costs, Census opted to reduce field operations by leaning on satellite imagery to survey America's approximately 11 million blocks and identify any changes made from the office, rather than traversing each block on foot.

In 2010, Census spent $444 million to hire almost 149,000 temporary workers to walk all 11 million blocks and identify every household where someone could live. By comparison, the new process, known interactive review, involved hiring 136 clerks at an estimated cost of $12.9 million.

The OIG report noted that the bureau did not complete a formal cost estimate for the interactive review operations and that its estimate "is not transparent and discrepancies between estimated and actual costs, if found, cannot be researched and identified."

The bureau confirmed to the OIG that, as of June 8, 2017, all 11 million or so blocks had been reviewed at least once.

Auditors, however, found that Census's quality assurance plan for the interactive review operation, which was drafted but never finalized, does not document the methodology used to calculate error rates and their potential impact on data quality.

To make sure the process is conducted correctly, interactive review entails a double-blind review -- meaning that after the initial review, a second clerk reviews the same block. If the two agree, no additional reviews are performed. If they don't, a third reviewer, called the adjudicator, breaks the tie and can charge errors to either clerk.

According to Census policy, if a clerk has an error rate below 5 percent after completing 500 blocks, only a sample of his or her future reviews will go through a double-blind check. But if a clerk's error rate exceeds 5 percent during or after the baseline period, he or she is supposed to have all future reviews checked.

However, auditors found, in practice, "every single clerk qualified for sampling after his or her first 500 blocks," calling into question the accuracy of the collected data.

Additionally, auditors found that the reason Census performed fewer quality assurance reviews was because bureau policy allows adjudicators to forgive errors when imagery data is "highly ambiguous."

Because these forgiven errors are not factored into error rates, auditors reported that this practice "interferes with business processes established to ensure" low error rates.

OIG recommends Census finalize the quality assurance plan, complete with clear explanations of how errors rates are calculated and descriptions of internal controls. The auditors also recommend ensuring current practices for error forgiveness are addressed when developing business rules to meet the 5 percent undetected error rate.

Census said it had no fundamental disagreements with the findings or recommendations and will prepare a formal action plan, clearly define all internal controls and revise its business rules.

About the Author

Chase Gunter is a staff writer covering civilian agencies, workforce issues, health IT, open data and innovation.

Prior to joining FCW, Gunter reported for the C-Ville Weekly in Charlottesville, Va., and served as a college sports beat writer for the South Boston (Va.) News and Record. He started at FCW as an editorial fellow before joining the team full-time as a reporter.

Gunter is a graduate of the University of Virginia, where his emphases were English, history and media studies.

Click here for previous articles by Gunter, or connect with him on Twitter: @WChaseGunter

Featured

  • Cybersecurity

    DHS floats 'collective defense' model for cybersecurity

    Homeland Security Secretary Kirstjen Nielsen wants her department to have a more direct role in defending the private sector and critical infrastructure entities from cyberthreats.

  • Defense
    Defense Secretary James Mattis testifies at an April 12 hearing of the House Armed Services Committee.

    Mattis: Cloud deal not tailored for Amazon

    On Capitol Hill, Defense Secretary Jim Mattis sought to quell "rumors" that the Pentagon's planned single-award cloud acquisition was designed with Amazon Web Services in mind.

  • Census
    shutterstock image

    2020 Census to include citizenship question

    The Department of Commerce is breaking with recent practice and restoring a question about respondent citizenship last used in 1950, despite being urged not to by former Census directors and outside experts.

Stay Connected

FCW Update

Sign up for our newsletter.

I agree to this site's Privacy Policy.