Oversight

Census dinged by IG for canvassing error rate

Census Bureau headquarters in Suitland, Md. 

Census Bureau headquarters in Suitland, Md.

A recent Department of Commerce inspector general report raises questions about the quality control procedures and data quality related to how the Census Bureau uses satellite imagery to count its blocks.

As part of its larger turn to more modern tech solutions to cut costs, Census opted to reduce field operations by leaning on satellite imagery to survey America's approximately 11 million blocks and identify any changes made from the office, rather than traversing each block on foot.

In 2010, Census spent $444 million to hire almost 149,000 temporary workers to walk all 11 million blocks and identify every household where someone could live. By comparison, the new process, known interactive review, involved hiring 136 clerks at an estimated cost of $12.9 million.

The OIG report noted that the bureau did not complete a formal cost estimate for the interactive review operations and that its estimate "is not transparent and discrepancies between estimated and actual costs, if found, cannot be researched and identified."

The bureau confirmed to the OIG that, as of June 8, 2017, all 11 million or so blocks had been reviewed at least once.

Auditors, however, found that Census's quality assurance plan for the interactive review operation, which was drafted but never finalized, does not document the methodology used to calculate error rates and their potential impact on data quality.

To make sure the process is conducted correctly, interactive review entails a double-blind review -- meaning that after the initial review, a second clerk reviews the same block. If the two agree, no additional reviews are performed. If they don't, a third reviewer, called the adjudicator, breaks the tie and can charge errors to either clerk.

According to Census policy, if a clerk has an error rate below 5 percent after completing 500 blocks, only a sample of his or her future reviews will go through a double-blind check. But if a clerk's error rate exceeds 5 percent during or after the baseline period, he or she is supposed to have all future reviews checked.

However, auditors found, in practice, "every single clerk qualified for sampling after his or her first 500 blocks," calling into question the accuracy of the collected data.

Additionally, auditors found that the reason Census performed fewer quality assurance reviews was because bureau policy allows adjudicators to forgive errors when imagery data is "highly ambiguous."

Because these forgiven errors are not factored into error rates, auditors reported that this practice "interferes with business processes established to ensure" low error rates.

OIG recommends Census finalize the quality assurance plan, complete with clear explanations of how errors rates are calculated and descriptions of internal controls. The auditors also recommend ensuring current practices for error forgiveness are addressed when developing business rules to meet the 5 percent undetected error rate.

Census said it had no fundamental disagreements with the findings or recommendations and will prepare a formal action plan, clearly define all internal controls and revise its business rules.

About the Author

Chase Gunter is a staff writer covering civilian agencies, workforce issues, health IT, open data and innovation.

Prior to joining FCW, Gunter reported for the C-Ville Weekly in Charlottesville, Va., and served as a college sports beat writer for the South Boston (Va.) News and Record. He started at FCW as an editorial fellow before joining the team full-time as a reporter.

Gunter is a graduate of the University of Virginia, where his emphases were English, history and media studies.

Click here for previous articles by Gunter, or connect with him on Twitter: @WChaseGunter

Featured

  • Contracting
    8 prototypes of the border walls as tweeted by CBP San Diego

    DHS contractors face protests – on the streets

    Tech companies are facing protests internally from workers and externally from activists about doing for government amid controversial policies like "zero tolerance" for illegal immigration.

  • Workforce
    By Mark Van Scyoc Royalty-free stock photo ID: 285175268

    At OPM, Weichert pushes direct hire, pay agent changes

    Margaret Weichert, now acting director of the Office of Personnel Management, is clearing agencies to make direct hires in IT, cyber and other tech fields and is changing pay for specialized occupations.

  • Cloud
    Shutterstock ID ID: 222190471 By wk1003mike

    IBM protests JEDI cloud deal

    As the deadline to submit bids on the Pentagon's $10 billion, 10-year warfighter cloud deal draws near, IBM announced a legal protest.

Stay Connected

FCW Update

Sign up for our newsletter.

I agree to this site's Privacy Policy.