Panel: Government data-mining programs need more scrutiny

The federal government should systematically evaluate the effectiveness and lawfulness of homeland security-related data mining and behavioral surveillance programs before deployment, according to a new report from a committee of the National Research Council of the National Academies.

“Protecting Individual Privacy in the Struggle Against Terrorists,” a 300-plus page report released today by a committee of privacy, national security and intelligence experts, lays out an extensive framework for how the government should assess and oversee data and information-based programs being used to fight terrorism.

The Committee on Technical and Privacy Dimensions of Information for Terrorism Prevention and Other National Goals also recommends that Congress reexamine existing laws to consider how privacy should be protected in current information-based counterterrorism programs.

"The framework is not designed simply, if you will, balance privacy and security, but rather with the intention of accomplishing both privacy and security," said Fred Cate, a commissioner and director of the Indiana University’s Center for Applied Cybersecurity Research.

The committee, which was supported by the Homeland Security Department and the National Science Foundation, was first convened in late 2005. The group sought to provide guidance to policy-makers, government officials and industry as they explore new surveillance tools for use in national security.

The years since the 2001 terrorist attacks have seen a proliferation of data mining and surveillance programs by government agencies. Some privacy advocates have questioned whether current laws adequately address the issues posed by these programs.

The committee’s findings stem from analysis of unclassified government programs and interviews with officials. However, members of the committee said that its conclusions apply to classified programs as well.

The report recommends evaluating proposed programs by the following criteria:

  • The program’s objective is clearly stated.

  • The program fully complies with applicable law.

  • The program’s false positive rate is acceptably low.

  • The program is audited at least annually.

  • Uses of personally identifiable information have received proper approval.

The report also emphasized the importance of data quality when it comes to protecting the privacy of people who are not terrorists.

"Individuals claimed by law enforcement officials to match prints found at a crime scene have sometimes turned out not to match upon further investigation," the report states. "Also, matching names or other forms of record linkage are error-prone operations, generally because of data quality issues."

According to the report, committee members believe current policies do not adequately address concerns with the shortcomings in current analytic techniques. The committee also concluded that automated terrorist identification through data mining was not feasible.

About the Author

Ben Bain is a reporter for Federal Computer Week.

The Fed 100

Save the date for 28th annual Federal 100 Awards Gala.


  • computer network

    How Einstein changes the way government does business

    The Department of Commerce is revising its confidentiality agreement for statistical data survey respondents to reflect the fact that the Department of Homeland Security could see some of that data if it is captured by the Einstein system.

  • Defense Secretary Jim Mattis. Army photo by Monica King. Jan. 26, 2017.

    Mattis mulls consolidation in IT, cyber

    In a Feb. 17 memo, Defense Secretary Jim Mattis told senior leadership to establish teams to look for duplication across the armed services in business operations, including in IT and cybersecurity.

  • Image from

    DHS vague on rules for election aid, say states

    State election officials had more questions than answers after a Department of Homeland Security presentation on the designation of election systems as critical U.S. infrastructure.

  • Org Chart Stock Art - Shutterstock

    How the hiring freeze targets millennials

    The government desperately needs younger talent to replace an aging workforce, and experts say that a freeze on hiring doesn't help.

  • Shutterstock image: healthcare digital interface.

    VA moves ahead with homegrown scheduling IT

    The Department of Veterans Affairs will test an internally developed scheduling module at primary care sites nationwide to see if it's ready to service the entire agency.

  • Shutterstock images (honglouwawa & 0beron): Bitcoin image overlay replaced with a dollar sign on a hardware circuit.

    MGT Act poised for a comeback

    After missing in the last Congress, drafters of a bill to encourage cloud adoption are looking for a new plan.

Reader comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group