'To purge or not to purge'

man studying data

It's not easy to nail down the rules of the road followed by U.S. spy agencies when it comes to how they store the data they collect. At a recent hearing of the Privacy and Civil Liberties Review Board, top intelligence community lawyers tried to explain the nuances of how they store data collected from Internet services and intercepted as it travels across global networks.

"To purge or not to purge -- that is the question," summed up Rajesh De, general counsel of the National Security Agency. The answer, as it turns out, is not very clear.

The collection practices, revealed to the public by the Edward Snowden leaks, are authorized by section 702 of the Foreign Intelligence Surveillance Act. They are not designed to collect information on people protected by U.S. law, including U.S. citizens whether at home or abroad, and persons inside the U.S.

But there are fears that the collection procedures, which give analysts fairly wide latitude in intercepting and storing  all manner of Internet communications, including email, social media, video and voice calls, chat, and more, can be used to target subjects ostensibly protected by U.S. law. The intelligence community points to minimization procedures, periodic reviews and oversight from Congress and  agency inspectors general as checks against unlawful collection.

Those internal checks are meant to lead to the purging of information on U.S. persons – citizens and permanent residents -- that was incidental to other collection, collected inadvertently or is outside the boundaries of the authorizing statute. Given the secrecy of the programs and differences in the rules about different modes of collection, it can be difficult to determine when an intelligence agency is required to expunge data from its systems.

Records collected from Internet companies through the Prism program are retained for five years. Data siphoned directly from the Internet backbone, called Upstream collection, are supposed to be expunged after two years, because of a Foreign Intelligence Surveillance Court determination that such information is more likely to contain incidental or unauthorized communications.

Both methods authorized by 702 collect the entirety of communications -- the metadata and the underlying content. According to De, about 10 percent of NSA collections come from Upstream targets, and the rest comes from Prism. Neither method is considered bulk collection by the intelligence agencies -- information is targeted using "selectors" like an email address or a phone number. It's not clear what else might constitute a selector, but presumably information such as a device's IP address or a mobile device’s unique identifier could be included. Intelligence agencies can also monitor communications for references to a third-party selector. These so-called "about" collections are supposed to target selectors pertaining to non-U.S. persons.

Robert Litt, general counsel for the Office of the Director of National Intelligence, told the board that selectors are focused on devices or accounts, not broad populations. For instance, an area code, he said, would not qualify as a selector.

The declassified version of the NSA's minimization rules indicates that the agency deposits data from 702 collections into a "segregated repository" where it can be tagged by specialists as being relevant to NSA analysts. Data collected under Prism can be queried for information pertaining to U.S. persons, as long as it is "first approved in accordance with NSA procedures." The NSA is charged with keeping records on all such cases. Data intercepted via Upstream collection may not be queried for information designed to target U.S. persons. 

According to a redacted compliance report declassified by the DNI, problems arise in one half of one percent of selectors, including collecting data on U.S. persons or collections outside the scope of what is authorized.

Collections and targets that are out of compliance are handled in different ways, according to De.

Intelligence agencies are supposed to "detask" or stop collection on selectors that are determined to have been targeted my mistake, or outside the scope of FISA. Data collected on those targets is supposed to be removed from NSA systems in such a way that it cannot be used. It's not clear if this means that the data is deleted or that it is tagged in such a way that it cannot be queried. De indicated that NSA systems are designed to take into account the requirements of the oversight and minimization regimes.

While De said he couldn't articulate all the exceptions to the NSA's minimization procedures, he said that wholly domestic communications as a default rule must be purged. At the same time, there is an exception for information that is said to have foreign intelligence value, as determined by an analyst.

Information that implicates U.S. persons in potential crimes or terrorist networks is also retained. If a terrorist is calling a U.S. person in Minneapolis, "it's of high interest to us," said Brad Wiegmann, deputy assistant attorney general in the National Security Division of the Department of Justice.

The minimization procedures and the exceptions "give the government broad authority to collect purely domestic communications," said Jameel Jaffer, deputy legal director of the American Civil Liberties Union.

Despite widespread and highly vocal criticism, the spy agencies have no desire to eliminate or pare back the programs. While Litt and others acknowledged a need for more transparency, they said the programs have been useful and worth the expense.

"You can see time and again in important intelligence reports provided to policymakers that it is derived from section 702 collection," Litt said.

About the Author

Adam Mazmanian is executive editor of FCW.

Before joining the editing team, Mazmanian was an FCW staff writer covering Congress, government-wide technology policy and the Department of Veterans Affairs. Prior to joining FCW, Mazmanian was technology correspondent for National Journal and served in a variety of editorial roles at B2B news service SmartBrief. Mazmanian has contributed reviews and articles to the Washington Post, the Washington City Paper, Newsday, New York Press, Architect Magazine and other publications.

Click here for previous articles by Mazmanian. Connect with him on Twitter at @thisismaz.


  • Workforce
    White House rainbow light shutterstock ID : 1130423963 By zhephotography

    White House rolls out DEIA strategy

    On Tuesday, the Biden administration issued agencies a roadmap to guide their efforts to develop strategic plans for diversity, equity, inclusion and accessibility (DEIA), as required under a as required under a June executive order.

  • Defense
    software (whiteMocca/Shutterstock.com)

    Why DOD is so bad at buying software

    The Defense Department wants to acquire emerging technology faster and more efficiently. But will its latest attempts to streamline its processes be enough?

Stay Connected