Unlocking history at State

The State Department's archives include more than 25 million records of telegram e-mail messages to and from the department's field offices around the world since 1973. The content ranges from the mundane — a 1975 record of the embassy in Suriname receiving a new Dodge truck was an example shown during a recent demonstration — to exotic, top-secret, we'd-tell-you-but-we'd-have-to-kill-you stuff.

The department's challenge is separating the wheat from the chaff when looking for specific items in the huge database. The department's Office of Information Resources Management Programs and Services (IPS) is charged with providing information sought under the Freedom of Information Act and with determining when documents can be declassified and released to the public.

The professional searchers who look for data in the State Archiving System use an advanced records-searching capability that helps them quickly find relevant information.

The State searchers use Verity Inc.'s Information Server software, with a sophisticated interface to perform complex searches. "Anything you read in the Washington Post or New York Times or see on CNN that is attributed to the State Department, chances are, somewhere in this department, we are researching that data," said an IPS team member, who spoke on condition of anonymity.

The department also maintains a publicly accessible database (foia.state.gov) that uses the same Verity search technology, but through a simplified interface to accommodate computer novices and people who speak little English.

Before the adoption of the Verity search software, one could search the telegram records only by their assigned number, the subject line, and the Traffic Analysis Geography or Subject information. Now, the entire database is searchable by text, date, sender or recipient, classification ("top secret," for example) and many other parameters.

The system returns results quickly. During the demonstration, a search for information about Suriname from 1975 was quickly narrowed in scope to a half- million records. A few seconds later, the search was narrowed further and the results were in: It seems that as pleased as the embassy workers were to have a new truck, they really wished Dodge would hurry up and ship an air conditioner for it as soon as possible.


  • Defense
    Ryan D. McCarthy being sworn in as Army Secretary Oct. 10, 2019. (Photo credit: Sgt. Dana Clarke/U.S. Army)

    Army wants to spend nearly $1B on cloud, data by 2025

    Army Secretary Ryan McCarthy said lack of funding or a potential delay in the JEDI cloud bid "strikes to the heart of our concern."

  • Congress
    Rep. Jim Langevin (D-R.I.) at the Hack the Capitol conference Sept. 20, 2018

    Jim Langevin's view from the Hill

    As chairman of of the Intelligence and Emerging Threats and Capabilities subcommittee of the House Armed Services Committe and a member of the House Homeland Security Committee, Rhode Island Democrat Jim Langevin is one of the most influential voices on cybersecurity in Congress.

Stay Connected


Sign up for our newsletter.

I agree to this site's Privacy Policy.