Watchdog group makes comments more searchable

Federal public comments on proposed regulations now are more easily searchable thanks to a new website created by Sunlight Labs, which is part of the Sunlight Foundation transparency watchdog group.

Sunlight Labs’ new InfluenceExplorer website allows for searches by name of person, company or lawmaker.

It displays nearly a million regulation-related comments culled from the federal e-rulemaking website, said Tom Lee, director of the labs.

“We have nearly 1.5 terabytes of data from, representing every current and historic docket,” Lee said in an interview.

The goal of Sunlight Labs’ project was to make the regulatory comments more easily accessible to the public so that special interest influence over regulations can be researched more readily, he said.

Researchers trying to determine the extent of industry and special interest influence on dockets within tend to have a difficult time searching for that information, he said.

“ is not ideal, and it is very difficult to extract the information,” Lee said. One of the complications for search is that comments are submitted in multiple formats, such as Adobe PDFs and Microsoft Word files.

To make searches and data extractions easier, Sunlight created its own searchable dataset using the comment data.

The Sunlight project also involved downloading comments on regulation from, the Environmental Protection Agency and from Federal Advisory Committees to a transparency website, where the data is available for bulk download.

“This data addition also allows, for the first time, a bulk download of the complete text of a docket or even entire agency from via our TransparencyData site to the public for the first time,” Sunlight officials said in a news release on Oct. 5.

About the Author

Alice Lipowicz is a staff writer covering government 2.0, homeland security and other IT policies for Federal Computer Week.

FCW in Print

In the latest issue: Looking back on three decades of big stories in federal IT.


  • Shutterstock image: looking for code.

    How DOD embraced bug bounties -- and how your agency can, too

    Hack the Pentagon proved to Defense Department officials that outside hackers can be assets, not adversaries.

  • Shutterstock image: cyber defense.

    Why PPD-41 is evolutionary, not revolutionary

    Government cybersecurity officials say the presidential policy directive codifies cyber incident response protocols but doesn't radically change what's been in practice in recent years.

  • Anne Rung -- Commerce Department Photo

    Exit interview with Anne Rung

    The government's departing top acquisition official said she leaves behind a solid foundation on which to build more effective and efficient federal IT.

  • Charles Phalen

    Administration appoints first head of NBIB

    The National Background Investigations Bureau announced the appointment of its first director as the agency prepares to take over processing government background checks.

  • Sen. James Lankford (R-Okla.)

    Senator: Rigid hiring process pushes millennials from federal work

    Sen. James Lankford (R-Okla.) said agencies are missing out on younger workers because of the government's rigidity, particularly its protracted hiring process.

  • FCW @ 30 GPS

    FCW @ 30

    Since 1987, FCW has covered it all -- the major contracts, the disruptive technologies, the picayune scandals and the many, many people who make federal IT function. Here's a look back at six of the most significant stories.

Reader comments

Wed, Oct 5, 2011 Kevin Koehler

Just a quick note to clarify. Influence Explorer has actually been around for over a year now. It launched with campaign finance and lobbying data. We at the Sunlight Foundation have continued to add new data sets since then, like the regulations comments made available today. And there will be more in the future. Thanks for writing about the regulations data! We hope folks out there find it useful.

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group