GAO: Security accreditation program a tough sell

The federal government's program for testing and accrediting the security of commercial technology has not been proven a success, according to a report by the Government Accountability Office.

The National Information Assurance Partnership (NIAP), which is sponsored by the National Security Agency and the National Institute of Standards and Technology, was created to make it easier for agencies to find products that meet basic industry standards for security.

NIAP officials are responsible for implementing the Common Criteria Evaluation and Validation Scheme, a rigorous set of security tests that adhere to international standards. Officials provide technical guidelines to commercial laboratories that conduct tests on the products vendors submit. Once approved, a product is listed on the NIAP Web site.

Unfortunately, agencies often find that the products they need are not on the list or that only older versions have been accredited, GAO’s report states.

The program has other problems, auditors said. Nearly 10 years after NIAP debuted, vendors still don’t know much about the evaluation process. And the number of qualified validating experts has dropped in the past year, which could lead to delays in evaluations.

On a more fundamental level, NIAP program managers have not established metrics by which to measure the program's effectiveness, GAO’s report states. For example, they have not collected data on the findings, flaws and fixes that resulted from NIAP testing.

The NIAP accrediting process does provide some benefits to the organizations that use it, the report states. It can improve agencies’ confidence that products will work as promised, and vendors can fix flaws identified during the independent testing and evaluation.

The process can also make life easier for vendors and agencies because it allows a broader range of international products, the report states. It can also improve the processes vendors follow when developing new products.

The report made two recommendations to help remedy existing problems. The first would have Defense Secretary Donald Rumsfeld order NSA and NIST to develop workshops for agencies and vendors participating in the NIAP program, the report states.

The Defense Department should also think about collecting, analyzing and reporting metrics on how effective NIAP tests and evaluations are, the report states. The metrics could include summaries of findings, flaws and fixes.

Priscilla Guthrie, DOD’s deputy chief information officer, agreed only partially with the report’s first recommendation. In a response letter to GAO, she agreed that improving awareness and training is important.

However, she added that both NIST and DOD have cut support for NIAP to fund other priorities, making it impossible to allot extra money to such efforts.

DOD should instead direct partner vendors, evaluation laboratories and industry associations to create workshops using existing resources, Guthrie said. They should also bring in help from outside organizations, she added.

She agreed fully with the report’s second recommendation. She said NIAP has been collecting such metrics since 2004 and is developing a template for an end-of-evaluation report that will review all changes to products and vendor procedures throughout the evaluation process.