GAO: Common Criteria is not common enough

Auditors say the process takes too long and its effectiveness is not well-understood

"Information Assurance: National Partnership Offers Benefits, but Faces Considerable Challenges"

Related Links

Many vendors understand the importance of getting products certified under a set of security standards called the Common Criteria Evaluation and Validation Scheme, but the organization that oversees the program has not done enough to educate agencies or vendors about it, according to a Government Accountability Office report released last week.

GAO also criticized the National Information Assurance Partnership (NIAP) for not providing metrics or evidence that the Common Criteria actually improves product security. In addition, the Common Criteria process takes so long to complete that agencies often find that the products they need are not on the list of certified offerings or that only older versions have been accredited, GAO’s report states.

Products undergoing certification and accreditation can be obsolete by the time they are approved, said Daniel Kent, director of systems engineering for U.S. federal sales at Cisco Systems.

Ideally, the certification and accreditation process should take no more than six months, Kent said. However, in reality, 10 to 18 months is common, he said.

The government should establish centers of excellence for testing so agencies wouldn’t have to duplicate their efforts and vendors wouldn’t waste time and resources, he said.

It is possible to complete the testing process in as little as two to four weeks, said Helmut Kurth, chief scientist and lab director at atsec, an information technology security consulting firm that performs Common Criteria testing. That is fast enough to ensure that state-of-the-art technology can get out in the field.

“It’s possible to do evaluation in parallel with development,” and labs and vendors must be prepared to do that, he said.

NIAP certification often is too slow for defense and intelligence agencies, said John Pescatore, vice president of Internet security research at Gartner. Only government labs can test at Common Criteria Evaluation Assurance Levels 5 through 7 — the highest levels of scrutiny. NIAP now has fewer experienced testing employees and is not replacing them, which will further lengthen the process, he added.

To help remedy existing problems, NIAP program managers should create metrics that measure the program’s effectiveness and collect data on the findings, flaws and fixes that resulted from NIAP testing, according to GAO’s report.

Priscilla Guthrie, the Defense Department’s deputy chief information officer, said in a written response to GAO’s report that NIAP has been collecting such metrics since 2004 and is developing a template for an end-of-evaluation report that will review all changes to products and vendor procedures throughout the evaluation process.

The GAO report adds that Defense Secretary Donald Rumsfeld should order the National Security Agency and the National Institute of Standards and Technology, NIAP’s sponsors, to develop workshops for agencies and vendors participating in the NIAP program.

Guthrie agreed that improving awareness and training is important. However, she added that NIST and DOD have cut support for NIAP to fund other priorities, making it impossible to allot extra money to such efforts.

DOD should instead direct partner vendors, evaluation laboratories and industry associations to create workshops using existing resources, Guthrie said. They should also get help from outside organizations, she added.

The problems the GAO report describes are not problems with NIAP itself, said Salvatore La Pietra, president and co-founder of atsec. “It’s easy for agencies to criticize NIAP, but they probably don’t use the processes correctly in the first place” because they’re not educated about them, he said. “They have to do their homework.”

Pescatore said GAO’s call for increased education and awareness of NIAP’s function is overblown. Large vendors already know the process well and can afford millions of dollars for tailor-made product evaluations, he said.

Any education efforts should target smaller vendors — with $10 million to $50 million a year in annual revenue — that don’t know about the NIAP process, don’t know how expensive it is and have trouble affording it, Pescatore said. NIAP must do more than educate, he added. It must provide subsidies or reduce prices so smaller vendors can participate, he said.


**********

Security experts on NIAP: A case of steel doors on grass huts

The Government Accountability Office’s report on the National Information Assurance Partnership missed at least two critical issues, security experts say.

The organization’s security criteria require products to have necessary security features, but they do not call for testing for exploitable weaknesses in other features, said John Pescatore, vice president of Internet security research at Gartner.

“This process could be used to drive all software to higher levels of security,” Pescatore said. “Now it’s just being used as a procurement checklist.”

Another problem that the GAO report does not sufficiently address is how to keep track of certifications for updated versions of certified products, said Helmut Kurth, chief scientist and lab director at atsec, an information technology security consulting firm that performs Common Criteria testing.

The Common Criteria Recognition Arrangement and the Common Criteria Development Board must define and agree on a scheme to maintain product certifications when products change, Kurth said.

Customers that need a new feature in a later version of a product currently must wait for that later version to go through the certification and accreditation process, said Daniel Kent, director of systems engineering for U.S. federal sales at Cisco Systems.

— Michael Arnone

The 2014 Federal 100

Get to know the 100 women and men honored this year for going above and beyond in federal IT.

Reader comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above