A measurement tool intended to help agencies analyze the management of their information security programs is being scaled back
A measurement tool intended to help agencies analyze the management of their
information security programs is being scaled back after agency officials
and security experts questioned whether it could also measure the effectiveness
of those programs.
The CIO Council's security committee started development of its Information
Technology Security Maturity Framework late last year after Rep. Stephen
Horn (R-Calif.) announced his intention to grade agencies' security postures.
Horn announced grades last week , giving a government- wide grade of D-minus, but the framework is far from complete.
After working with the National Institute of Standards and Technology's
Computer Security Division, the committee released a draft of the framework
for general comment in July. Reactions were generally favorable, and the
committee plans to release the first official version in October. But it
will likely cover only the first three levels of assessment — the agencies'
plans to secure their systems and early implementation — and the rest will
be left for future improvement, said John Gilligan, committee co-chairman.
"We will continue to evolve the framework," he said.
The first real criticism came from the General Accounting Office, which
has developed its own metrics for measuring agencies' security for audits
that focus not only on whether agencies have plans, but also on whether
the plans are working. The framework, GAO said, did not pay enough attention
to whether the plans are working. "They felt the initial draft had done
a good job of identifying process but could be stronger in identifying effectiveness,"
The Computer System Security and Privacy Advisory Board, a government/
industry group that advises NIST, Congress and the Office of Management
and Budget, last week sent a letter to the committee expressing concerns
with the framework's assumption that good processes equal good outcomes.
"The problem with that assumption is that demonstrating its truth would
require some kind of scientific experiment.... It is just very hard to prove,"
said board member Stephen Lipner, manager of Microsoft Corp.'s Security
The committee held a workshop Sept. 15 to review the changes to the
framework. Changes will include cutting back on the higher levels of assessment,
which are "too fuzzy" and need to be better defined, said Marianne Swanson,
a computer specialist at the NIST Computer Security Division who has been
leading much of the work on the framework.
NIST will also work to issue more specific guidance on the "basic requirements"
for different maturity levels, Swanson said. NIST has already provided draft
general guidelines, but the future requirements will likely be based on
several NIST special publications, GAO guidance, and OMB's Circular A-130
Appendix III, which outlines federal requirements for information security.
NEXT STORY: Ex-deputy CIO fined $30K