Improved FISMA scores don't add up to better security, auditor says

GAO official says metrics generally don't measure how well security controls are established

The government’s current choice of metrics is partly to blame for the fact that agencies are reporting improved compliance with security requirements even while government investigators continue to find security gaps, auditors say.

Part of the problem is that although the Office of Management and Budget requires agencies to establish information technology security controls, the metrics generally do not measure how well those controls are implemented, according to the Government Accountability Office.

“Developing and using metrics that measure how well agencies implement important controls can contribute to increased focus on the effective implementation of federal information security,” said Gregory Wilshusen, director of information security issues at GAO, testifying June 25 before the House Science and Technology Committee’s Technology and Innovation Subcommittee.

Wilshusen said the current metrics probably served a useful purpose when they were developed because, at that time, many agencies weren’t performing basic security controls. However, he said, it’s time to examine how agencies implement the controls and consider other types of metrics.

Wilshusen’s testimony echoed findings GAO released in May that said OMB should improve the guidance it gives agencies for complying with the Federal Information Security Management Act. To comply with that law, OMB collects annual reviews of agencies’ information security programs from chief information officers, inspectors general and other agency officials.

Meanwhile, lawmakers continue to ponder reforms to FISMA, which critics say relies too much on paper compliance reports and doesn’t fully address IT vulnerabilities. For example, Sen. Thomas Carper (D-Del.) introduced legislation in April designed to improve FISMA.

The Obama administration has also indicated it might be time to change how IT security is measured. Vivek Kundra, the federal CIO, has said the administration’s initial review of government information security showed that performance data currently collected under FISMA doesn’t accurately reflect the security posture of agencies.

The current collection process is cumbersome and takes time away from meaningful analysis, Kundra has said. Furthermore, there is too much focus on compliance and not enough on outcomes, and a risk-based approach to IT security is needed.

During the hearing, Rep. David Wu (D-Ore.), the subcommittee’s chairman, said a key finding of the Obama administration’s cyberspace policy review was the need for objectives and metrics to accurately measure cybersecurity performance.

“The development of these metrics would provide a base from which we could improve program assessment, budgeting, research and development prioritization and strategic planning,” Wu said.

About the Author

Ben Bain is a reporter for Federal Computer Week.

Featured

  • Cybersecurity

    DHS floats 'collective defense' model for cybersecurity

    Homeland Security Secretary Kirstjen Nielsen wants her department to have a more direct role in defending the private sector and critical infrastructure entities from cyberthreats.

  • Defense
    Defense Secretary James Mattis testifies at an April 12 hearing of the House Armed Services Committee.

    Mattis: Cloud deal not tailored for Amazon

    On Capitol Hill, Defense Secretary Jim Mattis sought to quell "rumors" that the Pentagon's planned single-award cloud acquisition was designed with Amazon Web Services in mind.

  • Census
    shutterstock image

    2020 Census to include citizenship question

    The Department of Commerce is breaking with recent practice and restoring a question about respondent citizenship last used in 1950, despite being urged not to by former Census directors and outside experts.

Stay Connected

FCW Update

Sign up for our newsletter.

I agree to this site's Privacy Policy.