By Steve Kelman

Blog archive

'Data-driven reviews' and agency performance

Shutterstock image (Macrovector): business management discussion brainstorming session.

Last week the Government Accountability Office issued its second report (the first was in 2013) on implementation of the Government Performance and Results Modernization Act of 2010. Given my ongoing interest in this topic, I checked it out, only to find there wasn't that much new in it, mainly summarizing earlier reports. But one of those cited reports, from this past July (which came out while I was either in or just getting out of the hospital with my cancer treatment), turned out to be filled with useful and surprising information.

That report is titled, in great GAO-ese, "Agencies Report Positive Effects of Data-Driven Reviews on Performance but Some Should Strengthen Practices." A search of the website suggested FCW did not cover this report last July, so it's probably not too late to write about it.

This report documents what I would contend is the single most significant change in the management of the federal government in quite a number of years: The spread of regularized meetings involving senior agency officials and career managers to examine using data, how the organization is doing in meeting its priority performance targets, and to deliberate about steps that can be taken when progress falls short. These kinds of meetings are an important staple of life in big companies, and are considered essential to driving corporate performance. But they have never really been a part of how most agencies are managed.

"Data-driven reviews" got their start with Commissioner William Bratton's COMPSTAT system for the New York City police in the early 1990s. They attracted significant attention from other police departments, and then from a growing number of cities, where they became known as "STAT" meetings. (My colleague Robert Behn published a book last year on common features of such city systems, The PerformanceStat Potential.)

Such meetings sporadically began in the federal government after passage of the Government Performance and Results Act (GPRA) in 1993, but in many agencies GPRA was a tick-the-box compliance exercise that produced reports for outside overseers to look at or, often, to just gather dust on a shelf.

The idea of data-driven "STAT"-style meetings got wind in its sails when the Obama administration announced what GAO calls a move "away from the passive collection and reporting of performance information to a model where performance information is actively used by agency officials to inform decision-making, which is more likely to lead to performance improvements." (Full disclosure: My wife, Shelley Metzenbaum, headed this effort in the first years of the administration.) Then Congress passed the GPRA Modernization Act, which called for data-driven meetings, quarterly or more often, for CFO Act agencies.

Now, a few years later, GAO finds that 20 of 23 agencies are actually holding such meetings, quarterly or more often. (The General Services Administration holds them once a week; the Department of Commerce, Department of Veterans Affairs and NASA monthly; and the Social Security Administration every other month.)

Amazingly, in four agencies, the meetings are led by the agency head; in another 10 the chief operation officer/deputy secretary takes the reins. Only three agencies' meetings are led by the performance improvement officer, a somewhat lower-ranking career official. This represents a whole new level of senior agency leader involvement in disciplined, focused, regularized discussion of performance improvement.

Since earlier GAO surveys examining what factors promote use of performance information at lower levels of the system found that top-leader support was one of the best predictors, this new senior leader involvement bodes well for using performance information at lower levels of the government.

GAO also attended reviews at several agencies, and interviewed performance improvement officers about the process and results. (Since I am skeptical of the objectivity of these officials in judging how successful their efforts have been, I am not reporting their subjective self-assessments here, and instead focusing on answers to factual statements.)

What happens at these meetings? They almost always go through the agency's priority goals, often with a red/yellow/green traffic-light rating of progress towards meeting each performance target. Nineteen of 22 agency PIOs reported that meetings are always used to identify goals at risk, and to have goal leaders (the official formerly in charge of working to meet each goal) explain why the goal is at risk, then discuss with those present possible strategies for improvement.

A somewhat smaller set of 12 agencies reported always identifying and agreeing on specific follow-up actions to be taken after the meeting, and nine said they check in between meetings on the status of follow-ups. And 19 of 22 agencies identified getting accurate and timely data as a challenge.

There is a significant overlap between these meetings and the movements for increased use of data analytics and for "evidence-based government." Sometimes data analytics underlie conclusions about what approaches will work best to drive a performance improvement.

"Evidence-based government" is often discussed in a legislative or OMB context with regard to choices about what programs to fund or not to fund, data-driven meetings often assume the underlying program (say, for example, the desirability of increasing production of renewable energy on federal lands the Interior Department manages, which has been one of the Department's priority goals) and use evidence to choose among alternative ways to implement the goal. All of these efforts fit into a broader attempt to improve how well government delivers results.

If somebody had asked me five years ago what I thought the prospects were that almost all cabinet agencies would be holding regular meetings focused on performance targets and led by senior agency leadership, I might not have asked the person asking what they were smoking, but I certainly would have been skeptical. These GAO findings represent big progress in a modest period of time. The next challenge will now be to make sure this new practice survives the change of administration in 2017, and become as normal to government as it is to GE or Walmart.

Posted by Steve Kelman on Oct 06, 2015 at 4:27 AM


  • IT Modernization
    shutterstock image By enzozo; photo ID: 319763930

    OMB provides key guidance for TMF proposals amid surge in submissions

    Deputy Federal CIO Maria Roat details what makes for a winning Technology Modernization Fund proposal as agencies continue to submit major IT projects for potential funding.

  • gears and money (zaozaa19/

    Worries from a Democrat about the Biden administration and federal procurement

    Steve Kelman is concerned that the push for more spending with small disadvantaged businesses will detract from the goal of getting the best deal for agencies and taxpayers.

Stay Connected