Kelman: An embarrassing question

During a recent trip to Brazil to speak at a conference on public management, I was contacted by a Brazilian public administration graduate student with a straightforward question: Could I direct him to data sources on the performance of the U.S. procurement system?

In speaking about the world’s richest country, I was forced to give an embarrassing answer: We collect surprisingly little such data. As a longtime fan of using performance measurement to improve government performance, I think this is a real shame.

We do collect some kinds of performance data. The government tracks cost growth on major weapons systems. However, cost growth is often a function of overly optimistic initial cost estimates. The degree of optimism varies over time, depending on how tight budgets are and how tight the competition is to get programs funded. Those factors make it difficult to get useful data for comparing the relative cost  of major weapons programs at different times. Inspectors general sometimes audit spare parts prices, but they typically do so to find outrageous examples rather than to examine overall trends.

Collecting good performance data, particularly trend data, on the procurement system is often difficult. Particularly for services, many contracts are unique and therefore hard to compare with those that come before or after.

Nonetheless, a procurement system that took performance measurement seriously would have no shortage of valuable data that could be collected. Just to cite two examples of low-hanging fruit:

  • Prices paid for IT hardware and commercial software, compared by agency and contract and with commercial retail prices. It is shocking that we don’t have historical data on this. We should be collecting data on the Top 10 agency information technology commodity purchases by category to learn how the government is doing and which contracts are delivering the best prices.

  • Customer satisfaction with vendor performance on randomly selected (stratified by dollar value) of IT services contracts. In 2004, two of my students examined customer satisfaction on 100 recent IT services purchases from schedules and found average satisfaction of 9.3 on a 10-point scale, compared with only 6.8 in the late 1980s based on research I did for a book before the procurement reforms of the 1990s occurred. It would be interesting to see if this satisfaction level has gone down since 2004. Satisfaction should also be tracked by contract and agency. Make the surveys simple — just ask customers to rate overall vendor performance between 1 and 10.

Can we justify spending money to collect this type of data when the contracting workforce is so stretched? Companies, which typically put more resources into collecting performance information than government does, find that such data collection more than pays for itself in improved performance.

Beyond that, if IGs redirected resources from endless audits of compliance with process rules to audits of procurement results, good government would get a double boost: Gathering performance information is worthwhile, and process compliance audits often are not. Given the importance of contracting, our failure to gather performance information is a scandal.

Kelman ( is professor of public management at Harvard University’s Kennedy School of Government and former administrator of the Office of Federal Procurement Policy.

About the Author

Kelman is professor of public management at Harvard University’s Kennedy School of Government and former administrator of the Office of Federal Procurement Policy. Connect with him on Twitter: @kelmansteve


  • Cybersecurity
    malware detection (Alexander Yakimov/

    Microsoft targets copycat influence websites

    Microsoft went to court to take down websites it believes to be part of a foreign intelligence operation targeting conservative think tanks and the U.S. Senate.

  • Cybersecurity
    secure network

    FAA explores shifting its network to FISMA high

    The Federal Aviation Administration is exploring an upgrade to the information security categorization of IT systems as part of air traffic control modernization.

  • Cybersecurity
    Shutterstock photo id 669226093 By Gorodenkoff

    The disinformation game

    The federal government is poised to bring new tools and strategies to bear in the fight against foreign-backed online disinformation campaigns, but how and when they choose to act could have ramifications on the U.S. political ecosystem.

Stay Connected

FCW Update

Sign up for our newsletter.

I agree to this site's Privacy Policy.