Kelman: An embarrassing question

During a recent trip to Brazil to speak at a conference on public management, I was contacted by a Brazilian public administration graduate student with a straightforward question: Could I direct him to data sources on the performance of the U.S. procurement system?

In speaking about the world’s richest country, I was forced to give an embarrassing answer: We collect surprisingly little such data. As a longtime fan of using performance measurement to improve government performance, I think this is a real shame.

We do collect some kinds of performance data. The government tracks cost growth on major weapons systems. However, cost growth is often a function of overly optimistic initial cost estimates. The degree of optimism varies over time, depending on how tight budgets are and how tight the competition is to get programs funded. Those factors make it difficult to get useful data for comparing the relative cost  of major weapons programs at different times. Inspectors general sometimes audit spare parts prices, but they typically do so to find outrageous examples rather than to examine overall trends.

Collecting good performance data, particularly trend data, on the procurement system is often difficult. Particularly for services, many contracts are unique and therefore hard to compare with those that come before or after.

Nonetheless, a procurement system that took performance measurement seriously would have no shortage of valuable data that could be collected. Just to cite two examples of low-hanging fruit:

  • Prices paid for IT hardware and commercial software, compared by agency and contract and with commercial retail prices. It is shocking that we don’t have historical data on this. We should be collecting data on the Top 10 agency information technology commodity purchases by category to learn how the government is doing and which contracts are delivering the best prices.

  • Customer satisfaction with vendor performance on randomly selected (stratified by dollar value) of IT services contracts. In 2004, two of my students examined customer satisfaction on 100 recent IT services purchases from schedules and found average satisfaction of 9.3 on a 10-point scale, compared with only 6.8 in the late 1980s based on research I did for a book before the procurement reforms of the 1990s occurred. It would be interesting to see if this satisfaction level has gone down since 2004. Satisfaction should also be tracked by contract and agency. Make the surveys simple — just ask customers to rate overall vendor performance between 1 and 10.

Can we justify spending money to collect this type of data when the contracting workforce is so stretched? Companies, which typically put more resources into collecting performance information than government does, find that such data collection more than pays for itself in improved performance.

Beyond that, if IGs redirected resources from endless audits of compliance with process rules to audits of procurement results, good government would get a double boost: Gathering performance information is worthwhile, and process compliance audits often are not. Given the importance of contracting, our failure to gather performance information is a scandal.

Kelman ( is professor of public management at Harvard University’s Kennedy School of Government and former administrator of the Office of Federal Procurement Policy.

About the Author

Kelman is professor of public management at Harvard University’s Kennedy School of Government and former administrator of the Office of Federal Procurement Policy. Connect with him on Twitter: @kelmansteve

FCW in Print

In the latest issue: Looking back on three decades of big stories in federal IT.


  • Anne Rung -- Commerce Department Photo

    Exit interview with Anne Rung

    The government's departing top acquisition official said she leaves behind a solid foundation on which to build more effective and efficient federal IT.

  • Charles Phalen

    Administration appoints first head of NBIB

    The National Background Investigations Bureau announced the appointment of its first director as the agency prepares to take over processing government background checks.

  • Sen. James Lankford (R-Okla.)

    Senator: Rigid hiring process pushes millennials from federal work

    Sen. James Lankford (R-Okla.) said agencies are missing out on younger workers because of the government's rigidity, particularly its protracted hiring process.

  • FCW @ 30 GPS

    FCW @ 30

    Since 1987, FCW has covered it all -- the major contracts, the disruptive technologies, the picayune scandals and the many, many people who make federal IT function. Here's a look back at six of the most significant stories.

  • Shutterstock image.

    A 'minibus' appropriations package could be in the cards

    A short-term funding bill is expected by Sept. 30 to keep the federal government operating through early December, but after that the options get more complicated.

  • Defense Secretary Ash Carter speaks at the TechCrunch Disrupt conference in San Francisco

    DOD launches new tech hub in Austin

    The DOD is opening a new Defense Innovation Unit Experimental office in Austin, Texas, while Congress debates legislation that could defund DIUx.

Reader comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group