the lectern banner

By Steve Kelman

Blog archive

Why a GAO report documenting improvement is not good news

man graphs performance

The Government Accountability Office recently issued a new report called "DOD Acts to Improve the Reporting of Past Performance Information", documenting progress the Defense Department has made getting past-performance report cards on contractors submitted into DOD's past-performance database on time. Now 74 percent of reports are being submitted on time, compared to 56 percent two years ago.

Sounds like a good news story, right?

I'm not so sure.

Done right – and the government currently is a long ways from doing it right, unfortunately – past-performance report cards are one of the most powerful tools in the government's toolkit to incentivize better vendor performance. Everybody knows that considering the past performance of people from whom we buy is one of the most powerful tools customers have to make the market system work better – suppliers treat us right in the hope of getting our repeat business.

There are serious problems with the current operation of the past-performance system in the government, as I have frequently written. The biggest problem is insufficient honesty in reports, so they don't differentiate enough between good performance that should be rewarded and bad performance that should be punished. I have frequently argued that the government should eliminate the contractor's ability to challenge a past performance evaluation they don't like, because it disincentivizes government honesty. Putting their own version of events into the system should be all contractors are entitled to.

Homeowners who have used the Angie's List service will know what I'm talking about. People who have had experiences with service companies post reviews about them on the website. The companies may post responses to the reviews, but have no power to have them changed or removed.

There is also an attitude by many, including the punishment-first caucus on the Hill and some so-called "watchdog" groups, that past performance should be used only to punish the bad, not to reward the good.

Unfortunately, the GAO report -- despite what I am sure are good intentions, and a title that sounds promising -- doesn't really help. In fact, it may hurt the important effort to improve the government's past-performance system.

Why?

What the GAO report discusses is the extent of compliance with the requirement to submit past-performance report cards, and they praise DOD because “compliance” has increased. The report repeatedly uses the word "compliance" with regard to past-performance report cards, sometimes several times in one paragraph. But when government people hear the word "compliance," most think about something unpleasant but (probably or at least possibly) necessary. It's not something we want to do, but a bitter pill that must be swallowed for the greater good.

The second problem is the performance measure the report uses to measure compliance. Although GAO itself has in the past complained about inadequate content in the report cards that do appear in the system, in particular with regard to descriptive narrative (they should also complain about the lack of differentiation in grades), there is none of that in this report. All they measure is the input of how many reports get submitted on time, not about whether they are any good or whether anyone is using them.

Between the emphasis on compliance and the use of a poor, input-style performance measure, the GAO report sends a dismal signal about what past performance report cards are about – they are a "drill," a "data call," imposed by "them" for their purposes, not for us on the front lines of the contracting system. If we fill in these report cards, the only reason is to tick a box. This is not for us, it is for them.

But the past-performance system absolutely should be for "us" – for the customers of the contracting system -- as a tool for getting better contracting performance on our contracts. Clearly, there will be a "free rider" problem – even in a system with honest, valuable report cards, most users of the procurement system would prefer that everyone else fill out their report cards while neglecting to complete their own. A procurement system that took past performance seriously would look for ways to deal with the free rider problem, perhaps with a feedback system similar to that consulting firms use for the lessons-learned reports consultants file for use by other consultants, perhaps including some kind of rewards for the reports that the most users say influenced their contract award decision.

My point is that we need some serious thinking (and action) about improving the way the past-performance system works. I don't think this GAO report helps.

A job for OFPP?

Posted on Jul 11, 2013 at 2:10 PM


The Fed 100

Save the date for 28th annual Federal 100 Awards Gala.

Featured

  • computer network

    How Einstein changes the way government does business

    The Department of Commerce is revising its confidentiality agreement for statistical data survey respondents to reflect the fact that the Department of Homeland Security could see some of that data if it is captured by the Einstein system.

  • Defense Secretary Jim Mattis. Army photo by Monica King. Jan. 26, 2017.

    Mattis mulls consolidation in IT, cyber

    In a Feb. 17 memo, Defense Secretary Jim Mattis told senior leadership to establish teams to look for duplication across the armed services in business operations, including in IT and cybersecurity.

  • Image from Shutterstock.com

    DHS vague on rules for election aid, say states

    State election officials had more questions than answers after a Department of Homeland Security presentation on the designation of election systems as critical U.S. infrastructure.

  • Org Chart Stock Art - Shutterstock

    How the hiring freeze targets millennials

    The government desperately needs younger talent to replace an aging workforce, and experts say that a freeze on hiring doesn't help.

  • Shutterstock image: healthcare digital interface.

    VA moves ahead with homegrown scheduling IT

    The Department of Veterans Affairs will test an internally developed scheduling module at primary care sites nationwide to see if it's ready to service the entire agency.

  • Shutterstock images (honglouwawa & 0beron): Bitcoin image overlay replaced with a dollar sign on a hardware circuit.

    MGT Act poised for a comeback

    After missing in the last Congress, drafters of a bill to encourage cloud adoption are looking for a new plan.

Reader comments

Tue, Aug 6, 2013 John Inman Williston, Vermont

We need to remember the basics, and it is a basic and fundamental truth that past performance is an extremely important factor in making a purchase decision, especially an important purchase decision. In this context, past performance = reputation, or past performance = confidence in a successful future outcome (based on the past). Government contracting officers and source selection officials need to make sound purchase decisions, and sound purchase decisions MUST include past performance -- at least, that's how I tend to see it.

Fri, Jul 12, 2013 Steve Kelman

Thanks for these comments from blog readers! Al, as a general matter, if something doesn't work, we should look to trying to improve or to scrap. I would be surprised if there were no connection between actual past performance and actual future performance but if, hypothetically, this were true, yes, wouldn't make sense to devote resources to this. Note that most of us in everyday buying decisions do go back to the same auto repair shop if we believe they did a good job last time.

Fri, Jul 12, 2013 Mike L.

We have been seeing more and more RFP evaluation criteria that read, "in the absence of past project references or performance reports, the bidder's past performance will be deemed acceptable." Pretty much defeats the whole purpose!

Thu, Jul 11, 2013 Molly G

While past performance is always listed as an evaluation factor, no matter how many times they say things like "best value" or "technical is significantly more important than price," in practice it always comes down to low price.

Thu, Jul 11, 2013 Al

A compromise for immunity on the content of those past performance reports would be to concede that any data over 5 years old is purged. Very old data should be discounted anyways (relevancy and *currency*), so this should not be a huge concession. This may also dissuade litigation based upon contested information in the database. I know you are a big picture-type thinker, Steve. What if we study past performance data and find very little correlation between past and future performance by firms? Should we scrap the past performance regime entirely in that case?

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group