Kelman: Let the data talk

More than data, anecdotes inform the debate about recent federal acquisition changes

Editor's note: This story was updated at 3:30 p.m. July 10, 2007. Please go to Corrections & Clarifications to see what changed.

Those who want to go back to the procurement methods of the 1980s believe subsequent reforms have led to rapacious contractors getting huge government giveaways. Because of defense spending’s visibility and its ties to the war in Iraq, much of the reform debate is about runaway increases in weapons costs.

I’ll explain, but first let me repeat an idea that is common wisdom among social scientists: The plural of anecdote is data.

Two studies offer data pertinent to the debate. The studies focus on the effect of procurement changes of the mid-to-late 1980s and acquisition reform in the ’90s on the growing weapons costs. Both look at weapons contracts completed in periods before and after the changes of the ’80s and ’90s. The studies examine several hundred contracts.

One study by Christensen, Searle and Vickery, published in 1999 in the Acquisition Review Quarterly, looks at the first period. That was the time of the Project on Government Oversight (POGO) holds up as the last era of true procurement reform. But actually it was a time when competition rules were bureaucratic, laws made it harder for government and industry to communicate, oversight was the watchword, and bid protests flourished.

The second study, a 2002 master’s thesis on cost analysis by Andrew Mosier, looks at the effect of acquisition reforms from the ’90s to the end of 2001.

Here’s how I would summarize the two studies’ findings: The so-called reforms of the mid-to-late ’80s produced an increase in the average cost growth across all weapons systems compared with the previous period. By contrast, the procurement reforms of the ’90s produced a decrease in average cost growth compared with the previous period.

The first study compared the growth of contract costs completed between 1988 and 1991 with those completed between 1992 and 1995. Given the time it takes for a weapons contract to be completed, one can surmise that contracts completed between 1988 and 1991 were undertaken before the reforms of the ’80s were implemented.

The study of the ’90s took a similar approach to contracts completed between 1993 and 1997, comparing those started before the ‘90s reforms had taken hold with those completed between 1998 and 2001.

One interesting finding is that cost growth following procurement changes in the ’80s was greater for development than for production contracts. By comparison, the drop in cost growth following acquisition changes in the ’90s was greater for development contracts. Those results are especially noteworthy because the success of development contracts depends on trust between the government and the contractor, a trust POGO dislikes.

The two papers don’t tell the complete story. A scholar would want to know whether the results are sensitive to small changes in the selected years. However, taking into account the studies’ limitations, I think they show that data is better than anecdotes. Instead, people have let anecdotes dominate the debate about procurement changes.

Kelman is professor of public management at Harvard University’s Kennedy School of Government and former administrator of the Office of Federal Procurement Policy. He can be reached at steve_kelman@
harvard.edu.

The Fed 100

Read the profiles of all this year's winners.

Featured

  • Then-presidential candidate Donald Trump at a 2016 campaign event. Image: Shutterstock

    'Buy American' order puts procurement in the spotlight

    Some IT contractors are worried that the "buy American" executive order from President Trump could squeeze key innovators out of the market.

  • OMB chief Mick Mulvaney, shown here in as a member of Congress in 2013. (Photo credit Gage Skidmore/Flickr)

    White House taps old policies for new government makeover

    New guidance from OMB advises agencies to use shared services, GWACs and federal schedules for acquisition, and to leverage IT wherever possible in restructuring plans.

  • Shutterstock image (by Everett Historical): aerial of the Pentagon.

    What DOD's next CIO will have to deal with

    It could be months before the Defense Department has a new CIO, and he or she will face a host of organizational and operational challenges from Day One

  • USAF Gen. John Hyten

    General: Cyber Command needs new platform before NSA split

    U.S. Cyber Command should be elevated to a full combatant command as soon as possible, the head of Strategic Command told Congress, but it cannot be separated from the NSA until it has its own cyber platform.

  • Image from Shutterstock.

    DLA goes virtual

    The Defense Logistics Agency is in the midst of an ambitious campaign to eliminate its IT infrastructure and transition to using exclusively shared, hosted and virtual services.

  • Fed 100 logo

    The 2017 Federal 100

    The women and men who make up this year's Fed 100 are proof positive of what one person can make possibile in federal IT. Read on to learn more about each and every winner's accomplishments.

Reader comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group