Kelman: Let the data talk

More than data, anecdotes inform the debate about recent federal acquisition changes

Editor's note: This story was updated at 3:30 p.m. July 10, 2007. Please go to Corrections & Clarifications to see what changed.

Those who want to go back to the procurement methods of the 1980s believe subsequent reforms have led to rapacious contractors getting huge government giveaways. Because of defense spending’s visibility and its ties to the war in Iraq, much of the reform debate is about runaway increases in weapons costs.

I’ll explain, but first let me repeat an idea that is common wisdom among social scientists: The plural of anecdote is data.

Two studies offer data pertinent to the debate. The studies focus on the effect of procurement changes of the mid-to-late 1980s and acquisition reform in the ’90s on the growing weapons costs. Both look at weapons contracts completed in periods before and after the changes of the ’80s and ’90s. The studies examine several hundred contracts.

One study by Christensen, Searle and Vickery, published in 1999 in the Acquisition Review Quarterly, looks at the first period. That was the time of the Project on Government Oversight (POGO) holds up as the last era of true procurement reform. But actually it was a time when competition rules were bureaucratic, laws made it harder for government and industry to communicate, oversight was the watchword, and bid protests flourished.

The second study, a 2002 master’s thesis on cost analysis by Andrew Mosier, looks at the effect of acquisition reforms from the ’90s to the end of 2001.

Here’s how I would summarize the two studies’ findings: The so-called reforms of the mid-to-late ’80s produced an increase in the average cost growth across all weapons systems compared with the previous period. By contrast, the procurement reforms of the ’90s produced a decrease in average cost growth compared with the previous period.

The first study compared the growth of contract costs completed between 1988 and 1991 with those completed between 1992 and 1995. Given the time it takes for a weapons contract to be completed, one can surmise that contracts completed between 1988 and 1991 were undertaken before the reforms of the ’80s were implemented.

The study of the ’90s took a similar approach to contracts completed between 1993 and 1997, comparing those started before the ‘90s reforms had taken hold with those completed between 1998 and 2001.

One interesting finding is that cost growth following procurement changes in the ’80s was greater for development than for production contracts. By comparison, the drop in cost growth following acquisition changes in the ’90s was greater for development contracts. Those results are especially noteworthy because the success of development contracts depends on trust between the government and the contractor, a trust POGO dislikes.

The two papers don’t tell the complete story. A scholar would want to know whether the results are sensitive to small changes in the selected years. However, taking into account the studies’ limitations, I think they show that data is better than anecdotes. Instead, people have let anecdotes dominate the debate about procurement changes.

Kelman is professor of public management at Harvard University’s Kennedy School of Government and former administrator of the Office of Federal Procurement Policy. He can be reached at steve_kelman@
harvard.edu.

The Fed 100

Save the date for 28th annual Federal 100 Awards Gala.

Featured

  • computer network

    How Einstein changes the way government does business

    The Department of Commerce is revising its confidentiality agreement for statistical data survey respondents to reflect the fact that the Department of Homeland Security could see some of that data if it is captured by the Einstein system.

  • Defense Secretary Jim Mattis. Army photo by Monica King. Jan. 26, 2017.

    Mattis mulls consolidation in IT, cyber

    In a Feb. 17 memo, Defense Secretary Jim Mattis told senior leadership to establish teams to look for duplication across the armed services in business operations, including in IT and cybersecurity.

  • Image from Shutterstock.com

    DHS vague on rules for election aid, say states

    State election officials had more questions than answers after a Department of Homeland Security presentation on the designation of election systems as critical U.S. infrastructure.

  • Org Chart Stock Art - Shutterstock

    How the hiring freeze targets millennials

    The government desperately needs younger talent to replace an aging workforce, and experts say that a freeze on hiring doesn't help.

  • Shutterstock image: healthcare digital interface.

    VA moves ahead with homegrown scheduling IT

    The Department of Veterans Affairs will test an internally developed scheduling module at primary care sites nationwide to see if it's ready to service the entire agency.

  • Shutterstock images (honglouwawa & 0beron): Bitcoin image overlay replaced with a dollar sign on a hardware circuit.

    MGT Act poised for a comeback

    After missing in the last Congress, drafters of a bill to encourage cloud adoption are looking for a new plan.

Reader comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group