TheLectern

By Steve Kelman

Blog archive

Experiments to improve public-sector management

In a column I just wrote, I proposed organizing an experiment to see whether providing dramatically increased resources for managing service contracts would improve contract cost and performance. Take a group of contracts and increase the number of contract management bodies devoted to the contract, say by threefold. Then take a control group of other contracts and don't change how they are managed. Look two years later and see if the first group have evolved in a more positive way than the second.
 
The proposal I made in the column was an example of an approach to improving government performance that deserves far more use than it's getting. We should be doing more actual experiments with an experimental group of organizations that receives some treatment in the form of some management practice or approach, a control group that doesn't receive the treatment, and an evaluaion after time has passed comparing performance of the two groups. If the treatment works, we should consider applying it more broadly; if it doesn't, we can try a different treatment to see what happens.
 
Of course, any time a Social Security senior manager in Washington compares the performance of various Social Security offices along some dimension, and looks to see which offices are performing better than others and why, that manager is in effect running an experiment. There are also a number of examples of doing large, expensive formal experiments (usually conducted for the government by university-based researchers) to evaluate various social policies -- such as giving poor people housing vouchers or sending children to charter schools -- and see whether they work or not.
 
The kinds of experiments I have in mind are somewhere in-between the informal comparisons that managers do using performance data available across organizational or team units on the one hand, and expensive experiments that often take years and cost millions of dollars, run by scholars. The former have sample sizes that may be too small and too many confounding factors that may explain observed differences across units to allow drawing conclusions with any degree of comfort. The latter apply rigorous standards of sample size, random selection, and so forth, but are very expensive and time-consuming.

We need something in the middle -- probably not rigorous enough to merit publication in an academic journal, but paying some attention to principles of experimental design to give the findings greater believability. Imagine experiments, say, of the impact of various recruitment techniques on the yield rate for highly sought-after candidates for federal jobs, or of the impact of changing some feature of an agency website on public usage or favorability.  Companies do experiments such as these regarding their products or services all the time, and we should apply this idea to government.
 
If we are going to improve the management of the public sector, we are going to need more efforts such as these. There is a constituency for formal program evaluations for government, especially in the Office of Management and Budget and the policy shops that exist in some cabinet agencies. We need a constituency of management experts and senior government executives who care about improving agency performance to start this inside government organizations. If there is interest in the government, I bet I could rustle up some academics who would like to help.

Posted on Jan 28, 2011 at 12:09 PM


Reader comments

Mon, Jan 31, 2011 Jacob

Ultimately, I believe that A. public management needs more of a scientific approach and B. scientific research needs better management. A. If the government doesn't experiment enough, then they do not have enough opportunity to actually test out their ideas before implementing them permanently. Currently, ideas for better management are often neglected because there is no definitive way of knowing if they will work. Or, ideas that are accepted are sometimes implemented before we can see how they would work. Experimentation would allow more good ideas to become implemented while stopping bad ideas before they are implemented. B. From personal experience in a Prostate Cancer research lab, management is often decentralized and inefficient. I am fortunate to have a good lab head, however many labs have many management issues that result in miscommunications, mislead research, and general confusion. I would support the creation of a National Lab to cosolidate, centralize, and prioritize pertinent national medical problems, while allowing private labs to do more specialized research. So research needs more centralized authority, while the government needs more discretion and experimentation.

Sat, Jan 29, 2011 Jacob

Correction to last comment: I exagerrated when I said the government wastes 100s of billions a dollars a year currently on inefficient contracting. As you said, we could more likely save 25 billion a year. In general, the government works a lot better than I, and the rest of America, give it credit for. By the way, I bet there's a correlation between Clinton's budget surplus and your procurement reform!

Sat, Jan 29, 2011 Jacob

We should not be too reliant on sole-source contracts and cost-reimbursement contracts, which potentially waste considerable money. I appreciate that the president has required agencies to submit spending and performance data to http://www.recovery.gov. Although your average joe will not look on this website, it does promote accountability and transaparency that allows me to make sure that the government is at the very least attempting to get the best deals for the American people. Since 100s of billions of dollars are probably currently being wasted from innefficient contracting, I do believe that Obama should hire more contracting workers. In the long run, this would save a lot of money. Unfortunately, Obama may have to wait until after re-election at this point because the media will frame him for wasting money, when in reality more contracting workers would save money in the long run. In general, the media does limit the government's ability to perform to its best. Hopefully after re-election, Obama will not have to worry about the media and will be able to save this country in the long-run. The long-term problems we face - the growing debt, losing our competitive edge, global warming, education, and social security, probably need to be solved by risky political moves that may indeed look bad in the short-run. Hopefully we can get beyond the media and solve our long-term problems realizing that the short-term is only a small piece of the bigger picture. Without enough experimentation, the government is blind to the numerous ways we can be more efficient and save money. Since presidents can serve no more than 2 terms, often it looks better for them to only think about the short-term. But I agree: let's experiment more now and save money in the long-term. This could significantly cut the national debt.

Fri, Jan 28, 2011 Front Liner District of Columbia

Thoughtful, but many service contracts are not suitable for your laudable statistical approach. Butts-in-seats contracts don't work because they are under government supervision, just like govt emps. (Whatever happened to the illegal personal services contracts?) Study contracts are unique. A real transactional contract, such as SAIC running USAF spare parts warehouses, might work. But we would really want to compare that with government performance (either prior, or another such govt-staffed operation), or perhaps with the private sector. However, what I don't like about your proposal is that it would take too many years (10-20, say). The reason is taht there are too many cooks and some of them will press for perfection in the analysis, or total consensus about the results. Neither is possible--and frankly the private sector rarely shoots that high, in part because they want results, (and now) and know the present-value of endless studying. The simplest solution is to ask better performance and productivity from career civil servants. I am afraid our expectations of civil servants have been dropping steadily. As a result, there is a lot of hand-wringing about contract management and oversight. You know better than many, or most, that it is not rocket science. What a CO or COR does is plain as day, has not changed much, and is not that hard. If we just got people to do their jobs--the old fashioned way--we would not need to pause a decade or two to inject some scientific aura that is not needed.

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above