the lectern banner

By Steve Kelman

Blog archive

The REAL regulatory challenge of agile development

steve kelman

Just a few days after my recent blog post on agile software development, the Office of Management and Budget published a document called TechFAR, designed to encourage use of agile. (No, I had no advance inside knowledge.)

The document is a mixture of two things OMB has tried to do in recent years, both in procurement and in other management areas.

The first is to act as a "mythbuster" (a phrase first associated with Dan Gordon as Office of Federal Procurement Policy administrator, explaining to government officials that the Federal Acquisition Regulation allowed, indeed encouraged, communication with industry prior to the government issuing an RFP). The second is to present templates with advice about how to do new things -- e.g. contract and solicitation clauses – that people may never have tried before and are not sure how to implement.

Both these elements of TechFAR are noteworthy, and should help the government move forward on agile, as well as provide encouragement and topside cover for agencies trying this.

OMB offers a link for making comments (there is no deadline -- this is not a regulation -- but they say that comments received by Sept. 1 can be incorporated in the next edition), and making it a living document is a nice feature.

Nonetheless, I have some problems with TechFAR's description of regulatory challenges for doing agile, and suggestions for how to deal with these -- which I will explain below and plan to submit (in the form of this blog text) as a comment.

The document presents the basic regulatory challenge of doing agile as being the requirement in FAR 15.203 that agencies "identify requirements in their requests for proposals." It continues that for agile "it is not realistic to expect users to know exactly what they need before they see it and rely on refinement of system requirements based on testing and customer feedback after the contract is awarded."

The mythbuster reply is that agencies can meet the FAR requirement "by identifying a Product Vision and coupling it with an explanation of how the Agile process will be used to achieve the Product Vision. Rather than providing a set of 'how to specifications' … the Product Vision will focus on a desired outcome, similar to performance-based contracting, which has been permitted by the FAR for many years."

I believe – though I may be wrong, I would like to hear from agile experts in the agencies – that this language misidentifies the regulatory challenge for agile contracting. This discussion refers to the award of the initial overall development contract for agile, which would presumably then almost always be a task order contract with individual orders representing agile "sprints."

In fact, though, the government contracting landscape is laden with RFPs with extremely vague specifications.

Think of most indefinite-delivery indefinite quantity contracts, which have extremely broad and vague requirements. Think of contracts for R&D or many contracts for staff augmentation, whose requirements (though never stated this way) are basically that the contractor employees do whatever specific jobs under a very broad rubric that the RFP mentions.

Traditional "waterfall" software development RFPs might represent a different culture with much more-detailed specifications, but this is not necessarily the rule in government. Indeed, contracting experts often criticize many government RFPs for being too vague.

So if people routinely get away with using vague requirements when it does not make sense, there should hardly be a problem if people use them when it does, such as here.

I think the problem is actually at the task order level, not the underlying contract level.

I believe there is pressure to develop strict requirements or specifications for a task order, corresponding in agile to a short sprint to develop code. But it is at least my understanding that many or even most agile sprints in the commercial world are not so specific, expressing a time set aside for the work and a direction of movement rather than specific requirements.

TechFAR says that for agile sprints "to ensure results, the Government must ensure that the 'definition of done' is clear, comprehensive, and objective. This definition is established post-award at the beginning of each sprint." Agile experts, please correct me if I'm wrong, but my understanding is that, at least in commercial agile sprints, this is typically not done. So the (probably) non-problem problem that TechFAR has solved at the contract level might still be there where it indeed is a problem, at the task order (sprint) level.

This, in turn, relates to the fact that TechFAR does not discuss evaluation of past performance on sprints. It is my understanding that the way commercial agile users incentivize good performance despite a lack of contractual requirements is through effective, non-bureaucratized use of past performance on earlier sprints. In another recent blog post, I suggested that to make it easier for agile to work, regulatory relief from the documentation and due process requirements for past performance evaluation at the task order level is needed.

One more request for the next edition of TechFAR: Clear guidance on whether the government should hire two or three vendors to compete with each other at the sprint level for business (using the multiple-award task order approach in FAR Part 16), or have the agile development done by a single vendor? I do not have a view on this -- though I have a general prejudice for bringing in a small number of vendors beyond one where the architecture and middleware allow for it -- but would love to see OMB discuss it.

Posted by Steve Kelman on Aug 12, 2014 at 12:03 PM


Reader comments

Mon, Aug 18, 2014 Robin Dymond

I also created an illustrated video to demonstrate how Agile could have saved billions for Healthcare.gov. It is geared to people unfamiliar with Agile. http://www.innovel.net/?p=497

Mon, Aug 18, 2014 Robin Dymond

Great Article Steve. I have been using these ideas for 13 years and currently train and coach people on Agile, Scrum and Lean. I created an Agile contract for a client using one of the big 5 consultancies. In that case we used 3 week sprints, and each sprint plan was an appendix to the agreement. After a Sprint the client had 5 days to accept the work. These days I would use product delivery milestones like Minimum Viable Product (MVP) in contracts, with time constraints to limit exposure, but no scope constraints.

Mon, Aug 18, 2014 John P Brown Virginia

Agile development demands a great deal from participants. Each person must always be aware of what they're doing and why. Teams must constantly reassess their goals and priorities and both managers and executives must give up their control of the process. Agile is a bottom-up process of constant learning and discovery. When traditional development habits creep in to an Agile process, the results are sure to be poor. I oversaw product development at a San Francisco supply chain startup and took away a core learning that every element of the system must be adapted to allow the benefits of Agile to come forth. This means how developers do their work, how teams plan, how organizations allocate resources and how customers, board members, and investors set expectations. Products are never "done" - they are constantly being tweaked and revised. Contracts that don't reflect this introduce chaos and significantly reduce the chances of success. If waterfall development can be compared to an traditional automobile assembly line, Agile development is akin to hiring a team of engineers to design and build something new. Perhaps there is language in NSF grants that allows for this sort of exploration and flexibility?

Fri, Aug 15, 2014

A key to Mark's concept rests at the end of his comments: "But, importantly, their work is constantly visible to the government." What Mark is saying, I believe, is that if the work is sufficiently visible (gov has access to code, and understands the code), then it is not so difficult for the government to switch vendors if performance is not optimal. That is a very powerful way to address the challenge that Steve Kelman cites in his comment, "unless it is relatively easy to turn that evaluation into positive or negative consequences for the vendor, I fear it will be difficult for the government to get the good performance it needs." I.e., relatively short task orders; a high level of transparency of the work, providing the government with flexibility; the incentive of future work. One assumption in this entire discussion that I would like to challenge is the concept of past performance. Past performance most likely exists to serve as evidence that a vendor will not bring large risk to a procurement. But with short duration task orders, this risk is small, and perhaps the agile concept of "fail early" is applicable: i.e., give new vendors a chance and see if they can prove themselves. The current vendor landscape is too much of a closed community: lowering the barrier to entry might make it easier to get the best people.

Fri, Aug 15, 2014 Mark Nelson Washington DC

In Agile, discovering requirements is part of the process, and, almost by definition, they cannot be unambiguously defined upfront, whether as part of an overall contract, or individual task order. Additionally, a key component of Agile is shared ownership, which would mean the contractor and the government are jointly responsible for the outcomes. Finally, using multiple contractors, and competing each iteration of a Scrum or Release, would go against another fundamental part of Agile, which is that teams, over time, develop their own velocity and ability to predict outcomes, and measures do not translate across teams.

Show All Comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above