Buy or build? For IT, it's custom vs. COTS

It's a timeless question — and the answer might not be as clear-cut as agencies think.

FCW magazine's feature story for July 30.

If agencies had all the money and time they needed, most would probably still prefer to build their own IT applications and house them on hardware located on agency premises. But that option is no longer viable.

Now, IT departments are governed by an ever-present squeeze on budgets and resources and a brave new world of 24/7 expectations — from the public to provide information and services and from their own employees for the up-to-date tools they need to do their jobs.

The appeal of building solutions in-house will likely never go away. The bespoke approach, after all, comes with the ability to exercise greater control over development and quality, integrate new technology more tightly with existing applications, better track documentation for ongoing support and react more quickly to requirement changes.

Some agency IT leaders have even pushed back against a headlong rush to the new. Last year, Terry Halvorsen, who was the Navy Department’s CIO at the time, cautioned that new is not always the answer.

“In today’s world, where technology changes in the blink of an eye and the race to be faster, more capable, bigger, etc., moves at lightning speed, we sometimes forget to ask ourselves that very simple question,” he wrote in a blog post. “As prudent stewards of [Navy] resources, we have to decide when it makes sense to invest in new technology” and when to stay with something that is still reliable and fully supports the Navy’s mission.

Nevertheless, new — even if that means embracing a modified version of old — is the goal for most agencies. The question then becomes whether to buy or find a way to efficiently and cost-effectively build.

From custom to COTS

Commercial off-the-shelf software has long been the answer for many government needs. For example, agencies have used Microsoft’s suite of word-processing, spreadsheet, collaboration and email applications for years.

However, the more complex the requirements that COTS software is expected to meet, the more that software must be customized. At some point, customization becomes so extensive and costly that it makes more sense for agencies to build the software themselves.

The Defense Information Systems Agency reached that point a few years ago when officials reviewed COTS applications and concluded that the available options were not appropriate, were too expensive or would have to be changed too much to meet the agency’s workflow requirements.

On the occasions when DISA officials turned to outside developers for help, they were told the applications could not be built at all.

So they decided to develop their own software, aimed initially at internal needs for human resources, security training and personnel management activities. The resulting Web-based application now comprises some 110 modules and is available to government and nongovernment entities as the Open Source Corporate Management Information System.

The open-source community’s credo of reusing software modules is becoming a standard way for agencies to spread the cost of software development among many users and thereby benefit from some of the volume-based advantages of COTS. The Department of Health and Human Services, for example, has made reuse part of its enterprise architecture principles, one of which states that HHS “evaluates investments against business requirements and service needs, with a philosophy of first reuse, then buy, then build.”

More recently, the General Services Administration latched onto reuse as a major driver for its IT projects. In 2014, then-CIO Sonny Hashmi made “platform reuse first” one of the nine key principles that would guide the agency’s IT modernization. In doing so, he essentially melded open-source with cloud-first policies.

Capitalizing on GSA’s existing investment in common application and infrastructure platforms “not only reduces IT costs and complexity, it also reduces the burden on GSA’s end users by reducing the number of disparate applications and tools they need to learn and access,” Hashmi said at the time.

He added that he hoped the approach would eventually lead GSA to build custom solutions for no more than 20 percent of projects.

Cloud can cut down on the one-offs

Deciding whether to borrow, buy or build requires a clear-eyed assessment of an agency’s particular needs and a broad understanding of what existing solutions can and cannot do, said Dave Zvenyach, director of acquisition and management at GSA’s 18F.

“In practice, much — if not most — of the things we need in government are very similar to needs experienced outside of government,” Zvenyach said. “In those cases, we borrow or buy. But where the government’s needs differ, we’ll look to borrow or build.”

Over time, he added, the hope is that the number of situations in which open-source solutions are borrowed will outnumber the situations in which they must be built.

However, some agencies will continue to need custom applications. For instance, the classified networks of intelligence agencies and some parts of the Defense Department cannot securely be connected to the Internet, which means that DOD and the intelligence community have missed out on some of the advantages of cloud services. That could be changing, though.

“The intelligence community has started to look at how to solve that problem by having public cloud providers come in and build a public cloud in a government facility,” said Stan Tyliszczak, staff vice president for technology integration and chief engineer at General Dynamics IT. “That gives them all the advantages of public cloud technology and the expertise associated with that, while still operating in a constrained environment where the physical and virtual access to resources is restricted for anyone who is not in the community.”

Last year, the CIA took the leap into the cloud after striking a $600 million deal with Amazon Web Services in 2013 to develop a private cloud for the 17 agencies that make up the intelligence community. Now users at those agencies can get a variety of computing and analytical services on demand from the CIA and the National Security Agency.

The deal was the result of a longtime effort by the CIA to boost the performance of its data centers and provide its analysts with better tools. It gives the agency “the flexibility to optimize resources across different workloads at different times,” agency CIO Doug Wolfe said in an interview last year with FCW’s sister publication GCN.

That should significantly improve the “time to mission,” he added, because it now takes just days to set up new data center environments and new analytics, rather than the months or years it took before.

NSA and DOD agencies with similar connectivity concerns are considering the same approach, Tyliszczak said.

Data vs. applications

Other agencies are discovering that they can get the advantages of the cloud while also having applications and services targeted to their specific needs, Tyliszczak said. For instance, they benefit from the economies of scale and operational efficiency that come with developing in the cloud rather than building an application themselves in a small-scale computing environment, and they shift the cost model from one that largely relies on upfront capital expenditures to one that has lower recurring costs.

“These days, you really should be able to put most of what an agency does into a cloud,” Tyliszczak said. “There will always be some legacy application that performs some unique function with unique interfaces that can’t be updated that way, but the reality is that much of the workload that agencies have today can be moved into come kind of cloud-based environment.”

There are other ways to look at the issues. Officials at Hitachi Data Systems Federal, for example, say the focus should be on the data that applications produce and where they need to go rather than on the applications themselves.

“In the past, agencies would buy a solution and then customize that to get the functionality they needed,” said Brian Houston, the company’s vice president of engineering. “What you would then find is multiple silos of infrastructure for the mainframe, for a [Network File System] type of environment, another for databases and so on.”

Instead, he said that if agencies have an infrastructure that allows them to move data to the appropriate performance tier, it typically doesn’t matter whether the application is built, bought or customized.

“If you build the infrastructure so that it is scalable and flexible enough, so that data can go wherever it’s needed no matter the protocol involved, you really take the risk out of the whole build/buy question,” Houston said. “The real issue is not about the application but what type of data needs to reside in the cloud, say, versus what needs to reside in an on-premises type of environment.”

He admitted that it isn’t always easy to convince agencies to follow that approach. But once they see the capabilities, it starts to make sense as far as budget and cost justifications are concerned. And it fulfills the mandate that agencies move to the cloud while allowing them to retain control over applications and data, which is still a fundamental concern.

Rephrasing the question

Officials are also rethinking how the often-turgid government procurement process could be revamped to meet the growing demand for new and better application functionality and the increasing speed at which applications and services must be produced. That’s why GSA has been developing special procurement processes for its upcoming blanket purchase agreement for agile services, which will enable the 18F program to get vendor help with user-centric design, agile software development and DevOps.

“Ultimately, we want access to the broadest pool of vendors possible in order to establish a marketplace,” Zvenyach said. “In this ‘alpha’ phase of the agile delivery BPA, we are using IT Schedule 70, which has thousands of vendors” and makes it relatively easy for other companies to qualify for an IT Schedule 70 award.

In the end, government technology development might no longer be a choice between buy or build.

“There are some things we’ll build, some we’ll buy, some we’ll commission to get certain parts,” said Aaron Snow, acting executive director of 18F. “Our mission is to help our agencies buy and build great digital experiences, however that may work out.”

Whether you use COTS, build custom solutions or do something in between is not really the question anymore, Tyliszczak said.

“It’s more a matter of how well the products you are buying off the shelf fit the agency’s need and how much customization is needed for that,” he said. “So it’s a hybrid of COTS, some customization and maybe some build, and the real challenge is in identifying how much of each is required.”

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.