5 pitfalls that derail EVM success

Earned value management is here to stay as a project management tool. But too many agencies still make common mistakes when using it.

No one understands better than the Federal Aviation Administration how hard it is to manage large-scale information technology projects. In a notable stretch of rough sailing, FAA officials saw their agency on the Government Accountability Office’s high-risk list for 14 years in a row.

“It’s not a reputation that anyone is proud of,” said James Washington, FAA’s chief acquisition officer.

Fortunately, that run came to an end this year, and FAA credits its adoption of earned value management and other best practices for its newfound success in managing projects. The agency now applies EVM to all large-scale projects that require an Office of Management and Budget Exhibit 300 business case form, or about one-third of its $3 billion in capital investments.

Since 2007, FAA’s EVM early warning system ended a couple of expensive IT initiatives that were beyond saving even if FAA had poured more money into them.

“We reached decisions fairly early on that we needed to cut our losses and channel our investments in other directions,” Washington said.

That success represents EVM at its best: A program management tool that sets baseline expectations for IT projects and then compares the value of work completed against expected progress and actual costs on an ongoing basis.

By using simple graphs to display data about schedules and spending, EVM systems show when projects risk becoming late or exceeding their budget. Most agencies base their approach on an industry-standard EVM model called ANSI/EIA 748. It specifies quarterly performance reviews by program managers, stakeholders and project implementers. Many government program managers go further and monitor performance continuously.

“EVM techniques are designed to make sure projects are performing the way they’re supposed to or identify early and frequently when something challenges their success,” said Kevin Plexico, senior vice president of research and analysis at government IT researcher Input.

With praise from GAO and other positive reviews, FAA is becoming a poster child for EVM success, but the agency also stands out because many of its peers still struggle to get EVM right. Common problems include lukewarm commitment by agency executives and contractors and problems with the accuracy of performance data.

Despite the inconsistencies in its application, EVM appears to be here to stay. The Defense Department has used EVM to some degree for decades, while Federal Acquisition Regulation rules and OMB mandates require that all federal agencies adopt the discipline for large-scale IT projects.

Last spring, some senators introduced legislation that would require OMB to create a Web site that uses EVM data to track the quarterly progress of major IT projects. Subsequently, OMB incorporated EVM elements into the IT Dashboard it launched last summer to summarize the performance of federal IT investments.

So with EVM appearing to gain steam, how can agencies circumvent its hazards and turn it into a valuable management tool? EVM veterans said organizations can start by steering clear of the following pitfalls.

Pitfall 1: Executive ambivalence

Compliance fatigue, personnel turnover, deadline pressures and other factors can keep senior executives and project managers from fully committing to EVM.

“Historically, earned value has had a bad knock as just a bunch of numbers that the finance geeks use,” said Daniel Butler, chairman of the National Defense Industry Association’s Program Management Systems Committee. The industry and government association promotes EVM and maintains the ANSI/EIA 748 standard.

Experts say executives who downplay the importance of EVM indirectly tell junior staff members and contractors to take a similarly casual approach, which is why EVM can be doomed unless it becomes ingrained in an agency’s or contractor’s operations.

“If the program manager is talking in the language of earned value, then obviously it’s part of the culture,” said David Kester, director of the Defense Contract Management Agency’s EVM Center, which certifies contractors for DOD projects worth $50 million or more according to their ability to meet ANSI/EIA 748 guidelines.

Solution: To achieve widespread buy-in, executives must first understand what EVM isn’t, said Jon Mihalic, a board member at the Project Management Institute, a professional association.

“It’s not an accounting tool. It’s not a financial management tool,” he said. “It’s part of your project management approach that can help drive your programs.”

PMI’s certification program and training manual, “A Guide to the Project Management Body of Knowledge,” include discussions of how earned value affects decisions about project and risk management.

Because EVM is more than just a regulatory check-off item, experts say senior managers must also understand the significant investments EVM requires.

“Most new folks coming [in] don’t understand the time and resources that it takes to stand something like this up,” Kester said. Many agencies devote 16 months or more to EVM training and system development, he added.

Pitfall 2: Poor data

EVM analyses are only as good as the numbers they crunch. Large IT projects produce volumes of rapidly changing data compiled by agency employees, contractors and subcontractors. As a result, there are plenty of opportunities for incorrect or outdated information to cloud performance reports.

Solution: After receiving sharp criticism for its handling of the problem-riddled FAA Telecommunications Infrastructure program, FAA re-examined its EVM data and found anomalies. With the help of an outside consultant, a 12-member team of FAA employees now reviews information submitted by contractors about planned work versus actual performance.

“Rather than taking the initial EVM data runs at face value, we validate the original set of data so we are certain what that data is telling us,” Washington said. “This gives us the assurance that we are looking for.”

Pitfall 3: Data overload

Accuracy isn’t the only problem when it comes to EVM data. Agencies also must avoid getting buried in data.

Too much detail can make it harder to spot the significant cost and schedule variances, said Fred Atwood, a certified Project Management Professional and program control manager at Vangent, an information management company that has used EVM for projects with the Labor, Commerce and Education departments and DOD’s Military Health System.

“Although you are getting more data, the additional work involved in setup, managing and reporting doesn’t always yield useful information,” Atwood said.

Solution: Focus on indicators that reveal trends for main project components. For example, project managers who oversee the development of a new software application might be expected to monitor progress on the user interface. But if they examine details at the level of individual menu items within that interface, they risk information overload.

“Rather than showing an agency 50 very small problems, [EVM analyses] should say, ‘Here are three major problems that are significant, and here’s why,’ to focus management attention on the things that are most critical,” Atwood said.

Pitfall 4: Poorly chosen software

A number of commercial software packages are available to automate data collection and analysis. Popular tools include Open Plan, Cobra and wInsight from Deltek; Oracle Primavera; and Caldera CostView.

However, misusing an application can undermine an organization’s management processes and skew EVM results.

Solution: Agencies should evaluate potential tools for their ability to integrate with the organization’s existing IT environment, Mihalic said. He added that even the best tools won’t guarantee success if they aren’t a proper fit.

“The biggest mistake I see is agencies buying a tool [and] building processes to fit the tool,” he said. “That’s when earned value becomes a problem.”

Instead, agencies should delay technology decisions until they’ve created well-defined processes that accommodate ANSI/EIA 748 guidelines and internal project management requirements. Then they should evaluate tools based on how easily they accommodate those processes.

Pitfall 5: Inexperienced contractors

Since OMB mandated the use of EVM, there has been no shortage of contractors that tout their EVM expertise.

“Everywhere you look, there are [EVM] consultants, but there is a huge disparity in their qualifications,” Butler said.

Solution: When drafting requests for proposals, agencies should include an excerpt of the EVM-related FAR clause in their solicitations. FAR requires contractors to show EVM certification, such as evaluations by DCMA, or document specifics about how they implement their EVM systems, Butler said.

The best candidates will have longtime EVM experience and perhaps DCMA certifications and projects in environments that adhere to recognized standards, Kester said.

“Find someone who is not just taking something off the Internet and walking through the basics of earned value,” he said.

As many agencies have learned, EVM is not a difficult concept to understand, but using it effectively requires more than a partial commitment.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.