EVM sounds the alarm

Smart agencies let earned value management lead them to better program management

Tim Young likes to tell audiences about being a project manager early in his career when he had a demanding boss. The boss told Young he would fire him if he couldn’t report at a moment’s notice whether projects were on schedule, on budget and within scope.

Young, who now serves as associate administrator for e-government and information technology at the Office of Management and Budget, said OMB won’t be as tough on federal agencies as his former boss was on him. New OMB policy guidelines, however, require agencies to become more disciplined about how they manage major IT projects. With the federal government spending about $65 billion a year on IT, agencies should make every effort to prevent IT projects from failing and wasting taxpayer money, Young said.

OMB officials have asked agencies to avoid costly failures by using a project management discipline known as earned value management. With few exceptions, EVM is not well-understood outside the Defense Department. One exception is the Office of Personnel Management. OPM, which recently made EVM a standard for managing IT projects, has learned valuable lessons about EVM’s benefits and limitations.

Setting up EVM departmentwide poses challenges. OPM officials found that EVM is worthwhile because it tells them when projects are in trouble, and it does so in time for them to fix problems. But EVM is not infallible. Most experts say EVM cannot help agencies that won’t accept bad news. They say EVM is most valuable if agencies use it to help people learn from their mistakes rather than to punish them.

OMB issued its EVM policy guidelines in two memos dated August 2004 and August 2005. In addition to requiring federal agencies and their contractors to use EVM for managing all major IT projects, OMB set new reporting requirements. Agencies must include EVM data when they submit Exhibit 300s, documents in which they present their business cases for major IT projects. OMB requires agencies to use EVM to calculate and report each project’s estimated total cost and completion date.

Agencies must answer other EVM questions. Is the date that project managers planned to complete certain work on a project months earlier than the date they actually completed it? If major gaps exist between the work that the agency planned to do and the work it actually did, OMB wants to know that, too.

Likewise, OMB asks for EVM data that reveals significant differences between how much the agency planned to pay for work and how much the work actually cost. OMB puts a major IT project on its watch list of projects requiring corrective action if it is off by more than 10 percent from the original schedule, budget and functional assumptions.

OMB’s 10 percent rule is reasonable for all but the largest projects, said Joel Koppelman, founder and chief executive officer of Primavera Systems, a project portfolio management software company. “If I were doing a $10 billion project, I wouldn’t want to wait until I was $1 billion off” before taking corrective action, he said.

The underlying principles of EVM are not new. DOD has been using cost and schedule controls on aerospace and defense projects since the mid-1960s. In 1991 DOD canceled the Navy’s A-12 stealth carrier aircraft project because EVM reports revealed the project — by then $1 billion over budget and a year behind schedule — could not recover. “It certainly got the attention of a lot of program managers who were not using EVM as a tool for managing their projects,” said David Christensen, accounting professor at Southern Utah University.

Despite early doubts, most experts now agree that EVM is suitable for managing major IT projects, Christensen said. “People for many years felt that IT projects were not measurable and that you couldn’t apply earned value to those projects,” he added.

EVM gives agency executives who sit on IT review boards a means of identifying IT projects that are likely to exceed their budgets and deciding whether those projects deserve to get additional funding to put them back on track, experts say. As a set of standard practices for managing projects, EVM makes it difficult for project team members to cheat or claim more progress than they have made.

EVM standard practices are described in an American National Standards Institute document known as ANSI Standard 748-A. “It’s a whole system of checks and balances,” said Elizabeth Mautner, OPM’s director of administration and finance for the human resources line of business, a governmentwide initiative to consolidate federal human resource services. But can someone put inaccurate data into the system? “We try to make it as hard as possible,” Mautner said.

Some experienced project managers say the benefits of EVM are offset by certain limitations. A common criticism of EVM is that it measures the quantity but not the quality of work performed. Another limitation of EVM is its underlying assumption that problems derive from poor project execution rather than inadequate project planning. Sometimes poor planning is the culprit, Christensen said. “If the project plan is not accurate or reasonable, then you’re comparing actual progress against a bogus plan,” he said.

Nevertheless, in various areas of the government, federal agencies are learning how to apply the principles of EVM to manage IT projects. OPM is among the first to make a commitment to use EVM agencywide. After successfully experimenting with EVM to manage five e-government initiatives in 2005, OPM transferred its EVM activities to the office of Janet Barnes, OPM’s chief information officer, and made EVM a requirement for managing all major IT projects.

“EVM has moved from being nice to have to being critical for daily operations at OPM,” said Norman Enger, program director of the human resources line of business initiative at OPM.

Getting to that point was not easy. “This is something that takes a lot of commitment and effort,” Enger said. In September 2004, OPM purchased Primavera’s EVM software, which included Primavera’s Budget Manager, Time Sheets, Cost Manager and myPrimavera, a personal workspace collaboration tool. Then OPM project managers spent a year learning how to get useful reports from the new system. “It takes that much time to put the infrastructure in place and change the culture,” Enger said.

One significant change was having OPM employees and contractors record all labor costs and other direct costs in the EVM system. “Contractors are not ecstatic about putting their labor hours and [other direct costs] into the EVM system,” Enger said.

EVM required integrating the Primavera system with OPM’s older financial systems — another significant effort. “This is the type of integration that takes a while, a lot of time and a lot of culture change,” Enger said. That change, he added, couldn’t have happened without senior-level support from OPM’s chief financial officer and CIO.

OPM now relies on EVM to manage IT projects budgeted at more than $30 million a year, and the number of projects is growing, Mautner said. The EVM system generates monthly reports that the agency’s investment review board uses when it meets quarterly to review and prioritize projects.

In April 2004, OPM began offering training to introduce EVM concepts to project management employees and senior executives. It continues to offer refresher courses to keep employees involved and ensure appropriate use of the EVM system.

Project managers use the term “change management” to refer to ways that managers can encourage employees to accept workplace changes. Change management is the most difficult aspect of bringing EVM into an agency or Cabinet-level department, Mautner said. “This is something you have to continue to pay attention to, and you’re never done with it,” she said.

Some companies include change management consulting with their project management services, although agencies typically do not include it in a contract’s statement of work. A change management consultation might begin with an assessment of employee attitudes and include strategies for communicating the value of proposed changes to employees and executives, said Greg Grosser, corporate development director at Macfadden, a professional services company based in Silver Spring, Md.

Change management is the responsibility of managers, some of whom blame an organization’s culture for their inability to get people to do something, Grosser said. When managers choose not to accept responsibility for the culture or even seek help in changing it, not much else can be done, he added.

People disagree about whether the difficulty of implementing EVM is comparable to the challenges of converting from older financial systems to an enterprise resource planning system, for example. “EVM is a lot easier than doing an ERP because it doesn’t involve nearly the number of people,” Koppelman said. But others say that when practiced on a large scale, EVM poses many of the same challenges as ERP. Both require the use of data dictionaries or agreement on definitions before people enter data into the systems. Reaching that kind of agreement among people from a variety of program offices requires many meetings.

Another challenge of using EVM is recognizing that although it is designed to provide objective data about a project’s progress, someone could generate false EVM reports. “If you’ve got powerful people and they refuse to be judged objectively, they’re going to find ways to screw the whole thing up,” said Dennis White, practice manager for performance management at Robbins-Gioia, a project management consulting company.

Other experts say the biggest danger in using EVM is that some senior executives will dismiss accurate but negative reports because they instinctively resist bad news and try to suppress it. EVM can be a problem for people with can-do attitudes, Christensen said. “They’re too optimistic about their ability to recover,” he added.

He said the way to deal with that problem is to ensure that early warning signs of a troubled project are not used to punish people. “They should be viewed as opportunities for improvement rather than as [occasions] for somebody to be fired,” he said.

EVM is valuable because it can identify problems early enough so that something can be done about them, said Neil Albert, president of the College of Performance Management at the Project Management Institute. Sometimes that means releasing reserve funding for a project, putting more people to work on a critical phase of a project or offering additional training to project staff.

EVM is also valuable at the end of a project when managers have time to review all the EVM data and learn from it, Koppelman said. “Good judgment is the result of experience,” he said. “The way to become a good project manager is by having bad experiences and learning from them.”

In brief: What is EVM and how does it work?

Earned value management is a rigorous project management discipline that puts a dollar value on work as organizations complete it.

EVM is not widely practiced outside the Defense Department. Therefore, these principles and terms are unfamiliar to many civilian information technology managers.

Here are a few facts about EVM. The discipline:

  • Works for any type of contract, including fixed-price contracts.

  • Can be simplified and tailored to manage relatively small IT projects.

  • Requires new accounting systems or modifications to existing accounting systems to enable monitoring and reporting of detailed project costs and scheduled work activities.

  • Requires extensive documentation of procedures and practices for project staff and contractors to follow.

  • Works best when used with automated project management software. Examples of companies that make such software include Artemis International Solutions, Compuware, Galorath, Microsoft, Proforma and Primavera Systems.

  • Works best when a project has at least one person for every $10 million of estimated project value.

  • Uses a work breakdown structure as the basic unit for measuring earned value.

  • Uses an organization breakdown structure as the basic unit for assigning people to do various project tasks.

  • Uses a 3-D performance management baseline for measuring whether a project is on track or in trouble. The three dimensions are schedule, budget and functional specifications.
Earned value management ranks low on awareness scale

Primavera Systems makes project management software that supports earned value management. The company recently hired O’Keeffe and Co. to ask federal information technology officials about their knowledge of EVM and their plans for using it to manage major new IT projects.

O’Keeffe and Primavera reported the following results from a survey of 109 federal IT officials.

  • 60% of survey respondents said EVM improves the likelihood of a project’s success.

  • 37% said their agencies use EVM.

  • 63% said their agencies do not use EVM or are unsure whether they use EVM.

  • 25% said their main challenge with EVM is not knowing much about it.

  • 24% said their main challenge with EVM is not having employees who are trained in it.

  • 21% said their main challenge with EVM is not having senior-level interest in it.

Source: O’Keeffe and Co.

NEXT STORY: Lockheed gets Sentinel contract

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.