EVM: A dash of cold water

Social Security Administration officials found that earned value management data is valuable only if it’s widely shared — so they publish EVM data on the agency’s intranet.

When officials at the Social Security Administration began to integrate earned value management into their project-management drill, they did something that few agencies have done. They established an EVM program management office that’s become a major hub of activity.  As part of the office of the deputy commissioner for systems, the EVM PMO doesn’t operate in a vacuum. It’s structured to push the data that EVM generates on information technology projects up through SSA’s management anks and throughout the agency. EVM reports are published on the agency’s intranet where employees at SSA can review them. The EVM PMO also routinely routes data to the agency’s Information Technology Advisory Board — its IT executive steering committee — and the SSA deputy commissioner’s office. And every quarter, senior-level executives review EVM performance for all the agency’s programs.“EVM data and the results of the reports are available all over the place,” said Otto Immink, senior IT specialist in the EVM PMO. “It is exposed across the agency and utilized by all levels of management.”SSA is one of the few agencies pursuing a program-level approach to EVM, said Karen Richey, senior cost analyst at the Government Accountability Office. “They’re not just doing it at the contract, but they’re trying to include all [management levels] in house and roll it up into a full-out program level,” she said. “Even DOD isn’t doing program-level EVM yet.” However, the Federal Aviation Administration has taken a similar program-level approach to EVM, she added.Getting EVM out of isolation in the project manager’s shop and into the executive domain is one of the fundamental challenges of deploying EVM in government, said David McClure, a research director at Gartner’s Government Group. SSA officials have done an exceptional job of achieving that, said McClure, former director of IT management issues at GAO. “It’s a good example of an organization that has taken EVM data and put it into the IT governance process so it’s not buried down at the project-management level,” he said. “It’s actually raised up to the level of discussion where it should appropriately be.”Moreover, SSA officials have added clarity to the EVM data so executives can understand the reports. “The formulas, calculations and indices are going to be Greek to somebody who doesn’t know what all this means,” McClure said.In its reports, the PMO explains what the indices mean with a simple system that uses red, green and yellow lines. “We measure the programs based on the standard EVM variances, such as cost and schedule variance and variance at completion, but we do it in nontechnical terms,” Immink said. For example, “if you have something that is plus or minus 10 percent off from where your established baseline was, you go yellow — that’s a caution. If it’s red, it better not be the first time management sees it. Red is an indicator that more serious attention needs to be placed on the program.”Presenting performance information to senior managers in this way increases executive accountability, McClure said. “If it’s that transparent, it’s going to be difficult for someone to say, ‘I never knew that this project was in trouble.’ ” In September 2006, SSA earned a good review from its inspector general for EVM. The IG’s audit found that SSA had implemented a system to manage major IT projects in accordance with guidance from the Office of Management and Budget. OMB mandated two years ago that all agencies use EVM for major new IT projects, reflecting the Bush administration’s emphasis on program performance and outcomes in the President’s Management Agenda.EVM is a performance management technique that measures a project’s progress against cost, schedule and technical, or performance, baselines. The EVM term for progress is “earned value.” Dennis White, Robbins-Gioia’s EVM practice area manager, said EVM centers on meeting project objectives, making cost only one element of the measurement metric. That approach contrasts with traditional project development, in which cost is the primary measure. “Once [an EVM] system — and system is defined as people, processes and tools — has been established and matured, the biggest benefit is going to be enhanced planning,” White said. “You will know what it takes to do things. You will get better at estimating and can do more realistic scheduling. You will be able to set up performance from the front end, as opposed to measuring everything from the back end.”Experts agree that for many agencies, implementing EVM has been a struggle, and the reasons for those situations vary. For some, it’s fearing the unknown, Richey said. “In civil agencies, a lot of times people are afraid of EVM,” she said. “There’s a lot of fear that it’s going to be a lot work, and there’s probably a misconception about the amount of overhead it creates for a program.”Richey added that many officials don’t understand that EVM is simply good project management practice. She said most of the 32 guidelines in the National Defense Industrial Association Program Management Systems Committee’s EVM System Intent Guide, a primary reference work for agencies, are steps agencies should take to manage any project, no matter how large or complex. “You’ve got to define what you are going to do, you’ve got to measure it and look at what’s causing any variances,” she said.Jennifer Echard, a senior cost analyst at GAO, said the absence of an oversight agency could be an impediment to civilian agencies’ EVM efforts. The Defense Department has the Defense Contract Management Agency, “which can go in and certify their EVM systems and do analysis,” she said. “There’s a lack of that on the civilian side.”A deficit of EVM training also is hindering agencies, some observers say. “It’s been a long and uphill road to get people adequately trained and experienced” in using EVM, McClure said. “There still is not enough training of EVM analysts and managers, and that continues to be an issue.”White concurred. “The lack of experienced and knowledgeable [employees] results in policies that aren’t enforceable and strong enough to make the whole set of management controls work,” he said.McClure said he is optimistic EVM will be demystified once agency officials gain experience in using EVM and understand how it works. “It’s not mystical. It’s not magical,” he said. “It’s been around for quite some time. It’s straightforward in terms of the calculations and the tracking of the project. It’s elevating the transparency of what’s actually happening with your resource expenditures and your scheduling and asking you if you need to make some changes to be more in sync with what you original expectations were.”Indeed, the potential benefits of using EVM are abundant and should outweigh what some agency officials see as the onerous task of having to meet an OMB compliance mandate, experts say. At SSA, for example, EVM provides the agency a reliable set of data that it uses to make management decisions and oversee IT programs, Immink said. “And it does so in a unique way. EVM is the only monitoring tool that I’m aware of that integrates your baselines schedule and performance against that schedule and costs incurred in the performance against the schedule.”  Overall, EVM has helped improve program management at SSA. “We’ve always had what we consider good program management practices here,” Immink said. “EVM has added another layer of rigor and discipline.”That is not to say EVM is a panacea. Experts agree that EVM has limitations.“It’s great [for] the transparency it gives in terms of cost and schedule and performance of projects, but [it] still doesn’t tell you whether the outcome is the right one from a business perspective,” McClure said. “I’ve seen lots of examples where projects are rebaselined from their original cost, schedule and performance targets, but there’s no explanatory factor given as to why the project is being rebased. So you’re just negating the value of EVM. You should be using it to educate your organization on why those variances are occurring.” Accurate data is also crucial for successful EVM efforts. “One haunting factor for earned value management is the quality of the baseline data for the project,” McClure said. “If you don’t have good, initial baseline data on cost and schedule, then this tool becomes more difficult to get value from. Without good data, you’ve got a great tool, but you’re not necessarily going to have an accurate picture of what’s going on.”Even accurate results, however, don’t guarantee EVM success. “A successful EVM implementation is measured by how heavily the resulting data is relied on,” Immink said. “You generate these fancy reports, but if it isn’t relied on for use in management and oversight decision-making, then you aren’t there yet.”




















































4 ways to get involved and learn more about EVMOtto Immink,  senior IT specialist in the Social Security Administration’s earned value management program office, and other experts offered the following tips for others interested in EVM.

1.  Get involved with the EVM community and draw from the expertise and experience of other people and agencies. “Over the past two years, a whole host of resources has become available,” Immink said. “Get engaged in the community and ask for help when you need it.” Here are some of the major players:

  • The National Defense Industrial Association’s Program Management Systems Committee. “That’s the group that develops and promulgates the criteria that define a compliant EVM system,” Immink said. “If you hang around those guys, you get to learn what exactly the criteria are that define  a compliant EVM” system.
  • The Earned Value Management Community of Interest.  EVM COI is a group of officials from civilian agencies who meet monthly to discuss EVM issues and compare experiences, Immink said. Volunteers from the Project Management Institute’s College of Performance Management founded EVM CoEye in 2005.
  • PMI’s  College of Performance Management has a strong focus on EVM. Membership in the college includes updates to the EVM system guidelines and related government documents; the latest changes and newest initiatives in government and industry;  developments in hardware, software and applications; and new techniques for team building and training.
2.  Work with the Office of Management and Budget. OMB officials “wrote the guidance as to what requirements we had to follow, so it seems like a logical step to check with them periodically to make sure you’re progressing,” Immink said. When SSA was deploying its EVM system, officials talked to OMB at least once a month. For example, SSA’s EVM policy defined what programs required EVM oversight. “If we didn’t check with OMB to ensure that the policy was acceptable, it made no sense to go forward with our implementation.”

The Government Accountability Office plans to publish “Best Practices for Developing and Managing Program Costs” later this month. In the works for two years, the guide will include three chapters on EVM, said GAO senior cost analyst Karen Richey. The guide states that cost estimating and EVM must be linked for successful project management.

3. Get formal training in EVM. The Defense Acquisition University’s online course in EVM is a valuable training resource, Richey said.

4.  Go to conferences and listen to what EVM advocates have to say, Immink said. The PMI College of Performance Management’s Web site contains a list of upcoming conferences and events.

— Richard W. Walker


X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.