IT dashboards: Picture perfect?

Dashboards can nudge agencies toward better program management — but they can't do it alone.

dashboard

Like the dashboard of a car, an IT dashboard provides data on many different things all in one place. (Stock image)

For many years, agencies have been tasked with measuring the performance and management of their technology programs, usually under the watchful eye of Congress and the White House. With agencies spending about $82 billion a year on IT, transparency is seen as a powerful motivator for keeping IT programs on track.

Over the years, report cards, watch lists and high-risk lists have been used to draw attention to systems that were over budget and behind schedule. In this era of open government, many of those tools have been superseded by online dashboards that display data in a visually compelling way to show how an organization or program is meeting specified performance goals.

Dashboards are proving to be useful tools in the campaign for better IT management. The most visible example is the IT Dashboard, launched by the Office of Management and Budget in 2009 to improve the oversight, transparency and accountability of federal IT programs.

The IT Dashboard offers a view into how agencies are managing 700 major IT programs that are worth a total of $40 billion. It tracks the progress of those programs by synthesizing and graphing data from Exhibits 53 and 300 (reports to OMB that detail agency IT investments and performance plans) along with other agency updates and investment information. If a project is running behind schedule or over budget, the dashboard will display that information, along with a color-coded risk rating on a spectrum from green for low risk to red for high risk.

OMB has added other data streams since launching the site to provide more agency-specific data or deeper views into trends or context, said an OMB official who spoke on background. The dashboard also offers insight into how well agencies are meeting Digital Government Strategy milestones and complying with PortfolioStat reviews, which detail agencies' processes for managing their IT portfolios and highlight duplicative programs that are ripe for consolidation.

However, the OMB official said the dashboard is more than a collection of data. It also helps improve IT management by increasing the quality of agency performance data, improving transparency and visibility regarding the health of IT investments, and increasing agencies' focus on project management and IT governance, according to the official.

Building a Better IT Dashboard

To create a good IT dashboard, start by prioritizing your organization's requirements, said Jason Wolan, professional services manager at vendor iDashboards.

1. Decide what metrics and key performance indicators you will use to measure performance, but keep it simple. "Don't bite off more than you can chew," he said. For instance, as agencies move to cloud technology, "one of the best things a dashboard can do is keep tabs on service-level agreements and make sure they're being met," Wolan said.

2. Put together a strong team and cover all your bases. "If you need...to find a subject-matter expert, then get them involved," Wolan said.

3. Make sure you are gathering the right information and that what you are publishing is an accurate reflection of your business.

4. Decide how you are going to use that data to improve your performance — something Wolan called a post-deployment strategy.

The PortfolioStat process is getting an upgrade as well. U.S. CIO Steven VanRoekel announced PortfolioStat 2.0 in a memo and accompanying blog post in March. The upgraded process streamlines agencies' data-collection requirements, reducing 30 reports down to three. VanRoekel wrote that it "improves analytics, consolidates the agency's strategic IT direction and management improvements into one central plan, and holds agencies accountable for the goals set through last year's process." The first reports — agencies' Information Resource Management Strategic Plans — were due to OMB on May 15.

Transparency helps

The IT Dashboard offers more insight into why a program is struggling than the watch lists of the past, which were incomplete and offered little transparency, said David Powner, director of IT management issues at the Government Accountability Office.

"The dashboard has transparency, and transparency doesn't hurt," he said.

In addition to making OMB's and Congress' IT oversight duties easier, the detailed data has bolstered the authority of agency CIOs, who are evaluated on their IT management success.

"There were some CIOs who, when the dashboard was rolled out, didn't have access to the data they needed" to send to OMB because they weren't involved in the IT investment process, Powner said. That had to change, which meant giving CIOs the appropriate authority and streamlining internal processes.

Nevertheless, Powner said there is room for improving the IT Dashboard. Indeed, many experts have criticized the accuracy and completeness of the data on which the dashboard is built. "The chatter is that the data is not very good, but it's not an excuse not to make it good," he said. "We think agencies need to keep working on it."

For example, Powner testified at a House hearing in January that the Defense Department had reported that it had no high-risk IT programs when in fact it had many. The GAO report on which Powner's testimony was based warned that "DOD is masking significant investment risks...and has not delivered the transparency intended by the dashboard."

"We want to continue to push for transparency and better mechanisms," he said. "The dashboard, although far from perfect, is one of the better models because it allows for comparisons. It's not only what is spent, but are we getting the right return on investment and who is accountable."

Larry Allen, president of Allen Federal Business Partners, said accuracy is always an issue when data is pulled from multiple sources, something the Federal Procurement Data System amply demonstrated. "You end up with an unclear picture," he said. Agencies across government collect and report their procurement data to FPDS, but it has been criticized in the past for containing inaccurate or incomplete information.

To improve the IT Dashboard's accuracy and value, OMB has added submission validations and dynamic data quality reports. Those tools help agencies correct their submissions or address issues by flagging potentially inaccurate information, such as unrealistically high or low values for certain data fields, according to OMB.

What dashboards can't do

For a dashboard to have real value, the garbage in/garbage out rule applies to more than the actual data provided, said Roger Baker, former CIO at the Department of Veterans Affairs. It determines what data you collect in the first place.

He said agency leaders need to ask themselves what they are measuring, "and is it something that is really essential to your business? The dashboard is of use when it is targeting information that you can use to manage your business."

VA, which hosts multiple IT dashboards, has a program management and accountability dashboard that tracks whether each IT project is meeting its milestones and aggregates the relevant data into a single view. "At VA, it's an important piece of how we manage because it tells us the status of our IT portfolio," said Baker, who became chief strategy officer at Agilex in April.

That data also feeds into the federal IT Dashboard, which Baker said has done a decent job of shining a light on problem programs. "The IT Dashboard has not by itself fixed the entire problem, but it is motivating agencies to make changes, which means that they are managing better, more tightly," he added. "If an agency receives a red grade, then they know someone will come and ask questions."

Stan Soloway, president and CEO of the Professional Services Council, said that although dashboards can be useful, they can't do all the work for agencies.

A dashboard won't help leaders make better management decisions if the information is not timely or the business rules are weak. "And knowing that the program is not going according to plan isn't enough," he said. "You need to know why."

In other words, a dashboard "doesn't eliminate the need for real analysis," Soloway said. "It's a good tool to help you find it, but you have to dig deeper."

Allen said a good IT dashboard requires a certain amount of central management, which can be tough for agencies to do. But overall, "I think it's more effective to have dashboards than not," he said. "It's better than flying totally in the dark. But there is a difference between a dashboard and a GPS."

Health IT Dashboard: Insight into industry improvements

The Department of Health and Human Services officially launched the first version of its Health IT Dashboard about a year ago to offer a one-stop shop for people interested in health IT and the impact it is having on the health care industry.

It lets the interested public know the impact and benefits of the American Recovery and Reinvestment Act of 2009 and delivers greater insight into the activities of grant recipients, state and federal agencies, and other stakeholders, said Victor Lazzaro, project director for the Health IT Dashboard at HHS' Office of the National Coordinator for Health IT.

Among its offerings, the dashboard allows users to generate maps of health IT adoption statistics, such as how many providers are using electronic health records and what states are offering courses in health IT through their community colleges. It also provides statistics on how grant money is being spent nationwide and how well states are meeting meaningful-use goals for EHRs.

More than just a place to find data, the dashboard is speeding the adoption and use of health IT, Lazzaro said. "We're not just summarizing data," he added. "We think people can look at the data and use it to amplify and accelerate their own efforts."

Most of the data is updated monthly, and the dashboard is built on open-source software so it is cheaper to maintain, Lazzaro said. In the next six to 12 months, he plans to add more datasets and enable better visual comparisons.

Health IT Dashboard

Click here for larger image

GWAC Dashboards: Business intelligence for agencies

The General Services Administration's GWAC Dashboards allow agencies and vendors to view information on governmentwide acquisition contracts, including task-order obligations, number of awards and the vendors that have been issued task orders by each agency. That information was hard to get before the dashboard was launched and in some cases required a Freedom of Information Act request.

The GWAC Dashboards, which are updated nightly, help vendors "better manage their contracts, leverage data to form business intelligence about their target accounts, and develop marketing plans," according to a GSA spokeswoman.

They can also help agencies conduct market research, evaluate task-order activity and vendor capability, and review small-business activity.

In the near future, GSA plans to improve download times and enhance the dashboards with more datasets and side-by-side chart displays.

Commerce Dashboard

Click here for larger image

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.