IT Management

IT dashboards: Picture perfect?

dashboard

Like the dashboard of a car, an IT dashboard provides data on many different things all in one place. (Stock image)

For many years, agencies have been tasked with measuring the performance and management of their technology programs, usually under the watchful eye of Congress and the White House. With agencies spending about $82 billion a year on IT, transparency is seen as a powerful motivator for keeping IT programs on track.

Over the years, report cards, watch lists and high-risk lists have been used to draw attention to systems that were over budget and behind schedule. In this era of open government, many of those tools have been superseded by online dashboards that display data in a visually compelling way to show how an organization or program is meeting specified performance goals.

Dashboards are proving to be useful tools in the campaign for better IT management. The most visible example is the IT Dashboard, launched by the Office of Management and Budget in 2009 to improve the oversight, transparency and accountability of federal IT programs.

The IT Dashboard offers a view into how agencies are managing 700 major IT programs that are worth a total of $40 billion. It tracks the progress of those programs by synthesizing and graphing data from Exhibits 53 and 300 (reports to OMB that detail agency IT investments and performance plans) along with other agency updates and investment information. If a project is running behind schedule or over budget, the dashboard will display that information, along with a color-coded risk rating on a spectrum from green for low risk to red for high risk.

OMB has added other data streams since launching the site to provide more agency-specific data or deeper views into trends or context, said an OMB official who spoke on background. The dashboard also offers insight into how well agencies are meeting Digital Government Strategy milestones and complying with PortfolioStat reviews, which detail agencies' processes for managing their IT portfolios and highlight duplicative programs that are ripe for consolidation.

However, the OMB official said the dashboard is more than a collection of data. It also helps improve IT management by increasing the quality of agency performance data, improving transparency and visibility regarding the health of IT investments, and increasing agencies' focus on project management and IT governance, according to the official.

Building a Better IT Dashboard

To create a good IT dashboard, start by prioritizing your organization's requirements, said Jason Wolan, professional services manager at vendor iDashboards.

1. Decide what metrics and key performance indicators you will use to measure performance, but keep it simple. "Don't bite off more than you can chew," he said. For instance, as agencies move to cloud technology, "one of the best things a dashboard can do is keep tabs on service-level agreements and make sure they're being met," Wolan said.

2. Put together a strong team and cover all your bases. "If you need...to find a subject-matter expert, then get them involved," Wolan said.

3. Make sure you are gathering the right information and that what you are publishing is an accurate reflection of your business.

4. Decide how you are going to use that data to improve your performance — something Wolan called a post-deployment strategy.

The PortfolioStat process is getting an upgrade as well. U.S. CIO Steven VanRoekel announced PortfolioStat 2.0 in a memo and accompanying blog post in March. The upgraded process streamlines agencies' data-collection requirements, reducing 30 reports down to three. VanRoekel wrote that it "improves analytics, consolidates the agency's strategic IT direction and management improvements into one central plan, and holds agencies accountable for the goals set through last year's process." The first reports — agencies' Information Resource Management Strategic Plans — were due to OMB on May 15.

Transparency helps

The IT Dashboard offers more insight into why a program is struggling than the watch lists of the past, which were incomplete and offered little transparency, said David Powner, director of IT management issues at the Government Accountability Office.

"The dashboard has transparency, and transparency doesn't hurt," he said.

In addition to making OMB's and Congress' IT oversight duties easier, the detailed data has bolstered the authority of agency CIOs, who are evaluated on their IT management success.

"There were some CIOs who, when the dashboard was rolled out, didn't have access to the data they needed" to send to OMB because they weren't involved in the IT investment process, Powner said. That had to change, which meant giving CIOs the appropriate authority and streamlining internal processes.

Nevertheless, Powner said there is room for improving the IT Dashboard. Indeed, many experts have criticized the accuracy and completeness of the data on which the dashboard is built. "The chatter is that the data is not very good, but it's not an excuse not to make it good," he said. "We think agencies need to keep working on it."

For example, Powner testified at a House hearing in January that the Defense Department had reported that it had no high-risk IT programs when in fact it had many. The GAO report on which Powner's testimony was based warned that "DOD is masking significant investment risks...and has not delivered the transparency intended by the dashboard."

"We want to continue to push for transparency and better mechanisms," he said. "The dashboard, although far from perfect, is one of the better models because it allows for comparisons. It's not only what is spent, but are we getting the right return on investment and who is accountable."

Larry Allen, president of Allen Federal Business Partners, said accuracy is always an issue when data is pulled from multiple sources, something the Federal Procurement Data System amply demonstrated. "You end up with an unclear picture," he said. Agencies across government collect and report their procurement data to FPDS, but it has been criticized in the past for containing inaccurate or incomplete information.

To improve the IT Dashboard's accuracy and value, OMB has added submission validations and dynamic data quality reports. Those tools help agencies correct their submissions or address issues by flagging potentially inaccurate information, such as unrealistically high or low values for certain data fields, according to OMB.

What dashboards can't do

For a dashboard to have real value, the garbage in/garbage out rule applies to more than the actual data provided, said Roger Baker, former CIO at the Department of Veterans Affairs. It determines what data you collect in the first place.

He said agency leaders need to ask themselves what they are measuring, "and is it something that is really essential to your business? The dashboard is of use when it is targeting information that you can use to manage your business."

VA, which hosts multiple IT dashboards, has a program management and accountability dashboard that tracks whether each IT project is meeting its milestones and aggregates the relevant data into a single view. "At VA, it's an important piece of how we manage because it tells us the status of our IT portfolio," said Baker, who became chief strategy officer at Agilex in April.

That data also feeds into the federal IT Dashboard, which Baker said has done a decent job of shining a light on problem programs. "The IT Dashboard has not by itself fixed the entire problem, but it is motivating agencies to make changes, which means that they are managing better, more tightly," he added. "If an agency receives a red grade, then they know someone will come and ask questions."

Stan Soloway, president and CEO of the Professional Services Council, said that although dashboards can be useful, they can't do all the work for agencies.

A dashboard won't help leaders make better management decisions if the information is not timely or the business rules are weak. "And knowing that the program is not going according to plan isn't enough," he said. "You need to know why."

In other words, a dashboard "doesn't eliminate the need for real analysis," Soloway said. "It's a good tool to help you find it, but you have to dig deeper."

Allen said a good IT dashboard requires a certain amount of central management, which can be tough for agencies to do. But overall, "I think it's more effective to have dashboards than not," he said. "It's better than flying totally in the dark. But there is a difference between a dashboard and a GPS."

Health IT Dashboard: Insight into industry improvements

The Department of Health and Human Services officially launched the first version of its Health IT Dashboard about a year ago to offer a one-stop shop for people interested in health IT and the impact it is having on the health care industry.

It lets the interested public know the impact and benefits of the American Recovery and Reinvestment Act of 2009 and delivers greater insight into the activities of grant recipients, state and federal agencies, and other stakeholders, said Victor Lazzaro, project director for the Health IT Dashboard at HHS' Office of the National Coordinator for Health IT.

Among its offerings, the dashboard allows users to generate maps of health IT adoption statistics, such as how many providers are using electronic health records and what states are offering courses in health IT through their community colleges. It also provides statistics on how grant money is being spent nationwide and how well states are meeting meaningful-use goals for EHRs.

More than just a place to find data, the dashboard is speeding the adoption and use of health IT, Lazzaro said. "We're not just summarizing data," he added. "We think people can look at the data and use it to amplify and accelerate their own efforts."

Most of the data is updated monthly, and the dashboard is built on open-source software so it is cheaper to maintain, Lazzaro said. In the next six to 12 months, he plans to add more datasets and enable better visual comparisons.

Health IT Dashboard

Click here for larger image

GWAC Dashboards: Business intelligence for agencies

The General Services Administration's GWAC Dashboards allow agencies and vendors to view information on governmentwide acquisition contracts, including task-order obligations, number of awards and the vendors that have been issued task orders by each agency. That information was hard to get before the dashboard was launched and in some cases required a Freedom of Information Act request.

The GWAC Dashboards, which are updated nightly, help vendors "better manage their contracts, leverage data to form business intelligence about their target accounts, and develop marketing plans," according to a GSA spokeswoman.

They can also help agencies conduct market research, evaluate task-order activity and vendor capability, and review small-business activity.

In the near future, GSA plans to improve download times and enhance the dashboards with more datasets and side-by-side chart displays.

Commerce Dashboard

Click here for larger image

About the Author

Colleen O'Hara is a freelance writer based in Arlington, Va.

Featured

Reader comments

Wed, May 29, 2013

The VA uses this Dashboard metric system to gather information regarding coding,billing, collections,etc....but for the life of me, the results never trickle down to the workers so they can improve processes....when information is gathered and not used to improve performance, then you are wasting taxpayer monies....If the VA would just get rid of their antiquated Vista (DOS) system, they could eliminate the overpaid bureaucratic management and let a 21rst century computing system do the analysis. Yes...costly in the short run...but necessary in the long run. In the end, government bureaucracies like the VA really needs to work more efficiently to have any credibility . The vets are not getting their benefits in a timely manner because their Vista computer systems are as antiquated as the VA "chain of command" system. I've always said that the problem with the VA is the lack of communication...both its people and its computer system. ALL

Wed, May 29, 2013 Joe

Ah yes, the IT Dashboard, which add enormous overhead to projects. If this were really open government, the IT Dashboard would collect and publish from every effort the cost associated with providing the information. Then OMB could do a REAL cost-benefit analysis, and we might see how well their rosy benefit claims hold up for the amount of data they require.

Tue, May 28, 2013

Ahh, the OPEN Data Policy. A policy which fails to mention "The Computer Matching and Privacy Protection Act of 1988". Why bother with the checks and balances that Congress believed were necessary? Under the guise of law enforcement (a legal exemption from the privacy act), one could really quickly go after your enemies list and easily bypass those pesky feds who might see a problem in aggregating information from multiple agencies that may or may not be collected with the same quality controls. Hmm, is the IRS scandal a mild prelude? As Americans, let us please re-read "1984"? Yes, Big Data is wonderfully efficient, but correlations with data stores that are littered with SSNs subject to rampant identity theft and used to apply for benefits should really put this measure on the slow boat.

Mon, May 27, 2013 Owen Ambur Hilton Head Island, SC

Consistent with the recent Executive Order and OMB policy memo making openness and machine-readability the default for government information, government "dashboards" should be based upon an open, standard, machine-readable format like StratML Part 2, Performance Plans and Reports (ANSI/AIIM 22:2011).

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above