Measurement kit: First aid for CIOs
- By Pat Plunkett
- Nov 08, 2000
New chief information officers will want to know several aspects of the
information technology operation they inherit.
At a minimum, CIOs need to know the resources spent on each major system,
which organizational units are the primary customers and how satisfied they
are with the systems. CIOs should also want to establish a baseline against
which to compare their performance during their tenures.
The CIO Measurement Kit below provides basic, yet essential, information
to help CIOs better manage an agency's IT assets and services. The kit consists
of the following components: cost, user satisfaction, alignment, baselines
Gathering and maintaining information on these components will provide
CIOs with insight to focus IT resources where they can have a measurable
impact on an agency's mission. The information collected becomes the backdrop
for effective capital planning. It also provides the measurement foundation
by which CIOs can determine to what degree IT contributes to their organization's
CIO Measurement Kit
Know what is being spent on whom, where and why. What is the allocation
of IT spending? How much is spent (for systems and support) on each major
system? How well does the spending match the priorities of the agency and
its performance goals?
According to Gartner Group Inc., in general governments spend about
80 percent of their IT budgets on supporting, maintaining and enhancing
their legacy systems. That only leaves 20 percent for new starts. Where
should IT funds be directed to have the most impact? What needs to be done
to achieve 90 percent of the cost estimates?
User satisfaction is an indicator of the quality and effectiveness of
a particular system. If users are dissatisfied, then it's questionable whether
that system will make any contribution to mission results. The more satisfied
users are, the more likely the system will be used and the greater likelihood
users will be productive and mission objectives will be achieved.
Satisfaction should focus on the quality of the system, the information
and the service provided. Obtaining user feedback often uncovers ways to
improve the system and can foster better relationships with the user community.
How satisfied are the users of the agency's major systems? What would they
like to see improved?
The closer IT systems are aligned to mission goals and objectives, the
greater likelihood the IT systems will contribute to mission results. If
business or organizational objectives are poorly defined, then alignment
will be difficult to ascertain. What are the measures the agency uses to
judge performance? Are they being used?
Logic models help determine alignment. A logic model is a diagram that
shows the if/then flow of results. For example: "If the system provides
this information, then the user will perform a particular task better, faster
or cheaper (BFC). If that task is performed BFC, then customer service will
improve. If customer service is improved, then mission performance will
improve." Logic models are more accurate if developed with program personnel.
By measuring key results (e.g. BFC) along the "flow of results," CIOs will
be able to determine whether the IT system is on track and can make adjustments
as necessary so that an IT initiative does contribute to mission results.
CIOs need to have measures for both the IT domain and the business domain.
Baselines are used to compare and evaluate future performance. Choose
meaningful measures for both the IT and business domains. If baselines do
not exist, create them by choosing measures and collecting the data. Make
sure the data match the measures. CIOs can jump-start or improve their existing
measurement efforts by considering performance measures used by other organizations
and comparing performance levels with organizations with similar business
Looking to the future, select targets that will contribute the most
to mission results. How are the targets for the IT initiatives aligned?
Do they match the performance goals and priorities of your organization
over the next three to five years? How will the achievement of those targets
contribute to the performance goals?
Measurement doesn't just happen. To provide useful and reliable information,
measurement must be systematic, iterative and important to management. The
information collected must be used for decision-making, e.g., during the
capital planning process; otherwise, people that collect the data will not
take it seriously.
For each major IT system, identify or determine the following:
* System type: Legacy or new start?
* Business owner: Internal organization that is currently using or will
use the system?
* Process supported: Core or non-core?
* Cost: Amount allocated annually and projected in the future?
* User satisfaction: For each owner, establish a baseline for the level
of satisfaction with the quality of the system, information and service.
* Alignment: Develop a logic model with users to show IT's link to the
mission. Users judge the degree the system aids them to complete their objectives.
Determine which performance measures your organization uses and how each
major system contributes to them.
* Baselines: Choose measures, use existing data or collect new data.
* Targets: Using a logic model, develop targets of performance that
will contribute to your organization's performance goals and match its priorities.
CIOs will be in a better position to:
* Determine IT's contribution to mission results.
* Determine if the IT budget is being allocated to best achieve IT and
* Improve relationships with users and management by greater attention
to user satisfaction.
* Align IT systems to mission objectives.
* Identify opportunities to streamline processes.
* Improve relations with their business counterparts.
Plunkett is the General Services Administration's program manager
for performance measurement.