'Hits' don't measure up
- By Timothy Sprehe
- Nov 12, 2000
"Performance measures" is a popular phrase these days, but agencies are
slow to apply performance measures in a systematic way.
Take agency Web sites, for example. The primary performance measures
that many agencies use for their Web sites seem to be more hits, more hits
and more hits.
But what does "more hits" measure? Simply that a lot of people out there
clicked on an Internet site. That kind of measure tells the agency exactly
nothing about whether the site is achieving its performance goals. For all
the agency knows, visitors could be clicking because they are furious at
Perhaps a better question is: What are the performance goals for agency
Web sites? And are the measures that agencies use telling them whether the
sites accomplish their intended jobs?
Some agencies have given a lot of thought to those questions. Charles
McClure of Florida State University, Kristin Eschenfelder of the University
of Wisconsin and I recently completed a study for three agencies in which
we developed a set of performance measures for assessing agency Web sites.
The three sponsoring agencies were the Defense Technical Information Center,
the Energy Information Administration and the Government Printing Office.
We found that the topic of Web site performance measures has at least
three dimensions when applied to government agencies. The first is that
of law and policy. Feds must be careful that they are complying with legal
and policy mandates regarding privacy (including children's privacy), freedom
of information, intellectual property, accessibility for people with disabilities
and security. They must be sure that their Web sites conform to relevant
parts of the Paperwork Reduction Act, the Government Paperwork Elimination
Act and the Federal Records Act.
Management factors are the second dimension for Web site performance.
Does a site have adequate infrastructure, including equipment and staffing,
and is the agency controlling who is posting content to the site and ensuring
the quality of the content?
Third, quantitative performance measures fall into a fivefold conceptual
grouping: extensiveness, efficiency, effectiveness, service quality and
usefulness. Although each concept has specific measures associated with
it, they overlap in practice.
One tip all study participants agreed on was that counting Web site
"hits" is a lousy measure of anything. Much better basic measures are units
of content retrieved such as number of documents downloaded or number
of user sessions per time period.
Check out the study report at fedbbs.access.gpo.gov/libs/measures.htm.
The sponsoring agencies and the authors will be happy if the study just
gets more agencies thinking about what they are trying to achieve with their
Web sites and talking about what are the best ways of measuring their goals.
Sprehe is president of Sprehe Information Management Associates in Washington,
D.C. He can be reached at firstname.lastname@example.org.