Know your data

A top challenge for the government's efforts to wring more value from the data it collects — through better analysis and sharing — is ensuring that information in agency repositories is consistent, current and comprehensible to users and programmers.

A growing number of agencies are finding that one way to attain this goal is by tapping the benefits of "metadata." This data about data contains valuable information about each piece of information in an agency's data holdings, such as its source and format and the changes it has undergone since entering the system.

When separated from the primary data and collected in a metadata repository, this information gives systems administrators the means to organize their data holdings similar to a card catalog in a library, said Wayne Eckerson, director of education and research at the Data Warehousing


This enables administrators to

introduce greater efficiency and accuracy into their data operations, eliminating inconsistencies, redundancies and irrelevant information. In turn, data growth and change can be more easily managed because new information can be filed using the metadata index.

Metadata also provides a foundation for greater data sharing and more advanced and potentially more lucrative information analysis when

using data mining applications, for example.

But metadata solutions also can pose significant cultural, technical and financial challenges, and "many people do as little as possible," Eckerson said. "You can build a data warehouse with minimal metadata and get by."

Resistance to data warehousing may be reflected in the marketplace, where metadata repository sales — chiefly by Computer Associates International Inc. and ASG (formerly Allen Systems Group Inc.) — total about $250 million per year, according to Michael Blechar, a vice president and research director at market research firm Gartner Inc.

The market is not very big or growing, he said, adding that Gartner's tally does not include sales of services and numerous general-purpose tools used by organizations to build metadata solutions.

Interest in metadata management "is growing in some places more than others," Blechar said. For

example, reflecting the increased popularity of Extensible Markup Language, more organizations are using metadata systems to provide their programmers with well-defined catalogs of their XML services and components.

Organizations are also using metadata to help add sophistication to data warehousing programs, such as advanced search and analysis


Federal agency efforts in this area generally have been hobbled by the lack of "metadata management policies or practices


  • Congress
    U.S. Capitol (Photo by M DOGAN / Shutterstock)

    Funding bill clears Congress, heads for president's desk

    The $1.3 trillion spending package passed the House of Representatives on March 22 and the Senate in the early hours of March 23. President Trump is expected to sign the bill, securing government funding for the remainder of fiscal year 2018.

  • 2018 Fed 100

    The 2018 Federal 100

    This year's Fed 100 winners show just how much committed and talented individuals can accomplish in federal IT. Read their profiles to learn more!

  • Census
    How tech can save money for 2020 census

    Trump campaign taps census question as a fund-raising tool

    A fundraising email for the Trump-Pence reelection campaign is trying to get supporters behind a controversial change to the census -- asking respondents whether or not they are U.S. citizens.

Stay Connected

FCW Update

Sign up for our newsletter.

I agree to this site's Privacy Policy.