There when you need it
- By John Moore
- Nov 02, 2003
Talk about a paradigm shift: Organizations that once fretted about paper piling up now struggle to manage ever-increasing amounts of electronic documents and records.
Such electronic information has been dubbed fixed content, which is data that is not subject to change or frequently accessed, such as old e-mail messages. Fixed-content data represents one-third to one-half of an organization's data and may become an even greater proportion in coming years, according to industry watchers. Federal agencies need to get a better grip on fixed content and apply knowledge management to it.
The pressure to manage fixed content has a couple of dimensions. Most obviously, the cost-effective management of thousands of e-mail messages and documents is challenging. Fixed content may occupy an organization's primary storage subsystems, which tend to be expensive, high-end devices designed for transactional data.
Enterprises are "storing information on very expensive real estate — primary storage — and that is probably not where it belongs," said Jean-Luc Chatelain, chief technology officer at and founder of Persist Technologies Inc., a storage appliance vendor.
Another dimension is the need to make fixed-content data readily accessible for users, which means archival tape storage is less desirable. Regulatory requirements involving the preservation of certain data are also part of the fixed-content mix.
Storage vendors have introduced near-line — sometimes called midline — storage devices that are less expensive than high-end primary disk storage but offer much faster access speeds than tape. Most of those storage subsystems specifically target fixed-content storage. Many use enterprise versions of Advanced Technology Attachment PC disks to minimize the price.
These storage vendors are allying with software providers to provide specialized storage platforms for e-mail and compliance data. The software firms, in turn, are beginning to integrate with records, document and knowledge management vendors. Application program interface (API) libraries and software development kits will provide a bridge for fixed-content storage systems and more encompassing solutions.
For organization officials hoping to claim fixed content as part of the knowledge base, the storage objective is threefold: save more data, save it longer and keep it accessible.
The e-mail mess
Knowledge management, however, is generally not the first priority for agencies looking to tame the fixed-content problem. Industry and government executives typically describe e-mail as the first issue to be addressed. The Radicati Group Inc. reported in July that the average user sends or receives 110 e-mail messages each business day, noting that e-mail traffic has grown 80 percent during the past year.
The Army's Acquisition, Logistics, and Technology Enterprise Systems and Services group is working on this issue. The group, which provides information management support to the Army and Defense Department, has been beta testing Persist Technologies' AppStor storage appliance. AppStor works with different types of reference data, but the Army group is using it primarily to manage e-mail.
Bobby Jones, a systems engineer with the group, said the organization has integrated Microsoft Corp.'s Exchange e-mail software with AppStor. He said AppStor provides a cheaper alternative for e-mail storage and retrieval.
"You don't have to have a huge Exchange server with direct-attached storage," he said. "We are looking at it as a way to consolidate some of our storage requirements."
Jones also said that AppStor provides e-mail replication, thus promoting continuity of operations. The solution mirrors e-mail operations to facilitate recovery in the event of component failure and also replicates local and remote e-mail functions.
Chatelain said this business continuity feature is popular among government buyers. "E-mail is probably the No. 1 carrier of business information," he said. The "criticality of e-mail has increased." Accordingly, access to an e-mail archive is important in the event of a catastrophe, he noted.
E-mail, in general, is one of the main factors behind the government's interest in fixed-content storage, said Mike Marchi, senior director of enterprise marketing at Network Appliance Inc.
"Organizations need the ability to retain information for an extended period of time" and retrieve it fairly quickly, he said. The company has been selling its NearStore near-line product for about 18 months and has found fixed-content storage to be a crucial application.
Storage vendors and specialty software firms are joining forces to address customers' e-mail concerns. Hewlett-Packard Co., for example, resells Persist Technologies' AppStor software and, more specifically, the AppStor for Enterprise Messaging solution set.
KVS Inc., meanwhile, markets Enterprise Vault, an e-mail archiving system for Exchange. The software runs on a dedicated Microsoft Windows NT server and can work with any storage solution that supports Windows NT files, said Mary Kay Roberto, vice president of KVS' North America division. Enterprise Vault can tap magnetic or optical media and network-attached storage or storage-area network solutions.
Roberto said that EMC Corp.'s Centera is a popular storage platform that integrates with the company's e-mail archiving software.
EMC introduced Centera in April 2002 specifically to address fixed- content storage. According to the company's market analysis, 75 percent of customer information will be fixed content by 2005, said Roy Sanford, vice president of marketing and alliances at EMC. He cited e-mail as a major market for Centera, which has been shipped to more than 300 customers.
The company has ties to another e-mail archiving solution: Legato Systems Inc.'s EmailXtender product line. EMC agreed to purchase Legato in July in a $1.3 billion transaction expected to close by the end of the year. EmailXtender supports such messaging products as Microsoft Exchange, IBM Corp.'s Lotus Notes/Domino and Unix Sendmail.
More than e-mail
But there's more to fixed-content storage than e-mail. Sanford defined fixed content as "any object that at some point in its life cycle stops changing and needs to be retained for long-term value for reference or repurposing." He said fixed content includes satellite photos, check images, scanned document images and X-rays.
Jones noted that Persist Technologies has "built in a great deal of document library-type ability," enabling Microsoft Word and PDF files to be logged into the storage application.
Solutions that broaden the scope of fixed-content storage couldn't have arrived at a better time. Laws and regulations addressing electronic records have agencies examining records management. National Archives and Records Administration officials have established the regulatory baseline for the treatment of electronic records. DOD officials have created a NARA-supported design standard for electronic records management software. In addition, the Health Insurance Portability and Accountability Act is among the congressional initiatives that contain records management directives.
Enterprise Storage Group analysts estimate that the worldwide market for compliance-related storage solutions, software and services will exceed $6 billon in the next few years. "I would say that DOD alone will account for several hundred million [dollars]," said Peter Gerr, senior research analyst at Enterprise Storage Group.
In the regulatory environment, "change is beginning to permeate throughout the federal government, and we're talking about the process and technology for storing, managing and protecting information," Gerr said. The change will most prominently affect e-mail and instant messaging, but surveillance video, office documents, and medical and employment records will change, too, he said.
Steve Krauss, business operations manager for the enterprise storage technology team at reseller GTSI Corp., said the regulatory environment has dictated the need to store records so that organizations can certify that they are unalterable.
Fixed-content solutions for managing this range of compliance data must contain specialized data integrity features, such as permanence. Regulations, in general, require electronic records to be preserved in an unalterable, nonrewriteable format. Optical- or tape-based write once, read many (WORM) products have traditionally offered data permanence, but now some disk storage vendors have introduced WORM products.
With Centera, for example, EMC offers "integrity features only seen before on optical media," Sanford said. The company offers a Compliance Edition of Centera to provide the additional data protection.
Similarly, Network Appliance provides features with NearStore that "enable you to guarantee that once it's written, it's permanent," Marchi said. Specifically, The company's SnapLock software can be used on a per-file system or per-volume basis. Designated SnapLock file systems or volumes are not rewriteable or erasable, he said
Marchi added that support for retention dates is also important for solutions geared to compliance data. With Network Appliance's product, an application may write a retention date when data is written to disk. The application may extend the retention date but not reduce it, he said. Once the retention period expires, the record can be deleted.
KM: A future consideration
Agencies want to better manage their e-mail and other forms of compliance data, but most are "still in an early stage of deciding to build a fixed-content archiving solution," Chatelain said.
The focus on corralling e-mail and compliance data pushes broader knowledge management efforts into the future.
However, Chatelain believes fixed-content storage is part of the foundation upon which knowledge management systems will reside. "It's a fundamental building block," he said.
And it's a building block that has only begun to fall into place in the federal market.
Moore is a freelance writer based in Syracuse, N.Y.
Points of reference
Enterprise Storage Group officials predict that reference information — such as old e-mail messages and records that are not subject to change and are infrequently accessed — will grow at a compound annual growth rate of 92 percent through 2005. It is the fastest-growing storage category. Currently, 37 percent of cumulative storage capacity is dedicated to reference information, according to the market research firm. In two years, reference information is expected to account for 54 percent of freshly produced information.
As for compliance data, the group expects the worldwide market for compliance-related storage solutions, software and services to exceed $6 billion in four years. Compliance records' capacity will grow at an annual rate of 64 percent. The global capacity of compliance records stored on disk-based solutions will increase at a compound annual rate of 172 percent.