Security means never having to say 'I love you'

New Mexico IT staff learns the hard way about what happens when security tools aren't in place

Frank Woods remembers his exact whereabouts when he found out just how painful

and costly the words "I love you" could be.

It was May 4, recalled Wood, chief operating officer of the New Mexico State

Highway Department. He was in a meeting in Albuquerque when he received

a priority call from his Santa Fe office. The news struck him like a freight

train: Massive amounts of IT infrastructure in his department had been infected

by the mercurial "love bug" e-mail virus, and it was spreading a file- and

JPEG-deleting worm.

"We use a lot of JPEG files, digital cameras, and scanned photographs in

our engineering design process," Wood said. "Our hardest-hit area was engineering.

We believe there may have been 50,000 to 60,000 files affected."

The task of cleaning up the infected machines was hampered by a tricky wild

card: The department was in the midst of integrating Microsoft Corp. Exchange

and Microsoft Outlook as its single e-mail program and installing a complete

Tivoli Systems Inc. framework infrastructure. Making matters worse, Wood

said, his organization did not have an emergency plan in place to deal with

the threat.

The first order of damage control was to shut down every piece of equipment

throughout the legacy system. It lay dormant for about two days.

"It's safe to say everything from design processes to having systems and

communications to our consultants and contractors — every part of our technology

was shut down. We literally went back to paper and pencils for many of our

processes," Wood said.

The second step was to set up an IT crisis management team; individuals

from Wood's staff, Tivoli, Compaq Computer Corp., Fiori Industries, and

the Foundation Consulting Group issued hourly reports.

Less than 24 hours after the initial lock down, individual parts of the

system were turned back on, only to be aborted after it became apparent

that the worm continued to propagate through outside sources. Because the

Tivoli environment was not yet fully implemented, any remote user not accounted

for could re-infect the network and therefore needed to be identified in

a hurry.

To start the process, Microsoft Information Scan was used to remove the

worm from Exchange. Next, Trend Micro Inc.'s ScanMail antivirus software

was deployed to stop the worm from breaching the perimeter. Wood said he

used Tivoli products to scan his enterprise's 100 servers and 1,750 desktops

in 128 locations for infected systems and to clean those in need with a

distributed package of scripts created by the team. Using Windows NT logging,

the scripts made sure an unknown machine could not log on to the network

until it verified its end-point location.

Following that, new Sophos antivirus software was installed on remote machines

from a centralized system. New rules were added to pay close attention to

potential worm-carrying agents such as Visual Basic Script files, ZIP files

and .exe files.

In less than a week, Wood said, his system was 90 percent to 99 percent

up and running and clean, without a single project timeline or federal appropriation

being jeopardized.

"I don't think any of us really expected this kind of virus attack to be

as smartly written — as virulent," Wood said. "I'll admit, personally, I

was caught by surprise."

Wood said security technology is playing a bigger role in his department

these days. He is determined to have checks and tools in place so he won't

have to dodge a similar bullet.

"A lot of us are getting a little bit long in the tooth and gray in the

beard," Wood said. "We need to pass something to our successors for them

to understand how we dealt with this process."