Government security managers recommend several techniques for evaluating the effectiveness of employee cybersecurity training and improving the odds that the lessons will sink in.
When Dennis Lauer joined the Millennium Challenge Corp. as chief information officer two years ago, the young federal program’s growing pains included a startling lack of security.
It was an almost free-for-all atmosphere, he recalled. Employees installed Apple iTunes on the agency’s network and regularly downloaded malware via pop-ups that harbored malicious code. “Almost every day we had [surreptitious] viruses, and people didn’t know not to click on" them, Lauer said.
The security situation began to change for the better when the office adopted new security policies and practices. Launched in 2004, MCC had adopted a few information technology shortcuts in the early years as the U.S. government corporation embarked on its mission of helping underdeveloped nations. When Lauer arrived at the agency, he had a list of more than 20 noncompliance items from Federal Information Security Management Act audits.
Now when users log on to the MCC network, they are greeted by a Tip of the Day awareness training application, which asks a question about IT security. The system then tracks the responses. Besides giving managers an easy way to assess the agency’s training program, the daily quizzes have also made employees more mindful of security.
“We’ve had a tremendous reduction in viruses,” Lauer said. “Instead of clicking on things, [users] call the help desk. They never used to do that before.”
But not every agency can report such success. Indeed, experts say the goals of user training efforts are still a long way from being realized.
“There is a gap, and the gap is costly because it undermines all the technology being thrown at security problems,” said Keith Rhodes, senior vice president and chief technology officer at QinetiQ North America’s Mission Solutions Group.
As with any technology solution, the disconnect is often found on the user's side of the keyboard. "No approach to training is infallible because human beings are fallible, and of course, human fallibility is what training tries to counter,” Rhodes said.
A recent survey by CDW Government underscored the challenges of security training. Four out of five federal IT managers said they provide ongoing classes on security policies and procedures. But even then, almost half had seen employees post passwords in public places, violating one of the most fundamental security proscriptions.
The survey highlights one of the hardest tasks in IT security: changing user behavior. Firewalls, intrusion-prevention systems, antivirus software and other security technologies provide some defense against attacks. But they don’t fully address the human dimension. For instance, firewalls won’t prevent an employee from stowing passwords under a mouse pad or engaging in other careless practices.
Agencies hope training programs will keep employees on the straight-and-narrow security path. But how can you tell whether — and to what extent — the message is sticking?
Security managers and industry consultants say there are a few basic techniques for evaluating the effectiveness of IT security training and improving the odds that the lessons will sink in.
Tip no. 1: Make employee testing simple and routine
At MCC, new employees receive IT awareness training as part of their orientation, and the security tip of the day provides ongoing reinforcement.
The U.S. Agency for International Development created the quiz application, which is available through the Information System Security Line of Business program. MCC officials keep tabs on employees’ security awareness by tracking responses to those daily quizzes via a monthly performance report.
In recent months, more than half of MCC’s 400 users earned A's, but about two dozen got F's. The monthly reports flag users who chronically fail the quizzes so supervisors can follow up with remedial action.
The Tip of the Day system also helps coordinate the training of MCC employees who work abroad in 20 countries. The automated training delivery system is available via MCC’s network — a more cost-effective option than trying to get hundreds of far-flung employees to gather in the same room for training.
“Delivering training can often be pretty expensive in that environment,” Lauer said.
Organizations with multiple locations always face a tough challenge when it comes to developing and measuring the success of training programs. In Colorado, officials are addressing the issue by undertaking an IT consolidation effort. The state is 18 months into a four-year initiative that will meld the IT operations of 16 executive branch agencies under the statewide Office of IT.
“To get metrics to prove that end-user security is working, you’ve got to be in a consolidated environment,” said Seth Kulakow, Colorado’s chief information security officer.
Consolidation will provide the consistency required to gather the correct metrics, he added.
Tip no. 2: Check what they do, not just what they know
Vulnerability assessments can help determine whether employees are learning their security lessons and complying with policies.
For instance, a vulnerability assessment tiger team composed of internal IT employees or external experts can scrutinize an organization’s security policies and test systems and network security postures, said Aaron Barr, chief executive officer at HBGary Federal, which provides malware analysis and incident response products.
The team can look for sticky notes bearing passwords and other potential security lapses. It might also simulate phishing scams by using bogus yet seemingly legitimate e-mail messages or phone calls to extract passwords and other personal information from users. The goal is to see how many passwords team members are able to get and how they are able to get them. That insight can then be integrated into evaluations of agencies’ training programs.
Barr recommends that agencies use internal IT security employees to conduct quarterly vulnerability assessments and external experts for annual vulnerability assessments.
“Auditing and assessing and evaluating has to be a continuous process,” said Patricia Titus, chief information security officer at Unisys Federal Systems.
Security managers can also infer the effectiveness of training through anecdotal evidence of employee behavior. At the Interior Department, a recent inspector general report notes an increase in reported security incidents, which indicates heightened awareness among employees, said Andrew Jackson, the agency’s deputy assistant secretary for technology, information and business services.
Tip no. 3: Put security in personal terms
To help the security message sink in with employees, Colorado IT officials plan to educate employees about best practices in home computer security. The idea is that employees who learn to protect personal and financial information at home will bring those good habits back to the office.
When IT security “actually means something financial or personal to them, that tends to stick a little longer,” Kulakow said.
“We have found the best return is to relate training topics to people in their personal lives,” agreed Carolyn Schmidt, program manager for IT security awareness, training and education at the National Institute of Standards and Technology’s CIO office. “If they pay attention in their home life, that will naturally translate to work life."
Tip no. 4: Invoke consequences for misbehavior
At MCC, a new policy prescribes a series of escalating actions when a user fails the Tip of the Day tests. After a single failing month, the user is contacted about his or her performance. After two consecutive failing months, the user’s supervisor is alerted. If poor performance extends beyond that, the user’s network access is disabled until he or she completes remedial training.
Elsewhere, Colorado’s Kulakow has recommended making an employee’s adherence to security policy part of his or her performance evaluation.
At NIST, some organizations already include IT security performance criteria in employee evaluations, Schmidt said. But she added, “I think the best practice for accountability is to have support from senior management and a good working relationship between security operations and human resources.”
Tip no. 5: Always remember the limits of training
Security vendors say user training — no matter how well-done — can only accomplish so much. Users might fall for particularly clever phishing schemes, or they might flagrantly disregard security training and policy directives.
“There is only so much we can do to change the behavior of end-users,” said Eddie Schwartz, chief security officer at NetWitness, a security vendor.
He said organizations would be better off investing in tools that enable them to monitor users and detect security lapses rather than spending more money on preventive training.
Content filtering and data loss prevention are among the products agencies can use to counteract human nature, said Keshun Morgan, a networking and security specialist at CDW-G. Content filtering lets agencies block prohibited Web sites, while data loss prevention seeks to keep confidential data from leaving the network.
As many experts point out, there is no silver bullet when it comes to cybersecurity, so a layered approach works best — one that relies on training and automated security solutions.
NEXT STORY: Better integrated intell could save lives