A word of caution for DARPA's cyber challenge champ

The Defense Advanced Research Projects Agency's three-year push to kick-start automated cyber defense innovation culminated in an Aug. 4 announcement that, pending final verification, a computer system dubbed Mayhem was the winner of DARPA's Cyber Grand Challenge. A technology watchdog group, however, has raised concerns about a possible downside to automated cybersecurity.

ForAllSecure, a team of Pittsburgh-based researchers, designed the Mayhem system and is set to receive $2 million in prize money.

The team was one of seven competing for almost $4 million in prizes. The final competition was an all-day affair in front of an audience of 5,000 computer security specialists in Las Vegas. TECHx's Xandra computer system came in second, while Shellphish's Mechanical Phish came in third, according to DARPA.

"I'm enormously gratified that we achieved CGC's primary goal, which was to provide clear proof of principle that machine-speed, scalable cyber defense is indeed possible," said Mike Walker, the DARPA program manager who launched the challenge in 2013, in an Aug. 4 statement.

The challenge was designed to accelerate development of advanced, autonomous systems that can detect, evaluate and patch software vulnerabilities before adversaries have a chance to exploit them.

The Electronic Frontier Foundation, however, tempered its enthusiasm for DARPA's innovation effort and called for a more cautious approach.

In an Aug. 4 blog post, EFF's Senior Staff Attorney Nate Cardozo, Chief Computer Scientist Peter Eckersley and Senior Staff Technologist Jeremy Gillula wrote that the work holds promise for automating flaw discovery and patching, "but this silver lining has a cloud."

Although people programmed the automated systems, the resulting tools mostly ferret out software and system vulnerabilities without human intervention. The group said such automation could potentially open a Pandora's box of autonomous, self-replicating viruses that attack machines and continuously exploit new vulnerabilities.

Automatic patching systems might not be able to keep up with autonomous viral development, especially in the complex world of the Internet of Things, the writers said.

However, they stressed the importance of DARPA's research on automated cybersecurity systems. "EFF is a pro-innovation organization, and we certainly wouldn't ask DARPA or any other research group to stop innovating," the authors wrote.

"We think the right thing, at least for now, is for researchers to proceed cautiously and be conscious of the risks," they continued. "When thematically similar concerns have been raised in other fields, researchers spent some time reviewing their safety precautions and risk assessments, then resumed their work. That's the right approach for automated vulnerability detection, too."