Is cloud computing fast enough for science?

Energy Department project offers cloud testbed

Early results from the Energy Department’s Magellan cloud computing testbed suggest that commercially available clouds suffer in performance when operating Message Passing Interface (MPI) applications such as weather calculations, an official has said.

“For the more traditional MPI applications there were significant slowdowns, over a factor of 10,” said Kathy Yelick, division director for the National Energy Research Scientific Computing division. NERSC is partnered on the Magellan cloud project with the Argonne National Laboratory.

Under the Magellan project, DOE is exploring many aspects of the application of clouds to scientific research, including performance, efficiency, and suitability for various types of research. At least six different tests currently are being carried out in the two-year $32 million project, Yelick said. About 3,000 NERSC scientists and other people have been invited to participate in Magellan.


Related stories:

Magellan explores the cloud as a research tool

Feds could get access to advanced cloud computing technologies


One of the tests compared performance between various commercial clouds and NERSC’s computers running in batch-style. Some of those tests involved MPI applications, in which there are many communications between processes and synchronizations between parallel processes. MPI applications typically are used for weather forecasting and for some chemistry research.

“The big performance differences were in the MPI applications,” Yelick said.

However, for computations that can be performed serially, such as genomics calculations, there was little or no deterioration in performance in the commercial cloud, Yelick said. Magellan directors recently set up a collaboration with the Joint Genome Institute to carry out some of the institute’s computations at the Magellan cloud testbed.

Magellan’s purpose is to explore a wide range of scientific issues regarding cloud computing, and ultimately to advise DOE how to incorporate cloud computing into its research, Yelick said.

“Some people want to understand the performance factor: seeing how fast the cloud will run. Others are really interested in the impact of virtualization and to what extent can we run multiple jobs on the same node without a performance impact,” Yelick said. Although commercial cloud providers give an “illusion of elasticity,” in reality there are limits on performance and some jobs are sharing the same processors, she added.

Currently, scientists with computational research needs typically purchase and configure computer clusters. But NERSC is hopeful that some of that research could be performed in a cloud, Yelick said.

“We’re really looking at this as an alternative to scientists buying their own clusters,” Yelick said. “Our goal is to inform DOE and the scientists and industry what is the sweet spot for cloud computing in science; what do you need to do to configure a cloud for science, how do to manage it, what is the business model, and do you need to buy your own cloud.”

About the Author

Alice Lipowicz is a staff writer covering government 2.0, homeland security and other IT policies for Federal Computer Week.

Featured

  • Defense
    Soldiers from the Old Guard test the second iteration of the Integrated Visual Augmentation System (IVAS) capability set during an exercise at Fort Belvoir, VA in Fall 2019. Photo by Courtney Bacon

    IVAS and the future of defense acquisition

    The Army’s Integrated Visual Augmentation System has been in the works for years, but the potentially multibillion deal could mark a paradigm shift in how the Defense Department buys and leverages technology.

  • Cybersecurity
    Deputy Secretary of Homeland Security Alejandro Mayorkas  (U.S. Coast Guard photo by Petty Officer 3rd Class Lora Ratliff)

    Mayorkas announces cyber 'sprints' on ransomware, ICS, workforce

    The Homeland Security secretary announced a series of focused efforts to address issues around ransomware, critical infrastructure and the agency's workforce that will all be launched in the coming weeks.

Stay Connected