Acquisition

Adopting new tech can lead to sleepovers

Shutterstock image (by Coprid): sleeping bag

(Coprid / Shutterstock)

FCC CIO David Bray says that he learned the hard way that moving to the commercial cloud is sometimes an actual physical undertaking akin to buying a new home.

Over the Labor Day weekend, his agency moved entirely to the commercial cloud as part of an ongoing IT modernization effort, powering down all of its servers and moving them to a public provider. The switch from having the facilities on FCC property to the public cloud provider’s entailed physically moving almost 200 servers.

“The Thursday before Labor Day at 2 p.m., we powered down everything that was at the FCC,” he said in remarks at an AFCEA Bethesda breakfast on Sept. 15. “By midnight everything was turned off. We had that moment of ‘what could possibly go wrong?’ I realized that I had just turned off the FCC completely. We moved everything onto seven trucks and hoped we didn’t get into any accidents.”

When the servers arrived at the provider’s location, Bray said, that’s when things got really interesting. It was apparent as the team began putting things back together that the new cable configurations for the systems that had been worked out before the move weren’t going to work, in spite of all the software analysis and pre-move due diligence. “This is thousands of cables,” he said. “It was like the Apollo 13 launch. Everything was cool -- at first.”

Then it wasn’t.

And contractors and FCC IT personnel worked for the next 55 hours to fix the problem on the spot. Some personnel, according to Bray, lived on the premises for days with sleeping bags, so they wouldn’t waste time going to and from the site. The problems delayed the switchover by a couple of days – but Brays says a less resilient response might have resulted in a months-long debacle.

Other panelists at the AFCEA event said such critical all-hands-on-deck thinking is key to the federal government’s adoption of the latest technology – and to dispensing with what is now an overwhelming reliance on legacy tech.

To help adapt to those changes, Greg Godbout, chief technology officer at the Environmental Protection Agency, said he is working on a tech development and acquisition group for EPA similar to the 18F effort he helped start at the General Services Administration. The idea for projects like GSA’s 18F is to seed a small area for people to try new things in a controlled environment and show others in the organization how they work.

About the Author

Mark Rockwell is a senior staff writer at FCW, whose beat focuses on acquisition, the Department of Homeland Security and the Department of Energy.

Before joining FCW, Rockwell was Washington correspondent for Government Security News, where he covered all aspects of homeland security from IT to detection dogs and border security. Over the last 25 years in Washington as a reporter, editor and correspondent, he has covered an increasingly wide array of high-tech issues for publications like Communications Week, Internet Week, Fiber Optics News, tele.com magazine and Wireless Week.

Rockwell received a Jesse H. Neal Award for his work covering telecommunications issues, and is a graduate of James Madison University.

Click here for previous articles by Rockwell. Contact him at mrockwell@fcw.com or follow him on Twitter at @MRockwell4.


Featured

  • Contracting
    8 prototypes of the border walls as tweeted by CBP San Diego

    DHS contractors face protests – on the streets

    Tech companies are facing protests internally from workers and externally from activists about doing for government amid controversial policies like "zero tolerance" for illegal immigration.

  • Workforce
    By Mark Van Scyoc Royalty-free stock photo ID: 285175268

    At OPM, Weichert pushes direct hire, pay agent changes

    Margaret Weichert, now acting director of the Office of Personnel Management, is clearing agencies to make direct hires in IT, cyber and other tech fields and is changing pay for specialized occupations.

  • Cloud
    Shutterstock ID ID: 222190471 By wk1003mike

    IBM protests JEDI cloud deal

    As the deadline to submit bids on the Pentagon's $10 billion, 10-year warfighter cloud deal draws near, IBM announced a legal protest.

Stay Connected

FCW Update

Sign up for our newsletter.

I agree to this site's Privacy Policy.