Watchdog dings contractors for IRS tax day outage

Shutterstock photo ID: photo ID: 245503636 By Mark Van Scyoc Sign outside the Internal Revenue Service building in downtown Washington, DC on December 26, 2014. 

An unpatched firmware bug, contractor errors and a lack of established communication protocols by top IT officials were responsible for an outage that shut down the Internal Revenue Service's e-file system and 58 other IRS processing systems for 11 hours on tax day, according to an Inspector General report.

The online e-file system for the IRS suffered an embarrassing Tax Day outage on April 17. The agency got the system back up and running within 11 hours, but wound up extending the tax filing deadline by another day.

Errors by two contractors, IBM and Unisys, were cited as contributing factors. When one of the high-availability storage arrays at IRS began detecting errors early in the morning on tax day, it sent an alert to the vendor, IBM. However, the company reportedly designated the error with a lower-level alert classification that only merited a response by the end of the next business day.

IBM actually identified the bug in 2017 and developed an updated version that fixed the flaw, but both IBM and IRS officials decided to go with an older version for the remainder of the filing season after determining it would be "more stable." Another client suffered a similar issue in January 2018, but IBM reportedly did not notify IRS or offer the preventative script developed to remediate the error.

The report also criticized Unisys because three of the company's storage arrays were not replicating all application and systems data from the main computing center in Memphis, creating "a single point of failure for tax processing operations." The IG also criticized Unisys for taking more than 20 hours to acknowledge and fix the problem – a process that was supposed to take at most 6.5 hours per the Unisys-IRS contract.

Auditors were largely contented with IRS' internal efforts to remedy problems. But the report notes the associate and deputy associate CIO were not notified until they arrived at work four hours after the problems began. Auditors advised the tax agency to document lessons learned during the process to incorporate for similar breakdowns in the future, formalize monthly meetings with Unisys and IBM to discuss microcode issues and build out more layers of approval for software updates.

They also recommended that IRS seek liquidated damages from Unisys for failing to meet performance levels during the outage. The IRS concurred and said it is seeking an unspecified amount and is seeking to modify several aspects of its contract to prevent similar failures.

In a response letter, IRS CIO Gina Garza agreed with remaining recommendations while noting that IRS believes its response to the outage "demonstrated our improved ability to rapidly and successfully respond to major incidents." She also noted that IRS had experienced a similar issue in 2016 and had cut down its response time from 30 hours to 11.

"This is significant and demonstrates our commitment to process improvements," said Garza. "We continue to develop and implement new technologies, refine our processes and conduct robust testing."

FCW has reached out to Unisys and IBM for comment.

About the Author

Derek B. Johnson is a senior staff writer at FCW, covering governmentwide IT policy, cybersecurity and a range of other federal technology issues.

Prior to joining FCW, Johnson was a freelance technology journalist. His work has appeared in The Washington Post, GoodCall News, Foreign Policy Journal, Washington Technology, Elevation DC, Connection Newspapers and The Maryland Gazette.

Johnson has a Bachelor's degree in journalism from Hofstra University and a Master's degree in public policy from George Mason University. He can be contacted at djohnson@fcw.com, or follow him on Twitter @derekdoestech.

Click here for previous articles by Johnson.


  • Defense
    The U.S. Army Corps of Engineers and the National Geospatial-Intelligence Agency (NGA) reveal concept renderings for the Next NGA West (N2W) campus from the design-build team McCarthy HITT winning proposal. The entirety of the campus is anticipated to be operational in 2025.

    How NGA is tackling interoperability challenges

    Mark Munsell, the National Geospatial-Intelligence Agency’s CTO, talks about talent shortages and how the agency is working to get more unclassified data.

  • Veterans Affairs
    Veterans Affairs CIO Jim Gfrerer speaks at an Oct. 10 FCW event (Photo credit: Troy K. Schneider)

    VA's pivot to agile

    With 10 months on the job, Veterans Affairs CIO Jim Gfrerer is pushing his organization toward a culture of constant delivery.

Stay Connected


Sign up for our newsletter.

I agree to this site's Privacy Policy.