Did USAJobs fix make matters worse?

In its efforts to fix significant operating and traffic problems at the USAJobs job search website, the Office of Personnel Management may have just made the problems worse with a misdirected fix, according to an industry critic.

On Oct. 18, OPM officials said they had installed a “governor” application on USAJobs to limit traffic, effectively blocking about 6 percent of the incoming visitors on their first try. The visitors were urged to hit the refresh button to keep trying. The purpose is to reduce the demand on the site's servers by forcing some users to wait for access.


Related stories:

OPM slows access to USAJobs to lighten load

OPM says USAJobs fixed, but complaints continue


The action was the latest effort to address technical problems in the eight days since OPM debuted the new job search site on Oct. 11. Thousands of users have reported searches returning incorrect results, timeouts and lost data. OPM has blamed the poor performance on extraordinary high traffic for the site.

However, OPM’s governor application may be shutting out more website visitors than anticipated, Linda Rix, co-chief executive of Avue Technologies Corp., which provides job application services to federal agencies, said on Oct. 19.

Instead of blocking 6 percent of incoming traffic—as OPM advertised—the governor application may be blocking a lot more, Rix said, based on her company’s authorized tests of the USAJobs system.

Avue’s tests conducted on Oct. 19 showed 86 failures to enter the USAJobs website in 100 tries, an 86 percent failure rate, Rix said.

“The governor may be hyperactive, and because of the way that OPM does caching, some people might be locked out for days,” Rix said. “That may be why some users are getting frustrated.”

Meanwhile, OPM officials said they were very focused on USAJobs concerns. The agency posted videos on its home page and the USAJobs Facebook page on Oct. 19 reassuring users that it is working around the clock addressing the system malfunctions.

OPM Director John Berry issued a statement emphasizing that the agency is making USAJobs operation a high priority. He also said many users have successfully submitted applications.

“During the last eight days, traffic has greatly exceeded previously reported volumes. To address this demand, we are adding additional server capacity to support this extremely high volume,” Berry said..

On Monday, OPM recorded a milestone: more than 100,000 applications submitted through the new system, Berry said.

"To be exact, as of 4 p.m. yesterday, Americans have submitted 141,289 applications for Federal jobs through the new USAJOBS. So we know the system is working for a lot of people,” he said.  “But I also know that’s cold comfort for those whose applications aren’t getting through. That's why we’re working with the Chief Human Capital Officers to extend non-emergency job openings for three weeks."

However, Rix suggested that the problems could go deeper and may not have been completely identified.

In an interview with Federal Computer Week, Rix outlined several other technical issues with USAJobs that her company has uncovered in testing and in conversation with industry experts and OPM staffers.

Avue manages job applications for several federal agencies and competes with OPM’s USAStaffing application, and Rix is considered a critic because of her writings on Huffington Post that preceded the USAJobs 3.0 problems. However, Rix also has substantial industry expertise in job search systems and Avue is authorized to perform certain operational tests on USAJobs.

Rix pointed to several other potential sources of bottlenecks in USAJobs. For one, OPM’s recent installation of additional capacity data transmission“pipes” for USAJobs may be inadequate, based on a comparison with similar tools used by Avue, Rix said.

While OPM, with the additional pipes, appears to have the equivalent of about 200 T1 lines to handle incoming traffic to the USAJobs website, Avue has the equivalent of about 2,700 T1 lines to handle similar or less traffic, she said.

“That suggests that OPM underestimated their capacity needs and essentially are bottlenecking the front end,” Rix said.

Rix also said she performed comparison tests using the Barracuda application that also suggest another capacity problem at USAJobs.

Her tests showed that for each visitor who visits the USAJobs home page, there are 16 hits on backend servers. By comparison, she tested the job search aggregator Indeed.com, which has a similar search configuration to USAJobs. Rix said the tests on Indeed.com returned a single hit per visitor.

“That suggests that their record high traffic on USAJobs really is not pent-up demand, it is an architecture issue,” Rix said. In essence, each visit results in 16 hits to the server, which she said strains the system and causes it to reach overcapacity on a frequent basis. It also could be overestimating traffic to the site by a factor of 16, she suggested.

Based on her testing, Rix said there are indications that OPM is not fully addressing all the capacity issues and operating issues yet. “They are not aware that they have more problems than what they are attempting to fix,” she said.

OPM officials did not respond on Oct. 19 to requests to comment about Rix’ test results and opinions.

The 2014 Federal 100

Get to know the 100 women and men honored this year for going above and beyond in federal IT.

Reader comments

Tue, Oct 25, 2011 Confused... Washington, DC

I tried to reach out to the help desk to let them know that I couldn't successfully execute a search in the area I entered....and they sent me instructions on how to create search.....very sad. I liked the old site MUCH better. It's interesting that they are seeing a sharp decline in applications...DUH! It's because people can't find the darn jobs to apply!

Mon, Oct 24, 2011 Dave Macon

God help us when our OPM makes something a top priority! Last February, John Berry told Sen. Barbara Mikulski, D-Md., that he had made Retirement Systems Modernization his #1 priority and promised "incremental and definable changes" over the next two years. The only incremental change was DEATH as Berry killed it shortly thereafter.
Is this another RSM?

Mon, Oct 24, 2011

I agree with many others!!! This new system is horrendous! with the old system at least we were able to search everything open in a given area. We had no problem with the way the site was originally - at least we could view open jobs easily and would get notifications and updates (when the agencies made them). Now we can't access it according to a state and city one is looking for a job in. The problem was not with the site!!! please reconsider because if they do not fix the site soon we will not be able to efficiently search for jobs in a timely manner! I agree with the previous post that we should not have to figure out what the job is posted as or what government job series it is especially for people unfamiliar with those series and the filters do not work nor pull up the appropriate jobs according to the GS rating one requests

Mon, Oct 24, 2011 Jack Las Vegas, NV

Another great job OPM! Thanks for hindering economic recovery efforts and wasting taxpayer money. Don’t worry, I can wait another few months for a job.

Sun, Oct 23, 2011 Tacoma, WA

I have tried repeatedly to access the site on both my computer and my iPhone. I am able to put in my login and password, but then I am taken to my security questions. Once I enter the proper answers, I end up back at the the sign-in page. I am right back where I started with no way to get passed this loop. This is not a good sign. I am supported to be up for a job next week. I hope this does not cause that process to grind to a halt. I hope the system will at least generate the lists for the job. I am not instilled with confidence on that matter with the performance to date.

Show All Comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above