Data Tools

Does your enterprise search engine stink? Here's why.

Search is so ubiquitous that the Oxford English Dictionary recognizes “Google” as a verb. There is not a single person who uses a computer who does not use a search engine on a regular basis. Search has changed the way we think, the way we research and even the way we process memory. From Lycos to Ask Jeeves, Yahoo, Bing and Google, we have been conditioned to believe that search will do the thinking for us.

That belief, however, is an illusion. What we do not see as consumers is the incredibly complex, extensive and intensive work that goes on behind the scenes to make search seem effortless. Although that illusion might not harm us in our personal lives, in business it presents a core challenge that must be overcome when properly implementing and maintaining an enterprise-level search engine.

Given the predisposition to think of search as preformatted to meet our needs, many IT managers and executives believe they can simply purchase, install and operate enterprise search software right out of the box. To a large extent, the leading search software vendors promote this plug-and-play mentality because it is a message customers want to hear. If you are familiar only with Web search as a personal tool, it makes sense to assume that a search engine for your business would operate the same way.

Most people simply do not realize the dramatic differences in the data landscape between Web and enterprise search. Most Web searches meet with normalized, high-quality Web pages, complete with excellent metadata. This high-quality content has been provided by an estimated $3.8 billion industry of diligent Web marketers and search engine optimization consultants. But behind the firewall, the 80 percent of corporate data that is unstructured is anything but normalized, and it typically carries little or no useful metadata. That huge quality gap matters.

Given their positive experience with Web search, people often believe that sophisticated algorithms can compensate for poor-quality data being input into the search engine. No matter how smart, sophisticated, nuanced or precise the search engine is, if the data is pulled from a series of lawless corporate file shares and dumped into the search engine, users are likely to be disappointed with the quality of the search results.

Don’t assume that data can be fed raw into the search engine. In most enterprise search implementations, some attention to detail for normalizing data, capturing existing metadata and automatically generating new metadata that is contextual to the application will go a long way. That requires thought, planning and configuration effort.

In other words, even the latest and greatest search engines need our help to better understand the data they are indexing. That content processing is aided by tools provided through most search engines, but it still requires a human hand to sculpt and properly implement.

Proper content processing — combined with productivity tools such as spell checking, query auto-completion and results sorting options, all of which are standard features of the latest search products — will help re-create the same illusion for your users as they experience with Web search. Providing this seemingly effortless search experience is the mark of a well-executed enterprise search engine implementation. To make it happen, we all need to overcome our conditioning that search will automatically do the thinking for us.

About the Author

Kamran Khan is co-founder, president and CEO of Search Technologies.

FCW in Print

In the latest issue: Looking back on three decades of big stories in federal IT.

Featured

  • Anne Rung -- Commerce Department Photo

    Exit interview with Anne Rung

    The government's departing top acquisition official said she leaves behind a solid foundation on which to build more effective and efficient federal IT.

  • Charles Phalen

    Administration appoints first head of NBIB

    The National Background Investigations Bureau announced the appointment of its first director as the agency prepares to take over processing government background checks.

  • Sen. James Lankford (R-Okla.)

    Senator: Rigid hiring process pushes millennials from federal work

    Sen. James Lankford (R-Okla.) said agencies are missing out on younger workers because of the government's rigidity, particularly its protracted hiring process.

  • FCW @ 30 GPS

    FCW @ 30

    Since 1987, FCW has covered it all -- the major contracts, the disruptive technologies, the picayune scandals and the many, many people who make federal IT function. Here's a look back at six of the most significant stories.

  • Shutterstock image.

    A 'minibus' appropriations package could be in the cards

    A short-term funding bill is expected by Sept. 30 to keep the federal government operating through early December, but after that the options get more complicated.

  • Defense Secretary Ash Carter speaks at the TechCrunch Disrupt conference in San Francisco

    DOD launches new tech hub in Austin

    The DOD is opening a new Defense Innovation Unit Experimental office in Austin, Texas, while Congress debates legislation that could defund DIUx.

Reader comments

Thu, Nov 20, 2014 DamianD

It is truly hard to find an enterprise search engine that fits to your business. You need to put a lot of money and effort in building up the infrastructure. Nonetheless I think you should try out Lookeen (www.lookeen.com). Especially for smaller businesses it is a great help. You can find anything from emails to files with an advanced search functionality. Note: The developer of Lookeen is my employer.

Wed, Nov 14, 2012

Thank you for helping dispel the myth that implementing a search solution is "easy". Finally a voice of reason in the search dialogue!

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group