Data Tools

Does your enterprise search engine stink? Here's why.

Search is so ubiquitous that the Oxford English Dictionary recognizes “Google” as a verb. There is not a single person who uses a computer who does not use a search engine on a regular basis. Search has changed the way we think, the way we research and even the way we process memory. From Lycos to Ask Jeeves, Yahoo, Bing and Google, we have been conditioned to believe that search will do the thinking for us.

That belief, however, is an illusion. What we do not see as consumers is the incredibly complex, extensive and intensive work that goes on behind the scenes to make search seem effortless. Although that illusion might not harm us in our personal lives, in business it presents a core challenge that must be overcome when properly implementing and maintaining an enterprise-level search engine.

Given the predisposition to think of search as preformatted to meet our needs, many IT managers and executives believe they can simply purchase, install and operate enterprise search software right out of the box. To a large extent, the leading search software vendors promote this plug-and-play mentality because it is a message customers want to hear. If you are familiar only with Web search as a personal tool, it makes sense to assume that a search engine for your business would operate the same way.

Most people simply do not realize the dramatic differences in the data landscape between Web and enterprise search. Most Web searches meet with normalized, high-quality Web pages, complete with excellent metadata. This high-quality content has been provided by an estimated $3.8 billion industry of diligent Web marketers and search engine optimization consultants. But behind the firewall, the 80 percent of corporate data that is unstructured is anything but normalized, and it typically carries little or no useful metadata. That huge quality gap matters.

Given their positive experience with Web search, people often believe that sophisticated algorithms can compensate for poor-quality data being input into the search engine. No matter how smart, sophisticated, nuanced or precise the search engine is, if the data is pulled from a series of lawless corporate file shares and dumped into the search engine, users are likely to be disappointed with the quality of the search results.

Don’t assume that data can be fed raw into the search engine. In most enterprise search implementations, some attention to detail for normalizing data, capturing existing metadata and automatically generating new metadata that is contextual to the application will go a long way. That requires thought, planning and configuration effort.

In other words, even the latest and greatest search engines need our help to better understand the data they are indexing. That content processing is aided by tools provided through most search engines, but it still requires a human hand to sculpt and properly implement.

Proper content processing — combined with productivity tools such as spell checking, query auto-completion and results sorting options, all of which are standard features of the latest search products — will help re-create the same illusion for your users as they experience with Web search. Providing this seemingly effortless search experience is the mark of a well-executed enterprise search engine implementation. To make it happen, we all need to overcome our conditioning that search will automatically do the thinking for us.

About the Author

Kamran Khan is co-founder, president and CEO of Search Technologies.

The Fed 100

Save the date for 28th annual Federal 100 Awards Gala.

Featured

  • computer network

    How Einstein changes the way government does business

    The Department of Commerce is revising its confidentiality agreement for statistical data survey respondents to reflect the fact that the Department of Homeland Security could see some of that data if it is captured by the Einstein system.

  • Defense Secretary Jim Mattis. Army photo by Monica King. Jan. 26, 2017.

    Mattis mulls consolidation in IT, cyber

    In a Feb. 17 memo, Defense Secretary Jim Mattis told senior leadership to establish teams to look for duplication across the armed services in business operations, including in IT and cybersecurity.

  • Image from Shutterstock.com

    DHS vague on rules for election aid, say states

    State election officials had more questions than answers after a Department of Homeland Security presentation on the designation of election systems as critical U.S. infrastructure.

  • Org Chart Stock Art - Shutterstock

    How the hiring freeze targets millennials

    The government desperately needs younger talent to replace an aging workforce, and experts say that a freeze on hiring doesn't help.

  • Shutterstock image: healthcare digital interface.

    VA moves ahead with homegrown scheduling IT

    The Department of Veterans Affairs will test an internally developed scheduling module at primary care sites nationwide to see if it's ready to service the entire agency.

  • Shutterstock images (honglouwawa & 0beron): Bitcoin image overlay replaced with a dollar sign on a hardware circuit.

    MGT Act poised for a comeback

    After missing in the last Congress, drafters of a bill to encourage cloud adoption are looking for a new plan.

Reader comments

Thu, Nov 20, 2014 DamianD

It is truly hard to find an enterprise search engine that fits to your business. You need to put a lot of money and effort in building up the infrastructure. Nonetheless I think you should try out Lookeen (www.lookeen.com). Especially for smaller businesses it is a great help. You can find anything from emails to files with an advanced search functionality. Note: The developer of Lookeen is my employer.

Wed, Nov 14, 2012

Thank you for helping dispel the myth that implementing a search solution is "easy". Finally a voice of reason in the search dialogue!

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

More from 1105 Public Sector Media Group