Better searching: Technology and people

If you're like me and many others surfing the internet, you reluctantly rely on search engines to help you find information.

By Len O'Neal September 1, 1999

If you’re like me and many others surfing the internet, you reluctantly rely on search engines to help you find information. You type in a word or two, hoping you’ve done so according to that particular search engine’s “tips” for improving your search results, click the search button, and pray that something relevant and useful appears within the first two or three pages of the 20,000 results you receive.

OK, perhaps this is an exaggeration. But as the number of web sites continues to reach mind-boggling numbers (over 5 million distinct web sites as of April 1999, according to Netcraft ), so do the number of web pages on the internet that can appear on your search results page. This dilemma is not lost on those who manage search engine web sites and the technologies behind searching; new and improved methods for finding your needle in the haystack are being introduced on almost a weekly basis, and many of them rely on people, not just technology.

The popularity contest

A first step in the goal of providing search results with better relevancy to your inquiries has been the use of link popularity, whereby technology monitors what web sites people are clicking on and moves the top choices higher on the search results list. This “popularity results” approach is used by several search sites, including, Hotbot. com,, and MSN Search (search.msn. com); on some search sites, you are not even aware that it’s there, while on others you will see a link entitled the “Top 10 Most Popular Sites.”

You too can be an editor

A more recent approach to improving the way we search the internet is the introduction of “open” directories, which are essentially Yahoo!-like web sites operated by volunteer editors. The more popular Open Directory ( has over 14,000 editors who organize web sites into categories. Many search sites are seeing a value to this approach, incorporating links from the Open Directory into their standard query results listings (see and

Others, such as, have decided to go it alone and are inviting web surfers to help manage its own directory of web sites into “Go Guides.”

Your (re)search assistant

Another technique–and it doesn’t get any more personal than this — is LookSmart Live at, where the concept is simple: You tell LookSmart what you are looking for, your request is sent to one of 80 editors working on this project, and you receive an e-mail with your answer within 24 hr.

Think of it as a focused discussion forum with a very knowledgeable moderator. Which reminds me: When you have a specific plant engineering-related question, use the resources available to you at Plant Engineering Online ( The discussion forums and the improved “Ask the Expert” feature provide you with the opportunity to use fellow plant engineers and industry experts to find the information you need.