Descriptions of Internet Search Tools
The sheer volume of information on the Internet makes it necessary to carefully consider what type of search tool best meets your needs. Searchers have a variety of options for conducting research on the Internet. Search tools serve different purposes: there are those that cast a wide net, indexing as much of the Internet as they can; and there are those which address more narrow interests, such as indexing only government sites.
Below you'll find descriptions of five major categories of search tools. No matter what kind of search you're doing, be sure to use more than one if you don't immediately get good results. Remember: whenever you use a Web search tool you are not searching the entire Web, you are searching only a portion as captured in that particular database.
Subject Directories | Gateways | General Search Engines | Meta-Search Engines | Specialized Databases
Directories have two major strengths: selectivity and human categorization. These Web guides use human intercession for filtering and quality control. Editorial decisions are employed in deciding what to include in the database and how to organize it. They are great for browsing, when your search is of a general nature, or for when you are using fairly common words. For example, if you want information on buying car insurance, a subject directory would be useful. While the keywords "car" and "insurance" would produce a flood of information in the larger Web search engines, they would produce a much more controlled set of results in directories.
These selective guides direct you to high quality information on the Web. If you're writing a research paper, you might want to start with this type of site. Listings may be compiled by librarians or subject experts and are often annotated. These guides tend to exercise editorial judgment which helps insure quality and relevance.
The databases of these engines are built by software - often called spiders or crawlers - which comb the Web by following links. After visiting a page, the spider may bring back the entire page to add to the database, or just the URL, title, headers and the first few words. Unlike directories and gateways, there is no filtering or quality control. These engines attempt to cover as much of the Web as possible but even the best doesn't cover everything on the Internet. Some search engines have added their own subject directories, which are a subset of the database.
Unlike the tools above, these search engines don't build their own database of Web pages. Instead they run a simultaneous search in several other engines. The results are then reported back in a single list. For example, MetaCrawler queries Lycos, WebCrawler, Infoseek, Excite, Thunderstone, AltaVista, GoTo, and Yahoo. These types of search engines allow the user only a few refinements, such as phrase searching, so you will need to go to individual engines to take advantage of the full range of search options. Also, they are subject to time-outs when search processing takes too long and they retrieve only the top hits from each search engine.
Use these when you want to search a limited part of the Web. These search tools tend to focus on a specific topic - this helps to limit the number of results and improve precision. You will find databases covering many special interests such as news, business, phone directories, health, government information, etc. They can be the stand-alone type (e.g., DayPop) or a part of a larger directory (e.g., Yahoo News).