Search Smarter, Not Harder, On the Web

Janet Foster, Contributiong Editor

Search engines and Web directories provide organization and structure for the vast amounts of information available on the Internet. Web directories such as Yahoo! include lists of resources that are classified into categories such as entertainment, computers, reference, and other subject headings. When looking for broad information on a general topic, directories are an excellent place to start. Search engines are more like book indexes in that they can pinpoint exact sites and help locate answers to specific questions. The two types of tools complement each other in the research process. With the proliferation of search engines, Web directories, and metasearches, the choices can seem overwhelming. A number of guides to deciphering search engine use and locating information more efficiently are available. The following Web sites and print recommendations highlight excellent resources for narrowing down the scope of searches and making online time more efficient.

Web Directories

Web directories like Yahoo! can be compared to a book’s table of contents. They provide an overview of information and help narrow the scope of your search within the boundaries of their virtual stacks. Yahoo!, “Yet Another Hierarchically Organized Oracle,” is the oldest major Web directory and a great place for patrons to start surfing the Web. Its large Web guide categorizes high-interest topics that are updated frequently, such as stock quotes and news, while its ease of navigation makes it popular with people new to the Web.

The Librarian’s Index to the Internet (LII) is a highly recommended searchable Web directory created and maintained by Carol Leita. Its hierarchy of categories is organized by subject matter with short annotations and no images facilitating more efficient search time. LII contents are excellent and its graphical user interface is easy to navigate.

Sites are continually added, making the searchable database extremely valuable. Bookmark the Librarian’s Index to the Internet and subscribe to receive weekly updates via e-mail. Send the message, subscribe liiweek yourFirstname yourLastname, to listproc@sunsite.berkeley.edu. Also, consider using the pathfinders at the Internet Public Library. IPL Pathfinders are created by the IPL staff and are intended to help people get started doing research on a particular topic, both online and at the local library. Current pathfinders are available in a multitude of subject areas including: arts and humanities, computers and Internet, education, entertainment, and just about any topic imaginable.

Search Engines

Unlike directories, search engines retrieve information from the entire Internet. AltaVista, Infoseek, Excite, HotBot, and Lycos are all examples of major search engines. Keywords are typed into the search engine, which then locates relevant matches on the Web. To keep records current, the search engines use programs called spiders or bots that follow links from page to page, recording all or parts of the contents of each page as they go. Most search engines work similarly.

AltaVista is one of the largest search engines on the Web, in terms of pages indexed. Its comprehensive coverage and advanced search commands make it an excellent choice for researchers. It also offers a number of features designed to appeal to basic users. In addition, for libraries with large ethnic populations, AltaVista offers translating services that allow people to search and read results in a wide selection of foreign languages.

Like Alta Vista, Hotbot is a favorite among researchers due to its large index of the Web and many power searching features. A drop-down menu allows searchers to refine searches to “all the words,” “any of the words,” and other phrases that limit the results. Another drop-down menu allows searchers to select images, sounds, or other methods of narrowing the scope of a search.

Infoseek is one of the more popular search engines and may be considered the best place to start a comprehensive search of the Web. Infoseek consistently provides quality results in response to general and broad searches. It also has an impressive directory of Web sites. Using Infoseek to refine searches is easily accomplished with an advanced search function that allows users to search within the results, thereby finding the most relevant results wih the least number of keywords. Placing keywords within quotation marks reduces the excessive number of irrelevant hits. This tip applies to all major search engines.

Switchboard is the site to use when looking for a person or a company. Switchboard compiles telephone listings from across the U.S. to narrow your search geographically as you might need to do in directory services offered by many telephone companies. Reference librarians can use services like Switchboard for patrons who need lists of companies limited by type or locale.

Switchboard and other Web directories like Bigfoot have replaced costly CD-ROM telephone databases that tend to become quickly outdated.

Google! is a search engine that makes heavy use of link popularity as a primary way to rank Web sites. This can be especially helpful in finding good sites in response to general searches, because users across the Web have in essence voted for good sites by linking to them. Google! is a new search engine still considered an “alpha test” with approximately 25 million Web sites indexed. The Google! Web page is structured simply without numerous advanced search features. Search results include an excerpt of the text containing the key words searched for.

Comparing Search Engines

Search Engine Watch provides a thorough compendium of available search engines and their technologies, including strategies for maximizing use of such services. The purpose of searchenginewatch.com is to compare different search engines and illustrate how to perform keyword and Boolean searches in each. A “Webmaster’s Guide to Search Engines” depicts how search engines index sites and explains search engine design plus strategies for improving ranking. “Search Engine Facts and Fun” gives an overview of leading search engines, commonly searched-for terms, and other details; “Search Engine Resources” reviews leading search engines and gives tutorials on using them. You can also subscribe to “Search Engine Report,” a monthly newsletter that covers developments with search engines and changes to the Search Engine Watch Web site.

The Spider’s Apprentice: A Helpful Guide to Web Search Engines provides exactly what its title indicates. Spidap is not another search engine. Rather, it offers tips on using search engines like AltaVista, Infoseek, Excite, WebCrawler, Lycos, Hotbot, and Yahoo! thereby helping you search the Web more efficiently. The Spider’s Apprentice includes a helpful guide on how search engines work, including a breakdown by type such as keyword searching and concept-based searching.

Created and maintained by Greg Notess, author of Government Information on the Internet and a colunist in Online and Database, Search Engine Showdown offers reviews, analyses, statistics, and strategies for many of the major search engines. Organized in four primary sections (Features, Analysis, Learning, News and Talk) the site explores topics such as search engine database size and capabilities, searching for phone number information, subject guides, multiple search engines, and tips on successful searching. The site is regularly updated and also includes a newsletter, which users may subscribe to at the site.

Marylaine Block’s “When and How to Search the Web” is a superb reference for librarians. This comprehensive guide discusses when to use indexes versus search engines, how to evaluate information, and tips on using advanced features. Part of the “Best Information on the ’Net,” Marylaine Block’s informative article should be printed out or bookmarked for future reference.

Meta Search Tools

Even the largest keyword indexes can’t catalog all the information on the Internet, and tend to specialize in certain types of information. Meta searches offer access to several databases at the same time ensuring more comprehensive results. Meta services query multiple search engines simultaneously and then return the results organized by search engine or combined into one set of returns. Although they can provide more complete search results, meta searches tend to be slower than conventional search tools.

Dogpile is a general meta engine that fetches data from multiple search engines and displays results on a single page making results easily accessible. It also features simultaneous searches of subject directories, usenet news, mailing lists, and FTP collections, as well as Web pages. With Dogpile, you can customize the search order and it will automatically use some of the advanced search operators if the database searched has them.

An excellent meta search engine for librarians can be found at the Research-It! Web Site. Research-It! is a great resource for searching dictionary, thesaurus, language translation, and other reference works. If you’ve attended any national library conferences lately, chances are good that you’ve encountered the Web butlers who “serve up answers” at the Ask Jeeves Web site.

Ask Jeeves is a subject directory that allows users to enter questions in plain English, making searching easier for searchers than conventional Boolean logic. Results are returned from pages found on various other engines. Ask Jeeves searches its database of already researched questions, returns several possible question matches, and runs questions through WebCrawler, Yahoo!, Infoseek, Excite, AltaVista, and Lycos. The results are presented in drop-down select boxes. A kid’s version, Ask Jeeves for Kids, is also available and includes special topics that can be highlighted each month. Both are worthy of inclusion on your bookmark list.

Taking the Quantum Cyber Leap

Recent articles in the New York Times and USA Today indicate that the growth of the Web has surpassed its search capability. Using a combination of search engines and directories in conjunction with one another can help find relevant sites. But when all else fails, consider taking the quantum leap. If the site you are researching is known as a company, educational institution, or nonprofit organization, consider trying the name and the appropriate domain ending (i.e., .com, .edu, .org). For example, Yale University would be www.yale.edu. Also, read books and journals to keep apprised of Web search trends.

Search Engines: For the World Wide Web by Alfred and Emily Glossbrenner is a clear, step-by-step book with chapters on each of the most popular Web search engines: AltaVista, Excite, HotBot, InfoSeek, Lycos, WebCrawler, and Yahoo! It also shows the reader how to quickly find particular people or companies via other search tools such as Four11, bigBook, and BigYellow. The authors provide tips and techniques for finding information on the Internet - quickly, easily, and without frustration.

Searcher, The Magazine for Database Professionals is available online or by subscription. Regular columns include Web Wise Ways and Searcher’s Voice. And Yahoo! features the The Surf Guru where you can pose questions or read answers to others’ queries.

The preponderance of Internet sites and search engines necessitates making wise Web choices. As a general rule, if your topic is general, try an Internet guide first. If your topic is very specific, a search engine is a better choice.

Be flexible and check out multiple choices. Internet guides and search engines vary greatly and none have the capacity to catalog the entire Web. Despite the enormous wealth of information on the Internet, in many cases print resources are still the best resources for librarians.