Searching | Evaluation | Ethical Use

Locating Resources on the Web

image of Featured Tips

Tip #10 Search the Invisible (Deep) Web

There are documents hidden in areas of the Internet that search engines simply can’t see. The search engine’s robotic ‘crawlers’ either miss or are locked out of these areas on the Internet. Behind the barriers lie treasure troves of quality information. Collectively this information is called the Invisible Web. There are many categories of invisible information missed by the popular search engines. The most common are: webpages that have been skipped by search engine crawlers, online database content, and password protected websites.

Searching the Invisible Web for information is a two-step process. First you search for online resources likely to hold the 'invisible' information. Once you find such a site, you go there and use local tools to search the site itself.

Without a search strategy, the size of the Invisible Web creates information overload. To search the Invisible Web you need a plan that helps you quickly find the most likely sources of quality information. As you develop a personal search strategy that is both flexible and focused you become a more efficient researcher

Strategies to Consider:

If you already know a site that has the kind of information you are looking for, go directly to the source. If you are looking for the population of a US city, instead of using a traditional search engine, search the database available at http://www.census.gov. When seeking information on the Invisible Web, use a search engine to find a websites with databases about your topic. Go to the website and query the database directly. Use websites that list Invisible Web collections and provide easy access to search forms. When visiting a site with Invisible Web resources use the site's search tools to refine your search to the specific information you are looking for.

Invisible (or Deep) Web Resources These sites lead you to invisible web content:

CompletePlanet - BrightPlanet claims to have103,000 search sites organized into more than 4,000 subject headings.
Online Learning Module: Invisible Web: How Can You Search It?
Online Learning Module: The Opaque Web
Online Learning Module: Vanishing Web