Companies like Google, Yahoo and MSN use software to build databases of Web pages and then provide a powerful process - based on filling in specific fields with keywords and operators - with which to search that data. We call these tools search engines. This article explores some of the differences between search engines and the direct implications these have on what we see when we type a query into a search box.

Each search engine uses a unique algorithm to collect data from Web pages and build a database. The search engines don't index exactly the same Web pages. So how much overlap is there between the snippet results in Google and Yahoo? A study conducted by Jux2 in April 2004 ran 500 queries and discovered that only 3.5 out of the top 10 results in Google were also in the top 10 results in Yahoo. So if you search in only one database, there's a good chance that the quality of information you need may not be where you are looking.

We can see this in action using a meta search engine called Jux2. Jux2 is designed for comparative searching of three Web page databases - Google, Yahoo and MSN - all at the same time. When you enter a query, it shows you which snippet results are common and which are unique to each search engine for that query.

I wanted to find the most pertinent and quality source that provides schedules and stories about the 2006 Soccer World Cup. Assuming that I didn't know anything specific about soccer and the organization hosting the event, I simply used the keywords [soccer world cup] in jux2 and I got some fascinating results. A search using those three keywords in jux2 showed me that the Fifa World Cup site http://fifaworldcup.yahoo.com was the #2 result on Google and the #1 result on Yahoo. After a quick look at that website I knew this is what I was looking for - a comprehensive and authoritative site created by the official event sponsor, Fifa.

I selected the button to "Compare MSN’s results” to see which webpages MSN missed for this query that the other two search engines had found. To my surprise, MSN didn't show the Fifa World Cup website in the top 10 results from its database for this query.

I wondered whether I could find the Fifa World Cup Web page in MSN’s Web page database so I broadened my query by leaving out the word [soccer] and used only [world cup]. Then the Fifa World Cup site appeared as #1 in MSN’s snippet results. It turns out that the word [Fifa] is a much more specific keyword and had I used that, I would have the website I was looking for at the top of the list in all three search engines.

This comparison of search engine results suggests a couple strategies for our own practice:

• For any given query there is very little overlap between the results that the search engines pull from their databases.  If I had been searching exclusively in MSN and had only used my initial query, I would have not found the most authoritative resource for the 2006 World Cup.  It is essential to use more than one search engine to find the best resources. 
• Each search engine uses its own algorithm to index webpages, so we can’t rely on just one set of keywords in our research.  Revising and refining our query by either broadening or narrowing the keywords is necessary to drive the highest quality snippets to the top of the list.

Leaving either one of these approaches out of your search strategy means that some of the best websites will not be retrieved.

Try It!
10 minute Mini-lesson for the Classroom

Step 1 The next time you have a group of students who are preparing to do a research project, have them brainstorm keywords for the topic. State that using just one search engine database to look for information often means that you miss relevant and useful resources.

Step 2 Using a multimedia projector, go to Google and plug in the query they have crafted as a group.

Determine if any of the top 5 are worthwhile. If not, revise the query as a group until you have 1 or 2 snippets that would be worth using. Note these top 5 results.

Step 3 Have the students vote on which of the 5 snippets in

Google will show up in the top 5 in Yahoo using the same query. Go to Yahoo and plug in the query. Compare which results were the same and which were different.

Step 4 Debrief with the students. Indicate that search engines build their databases using different rules and that they don't all index the same webpages. Discuss when it would be useful to use more than one database in their research.

Step 5 Show them the jux2.com metasearch engine and plug in the query that they crafted. Use the “

Compare results” feature to show them what each search engine missed in the top 10 snippets.

Step 6 Suggest they use jux2.com at least once as they do their research project.