Query Assessment Guide

Home > Question to Query > Query Assessment Guide

The Query Challenge assessment is intended to measure students' understanding of effective queries. Four multiple choice challenges make up the assessment. Each item and the responses have been tested to determine what is effective in actual searching.

Use this guide to interpret students' responses and select helpful interventions, especially if students make the same type of mistakes more than once. The report page of the assessment also provides user feedback. Have students print out the page to provide feedback about their performance. Instruct students to fill in the name and course/section boxes with the appropriate information to get started.

Typical mistakes may fall into one or more of the following groups:

Too few words: A one word query is rarely effective, especially since the search question may consist of multiple important concepts. If a question has two important concepts, search for both of them; if a question has three important concepts, search for all three; if a question has four or more important concepts, search for the three most important--going beyond three or four terms limits the effectiveness of a query due to the 1 in 5 rule (also see below).

Too many words: More than three or four keywords becomes problematic due to the 1 in 5 rule. On average, your ability to guess the word used by the person who authored the information you want is 1 in 5. For two words, the odds could be as poor as 1 in 25; for three words only randomly related, the odds are 1 in 125, and so on. Since words tend to be used in meaningful contexts, the odds are probably not as bad as this. Nonetheless, the more words you use, the less likely you will match all the words used. Using natural language really inflates the word count and tends to mask two types of particularly ineffective keywords:

• Useless and redundant words -- words that duplicate meaning of another word should be avoided (for example, in the query fastest speed, speed is unnecessary if fastest is used). The same is true for useless words that really don't add to the meaning (for example, in the query China's Great Wall, China's is useless--the Great Wall is in China.).

• Stop words -- Common parts of speech (pronouns, prepositions, intransitive verbs) are ignored by many search engines. It doesn't hurt to add them, but it wastes time and may give the erroneous impression that a query makes more sense than it does. Remove the stop words and see if the words that remain are essential.

Challenge 1 choices	What is the top speed of earth's fastest animal?
a. fastest animal	Correct.
b. speed fastest animal	Incorrect. The word speed is redundant when fastest is used. This query is still effective in finding relevant information, however, so 1 point is awarded. Instruct students to examine keywords carefully to determine if all the words are really necessary.
c. top speed earth's fastest animal	Incorrect. Two redundant terms (top and speed --both are captured in the term fastest) and one useless term (earth's -- where else would you look for animals?) are contained in this query. Instruct students to examine keywords carefully to determine if all the words are really necessary and reduce the query to its essential ideas.
d. what is the top speed of earth's fastest animal	Incorrect. Several stop words (what, is, the, of), two redundant words and one useless word (see above) are included in this query. Caution students about using natural language in queries: stop words are ineffective because search engines ignore them, redundant words and useless words add to the word count and reduce the likelihood that relevant answers will be found in the top snippets.

Challenge 2 choices	In what ancient country was an abacus first used?
a. abacus	Incorrect. A one-word query omits an important search concept (in this case, first).
b. first abacus	Correct.
c. ancient country abacus	Incorrect. The word country complicates the search, especially by including the word ancient just before it. Instruct students to consider if all the important concepts are captured by their keywords and remove any unnecessary words.
d. in what ancient country was an abacus first used	Incorrect. Several stop words (in, what, was, an), one redundant word (ancient--when first is used) and two useless words (country and used) are included in this query. Verbs tend to make ineffective search terms. Caution students about using natural language in queries: stop words are ineffective because search engines ignore them, redundant words and useless words add to the word count and reduce the likelihood that relevant answers will be found in the top snippets.

Challenge 3 choices	How many years does an average Chinese water dragon live?
a. Chinese water dragon	Incorrect. Although all these words are necessary, an important concept is missing (years). Instruct students to identify all the important concepts and include them in the query.
b. Chinese water dragon live	Incorrect. An important concept is missing (years). The word live is useless (and it has multiple meanings) Instruct students to identify all the important concepts and include them in the query.
c. years Chinese water dragon	Correct.
d. many years average Chinese water dragon live	Incorrect. The query is complicated by including a redundant word (many--when years is used) and two useless words (average and live). Caution students about using this many words in a query. Redundant and useless words add to the word count and reduce the likelihood that relevant answers will be found in the top snippets.

Challenge 4 choices	What animal can hold its breath the longest time underwater?
a. animal breath	Incorrect. Although both words are necessary, an important concept is missing (longest). Instruct students to identify all the important concepts and include them in the query.
b. animal hold time	Incorrect. Two important concepts are missing (breath and longest). The word time has multiple meanings and is not as powerful as longest. Instruct students to identify all the important concepts and include them in the query. Avoid using verbs in a query ("hold").
c. animal breath longest	Correct.
d. animal hold breath longest time underwater	Incorrect. The query is complicated by including two useless words (hold and underwater) and one redundant one (time--when longest is used). These words add little to the overall meaning. An animal usually doesn't hold its breath unless it is underwater. Caution students about using this many words in a query--have them reduce the number of words to equal the number of essential concepts. Redundant and useless words inflate the word count and reduce the likelihood that relevant answers will be found in the top snippets.

Query Challenge (assessment instrument)

Challenge 1 choices

What is the top speed of earth's fastest animal?

Challenge 2 choices

In what ancient country was an abacus first used?

Challenge 3 choices

How many years does an average Chinese water dragon live?

Challenge 4 choices

What animal can hold its breath the longest time underwater?