Outside the Search Box

Full Circle > Getting Started >

Confessions of a Search Challenge Designer

by Carl Heine

As a researcher and a challenge designer, I spend quite a bit of time searching outside the box. Sometimes this involves breaking the rules: not following the advice found in the Resource Kits.

When designing a new challenge I usually enter a single term into Google (e.g., rollercoasters) and scan the results for something interesting. Anyone familiar with effective keyword strategies will recognize the value of searching with multiple words to narrow the results. Using a single word, a practice popular with digital natives, yields unexpected results, which works well for an exploratory search. I've done the same thing by selecting a single word in a subject directory and using it to springboard to another interesting word, and so on, until I arrive at something I think would make an appealing search. If you start out uncertain what you want to look for, the single word search works a lot like brainstorming: retrieving a wealth of information through discovery. This can be a very valuable exercise when looking for an interesting subject to research. The Kermit, Pump Price and Lego Challenges all began this way.

I break with another common practice by starting with results after the first few pages. Few individuals (about 4%) ever look at the second page of returns. Google reports that the number of searchers who open the third page (results numbered 21 to 30) is so low that it is statistically insignificant.<need source> What's the deepest you've ever gone? How deep can you go? You may be surprised to learn that although Google says it found millions or billions of results for a query, you can retrieve--at most--only 800 to 900 of them. The list ends around page 85 (give or take a few) when 10 results are displayed per page. With 100 results per page the list stops at page 9. I imagine this helps speed up the retrieval process; besides, who ever looks that far down the list anyway? I tend to only because there's often hard-to-find search material down there.

Once something interesting catches my eye, I see if I can raise it to the first page of results. This is easy to do with the right combination of keywords pulled from the page. If the search challenge is about trying alternate keywords (nyms), I test the new words in a query to see if what happens to the target page. If it still comes to the top, other terms are tried until the page is buried once again. Those become the words used in the search challenge. It requires repeated testing of combinations, but because of the 1 in 5 rule, a nyms search is relatively easy to create.

A search that requires operators is much harder to design. Most of the time, operators just aren't needed. Quotes are the exception and I don't create many of those searches. Once you recognize that a search challenge contains an exact phrase, it can be solved in seconds (the TV Challenge, the Poetry Challenge). Harder to recognize are a couple of keywords that must be kept together in the order given. Most of the time Google does a nice job without having to use quotes around word pairs. As long as the words are entered in the order they normally occur, it's practically the same as using quotes. But the more common the words, the more helpful quotes tend to be. And then if you need to find a reversed word pair such as 'America North', quotes help to eliminate the more frequently used 'North America'.

The hardest challenges to design are those that require OR ( | ) or NOT ( - ). There are just so many ways to find something using words that operators simply represent one more way. Even though the Apollo 8 and Piranha Challenges can be solved by using NOT, I realize there are other, less efficient ways to solve it. Individuals who think outside the search box will find other ways. Because of this, I've concluded that the only way not to find something is by looking for the wrong thing from the start.

Here are a few more thoughts about getting outside the search box to find what you need:

Think about where to look. Another database besides Google probably has the answer too--and it might not be as buried as deeply there.
Think about the answer, not the question. Most of the time, new terms will emerge by imagining how an answer may be worded.
Use advanced search whenever you can. Don't just stick with the AND search box (the simple Google search). Try the other options: eliminate a pesky domain with the -site: search box, look for synonyms with the ~ search box. Each database's advanced search box may offer different controls.
Browse without getting lost. While the least efficient method of searching, browsing has a lot going for it when used wisely. Keep a set of keywords in mind and follow links that match those or better terms you discover. Turn around whenever the trail goes cold. A persistent, determined browser can and will find information.
The same browsing skills apply to using a subject directory: start with known keywords, be on the lookout for better ones and turn around whenever a trail peeters out.
Ask someone else what they did. You don't have to discover everything the hard way.
Try breaking the rules of good search. Searching the Internet is all about the unknown and probability. The most seasoned searchers cannot always predict what they will find or the best way to get there. If the high probability techniques don't work, try experimenting. That's how I've discovered much of the information that makes up the search challenges on this site.