Maybe if I Just Add Some Quotes…

Perhaps lost in all the coverage surrounding the extraordinary financial successes of search-related companies is the story regarding growing dissatisfaction with internet search. These stats from an InformationWeek article entitled “The Ultimate Search Engine” are nonetheless becoming more familiar:

“People search for 11 minutes on average before finding what they’re looking for, and half abandon searches without getting that far, according to Microsoft. By Gartner’s estimate, half of potential Web sales are lost because visitors simply can’t find what they want.”

Furthermore, a May 2007 study by Autobytel and Kelton Research asserts that “72% of consumers report experiencing ‘search engine fatigue’ (i.e., a high level of frustration) when researching a topic on the Internet” and that “65% of consumers say they’ve spent two or more hours in a single sitting searching for specific information on search engines.”

Search is not “broken,” but if half of users abandon searches without ultimately being satisfied, half of potential sales are lost and 3 out of 4 people are feeling frustration, then clearly there is substantial room for improvement.

It is our contention that there are two major factors driving this dissatisfaction:

1. The quantity of content on the internet is exploding. Google announced 6 billion items in their index way back in 2004, and one may only speculate at where it is today. Yahoo! has since proclaimed 20 billion items and, more recently, Microsoft declared a quadrupling of their index to 20 billion items. As discussed previously, it is virtually impossible to navigate that much information with two- and three-word queries.
2. As users become more familiar with search and the internet, their expectations increase. As John Battelle noted in his book “The Search”, “Surfers [are moving] from a stance of exploration (‘What’s out there?’) to expectation (‘I want to find something that I know is out there’).” When people know that the information is out there and expect to find it, but yet they can’t because it’s buried under an avalanche of irrelevant results, frustration naturally ensues.

There are, not surprisingly given the enormity of the stakes at hand, many strategies for combating this problem. Yet it is particularly instructive to note what users are actually doing during those 11 minutes to over two hours. Since most everyone has already been there, we know that it consists primarily of clicking on various results and, most importantly, query reformulation.

Query reformulation is a major challenge facing internet searchers today. 28% of all queries are reformulations of a previous query and, in such cases, the average query is reformulated 2.6 times. While experienced users know how to manipulate terms, use quotes, employ Boolean operators and even filter out documents, the outcome is that there is a direct relationship between the quality of the results and the skill of the searcher.

The major search engines, recognizing this problem, have implemented functionality to facilitate query reformulation. Yahoo!’s Search Assist will suggest keywords and related concepts as users enter queries. Google will often provide links to “related searches” at the top or bottom of their results page and they are also working on Google Suggest, a feature similar to Search Assist. Live Search has “related searches” on their results page and Ask.com has “search suggestions” during the entering of the query as well as “narrow your search” suggestions on the results page.

Nevertheless, these features and others like them presuppose a certain level of knowledge on the part of the user and require user time and effort to explicitly refine the query. Even experienced users with the aid of query suggestions might not be able to find the exact, magical combination of words and operators to accurately express intent, either because the vocabulary is unknown or the objective is imprecise.

As such, current search technologies do not yet do enough to assist users during the search process. The search results page, with its results, maps, photos, query suggestions and other offerings, is treated as a static creation presented to the end user as a final product. Once delivered, the onus is upon the user to dig through all of the pages of results and either find the information sought or reformulate the query.

It is precisely for this reason that Surf Canyon has developed Discovery for Search™ technology to work with searchers to help them find what they need. There is great benefit to the user in allowing the search results page to be dynamic, altering and refining the presentation of information as a search is being conducted. What might have been the most efficient ranking of millions of matched documents at the moment the user pressed the ‘search’ button is most likely not the case after the user’s very next action. Personalizing the search results in real time, on the fly, not only enables people to find information faster and with less effort, diminishing ‘search engine fatigue,’ but actually encourages people along the way by actively assisting with their efforts.