First Do No Harm - The Education of Dr.Search, a Search Results Expert System

FIrstSearch is a boolean-based search system in which databases, each with millions of records, are indexed in many different ways. Although there are many indexes, almost all searches use a few indexes: keyword, author, and title.

About 25% of searches in FirstSearch produce no records.

How many find too many records?

In usability tests, we often observed untrained users being perfectly happy skimming through the first few records of a result set of hundreds of thousands of records. This may be an effect of searching on the Web, where thousands of matches are expected. Trained searchers (e.g., librarians) would usually comment on the number of matching records, which would usually be lower because they have been trained to use more terms and to apply limits (e.g., language and material type) before a search.

Based on experiences with the HCI Bibliography.

Given these concerns about no matches and too many matches, we set out to build a simple expert system to use properties of the queries and results to suggest ways to improving a query to improve results. Dr.Search would begin by diagnosing the problem, based on a list of known problem conditions for no matches:

and for too many matches (defined as more than a user would look at): Sometimes, the diagnosis and prescription for no matches is the oppostite to that for too many matches. Sometimes, it is possible that a problem for no matches (e.g., term not in index) also applies to searches with too many matches (e.g., ORed terms).

Prescriptions