Boolean Searching in freeWAIS-sf

Table of Contents

1. Introduction and Credits
2. Summary
3. Upper/Lowercase for Search Terms & Boolean Operators
   1. Search terms
   2. Search terms and Boolean operators
4. Hierarchy of Boolean Operators
5. Truncation of Search Terms
6. Stemming

1. Introduction and Credits

Boolean searching capabilities were not part of the original WAIS release, but were soon introduced into WAIS indexing packages due to user demand. All WAIS packages but the earliest versions support AND, OR, NOT, right-hand truncation using the asterisk (*), and the use of parentheses in search statements. How these are implemented differ among the various WAIS versions.
Here we report about freeWAIS-sf. More information about this package can be obtained by connecting to the University of Dortmund where the package was developed.

This document is a free adaptation from four different documents prepared by:
Natalie Oakes Sturr
Systems Librarian, Penfield Library,
SUNY Oswego, Oswego, NY 13126
E-mail: sturr@oswego.edu

By navigating the Web, you can see these four original documents:

• Boolean Searching in WAIS Databases
• Hierarchy of Boolean Operators
• Truncation & Stemming
• Use of Upper or Lowercase for Search Terms and Boolean Operators
  

2. Summary

1. Search Terms: only lowercase terms search works consistently
2. Boolean Operators:
   • Both upper and lowercase operators work consistently
   • Evaluated in the following order:
     1. Operations in parentheses
     2. NOT
     3. AND
     4. OR
3. Truncation:
   • Works correctly and consistently when Porter stemming is turned off
   • When Porter stemming is turned on, search results can be misleading
4. Stemming: Porter stemming available as an option

3. Upper/Lowercase for Search Terms & Boolean Operators

The use of upper or lowercase for either search terms or Boolean operators can affect the results of WAIS searches.

a. Search terms

freeWAIS-sf processes lowercase search terms correctly. However it returns inconsistent search results when search statements include uppercase search terms.
In fact freeWAIS-sf returns 0 (zero) documents when uppercase search terms are combined with either the AND or NOT operators.

b. Search terms and Boolean operators

A major challenge when searching a WAIS database is whether to use uppercase or lowercase Boolean operators. freeWAIS-sf correctly handle both upper and lowercase operators.

4. Hierarchy of Boolean Operators

Boolean operators and the use of parentheses are supported in freeWAIS-sf 2.x. Since operations in parentheses are evaluated first, it is advisable to use parentheses in search statements to assure consistent results.

In freeWAIS-sf, the hierarchy of Boolean operators is:

1. Operations in parentheses
2. NOT
3. AND
4. OR

NOT is evaluated before AND, which is evaluated before OR. Operations in parentheses are evaluated first of all.

Examples:

5. Truncation of Search Terms

Right-hand truncation, denoted with an asterisk (*), is listed as a feature of freeWAIS-sf.
This package also provides stemming, similar to automatic right-hand truncation, as an option (see later).

freeWAIS-sf, right-hand truncation produces correct results as long as stemming is NOT turned on.

• Play* retrieves documents containing the words play, plays, player, playful, and playground
• Plain* retrieves documents containing the words plain, plains, plainly, and plaintiff.
• Set*, retrieves documents containing the words set, settle, settled, settler and settlers.

6. Stemming

Two types of stemming are available in various versions of WAIS: Porter and plural.
freeWAIS-sf offers only Porter stemming.

Porter stemming attempts to identify and index the word stem. If a word and its stem are different, only the word stem is indexed. Thus, it appears to the user that search terms are automatically truncated. For example, both physics and physical stem to physic.

Plural stemming attempts to identify and index the singular form of a search term. When searching either the singular or plural form of a term, both are retrieved:

Although Porter stemming is based on a computer algorithm, the English language is not! This can cause search statements to return unexpected results. For example, play is stemmed to plai. These inconsistencies are compounded when combined with right-hand truncation.

Data used to determine how truncation and stemming are implemented in freeWAIS-sf are presented below. They refer to freeWAIS-sf 1.1 that is the more recent version tested.
All searches were performed on a small database constructed for the purpose of testing truncation. The database has 10 records (one word per record) and was indexed using the paragraph format (-t para). waissearch was used to search the databases. The words listed are those retrieved with each search statement. The database contains the following words:

• play, plays, player, playful, playground, plain, plains, plainly, plaintiff, plait

© 1996-2003 BioPD - University of Padova (Italy) - Author: Leopoldo Saggin - Last version: January 28, 2003
Best efforts were made to provide correct information, however this document may contain technical inaccuracies and/or typographical errors.
The author declares that this material is provided "as is" without any warranty even in the implied warranty of merchantability or fitness for a particular purpose.
All trademarks cited inside this document are property of their respective owners