Title: The FirstSearch User Interface Architecture
Author: Gary Perlman, OCLC, perlman@acm.org, www.acm.org/~perlman/
Location: http://www.acm.org/~perlman/fsmulti.html
Modified:
Contents:
Abstract: how to design the UI architecture so that developers, writers (including translators), graphic and UI designers, and even marketing can work on the same functional system at the same time without affecting other. I'd cover the issues of adapting to multiple hardware, OS and browser platforms, multilingual and visually impaired user access, time permitting.
Disclaimer: These notes represent my views, not necessarily those of OCLC. Any features described may or may not appear in software or services provided by OCLC.

[1] Problem: Second System Effect

The OCLC FirstSearch® service is a Web-based bibliographic and full text retrieval system (at NewFirstSearch.oclc.org). FirstSearch delivers to libraries and their patrons over 80 databases, each with about 10-30 indexes (e.g., keyword, author, title, subject, date, ...), to access a combined total of over 200 million records and millions of full text articles.

Originally delivered as a text-based system in 1991, the Web-based version of FirstSearch, released in 1996, steadily gained popularity over the text version. The system was built on the OCLC SiteSearch® Z39.50 Web-server, which maintained persistent information about a user's search/retrieval session. The original SiteSearch was developed in C and contained several embedded proprietary languages developed by the OCLC Office of Research. The desire to add new features and to be able to adapt more easily to user needs motivated the development of a new version of FirstSearch, built on the new Java-based version of SiteSearch. The new FirstSearch would have many new features:

The difficulty of adding desired features in the old system, and the relative liberation of a dynamic Java environment, led to what Brooks (1975/1995, Ch.5) would call a second-system effect of every feature for which there was a desire.

There was a high company priority to integrate the functionality of existing systems and retire those systems:

Over a year was spent developing the detailed requirements, during which time design options were unclear because we were in areas of little experience:

There were several general requirements for the user interface of new FirstSearch. Although we had devoted considerable effort to the detailed requirements, the cross-platform, text-only, multilingual, and accessible requirements were expressed with little more than a sentence each (i.e., less than above). Also, there was a one-line requirement for a context-sensitive help system (which would meet all the above general requirements).

[1.1] Multi-Platform: Run on Everything

The system would work on all "current" browsers. Initially, this was officially the 4.x versions of Initially, we wanted to require JavaScript, and even CSS (Cascading Style Sheets), but Navigator 4.0 had limited support for features we could provide by other means. It did not take long to realize that many sites (e.g. libraries with hundreds of old machines) would not be willing or able to upgrade their browsers, so we committed to support the 3.x versions of Navigator and Explorer, although with some compromises due to limited functionality (e.g., MSIE 3). So we committed to support: Additionally, we committed to support different screen sizes: across different hardware/operating system: and to test 256-color screens and for grayscale contrast.

[1.2] Multilingual Interface: Run in Many Languages

We planned to translate the interface and online help into three languages initially: We hoped this would be easier than our multilingual effort in the old Web-based version (Hysell & Perlman, 1999) which was not developed with internationalization in mind.

[1.3] Levels of Users: Usable and Useful to Everyone

We planned to support three search modes:

[1.4] Text-only Version: Run on any Hardware

In addition to the graphical UI browsers, we wanted to support a version that could run in a telnet window because for a small number of high-frequency users, telnet is the only access. We chose to create a version of the interface that would work reasonably well with Lynx, a text-based HTML browser. This would replace the TTY version of FirstSearch.

[1.5] Universally Accessible: Run for Everyone

There was a requirement to be "ADA compliant" (Americans with Dissabilities Act) although with no knowledge of what that entailed, not even that, at the time, there were no defined standards. Initially, we thought that the text-only Lynx interface would serve that purpose, but later found that text-screen-readers serve some users, while specially adapted graphical browsers serve others.

[1.6] Group Coordination Issues

Different groups of people were given primary responsibility for the different dimensions of the user interface:

This is an over-simplification because many groups contributed to many dimensions. Still, few individuals think about all the above concerns when working on a specific task, so it was a user interface coordination goal to make sure that, for example,:

[2] Approach: Cross-Products of Partitioned Information to Defer Decisions

My highest priority goal in developing the user interface architecture for FirstSearch was to be able to adapt the interface to inevitable requests for changes (due to as yet unknown usability, performance, functional, etc. considerations). Because all the development was in an environment with so many unknowns, the architecture had to be incrementally scaleable starting with a simple model of the user interface, but able to expand as we understood more. Previous experience (Perlman, 1985) suggested that partitioning the system into orthogonal sets of information and building the system by forming the cross-product of those sets would allow incremental elaboration and optimal redesign. The method is similar to word-processing mail merge except that instead of inserting address information into letter/label templates, attributes of functions are inserted into platform-specific templates for Web pages.

The partitioning chosen included:

[2.1] Functional Partition: The FSPage Model

The first step in developing the UI architecture for FirstSearch was to apply information design to identify some parts (attributes) of pages. A page is an object with the information used to construct what a user observes and does on a single Web page. Initially, the specific pages in the system were unimportant because we knew that new pages would be added and some existing pages would be merged with others or deleted. A canonical sequence of pages in a search/retrieval session is database-selection, search, results. Similar to pages, the specific attributes of pages were unimportant because it should be easy to add/delete/change attributes. Each page in FirstSearch has:

Individual pages could have any of about 10 other specialized attributes (e.g., the name of the form element in the panel where the cursor should be focused when a form is displayed). Attributes can contain constant text and any number of entities (SiteSearch constants, variables, and Java method calls), so they are highly dynamic.

To make these pages platform-independent and language-independent, we extracted the platform-dependent parts into a style file and the language-dependent parts into a language file. We replaced what was extracted with entities (basically variables) defined in configuration files (which we call INI files). The resulting platform- and language-independent page definitions were left in page.ini, a configuration file with a section for each page. For example, the definition for the expert search page looks something like (some details are hidden here for a more focused presentation):

   [expert]
   pagename     = expert
   pagetitle    = &Lang.pagetitle.expert;
   pagelabel    = &Lang.pagelabel.expert;
   tips         = &Lang.tips.expert;
   status       = &Lang.status.expert;
   controls     = &Style.thesaurus.gadget;
   action       = /&namespace;/FSQUERY:searchtype=expert
   term         = termexpert
   index        = indexexpert
   focus        = termexpert
   panel        = 
      &Style.dialog.begin;
         &Pages.basic.submit;
         &Pages.expert.searchbox;
         &Pages.expert.index;
         &Pages.advanced.limits;
         &Pages.advanced.options;
         &Pages.basic.submit;
      &Style.dialog.end;
   searchbox    = 
      &Style.dialog.rowbegin;
         &Style.font.labelbegin;
            <label for=termexpert>
               &Lang.label.find;
            </label>
         &Style.font.labelend;
         &Style.dialog.elementbegin;
            <textarea name=termexpert id=termexpert>
               &termexpert;
            </textarea>
         &Style.dialog.elementend;
      &Style.dialog.rowend;
Many of the attributes are references to language entities defined in language files (one for each language). Some of the attributes are style entities, used to mark the beginning and end of structurally meaningful parts. Some of the entities include references to other parts of pages, so that definitions can be modular and reused (e.g., all search screens use the Basic submit buttons). Special purpose attributes indicate the names of terms and indexes used on search screens, and where to focus the cursor (if JavaScript is available).
   [record]
   pagename     = record
   pagetitle    = &Lang.pagetitle.record;
   pagelabel    = &Lang.pagelabel.record;
   tips         = &Lang.tips.record;
   status       = &Lang.status.record;
   controls     = 
      &Style.thesaurus.gadget;
      &Style.ill.gadget;
      &Style.holdings.gadget;
      &Style.email.gadget;
      &Style.print.gadget;
   action       = /&namespace;/FSFETCH:fetchtype=record
   panel        = 
      &Style.dialog.begin;
         &Style.dbsuggest.gadget;
         &Style.navigate.gadget;
         &Style.record.gadget;
         &Style.navigate.gadget;
      &Style.dialog.end;

[2.2] Practical Partitioning

With the page object defined, the most difficult aspect of partitioning the language and style information was locating the information to be partitioned and then doing it consistently. We found that it was nearly impossible to explain to developers why and how to keep this information separate, perhaps because it required an understanding of translation, cross-platform development, accessibility issues, and general usability considerations. Another problem with having developers create language-independent and platform-independent code was that the language and the formats were being developed as the system was being built.

Our approach was to have developers create screens using untamed English and HTML and then partition the information for them. To internationalize the design, we moved language strings into a language file, replacing them with language entities to create language-independent HTML; later, the language-specific values would be inserted into the HTML by substituting language entity values in the user's language. Platform-specific parts were extracted and replaced with style entities for runtime substitution based on the user's platform and preferences. During the process, choice of language and interface design could be reviewed, and later, design decisions were centralized so that decisions could be changed.

Internationalization/
Localization
Style Abstraction/
Presentation
replace
language
strings
Language-
independent
HTML file
Language
file
language
entity
substitution
extract
formats/
styles
Platform-
independent
HTML file
Style
file
style
entity
substitution

[2.3] Platform-Dependent Partition

The FSPage object contains platform-dependent parts, parts that will display differently on different platforms, and which will require different HTML. There are many factors that may affect how the interface design is presented to users, including: For example, if JavaScript is available, a Help window can pop up, be sized to the screen dimensions, and show shortcut keys that are based on the operating system.

There are many methods to adapt to different displays, ranging from using lowest-common denominator features, to unique sub-sites for different displays, to dynamically-customized displays. Given the number of platforms planned for FirstSearch, and the many differences among these platforms, dynamic generation of HTML was an obvious choice.

There are many ways to implement dynamic generation of HTML.

The one chosen for FirstSearch was designed to abstractly represent the structure of displays separate from the final rendering. For example, an error message might be marked up as:
   &ErrorBegin;
      Something bad happened
   &ErrorEnd;
or a line in a search form might be marked up as:
   &SearchFormBegin;
      ...
      &SearchLineBegin;
         &LabelBegin;
            Find:
         &LabelEnd;
         &FormElementBegin;
            <input type=text name=terms>
         &FormElementEnd;
      &SearchLineEnd;
      ...
   &SearchFormEnd;
These examples have been simplified to better explain the methods used in FirstSearch; the real versions have many gory details. SiteSearch, on which FirstSearch is built, allows the definition of entities that are substituted into the outgoing HTML. So by changing the definition of these structural entities, we can change the HTML that will be generated. For example: The fine granularity of control and the likelihood of editorial changes made it undesirable to code these changes in Java. Instead, a declarative method of specifying custom values was adopted.
  1. When a session starts, all the potential customizing variables, including browser attributes, are stored in about 30 entities.
  2. Default entities are read from an initializing configuration file (called an INI file).
  3. Customizing entities are set based on the values of other entities. These are read from conditional INI-file sections.

Ordinary INI files contain named sections and entity definitions in those sections.

   [styles]
   ErrorBegin = <font color=red size=5><b>
   ErrorEnd   = </b></font>
Conditional sections are named by entities and conditional values. For example, the browser entity may be equal to Mozilla, MSIE, Lynx or some other value. Conditional styles could be defined for each browser value:
   [styles]
   section*   = browser

   [browser=Lynx]
   ErrorBegin = <hl><b>
   ErrorEnd   = </b></hl>

   [browser]
   ErrorBegin = <font color=red size=5><b>
   ErrorEnd   = </b></font>
The reference to section* causes the system to read the conditional section named browser. If browser is Lynx, the section called [browser=Lynx] is used. If no value of browser has been set, the default [browser] section is used. This can be further elaborated based on sets of conditional sections. For example, if we wanted the error message font size to depend on the screen size:
   [styles]
   section*   = browser
   section*   = screensize

   [browser=Lynx]
   ErrorBegin = <hl><b>
   ErrorEnd   = </b></hl>

   [browser]
   ErrorBegin = <font color=red size=&ErrorSize;><b>
   ErrorEnd   = </b></font>

   [screensize=large]
   ErrorSize  = 5

   [screensize=medium]
   ErrorSize  = 4

   [screensize]
   ErrorSize  = 3
Once conditional sections are set up, it is easy to add conditional entities. A major advantage of setting these in INI files is that the changes can be viewed while the system is running. The INI files can be re-read and entities re-set without changing any code. Another advantage is that the all the peculiarities of particular platforms are specified together. For example, the entire color scheme for MSIE 3 is different than for the rest of the system because that browser version does not support changing the color of text if it is in a hot link. Another example is that only MSIE 4+ handles greek entities like &alpha;; for browsers that do not display them properly, we remove the entity delimiters (i.e., show "alpha" instead of &alpha;).

[2.4] Language Partition: Internationalization and Localization

Internationalization in FirstSearch is accomplished by moving all language-specific terms into INI files, and by replacing those terms by entities that refer to the section and entity name in that section. In FirstSearch language files, sections are used to distinguish how or where some text will be used. For example, all diagnostic message are stored in a section called msg:
   [msg]
   bad       = Something bad happened
   nohits    = Your search matched no records
   nojs      = Your browser does not support JavaScript
Entities in language files are accessed by naming the entity holding all the sections (Lang), the section (msg), and the variable name (bad) like this: &Lang.msg.bad;. So the patform-independent, language-independent error message above becomes:
   &ErrorBegin;
      &Lang.msg.bad;
   &ErrorEnd;
When a user chooses a different language, entities in a different language INI file are associated with their session.

Structure: Referring back to the FSPage object, pagetitle, tips, and status are all sections in the language file. Each section contains variables for each page defining the page title, on-screen help tips, and status. This proximity makes it easier to make the text for different screens consistent, both in English and when translating. It requires, however, that developers place page attributes in different sections of different INI files. To make it easier for the user interface and database groups to work together, we separated the user interface language INI file from a language file for database-specific terminology (which accounted for about two-thirds of the language used in the system). This reduced contention while both files went through hundreds of revisions.

English as the Second Language: Although the development was in English, with English strings being moved into the English language files for later translation, there was an initial translation step that took almost as long as the translation into Spanish and French. The initial language was a dialect of English used by librarians and developers of systems for searching library materials, call it Jargonese. Some of the terminology was inappropriate for naive users, all the more common because of the advent of the Web. So internally, a screen might be called "history", but to the users, it would be known as "Previous Searches", and we would actually display &Lang.pagetitle.history; so that if the name changed, the change would be propagated throughout the entire system, including documentation.

Embedded HTML: There was some difficulty of translating text with embedded entities and HTML. The English text would include entities to refer to values defined elsewhere (e.g., page titles, the current year), and entities for formatting (e.g., highlight parts of the status message). Our translators had to be aware of these and sometimes need to change the order to have correct syntax in the target language. At the same time, they had to be careful to avoid breaking the HTML and/or entities (in some cases, entities like &year; themselves were translated).

Finding Entities and Previewing Translation: To help translators and documentation writers determine where an entity was defined, we created an entity language in which the value of an entity was the section and variable name where it was defined (e.g., &Lang.pagetitle.history; would be displayed as pagetitle~history). Then they would know that the string displayed in English as "Previous Searches" was the history variable in the pagetitle section. To help translators see their translations, we provided a facility for dynamic reloading of entity values on the current screen so they would not need to start a new session. This was also used by marketing to preview new English terminology to replace Jargonese.

[2.5] Template-Based Page Generation

The first prototype systems were based on ad hoc format flat-file databases accessed with perl scripts to generate HTML files. Eventually, the information about pages migrated into semi-structured INI files, and HTML files evolved into dynamically-generated HTML. The pages and attributes evolved over time, gradually increasing in complexity, and keeping them in an easily editable format was a positive feature.

HTML pages are generated in FirstSearch by inserting page-specific entities into templates. There are templates for the graphical-browser-based version, the text-only Lynx-based version, a printable (cleaned up) version, etc. Templates can (and perhaps should) start as simple renderings of some attributes, but are scaleable in that they can be augmented easily. They can also accommodate major changes. Initial versions of the FirstSearch interface were framed, but because of transaction costs, we decided to evaluate an unframed version. Creating an unframed version of the whole interface took about an hour, and the results motivated us to change to an unframed interface. This change was made by one person, changing one set of templates into a single template, in less than a day. At various times, we have been able to design, create, and view completely different interface designs that were actual running systems.

Here is a simplified template for the Lynx interface, which is much more vertical than the graphical version. Note that most page attributes have been assigned to entities (e.g., the pagetitle is in &FSpagetitle;).

   <html lang="&language;" pagename="&FSpagename;">
   <head>
      <title>&FSpagetitle;</title>
   </head>
   <body bgcolor="#FFFFFF" &linkcolors;>
      <h1>&FSpagetitle;</h1>
      &FSpagetips;
      &FSpagestatus;
      <form name="main" method="POST" action="&FSpageaction;">
         &FSpagepanel;
         &FSpagecontrols;
         &StyleTable.FSMenu.gadget;
      </form>
   </body>
   </html>

This Lynx version is one of many possible renderings of the parts of pages. One advantage of the framed version of the interface was that the main frame contained all and only the information that users would want to print; when the framing was removed, it took about an hour to create a template for a printer-friendly format that did not show menus and controls.

Templates are easy to evolve. To move the controls for all pages, move one line. To duplicate the controls above and below the main panel, copy one line. Many changes are unexpected, so the flexibility of being able to make global changes is highly desirable. One developer wanted to place an entity value on every page in the system; it was a one-line change. Quality assurance wanted to add specially formatted comments to delimit logical sections of the screens (to help highlight differences in regression scripts); the change took less time than it took to write this paragraph.

Because templates can be defined hierarchically, they can share reusable parts. This can minimize the cost of new versions of templates, say, for a version that takes full advantage of cascading style sheets.

[2.6] Levels of Users

FirstSearch is designed for different levels of users with three search levels: basic, advanced, and expert. These differ in the number of search boxes, number of indexes offered, number of limits shown, and the type of help offered.
Adapting to User Level
Basic Advanced Expert
Search box 1 small 3 small 3 large
Indexes 3 10-15 20-30
Limits full-text
library
all all
Help simple examples examples of more features reference material on-screen

[2.7] Text-Only Version Issues

As discussed earlier, the text-only version for the Lynx browser was built with a customized (more vertical) HTML page template. Beyond that, most of the HTML constructs displayed adequately well, even tables, because Lynx recognizes that a table row tag starts a new line. Where the format needed minor changes, we added Lynx-specific entities for simple formatting (space, vertical bar, break, horizontal line, paragraph, comment, button). These were empty strings for the non-Lynx version, so the same tables used to format a form or data display could be used by Lynx. So, if a part of a screen did not loo right in the Lynx version, it could be tweaked with Lynx-specific entities without affecting the graphical interface.

[2.8] Accessibility Issues

Initially, we thought that the text-only Lynx version would be the best platform for a screen-reader for a sight-impaired user. After interviewing one of our staff, who is blind and uses Web-aware HTML-reading software, we broadened our approach to include all browsers.

Because the HTML for formatting the display is localized in style files, most changes to adapt to the WAI Guidelines could be added centrally. Microsoft's Internet Explorer 4+ (MSIE4+) provides substantial support for accessibility-oriented tags, including some visible features useful for all users:

[2.9] Coordination Issues

The partitioning of the user interface allowed non-developers to make changes to the developing system without involving developers.

easy to edit ini files non-pgmr input: prasse UI, Rumbaugh, lang

disappointment central: 200 MHz 64 Meg Win 95 256 color 640x480 Explorer 3.0 Navigator 3.0

Partitioning also took UI decisions out of the hands of developers.

You don't get to choose how it gets displayed. show partitioning for:

problems of enforcement on programmers
	direct English + HTML
	commandments + checking tools
problems of indirection
	include entities referring to images
problems of translating HTML with embedded entities
	checkhtml and checkini

[3] Observations and Conclusions

mail merge, report generation - very scaleable, incremental
  • multiple platform adaptation (hardware, browser, OS)
  • impaired user access More dimensions: