How to Personalize the Web

ABSTRACT

Keywords

© Copyright ACM 1997

Download

INTRODUCTION

URLs make all information equally available to a user. Although the prospect of having 100 million pages close at hand is staggering, hypertext simplifies access to this huge information repository. Web authors arrange information into directed graphs that link related pages together. In this way, hypertext reduces the user's decision space from a choice among millions of URLs to a choice among tens of hyperlinks on a particular page. Thus, hypertext transforms a large number of pages that are logically proximate into an accessibility space in which some pages are practically closer than others. Put differently, the web combines universal proximity (URLs) with tractable accessibility (hypertext). Given this view, it seems that the accessibility of information is completely impersonal. Every user sees the same web connected together in the same way. Web authors build the structure of the web according to their interests, concerns, and tastes, and then all users rely on that same structure.

To overcome the impersonal feel of the web, browser software generally provides "Hot Lists" or "Bookmarks" which allow users to record often-used URLs for quick access. These lists of URLs begin to make the web more personal, pulling certain web pages closer to individual users, and thus enabling easier access. They add a personal structure to the impersonal structure of the web. Of course, some users go further and author their own home pages which can also provide launching points for web access. In addition, browsers can generally be configured to automatically start up with a particular page loaded. Creating hot lists, home pages, and setting start pages are steps individual users can take toward personalizing the web.

The impersonal organization of information on the web has some negative consequences for users. In a recent survey, Pitkow and Kehoe found the main problems people report when using the web are (a) slow network or connection speeds; (b) not being able to find particular pages, even after they have been found before; (c) not being able to manage or organize retrieved information; and (d) not being able to visualize where they have been [16]. The first problem (beyond technical issues of modem speed and so on) results from the perception that all information is equally proximate. Because network delays routinely slow user access to particular information sources, the perceived logical proximity of information does not correspond to actual time needed to obtain information. The second and third problems result from users having to manually manage their personal information. The fourth problem results from the impersonal organization of the web . We believe all these problems can be solved by providing automatic mechanisms that tailor the web for individual users.

In this paper, we describe a fully implemented system, Web Browser Intelligence (WBI, pronounced "WEB-ee"), that automatically personalizes the web using agent technology [9]. WBI provides an architecture which taps into the communication stream between a user's web browser and the web itself. Small programs, or agents, attach themselves to this stream, observe the data flowing along the stream, and alter the data as it flows past. These agents can learn about the user, influence what the user sees by marking-up pages before passing them on, and provide entirely new functions to the user through the web browser. In what follows, we describe the WBI architecture, the style of interaction between the user and the web that we advocate, the specific functionality we have provided, and the future of WBI.

WBI ARCHITECTURE

Figure 1. WBI stands between a user's web browser and the web, monitoring, editing, and generating documents.

Four Kinds of Agents

An editor agent intercepts the communication stream, receiving information and then delivering a modified version of it. This edited data stream can be created from the incoming stream and other available information, such as a user's past history, system status, or any information obtained from the web. Editors can connect to either the request part of the stream or to the response part. Request editors can transform a document request from the browser into another request. One simple application of a request editor is to fetch documents that WBI knows have been moved to a different URL. Response editors can modify the actual content of documents that users see. For example, response editors are used to add extra buttons or additional links to a web page, or to change colors and backgrounds. Alternatively, editors can choose to simply pass their input to their output, effecting no change in the stream. Any number of editor agents may alter a transaction.

A generator is an agent that converts a request into a response. Every transaction activates exactly one generator. The default generator is simply a web communication program that passes the request on to the appropriate web server, retrieves the response and passes it back to the browser (i.e., simply performing the job of an HTTP proxy). Other generators are used to provide new WBI functions, intercepting requests from the browser and generating documents for the user to see in response. For example, a generator could provide a web-based way to view the state of the printer queues on a user's workstation. When the generator is activated, it produces an HTML document which describes the printer queues. Likewise, a generator can handle requests with special communication protocols, such as communicating with a firewall.

Finally, an autonomous agent is executed based on a trigger independent of the communication stream, such as a time interval. An autonomous agent simply performs its task and then terminates. For example, an autonomous agent might perform housekeeping actions, digest recorded transactions to develop user models, or explore the web for new information.

How Agents are Triggered

How Agents Work Together

Figure 2. A collection of agents that (a) monitors web access, (b) clusters retrieved pages and extracts keywords to identify clusters, (c) searches for new pages containing the keywords, (d) edits retrieved pages that are part of known clusters to point to alert user of related pages, and (e) generates pages containing lists of related URLs.

Performance Requirements

Fast performance is a key requirement. Because WBI stands in the middle of every web transaction, if it noticeably slows down communications, people will not use it. Typical web transactions take between 0.5 and 20 seconds. We designed WBI and its associated agents to add no more than 1 second of overhead to these transactions. To perform at this level, WBI was built in a highly multi-threaded fashion so that no byte that ought to be delivered to the browser gets stuck somewhere in the agent network. Generators and editors control what is delivered to the browser so they are run at high priority. Monitors are run at a low priority, as they do not affect what a user sees.

Because of performance considerations editor agents must always pass data along the stream as quickly as possible. This implies that editors should not depend on the contents of whole documents to make their modifications, but should depend only on small, localized sections of documents. For example, an editor that annotates the top of a web page only if the page contains a certain word or phrase or link might search the entire contents of page before it passes the page along. Given large web pages that contain hundreds of kilobytes of text, the delay incurred by such an editor could be substantial, lasting tens of seconds. One way to put information at the top of a page that depends on information later in the page is to use image embedding. An editor can insert an embedded image that points to a generator that produces the appropriate image. This trick enables the editor to pass the stream along while processing the data, effectively delaying the decision about whether to annotate until enough data has been seen. Of course, this only works for graphical browsers that incrementally display pages as they load.

Related Work

A second difference between WBI and OreO is that OreO does not provide any generator function. OreO expects an independent HTTP proxy server at the end of the chain that will actually retrieve a web document. Thus, OreO cannot serve dynamic documents itself, but must rely on a separate server. As mentioned, WBI incorporates generator agents which can produce documents in response to HTTP requests. The example applications we have built often required generator agents for producing search forms and results, dynamic images, configuration pages, and so on. Thus, generators are integral to agent-user interaction.

Third, OreO provides no special support for monitor functions. Although monitors can be written as editors that simply pass their data along while storing what they need, many such monitors chained along the HTTP stream will inevitably slow the movement data along the stream. Because WBI distinguishes monitors from editors, monitors can run at a lower process priority, maintaining the throughput from web to user.

Finally, an OreO-based proxy acts as a single editor for transforming HTTP requests and responses. To create a set of agents using OreO, a set of independent proxies must be created. Using WBI, a single proxy which contains a series of separate transformations can be used instead. The processor overhead of creating a series of socket connections to chain OreO proxies can be substantial.

Before describing details of our agents, we next discuss several human-computer interaction guidelines that have influenced the design of WBI.

USERS SEE THE WEB, NOT WBI

1. WBI must maintain the standard user interface of the web---the user communicates principally and directly with the web rather than with agents. This means that all of WBI's output and all user input must be done via HTTP and HTML. Users click on links or type in URLs. WBI's agents display their results on ordinary web pages.
2. Agents act both proactively and reactively. When they have nothing to say, agents must remain silent and hidden. When an opportunity for augmentation arises, however, proactive assistance and advice can be offered. A reactive response results from a direct request for an agent's assistance. For example, when a user requests http://wbi/printq to display workstation printer queues, the printer queue generator agent reacts by reporting on the queue state.
3. Agents cannot take control from the user. Agents can offer advice, but the user remains in charge. This reduces the problem of user-agent trust which can hamper agent systems.
4. Agent activity must be peripheral (not central) to the task at hand so the user can easily ignore errant agent activity.
5. Agents build user models which generalize, recognize, and classify user activity to predict future activity. For example, in recognizing that a user reads a particular web page every morning, one agent has built a model of the user that enables another to prefetch that page or to notify the user if he or she forgets to read that page one day.

We now turn to details of WBI's agents.

SIMPLE AGENTS SOLVE COMMON PROBLEMS

We describe each of WBI's basic functions in turn, and then discuss more complicated functions.

Personal History

Figure 3. This screenshot shows part of WBI's personal history function. The form near the top of the page can be used to limit the displayed URLs to those that contain a specific word or phrase.

Shortcuts

Figure 4. This screenshot shows the top of a web page on which one editor agent has added a toolbar for accessing WBI's main functions (History, Path, Watch, and Help), and another editor has added shortcut links. The hyperlinks under the arrow to the left of the toolbar are URLs that this user has often visited shortly after visiting the current URL, as sketched in the text.

Page Watching

Traffic Lights

All these functions personalize the web for individual users. Frequently or previously visited pages are more easily accessible than the web at-large, and web performance (network speed) is determined in real-time and displayed in a useful way. The user's normal interaction with the web is maintained but with new, unobtrusive functionality.

COMPLEX AGENTS BUILD USER MODELS

We are currently working on a set of agents that allow users to annotate and group web browsing activity into useful collections with minimal extra effort (cf. [5]). These collections capture some of the thought and understanding that went into a particular searching or browsing session [7]. They can then be referred to later so that the abstraction underlying web history can be raised from single pages to reasoned sessions [4]. We are also looking toward sharing histories among users so that expertise in a subject area can be transferred.

Finally, we have implemented agents that make a user's workstation desktop available through the web browser; these functions include launching applications, querying system state, managing files, and so on. Through networking, such agents can enable sharing work via the web.

HAS WBI SUCCEEDED?

At first, we were surprised the survey also indicated that no single WBI function was most popular, and that most WBI user's clearly had a favorite function. Each individual web user seemed to find a single function to resonate with his or her standard way of using the web; for instance, whereas one person preferred to query the personal history using a keyword search, another preferred to scan the most recently visited sites on the personal history page. Yet the suggestion that individual users stick with a single method accords with data we recently collected by watching experienced web user search for specific information [11]. In that study, we found that individuals rely on standard routines or patterns of behavior. These behavior patterns are so strong that people not only plan and execute their searches according to their patterns, but they also recall their behavior as fitting their standard pattern even when it did not.

Further study will be required to understand how users are affected by the new functions WBI provides. The results of such studies will help us design the best functions. The remaining challenge is to unobtrusively encourage users who seem set in their ways to broaden their range of search strategies. The simple fact that WBI has been downloaded more than 25,000 times in six months suggests that people are actively looking for tools to enhance web use.

FUTURE DIRECTIONS: SCENARIOS & MODELS

User modeling will become increasingly important as our agents become more sophisticated. Modeling consists primarily of taking information from the communication stream and recognizing, generalizing, and classifying that information to produce useful results for other agents. This activity requires: (a) inferring what the user is thinking based on what the user is doing on the computer, and (b) determining what the computer should do based on what the user is (presumably) thinking. We have built a few simple examples of these transformations, such as the agent that derives clusters and keywords from the personal history and then finds documents containing those keywords. This agent tries to determine what is of interest to the user---under the assumption that keywords identify interesting documents [1]---and then translates those keywords into undiscovered documents using a search service. This is a relatively simple example of user modeling, but more sophisticated techniques will be required as user-interaction scenarios become more complex.

CONCLUSION

As a final note, we believe the ideas behind WBI are not limited to the world-wide web but apply equally to any communication stream between a user and a computational system. Other candidates for study include workstation window environments, conventional information retrieval and database applications, online public-access catalogs, electronic mail, network news, command shells, word processors, and other applications.

REFERENCES