CHI 97 Electronic Publications: Late-Breaking/Short Talks
LICAI+: A Comprehension-Based Model of Learning for Display-Based Human-Computer Interaction
Muneo Kitajima
National Institute of Bioscience and Human-Technology
1-1 Higashi Tsukuba Ibaraki 305, JAPAN
Tel: +81 (298) 54-6730
E-mail: kitajima@nibh.go.jp
Peter G. Polson
Institute of Cognitive Science
University of Colorado
Boulder, CO 80309-0345, USA
Tel: +1 (303) 492-5622
E-mail: ppolson@psych.colorado.edu
ABSTRACT
This paper describes a model of comprehension-based learning, LICAI+,
an extension to the comprehension-based model of display-based HCI,
LICAI [5], that simulates a user who performs tasks given as instructions.
LICAI+ models users' learning of task performance
by incorporating a process for encoding events
during the task performance.
A simulation of encoding and recalling events is described.
Keywords
cognitive model; learning; display-based human-computer interaction; construction-integration theory.
© 1997 Copyright on this material is held by the authors.
INTRODUCTION
This paper describes a model of how experienced users
of an environment (e.g., Mac OS or Windows 95) learn
to use a novel graphing application like Cricket Graph III.
We assume that the user has an explicit task goal (e.g., Hide Legend),
and that she generates the correct action sequence
by being given hints of the form "Pull-down the Options menu"
or successfully discovering steps by exploration.
After successfully performing the task for the first time,
the user is asked to perform the task again after a delay.
The task goal and the displays generated
by the application interface are the only retrieval cues given to the user.
In a previous paper [5], we described LICAI,
LInked model of Comprehension-based Action planing
and Instruction taking.
It simulates the cognitive processes
involved in comprehending and following hints
and successfully performing steps by exploration.
This paper extends LICAI by incorporating learning processes.
We want to understand the relationship
between users' recall performance
and their representation of the task and the application displays.
Franzke [1] found that both the probabilities of successful exploration
and later recall were determined
by overlap of users task goal descriptions
with the labels of correct menu choices.
THE LICAI+ MODEL
LICAI+ incorporates learning mechanisms
derived from the memory for text [3].
We start by describing LICAI
for comprehending hints and successful exploration
(See [4] and [5] for details).
Then, we describe how LICAI+ encodes successful actions,
and later retrieves them using a task goal
and application displays as retrieval cues.
The Construction-Integration Architecture
LICAI is comprehension-based model of
instruction following and exploration.
The cognitive processes specified in LICAI are implemented
using the construction-integration (C-I) architecture
developed by Kintsch [2]
which has been applied successfully to model cognitive processes
involved in text comprehension [2], word problem solving [2],
and action planning [4].
In the construction phase,
a C-I cycle generates a connectionist network
that represents alternative meanings of a sentence or alternative actions
that can be performed on a given step in a computer-based task,
and the knowledge necessary to select among the alternatives.
The integration phase uses spreading activation
to implement a constraint satisfaction process
that selects a contextually appropriate alternative
consistent with the users' goals.
The nodes in the network are propositions.
Links in the network are established
by common arguments of propositions;
when two nodes share a common argument, they are connected.
The constraint satisfaction process is controlled
by the pattern of interconnections.
Comprehending Hints and Exploration
Kitajima and Polson [5] describe in detail the processes in LICAI
that simulate comprehension of hints like
"Pull-down the Options menu."
This hint specifies actions to be performed on an object on the screen.
LICAI transforms the propositional representation of the hint
into a representation that controls the action planning process,
a do-it goal.
However, even without hints,
the action planning process can discover correct actions by exploration.
A task goal like Hide Legend does not contain any information
about the correct action sequence.
However, the action planning process can discover
the correct action sequence
if labels of screen objects like a correct menu choice
overlap with the task goal.
If the interface to a graphing program had a Hide menu
and Legend was a menu item,
the action planning process would be able to discover
the correct actions utilizing knowledge about the interface
stored in long-term memory.
For example,
the model knows that Hide is a menu label,
that menus can be pointed at, that press-and-hold is a legal action,
and that press-and-hold will pull-down the menu.
Encoding Results of Performing Actions
LICAI+ defines an encoding process
that operates after LICAI's action planning process.
This encoding process comprehends the results of the action
just executed by the model,
generating a memory trace as by-product of the C-I cycle
that comprehends the result of the last action.
When an action causes a significant display change
(i.e., appearance of a pull-down menu, of an application window, etc.),
a specialized comprehension schema is used
for constructing a propositional representation of the event.
The comprehension schema generates a proposition
including the following arguments:
1) the current task-goal,
2) the current do-it goal,
3) the label of the acted-on object, and
4) its current state.
This proposition is incorporated into the network during the construction
phase.
During the integration phase,
the amount of activation received by this proposition is determined
by the pattern of links in the network which is determined by overlapping
labels.
The pattern of activation then determines
the strength of the memory trace of this event.
SIMULATION: ENCODING AND RECALLING
We modeled learning of the steps for the task Hide Legend
using Cricket Graph III.
We assume that a user is an experienced user of the Macintosh OS
who has had no experience with this application.
Rodriguez and Polson [6] have shown
such users cannot perform the first two steps of the task
(Pull-down the Options Menu, Select Show Graph Items...)
without hints.
We gave the simulation hints
that enabled it to perform these two steps successfully.
LICAI+ comprehended the hints and converted them to do-it goals,
followed by execution of the first two steps.
Display changes triggered the encoding processes.
Figure 1 shows the elements in the network
that participated in the encoding process.
Releasing on "Show Graph Items..." caused a dialog box to appear
for showing or hiding elements of the graph like the legend.
Each element had a check box next to an associated label (e.g., Legend).
The graph item described by a label was hidden
by clearing the associated check box and clicking
on the button labeled "OK."
These two steps can be successfully performed
by the model and subjects by exploration.
We simulated doing these steps by exploration
where the original task goal Hide Legend controlled the process.
The resulting encoding of the outcome of an action
has no do-it goal (see Figure 1).
| Step No. | Task Goal | Do-It Goal | Label of Acted-on Object | State of Acted-on Object
|
|---|
| 1 | Hide Legend | select 'Options' | 'Options' | Highlighted
|
| 2 | Hide Legend | select 'Show Graph Items...' | 'Show Graph Items...' | not visible
|
| 3 | Hide Legend | none | 'Legend' | Legend Check Box Cleared
|
| 4 | Hide Legend | none | 'OK' | not visible
|
Figure 1. The elements in the network used for encoding events
during Hide Legend task using Cricket Graph III.
Recalling Memory of Events
Retention of the four steps performed during the learning session
was tested by using the task goal, Hide Legend,
and the initial display as retrieval cues.
Figure 2 shows the activation values of the encoded events
in the recall session.
The most highly activated event representation is
the representation of Step 3, clearing the legend check box.
| Actions Involved in Events | Activation Values
|
|---|
| Pulling-Down "Options" | 0.0275
|
| Releasing "Show Graph Items..." | 0.0330
|
| Clearing Legend Check-Box | 0.1631
|
| Clicking "OK" | 0.0329
|
Figure 2. Retrieval of events from memory cued by the task
description, "Hide Legend" and the initial display.
The memory retrieval process,
like the action planning process is dominated by the overlap
between the goal and labels of objects on the display.
Observe that the model retrieved this step
even though the object to be acted on, the check box,
was not yet on the display.
Additional cognitive processes would be required
for selecting the encoding of the first step of the task,
pulling down the Options menu.
This suggests that performance of the actions
that were hinted in the learning session
can be still difficult to recall consistent with Franzke [1].
To conclude,
the encoding process in LICAI+ generates weak memory traces
for the actions on objects
that do not have labels that overlap with the task goal.
This result suggests that subjects will have difficulty
recalling the steps from memory in a task that required hints
(i.e., cannot be successfully performed by exploration.)
REFERENCES
- Franzke, M.
Turning research into practice: Characteristics of display-based interaction. In Proceedings of CHI '95 (Denver CO,
May 1995), ACM Press, 421-428.
- Kintsch, W. The role of knowledge in discourse comprehension: A
construction-integration model. Psychological Review, 95,
(1988), 163-182.
- Kintsch, W., and Welsch, D. M. The construction-integartion model: A
framework for studying memory for text. In W. E. Hockley and S. Lewandowsky
(Eds.), Relating theory and data: Essays on human memory (pp.
367-385). Erlbaum, Hillsdale NJ, 1991.
- Kitajima, M., and Polson, P. G. A comprehension-based model of correct
performance and errors in skilled, display-based human-computer
interaction. International Journal of Human-Computer Systems,
43, (1995), 65-99.
- Kitajima, M., and Polson, P. G. A comprehension-based model of
exploration. Human-Computer Interaction. (in press).
- Rodriguez, M., and Polson, P. G. Expert exploration. 1996. unpublished.
CHI 97 Electronic Publications: Late-Breaking/Short Talks
