Barbara Hayes-Roth

Computer Science Department
220A Gates Hall
Stanford University
Palo Alto, CA 94305

Phone: (415) 723-0506
Fax: (415) 725-5850
E-mail: bhr@cs.stanford.edu
URL: http://www-ksl.stanford.edu/people/bhr
Research Topic: Multi-agent human-computer collaboration
Keyword: Improvisation


Summary:
Directed Improvisation: A Paradigm for Multi-Agent Human-Computer Collaboration


We are investigating directed improvisation, a new paradigm for multi-agent human-computer collaboration. Basically, one or more users may direct one or more computer characters with abstract instructions and constraints. The characters improvise courses of behavior that follow the directions, express distinctive styles of behavior, and meet other objectives. Additional characters may improvise autonomously, without user direction. The resulting "work," comprising the individual and interactive behaviors of the characters, reflects a collaboration among all of the human and computer participants. Human users collaborate with one another to determine the abstract structure and quality of their characters' behaviors. Computer characters collaborate with users and with one another to improvise the details of their individual and interactive behaviors.

Distinctive features of directed improvisation include: (a) It provides a framework in which intelligent computer characters cooperate with users and one another to achieve the users' goals. Minimally, it offers an efficient mode of communication. Users need not explicitly instruct characters in all of the details of performing a task. Ideally, it offers characters who bring expertise to the task at hand and enhance achievement of overall objectives. (b) It explicitly characterizes flexibility in the manner in which goals can be achieved. Many important jobs (e.g., medical monitoring, process management, autonomous robot control) carry intrinsic uncertainty in critical run-time conditions, such as resource availability, operating constraints, and performance requirements. Directed improvisation allows users to give abstract directions, within which computer characters can improvise behavior that is compatible with run-time conditions. (c) It provides a natural, familiar style of interaction that mimics the improvisational quality of human behavior and interaction. In contrast to the stereotypic "feel" of most human-computer interaction, directed improvisation will have the variable, idiosyncratic, slightly unpredictable, "give-and-take" feel of human interaction. (d) It introduces a potentially amusing, engaging, and delightful quality to interactive experience in the form of computer characters who combine task-relevant collaboration with task-compatible improvisation.

We believe that directed improvisation is a useful and powerful framework for multi-agent human-computer collaboration in many application domains. For example, in the past, we have studied similar interaction paradigms in applications to ICU patient monitoring and office robots. In the present project, we are studying directed improvisation in applications related to the arts, learning through play, and social interaction.

We have developed and continue to develop a series of experimental systems, including:

  1. Improv Puppets. Two animated puppets improvise actions and dialogue in a shared virtual world, as directed by two children through their respective graphical director’s tools.

  2. Master-Servant. Two actors improvise user-specified scenarios comprising directions for role, characterization, and plot.

  3. Cyber Café. An autonomous actor plays the role of waiter (with variable personality), while users direct the behavior of their animated avatars in a shared virtual café.

  4. Merlyn. An autonomous character in the persona of Merlyn the Magician, tutor of the young King Arthur, guides a child through customized experiential learning experiences in a MOO.

  5. Lhasa. An autonomous character functions as a "receptionist" on a Web site.

We have done and continue to do empirical evaluations of our experimental systems, including:

  1. We evaluated 4-5 different interfaces for the puppets.

  2. We did observational studies of children’s play with the puppets.

  3. We currently are studying the effectiveness of certain kinds of play with puppets to facilitate children’s development of a "theory of mind."

  4. We are planning additional studies of children's play & learning with the puppets.

  5. We plan to do similar studies of the effectiveness of Merlyn and Lhasa.


Return to ITO Workshop Home Page