skip to main content
Department of Computer Science University of Colorado Boulder
cu: home | engineering | mycuinfo | about | cu a-z | search cu | contact cu cs: about | calendar | directory | catalog | schedules | mobile | contact cs
home · events · colloquia · 1999-2000 · 

Colloquium - Precup

ECCR 105

Options: A Framework for Temporal Abstraction in Reinforcement Learning
Department of Computer Science, University of Massachusetts

Decision making routinely involves choosing among different courses of action over a broad range of time scales. For instance, a person planning a trip to a distant location makes high-level decisions regarding what means of transportation to use, but also chooses low-level actions, such as the movement units for getting into a car. The problem of picking an appropriate time scale for reasoning and learning has been explored in artificial intelligence, control theory and robotics.

Doina Precup photo

In this talk I will present a novel approach to this problem, in the context of Markov Decision Processes (MDPs) and reinforcement learning. I will present a formal framework for representing temporally extended actions, called options. Options are a minimal extension to MDPs, allowing the incorporation of existing controllers, heuristics for picking actions, or learned courses of action. The outcomes of options can be predicted using multi-time models, learned by interacting with the environment. Such models can then be used to produce plans of behavior very quickly, using classical dynamic programming or reinforcement learning techniques.

The most interesting feature of the framework is that it allows an agent to work simultaneously with high-level and low-level temporal representations. The interplay of these levels can be exploited in order to learn and plan more efficiently and more accurately. I will present new algorithms that take advantage of this structure to improve the quality of plans, and to learn in parallel about the effects of many different options.

Hosted by Clayton Lewis.

The Department holds colloquia throughout the Fall and Spring semesters. These colloquia, open to the public, are typically held on Thursday afternoons, but sometimes occur at other times as well. If you would like to receive email notification of upcoming colloquia, subscribe to our Colloquia Mailing List. If you would like to schedule a colloquium, see Colloquium Scheduling.

Sign language interpreters are available upon request. Please contact Stephanie Morris at least five days prior to the colloquium.

See also:
Department of Computer Science
College of Engineering and Applied Science
University of Colorado Boulder
Boulder, CO 80309-0430 USA
Send email to

Engineering Center Office Tower
ECOT 717
FAX +1-303-492-2844
XHTML 1.0/CSS2 ©2012 Regents of the University of Colorado
Privacy · Legal · Trademarks
May 5, 2012 (13:29)