home · mobile · calendar · colloquia · 1996-1997 · 

Colloquium - Sutton

Knowledge Representation and Reinforcement Learning
University of Massachusetts, Amherst

Reinforcement learning has been a principled but impoverished approach to artificial intelligence -- principled because it is grounded in experience and in the mathematics of Markov decision processes (MDPs), but impoverished in that it is only able to use world knowledge when represented a constrained way, in particular, at a uniform temporal scale. The reinforcement learning agent that learns to throw a baseball cannot then learn where to throw it, or how to find its way to the playing field. The challenge is to find a knowledge representation language that is expressive and flexible, not unlike the rules of classical symbolic AI, and yet has a mathematically explicit semantics, like the state-transition probabilities of MDPs.

In this talk I propose "multi-time models," a mathematical framework for representing the dynamics of the world in a useful and temporally abstract way. The form of multi-time models is dictated, apparently uniquely, by the requirements for 1) temporal flexibility and expressiveness, 2) suitability for MDP-style planning, and 3) learnability. I present theoretical and conceptual results, illustrated by computational examples. This is joint work with Doina Precup.

Refreshments will be served immediately before the talk at 3:30pm.
Hosted by Satinder Singh.

Department of Computer Science
University of Colorado Boulder
Boulder, CO 80309-0430 USA
May 5, 2012 (14:13)