|
|
| Record Number | 10305 |
| Reference Type | Conference Proceedings |
| Author(s) | Evangelos A. Theodorou, Jonas Buchli, Stefan Schaal |
| Year | submitted |
| Title | Reinforcement Learning of Motor Skills in High Dimensions: A Pah Integral Approach. (MANUSCRIPT UNDER REVIEW. SUGGESTIONS WELCOME)) |
| Keywords | reinforcement learning, stochastic optimal control |
Abstract |
Reinforcement learning (RL) is one of the most general approaches to
learning control. Its applicability to complex motor systems,
however, has been largely impossible so far due to the computational
difficulties that reinforcement learning encounters in high
dimensionsal continuous state-action spaces. In this paper, we derive
a novel approach to RL for parameterized control policies based on
the framework of stochastic optimal control with path
integrals. While solidly grounded in optimal control theory and
estimation theory, the update equations for learning are surprisingly
simple and have no danger of numerical instabilites as neither matrix
inversions nor gradient learning rates are required. Empirical
evaluations demonstrate significant performance improvements over
gradient-based policy learning and scalability to high-dimensional
control problems. Finally, a learning experiment on a robot dog
illustrates the functionality of our algorithm in a real-world
scenario. We believe that our new algorithm, {\bf P}olicy {\bf
I}mprovement with {\bf P}ath {\bf I}ntegrals (${\bf PI}^2$), offers
currently one of the most efficient, numerically robust, and easy to
implement algorithms for RL in robotics.
|
| Notes | clmc |
| Link to PDF | http://www-clmc.usc.edu/publications//E/PI2.pdf |
| Papers are available as Adobe PDF ".pdf" files. Adobe Reader is available for free for all computer platforms.
|
|
|
|
|
Page last modified on August 10, 2006, at 06:47 PM
|
|
