Main » Home
Please join our IEEE Technical Committee on Robot Learning if you are interested in robot learning!!!
Welcome
Welcome to my homepage! My name is Jan Peters. My research centers around the goal of bringing advanced motor skills to robotics using techniques from machine learning and optimal control. You can check out my research interests and my publications for further information.
I have joined the Max-Planck Institute of Biological Cybernetics in 2007 as a Research Scientist and as Robot Learning Group Leader in the Department of Bernhard Schoelkopf. Before doing so, I completed a Ph.D. at the Department of Computer Science at the University of Southern California in sunny Los Angeles. There, I have been working with Stefan Schaal, Sethu Vijayakumar (now at U. Edinburgh, UK), and Firdaus Udwadia (Department of Mechanical Engineering). Chris Atkeson (Robotics Institute at CMU) and Gaurav Sukhatme also guided me to my thesis. Even longer ago, before joining USC, I studied computer science (with a focus on artificial intelligence), and electrical engineering (majoring in automation & control) in Germany and Singapore, worked in Germany, Japan, and Singapore. Furthermore, I obtained a Dipl.-Ing. (German MSEE) from Munich University of Technology and a Dipl.-Inform. (German MSCS) from Hagen University. At University of Southern California, I have obtained yet another Masters in Computer Science and, more recently, completed a Masters in Mechanical Engineering (Major: Dynamics & Nonlinear Control). Check out my curriculum vitae for more information.
At the Max-Planck Institute of Biological Cybernetics I have build up the new RObot Learning Lab (RoLL) working with four terrific robot learning students: Duy Nguyen-Tuong, Jens Kober, Katharina Muelling and Oliver Kroemer. We also had a couple of excellent research interns/external students collaborating with us: Gerhard Neumann (TU Graz), Hirotaka Hachiya (Tokyo Tech) and Marc Deisenroth (Cambridge Univ.).
As my research lies at the intersection between two fields, i.e., machine learning and robotics, I am always keen to bring members of both fields together. To do so, I have organized two NIPS workshops (Towards a New Reinforcement Learning! and Robotics Challenges for Machine Learning), one R:SS Workshop ( Learning for Locomotion), two IROS workshops (From motor to interaction learning in robots and Robotics Challenges for Machine Learning II), one ICRA workshop (Approaches to Sensorimotor Learning on Humanoid Robots) and one ECAI workshop (The 6th International Cognitive Robotics Workshop). My Co-Organizers included Pieter Abeel (U. Berkeley), Drew Bagnell (CMU), Dana Kulic (U. Waterloo), Jun Morimoto (ATR), Nick Roy (MIT), Stefan Schaal (USC), Olivier Sigaud (U.Paris 6), Russ Tedrake (MIT), Marc Toussaint (TU Berlin), Sethu Vijayakumar (U.Edingburgh), Gerhard Lakemeyer (RWTH Aachen, Germany), Yves Lespérance (York University, Canada), Fiora Pirri (University of Rome "La Sapienza", Italy), Ales Ude (Josef Stefan Institute, Slovenia), Tamim Asfour (U.Karlsruhe).
In 2008, Nick Roy (MIT), Russ Tedrake (MIT), Jun Morimoto (ATR) and I founded the IEEE Technical Committee on Robot Learning.
In 2009, Andrew Y. Ng (Stanford) and I have edited a Special Issue on Robot Learning in the Autonomous Robots journal. We have received 46 submissions and only accepted the best 8 papers. It required altogether 180 reviews written by roughly 100 colleagues to achieve this excellent selection.
In case that you are searching for my address or for directions on how to get to our lab and my office, look at my contact information. Alternatively, you can visit my official website.
Upcoming Events
- Plenary talk at LEMIR 2009 (Learning and Mining for Robotics workshop at ECML-PKDD).
- Dagstuhl Seminar: Cognition, Control and Learning for Robot Manipulation in Human Environments with Michael Beetz (TU München), Oliver Brock (U.Mass. in Amherst/TU Berlin), Gordon Cheng (ATR) from 16.08.09 to 21.08.09.
- Lecture at the RLSS Robot Learning Summer School at IST Lisbon.
- Invited Speaker at ISRR in Luzern in September 2009.
Recent News
- Invited talks at R:SS 2009 Workshops Regression in Robotics - Approaches and Applications and Creative Manipulation: Examples using the WAM.
- R:SS 2009 Workshops Bridging the gap between high-level discrete representations and low-level continuous behaviors with Dana Kulic, (U.Tokyo) and Pieter Abbel (U.California in Berkeley).
- Invited plenary lecture at 4th XVR Workshop & Joint PRESENCCIA and SKILLS PhD Symposium.
- ICRA 2009 Workshop: Approaches to Sensorimotor Learning on Humanoid Robots with Ales Ude, Tamim Asfour, Jun Morimoto and Stefan Schaal on 17.05.2009.
- Invited plenary lecture at Premières Journées Annuelles du GDR Robotique 2008 (French National Conference on Robotics).
- Duy Nguyen-Tuong's and my paper was finalist for the IROS 2008 Best Paper Award.
- IROS 2008 Workshop: From motor to interaction learning in robots with Olivier Sigaud (U.Paris 6) and Sethu Vijayakumar (U.Edingburgh).
- IROS 2008 Workshop: Robotics Challenges for Machine Learning II with Russ Tedrake (MIT), Nick Roy (MIT), Jun Morimoto (ATR).
- Invited talk on Motor Skill Learning for Cognitive Robotics at The 6th International Cognitive Robotics Workshop at ECAI 2008.
- Invited plenary lecture/Keynote Reinforcement Learning for Robotics at the European Workshop on Reinforcement Learning (EWRL).
- Keynote Unifying Imitation and Reinforcement Learning for Robotics at the Robotics: Science & Systems (R:SS), Workshop on Interactive Robotic Learning.
Current Publications
| Record Number | 10271 |
| Reference Type | Journal Article |
| Author(s) | Peters, J.;Schaal, S. |
| Year | 2008 |
| Title | Reinforcement learning of motor skills with policy gradients |
| Journal/Conference/Book Title | Neural Networks |
| Keywords | Reinforcement learning, Policy gradient methods, Natural gradients, Natural Actor-Critic, Motor skills, Motor primitives |
| Abstract | Autonomous learning is one of the hallmarks of human and animal behavior, and understanding the principles of learning will be crucial in order to achieve true autonomy in advanced machines like humanoid robots. In this paper, we examine learning of complex motor skills with human-like limbs. While supervised learning can offer useful tools for bootstrapping behavior, e.g., by learning from demonstration, it is only reinforcement learning that offers a general approach to the final trial-and-error improvement that is needed by each individual acquiring a skill. Neither neurobiological nor machine learning studies have, so far, offered compelling results on how reinforcement learning can be scaled to the high-dimensional continuous state and action spaces of humans or humanoids. Here, we combine two recent research developments on learning motor control in order to achieve this scaling. First, we interpret the idea of modular motor control by means of motor primitives as a suitable way to generate parameterized control policies for reinforcement learning. Second, we combine motor primitives with the theory of stochastic policy gradient learning, which currently seems to be the only feasible framework for reinforcement learning for humanoids. We evaluate different policy gradient methods with a focus on their applicability to parameterized motor primitives. We compare these algorithms in the context of motor primitive learning, and show that our most modern algorithm, the Episodic Natural Actor-Critic outperforms previous algorithms by at least an order of magnitude. We demonstrate the efficiency of this reinforcement learning method in the application of learning to hit a baseball with an anthropomorphic robot arm. |
| Notes | clmc Journal Article United States the official journal of the International Neural Network Society |
| Volume | 21 |
| Number | 4 |
| Pages | 682-97 |
| Date | May |
| Short Title | Reinforcement learning of motor skills with policy gradients |
| ISBN/ISSN | 0893-6080 (Print) |
| Accession Number | 18482830 |
| URL(s) | http://www-clmc.usc.edu/publications/P/peters-NN2008.pdf |
| Address | Max Planck Institute for Biological Cybernetics, Spemannstr. 38, 72076 Tubingen, Germany; University of Southern California, 3710 S. McClintoch Ave-RTH401, Los Angeles, CA 90089-2905, USA. |
| Language | eng |