Please join our IEEE Technical Committee on Robot Learning if you are interested in robot learning!!!

News:

> The Autonomous Robots Special Issue on Robot Learning is on its way. We have received 46 interesting submissions and have only accepted 8 papers making this special issue highly competitive.

Welcome

Welcome to my homepage! My name is Jan Peters. My research centers around the goal of bringing advanced motor skills to robotics using techniques from machine learning and optimal control. You can check out my research interests and my publications for further information.

I have joined the Max-Planck Institute of Biological Cybernetics in 2007 as a Research Scientist and as Robot Learning Group Leader in the Department of Bernhard Schoelkopf. Before doing so, I completed a Ph.D. at the Department of Computer Science at the University of Southern California in sunny Los Angeles. There, I have been working with Stefan Schaal, Sethu Vijayakumar (now at U. Edinburgh, UK), and Firdaus Udwadia (Department of Mechanical Engineering). Chris Atkeson (Robotics Institute at CMU) and Gaurav Sukhatme also guided me to my thesis. Even longer ago, before joining USC, I studied computer science (with a focus on artificial intelligence), and electrical engineering (majoring in automation & control) in Germany and Singapore, worked in Germany, Japan, and Singapore. Furthermore, I obtained a Dipl.-Ing. (German MSEE) from Munich University of Technology and a Dipl.-Inform. (German MSCS) from Hagen University. At University of Southern California, I have obtained yet another Masters in Computer Science and, more recently, completed a Masters in Mechanical Engineering (Major: Dynamics & Nonlinear Control). Check out my curriculum vitae for more information.

At the Max-Planck Institute of Biological Cybernetics I have build up the new RObot Learning Lab (RoLL) working with four terrific robot learning students: Duy Nguyen-Tuong, Jens Kober, Katharina Muelling and Oliver Kroemer. We also had a couple of excellent research interns/external students collaborating with us: Gerhard Neumann (TU Graz), Hirotaka Hachiya (Tokyo Tech) and Marc Deisenroth (Cambridge Univ.).

As my research lies at the intersection between two fields, i.e., machine learning and robotics, I am always keen to bring members of both fields together. To do so, I have organized two NIPS workshops (Towards a New Reinforcement Learning! and Robotics Challenges for Machine Learning), one R:SS Workshop ( Learning for Locomotion), two IROS workshops (From motor to interaction learning in robots and Robotics Challenges for Machine Learning II), one ICRA workshop (Approaches to Sensorimotor Learning on Humanoid Robots) and one ECAI workshop (The 6th International Cognitive Robotics Workshop). My Co-Organizers included Pieter Abeel (U. Berkeley), Drew Bagnell (CMU), Dana Kulic (U. Waterloo), Jun Morimoto (ATR), Nick Roy (MIT), Stefan Schaal (USC), Olivier Sigaud (U.Paris 6), Russ Tedrake (MIT), Marc Toussaint (TU Berlin), Sethu Vijayakumar (U.Edingburgh), Gerhard Lakemeyer (RWTH Aachen, Germany), Yves Lespérance (York University, Canada), Fiora Pirri (University of Rome "La Sapienza", Italy), Ales Ude (Josef Stefan Institute, Slovenia), Tamim Asfour (U.Karlsruhe).

In 2008, Nick Roy (MIT), Russ Tedrake (MIT), Jun Morimoto (ATR) and I founded the IEEE Technical Committee on Robot Learning.

In 2009, Andrew Y. Ng (Stanford) and I have edited a Special Issue on Robot Learning in the Autonomous Robots journal. We have received 46 submissions and only accepted the best 8 papers. It required altogether 180 reviews written by roughly 100 colleagues to achieve this excellent selection.

In case that you are searching for my address or for directions on how to get to our lab and my office, look at my contact information. Alternatively, you can visit my official website.

Upcoming Events

• Plenary talk at LEMIR 2009 (Learning and Mining for Robotics workshop at ECML-PKDD).
• Dagstuhl Seminar: Cognition, Control and Learning for Robot Manipulation in Human Environments with Michael Beetz (TU München), Oliver Brock (U.Mass. in Amherst/TU Berlin), Gordon Cheng (ATR) from 16.08.09 to 21.08.09.
• Lecture at the RLSS Robot Learning Summer School at IST Lisbon.
• Invited Speaker at ISRR in Luzern in September 2009.

Recent News

• Invited talks at R:SS 2009 Workshops Regression in Robotics - Approaches and Applications and Creative Manipulation: Examples using the WAM.
• R:SS 2009 Workshops Bridging the gap between high-level discrete representations and low-level continuous behaviors with Dana Kulic, (U.Tokyo) and Pieter Abbel (U.California in Berkeley).
• Invited plenary lecture at 4th XVR Workshop & Joint PRESENCCIA and SKILLS PhD Symposium.
• ICRA 2009 Workshop: Approaches to Sensorimotor Learning on Humanoid Robots with Ales Ude, Tamim Asfour, Jun Morimoto and Stefan Schaal on 17.05.2009.
• Invited plenary lecture at Premières Journées Annuelles du GDR Robotique 2008 (French National Conference on Robotics).
• Duy Nguyen-Tuong's and my paper was finalist for the IROS 2008 Best Paper Award.
• IROS 2008 Workshop: From motor to interaction learning in robots with Olivier Sigaud (U.Paris 6) and Sethu Vijayakumar (U.Edingburgh).
• IROS 2008 Workshop: Robotics Challenges for Machine Learning II with Russ Tedrake (MIT), Nick Roy (MIT), Jun Morimoto (ATR).
• Invited talk on Motor Skill Learning for Cognitive Robotics at The 6th International Cognitive Robotics Workshop at ECAI 2008.
• Invited plenary lecture/Keynote Reinforcement Learning for Robotics at the European Workshop on Reinforcement Learning (EWRL).
• Keynote Unifying Imitation and Reinforcement Learning for Robotics at the Robotics: Science & Systems (R:SS), Workshop on Interactive Robotic Learning.

Current Publications

Peters, J.; Morimoto, J.; Tedrake, R.; Roy, N. (in press). Robot Learning, IEEE Robotics & Automation Magazine.
[Keywords: robot learning]
[Details]

Hachiya,H.; Akiyama, T.; Sugiyama, M.; Peters, J. (accepted). Adaptive Importance Sampling for Value Function Approximation in Off-policy Reinforcement Learning, Neural Networks.
[Keywords: off-policy reinforcement learning; value function approximation; policy iteration; adaptive importance sampling; importance-weighted cross-validation; efficient sample reuse]
[Details]

Wierstra, D.; Foester, A.; Peters, J.; Schmidhuber, J. (accepted). Recurrent Policy Gradients, Journal of Algorithms.
[Details]

Morimura, T.; Uchibe, E.; Yoshimoto, J.; Peters, J.; Doya, K. (accepted). Derivatives of Logarithmic Stationary Distributions for Policy Gradient Reinforcement Learning, Neural Computation.
[Details]

Hachiya, H.; Peters, J.; Sugiyama, M. (2009). Efficient Sample Reuse in EM-based Policy Search, Proceedings of the 16th European Conference on Machine Learning (ECML 2009).
[Details]

Nguyen-Tuong, D.; Seeger, M.; Peters, J. (2009). Local Gaussian Process Regression for Real Time Online Model Learning and Control, Advances in Neural Information Processing Systems 22 (NIPS 2008), Cambridge, MA: MIT Press.
[Details] [PDF]

Neumann, G.; Peters, J. (2009). Fitted Q-iteration by Advantage Weighted Regression, Advances in Neural Information Processing Systems 22 (NIPS 2008), Cambridge, MA: MIT Press.
[Details] [PDF]

Kober, J.; Peters, J. (2009). Policy Search for Motor Primitives in Robotics, Advances in Neural Information Processing Systems 22 (NIPS 2008), Cambridge, MA: MIT Press.
[Details] [PDF]

Chiappa, S.; Kober, J.; Peters, J. (2009). Using Bayesian Dynamical Systems for Motion Template Libraries, Advances in Neural Information Processing Systems 22 (NIPS 2008), Cambridge, MA: MIT Press.
[Details] [PDF]

Deisenroth, M.P., Rasmussen, C.E.; Peters, J (2009). Gaussian Process Dynamic Programming, Neurocomputing, 72, pp.1508-1524.
[Details]

Hoffman, M.; de Freitas, N. ; Doucet, A.; Peters, J. (2009). An Expectation Maximization Algorithm for Continuous Markov Decision Processes with Arbitrary Reward, Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AIStats).
[Details]

Peters, J.; Kober, J. (2009). Using Reward-Weighted Imitation for Robot Reinforcement Learning, Proceedings of the 2009 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning..
[Details]

Hachiya, H.; Akiyama, T.; Sugiyama, M.; Peters, J. (2009). Efficient Data Reuse in Value Function Approximation, Proceedings of the 2009 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning..
[Details]

Kober, J.; Peters, J. (2009). Learning Motor Primitives for Robotics, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA).
[Details]

Neumann, G.; Maass, W; Peters, J. (2009). Learning Complex Motions by Sequencing Simpler Motion Templates, Proceedings of the International Conference on Machine Learning (ICML2009).
[Details]

Detry, R; Baseski, E.; Popovic, M.; Touati, Y.; Krueger, N; Kroemer, O.; Peters, J.; Piater, J; (2009). Learning Object-specific Grasp Affordance Densities, Proceedings of the International Conference on Development & Learning (ICDL 2009).
[Details]

Kober, J.; Peters, J. (2009). Reinforcement Learning fuer Motor-Primitive, Kuenstliche Intelligenz.
[Details]

Peters, J.; Ng, A. (2009). Guest Editorial: Special Issue on Robot Learning, Part A, Autonomous Robots.
[Details]

Lampert, C.H.; Peters, J. (2009). Active Structured Learning for High-Speed Object Detection , Proceedings of the DAGM (Pattern Recognition).
[Details]

Deisenroth, M.; Peters, J.; Rasmussen, C. (2008). Approximate Dynamic Programming with Gaussian Processes, American Control Conference.
[Details] [PDF]

Nguyen-Tuong, D.; Peters, J.; Seeger, M.; Schoelkopf, B. (2008). Computed Torque Control with Nonparametric Regressions Techniques, American Control Conference.
[Details] [PDF]

Deisenroth, M.P., Rasmussen, C.E.; Peters, J (2008). Model-Based Reinforcement Learning with Continuous States and Actions, Proceedings of the European Symposium on Artificial Neural Networks (ESANN 2008).
[Details] [PDF]

Steinke, F, Hein, M., Peters, J., Schölkopf, B (2008). Manifold-valued Thin-Plate Splines with Applications in Computer Graphics, Computer Graphics Forum (Special Issue on Eurographics 2008), 27, 2.
[Details] [PDF]

Nguyen-Tuong, D.; Peters, J.; Seeger, M.; Schoelkopf, B. (2008). Learning Inverse Dynamics: a Comparison, Proceedings of the European Symposium on Artificial Neural Networks (ESANN 2008).
[Details] [PDF]

Peters, J., Nguyen, D.; (2008). Real-Time Learning of Resolved Velocity Control on a Mitsubishi PA-10, International Conference on Robotics and Automation (ICRA).
[Details] [PDF]

Hachiya, H.; Akiyama, T.; Sugiyama, M.; Peters, J. (2008). Adaptive Importance Sampling with Automatic Model Selection in Value Function Approximation, Proceedings of the Twenty-Third National Conference on Artificial Intelligence (AAAI 2008).
[Details] [PDF]

Nguyen, D.; Peters, J. (2008). Local Gaussian Processes Regression for Real-time Model-based Robot Control, International Conference on Intelligent Robot Systems (IROS).
[Details] [PDF]

Kober, J.; Peters, J. (2008). Learning Perceptual Coupling for Motor Primitives, International Conference on Intelligent Robot Systems (IROS).
[Details] [PDF]

Lespérance, Y.; Lakemeyer, G.; Peters, J.; Pirri, F. (2008). Proceedings of the 6th International Cognitive Robotics Workshop (CogRob 2008), July 21-22, 2008, Patras, Greece, IOS Press, ISBN 978-960-6843-09-9.
[Details]

Nakanishi, J.;Cory, R.;Mistry, M.;Peters, J.;Schaal, S. (2008). Operational space control: A theoretical and emprical comparison, International Journal of Robotics Research, 27, 6, pp.737-757.
[Keywords: task space control, operational space control, redundancy resolution, humanoid robotics]
[Details] [PDF]

Wierstra,D.; Schaul,T.; Peters, J.; Schmidhuber, J. (2008). Episodic Reinforcement Learning by Logistic Reward-Weighted Regression, Proceedings of the International Conference on Artificial Neural Networks (ICANN).
[Details] [PDF]

Sehnke, F.; Osendorfer, C; Rueckstiess, T; Graves, A.; Peters, J.; Schmidhuber, J. (2008). Policy Gradients with Parameter-based Exploration for Control, Proceedings of the International Conference on Artificial Neural Networks (ICANN).
[Details] [PDF]

Peters, J. (2008). Machine Learning for Robotics, VDM-Verlag, ISBN 978-3-639-02110-3.
[Details] [PDF]

Peters, J.; Kober, J.; Nguyen-Tuong, D. (2008). Policy Learning – a unified perspective with applications in robotics, Proceedings of the European Workshop on Reinforcement Learning (EWRL).
[Keywords: reinforcement learning, policy gradient, weighted regression]
[Details] [PDF]

Kober, J.; Peters, J. (2008). Reinforcement Learning of Perceptual Coupling for Motor Primitives, Proceedings of the European Workshop on Reinforcement Learning (EWRL).
[Details]

Peters, J. (2008). Machine Learning for Motor Skills in Robotics, Künstliche Intelligenz, 3.
[Keywords: motor control, motor primitives, motor learning]
[Details] [PDF]

Nguyen, D.; Peters, J. (2008). Learning Robot Dynamics for Computed Torque Control using Local Gaussian Processes Regression, Proceedings of the ECSIS Symposium on Learning and Adaptive Behavior in Robotic Systems, LAB-RS 2008.
[Details] [PDF]

Peters, J.;Schaal, S. (2008). Natural actor critic, Neurocomputing, 71, 7-9, pp.1180-1190.
[Keywords: reinforcement learning, policy gradient, natural actor-critic, natural gradients]
[Details] [PDF]

Peters, J.;Schaal, S. (2008). Learning to control in operational space, International Journal of Robotics Research, 27, pp.197-212.
[Keywords: operational space control, learning, EM ALGORITHM, redundancy resolution, reinforcement learning]
[Details] [PDF]

Peters, J.;Schaal, S. (2008). Reinforcement learning of motor skills with policy gradients, Neural Networks, 21, 4, pp.682-97.
[Keywords: Reinforcement learning, Policy gradient methods, Natural gradients, Natural Actor-Critic, Motor skills, Motor primitives]
[Details] [PDF]

Peters, J.;Mistry, M.;Udwadia, F. E.;Nakanishi, J.;Schaal, S. (2008). A unifying methodology for robot control with redundant DOFs, Autonomous Robots, 24, 1, pp.1-12.
[Keywords: operational space control, inverse control, dexterous manipulation, optimal control]
[Details] [PDF]