News:

> The deadline for our Autonomous Robots Special Issue on Robot Learning has passed. We have received many interesting submissions.

> Duy Nguyen-Tuong's and my paper was finalist for the IROS 2008 Best Paper Award.

Welcome

Welcome to my homepage! My name is Jan Peters. My research centers around the goal of bringing advanced motor skills to robotics using techniques from machine learning and optimal control. You can check out my research interests and my publications for further information.

I have joined the Max-Planck Institute of Biological Cybernetics in 2007 as a Research Scientist and as Robot Learning Group Leader in the Department of Bernhard Schoelkopf. Before doing so, I completed a Ph.D. at the Department of Computer Science at the University of Southern California in sunny Los Angeles. There, I have been working with Stefan Schaal, Sethu Vijayakumar (now at U. Edinburgh, UK), and Firdaus Udwadia (Department of Mechanical Engineering). Chris Atkeson (Robotics Institute at CMU) and Gaurav Sukhatme also guided me to my thesis. Even longer ago, before joining USC, I studied computer science (with a focus on artificial intelligence), and electrical engineering (majoring in automation & control) in Germany and Singapore, worked in Germany, Japan, and Singapore. Furthermore, I obtained a Dipl.-Ing. (German MSEE) from Munich University of Technology and a Dipl.-Inform. (German MSCS) from Hagen University. At University of Southern California, I have obtained yet another Masters in Computer Science and, more recently, completed a Masters in Mechanical Engineering (Major: Dynamics & Nonlinear Control). Check out my curriculum vitae for more information.

At the Max-Planck Institute of Biological Cybernetics I have build up the new RObot Learning Lab (RoLL) working with four terrific robot learning students: Duy Nguyen-Tuong, Jens Kober, Katharina Muelling and Oliver Kroemer. We also had a couple of excellent research interns/external students collaborating with us: Gerhard Neumann (TU Graz), Hirotaka Hachiya (Tokyo Tech) and Marc Deisenroth (Cambridge Univ.).

As my research lies at the intersection between two fields, i.e., machine learning and robotics, I am always keen to bring members of both fields together. To do so, I have organized two NIPS and one RSS workshop with my Co-Organizers Drew Bagnell (CMU), Russ Tedrake (MIT), Marc Toussaint (TU Berlin) and Stefan Schaal (USC). I am also currently helping out at organizing two IROS workshops and one ECAI workshop. Also, in 2008, Nick Roy (MIT), Russ Tedrake (MIT), Jun Morimoto (ATR) and I founded the IEEE Technical Committee on Robot Learning.

In 2009, Andrew Y. Ng (Stanford) and I will edit a Special Issue on Robot Learning in the Autonomous Robots journal. We are welcoming submissions - please contact me if you are interested! Early submissions are encouraged, we will make sure that they get reviewed directly upon submission.

In case that you are searching for my address or for directions on how to get to our lab and my office, look at my contact information. Alternatively, you can visit my official website.

Upcoming Events

• Dagstuhl Seminar: Cognition, Control and Learning for Robot Manipulation in Human Environments with Michael Beetz (TU München), Oliver Brock (U.Mass. in Amherst), Gordon Cheng (ATR) from 16.08.09 to 21.08.09.
• ICRA 2009 Workshop: Approaches to Sensorimotor Learning on Humanoid Robots with Ales Ude, Tamim Asfour, Jun Morimoto and Stefan Schaal on 17.05.2009.

Please join our IEEE Technical Committee on Robot Learning if you are interested in robot learning!!!

Recent Past Events

• Invited plenary lecture at Premières Journées Annuelles du GDR Robotique 2008 (French National Conference on Robotics).
• IROS 2008 Workshop: From motor to interaction learning in robots with Olivier Sigaud (U.Paris 6) and Sethu Vijayakumar (U.Edingburgh).
• IROS 2008 Workshop: Robotics Challenges for Machine Learning II with Russ Tedrake (MIT), Nick Roy (MIT), Jun Morimoto (ATR).
• Invited talk on Motor Skill Learning for Cognitive Robotics at The 6th International Cognitive Robotics Workshop at ECAI 2008.
• Invited plenary lecture/Keynote Reinforcement Learning for Robotics at the European Workshop on Reinforcement Learning (EWRL).
• Keynote Unifying Imitation and Reinforcement Learning for Robotics at the Robotics: Science & Systems (R:SS), Workshop on Interactive Robotic Learning.
• Together with Marc Toussaint, I organized the NIPS 2007 Workshop: Robotics Challenges for Machine Learning!!!.
• Lecture on Policy Learning at the IEEE RAS / IFRR Summer School on Robot Learning.

Current Publications

Hachiya,H.; Akiyama, T.; Sugiyama, M.; Peters, J. (accepted). Adaptive Importance Sampling for Value Function Approximation in Off-policy Reinforcement Learning, Neural Networks.
[Keywords: off-policy reinforcement learning; value function approximation; policy iteration; adaptive importance sampling; importance-weighted cross-validation; efficient sample reuse]
[Details]

Deisenroth, M.P., Rasmussen, C.E.; Peters, J (accepted). Gaussian Process Dynamic Programming, Neurocomputing.
[Details]

Nguyen-Tuong, D.; Seeger, M.; Peters, J. (2009). Local Gaussian Process Regression for Real Time Online Model Learning and Control, Advances in Neural Information Processing Systems 22 (NIPS 2008), Cambridge, MA: MIT Press.
[Details]

Neumann, G.; Peters, J. (2009). Fitted Q-iteration by Advantage Weighted Regression, Advances in Neural Information Processing Systems 22 (NIPS 2008), Cambridge, MA: MIT Press.
[Details]

Kober, J.; Peters, J. (2009). Policy Search for Motor Primitives in Robotics, Advances in Neural Information Processing Systems 22 (NIPS 2008), Cambridge, MA: MIT Press.
[Details]

Chiappa, S.; Kober, J.; Peters, J. (2009). Using Bayesian Dynamical Systems for Motion Template Libraries, Advances in Neural Information Processing Systems 22 (NIPS 2008), Cambridge, MA: MIT Press.
[Details]

Peters, J., Schaal, S. (2008). Learning to Control in Operational Space, The International Journal of Robotics Research, 27, 2, pp.197-212.
[Keywords: operational space control, robot learning, reinforcement learning, reward-weighted regression]
[Details] [PDF]

Peters, J.; Schaal, S. (2008). Natural Actor-Critic, Neurocomputing, 71, 7-9, pp.1180-1190.
[Keywords: reinforcement learning, policy gradient, natural actor-critic, natural gradients]
[Details] [PDF]

Deisenroth, M.; Peters, J.; Rasmussen, C. (2008). Approximate Dynamic Programming with Gaussian Processes, American Control Conference.
[Details] [PDF]

Nguyen-Tuong, D.; Peters, J.; Seeger, M.; Schoelkopf, B. (2008). Computed Torque Control with Nonparametric Regressions Techniques, American Control Conference.
[Details] [PDF]

Peters, J.;Mistry, M.;Udwadia, F. E.;Nakanishi, J.;Schaal, S. (2008). A unifying methodology for robot control with redundant DOFs, Autonomous Robots, 24, 1, pp.1-12.
[Details] [PDF]

Deisenroth, M.P., Rasmussen, C.E.; Peters, J (2008). Model-Based Reinforcement Learning with Continuous States and Actions, Proceedings of the European Symposium on Artificial Neural Networks (ESANN 2008).
[Details] [PDF]

Steinke, F, Hein, M., Peters, J., Schölkopf, B (2008). Manifold-valued Thin-Plate Splines with Applications in Computer Graphics, Computer Graphics Forum (Special Issue on Eurographics 2008), 27, 2.
[Details] [PDF]

Nguyen-Tuong, D.; Peters, J.; Seeger, M.; Schoelkopf, B. (2008). Learning Inverse Dynamics: a Comparison, Proceedings of the European Symposium on Artificial Neural Networks (ESANN 2008).
[Details] [PDF]

Peters, J., Nguyen, D.; (2008). Real-Time Learning of Resolved Velocity Control on a Mitsubishi PA-10, International Conference on Robotics and Automation (ICRA).
[Details] [PDF]

Hachiya, H.; Akiyama, T.; Sugiyama, M.; Peters, J. (2008). Adaptive Importance Sampling with Automatic Model Selection in Value Function Approximation, Proceedings of the Twenty-Third National Conference on Artificial Intelligence (AAAI 2008).
[Details] [PDF]

Nguyen, D.; Peters, J. (2008). Local Gaussian Processes Regression for Real-time Model-based Robot Control, International Conference on Intelligent Robot Systems (IROS).
[Details] [PDF]

Kober, J.; Peters, J. (2008). Learning Perceptual Coupling for Motor Primitives, International Conference on Intelligent Robot Systems (IROS).
[Details] [PDF]

Lespérance, Y.; Lakemeyer, G.; Peters, J.; Pirri, F. (2008). Proceedings of the 6th International Cognitive Robotics Workshop (CogRob 2008), July 21-22, 2008, Patras, Greece, IOS Press, ISBN 978-960-6843-09-9.
[Details]

Peters, J.;Schaal, S. (2008). Reinforcement learning of motor skills with policy gradients, Neural Networks, 21, pp.682–697.
[Keywords: Reinforcement learning, Policy gradient methods, Natural gradients, Natural Actor-Critic, Motor skills, Motor primitives]
[Details] [PDF]

Nakanishi, J.;Cory, R.;Mistry, M.;Peters, J.;Schaal, S. (2008). Operational space control: A theoretical and emprical comparison, International Journal of Robotics Research, 27, 6, pp.737-757.
[Keywords: task space control, operational space control, redundancy resolution, humanoid robotics]
[Details] [PDF]

Wierstra,D.; Schaul,T.; Peters, J.; Schmidhuber, J. (2008). Episodic Reinforcement Learning by Logistic Reward-Weighted Regression, Proceedings of the International Conference on Artificial Neural Networks (ICANN).
[Details] [PDF]

Sehnke, F.; Osendorfer, C; Rueckstiess, T; Graves, A.; Peters, J.; Schmidhuber, J. (2008). Policy Gradients with Parameter-based Exploration for Control, Proceedings of the International Conference on Artificial Neural Networks (ICANN).
[Details] [PDF]

Peters, J. (2008). Machine Learning for Robotics, VDM-Verlag, ISBN 978-3-639-02110-3.
[Details] [PDF]

Peters, J.; Kober, J.; Nguyen-Tuong, D. (2008). Policy Learning – a unified perspective with applications in robotics, Proceedings of the European Workshop on Reinforcement Learning (EWRL).
[Keywords: reinforcement learning, policy gradient, weighted regression]
[Details] [PDF]

Kober, J.; Peters, J. (2008). Reinforcement Learning of Perceptual Coupling for Motor Primitives, Proceedings of the European Workshop on Reinforcement Learning (EWRL).
[Details]

Peters, J. (2008). Machine Learning for Motor Skills in Robotics, Künstliche Intelligenz, 3.
[Keywords: motor control, motor primitives, motor learning]
[Details] [PDF]

Nguyen, D.; Peters, J. (2008). Learning Robot Dynamics for Computed Torque Control using Local Gaussian Processes Regression, Proceedings of the ECSIS Symposium on Learning and Adaptive Behavior in Robotic Systems, LAB-RS 2008.
[Details] [PDF]