Motor Primitives
Movement coordination requires some form of planning: every degree-of-freedom (DOF) needs to be supplied with appropriate motor commands at every moment in time. The commands must be chosen such that they accomplish the desired task, but also such that they do not violate the abilities of the movement system. Due to the numerous DOFs in complex movement systems and the almost infinite number of possibilities to use the DOFs over time, there exist actually an infinite number of possible movement plans for any given task.This redundancy is advantageous as it allows a movement system to avoid situations where, for instance, the range of motion of DOFs is saturated, or where obstacles need to be circumvented to reach a goal. But, from a learning point of view, it also makes it quite complicated to find good movement plans since the state spaces spanned by all possible plans it extremely large. What is needed to make learning tractable in such high dimensional systems is some form of additional constraints, constraints that reduce the state spaces in a reasonable way without eliminating good solutions.
The classical way to constrain solution spaces is to impose optimization criteria on the movement planning, for instance, by requiring that the system accomplishes the task in minimum time or with minimal energy expenditure. However, it is not trivial to find the correct cost function that result in an adequate behavior. Thus, our research on trajecotry planning has been focussing on an alternative method of constraining movement planning by requiring that movements are built from movement primitives. We conceive of movement primitives as simple dynamical systems that can generate either discrete or rhythmic movements about every DOF. Only speed and amplitude parameters are initally needed to get a movement started. Learning is required to fine-tune certain additional parameters to improve the movement. This approach allows us to learn movements by just adjusting a relatively small set of parameters. We are currently exploring how these dynamical systems can be used to generate full body movement, how their parameters can be learned with novel reinforcement learning methods, and how such movement primitives can be sequenced and superimposed to accomplish more complex movement tasks. We also consider how our developed models compare to biological behavior to find out which movement primitives biological systems employ, and how such movement primitives are represented in the brain.
Inspiration from biology also motivates a related trajectory planning project that we conduct. A common feature in the brain is to employ topographic maps as basic represenation of sensory signals. Such maps can be built with various neural network approaches, for instance Kohonen's Self-Organizing Maps or the Topology Representing Network (TRN) by Martinetz. From a statistical point of view, topographic maps can be thought of as neural networks that perform probability density estimation with additional knowledge about neighborhood relations. Density estimators are very powerful tools to perform mappings between different coordinate systems, to perform sensory integration, and to serve as basic representation for other learning systems. But in addition to these properties, topographic maps can also peform spatial computations that can generate trajectory plans. For instance, by using diffusion-based path planning algorithms, we demonstrated the feasability of such an approach by learning obstacle avoidance with a pneumatic robot arm. Learning motor control with topographic maps is also highly interesting from a biological point of view, as, in contrast to visual information processing, the usefulness of topographic maps in motor control is far from understood so far.
Contact persons: Jun Nakanishi, Jan Peters, Stefan Schaal
Publications
Hoffmann, H.;Pastor, P.;Park, D.-H.;Schaal, S. (2009). Biologically-inspired dynamical systems for movement generation: automatic real-time goal adaptation and obstacle avoidance, International Conference on Robotics and Automation (ICRA2009).
[Keywords: movement primitives, dynamic systems, obstacle avoidance, generalization]
[Detail] [BibTeX] [PDF]
Pastor, P.;Hoffmann, H.;Asfour, T.;Schaal, S. (2009). Learning and generalization of motor skills by learning from demonstration, International Conference on Robotics and Automation (ICRA2009).
[Keywords: movement primitives, dynamic systems, obstacle avoidance, generalization, affordances]
[Detail] [BibTeX] [PDF]
Park, D.-H.;Hoffmann, H.;Schaal, S. (2008). Combining dynamic movement primitives and potential fields for online obstacle avoidance, Adaptive Motion of Animals and Machines (AMAM).
[Keywords: movement primitives, potential fields, obstacle avoidance, dexterous manipulation]
[Detail] [BibTeX] [PDF]
Mohajerian, P; Hoffmann, H.; Mistry, M.; Schaal, S. (2007). A Computational Model of Arm Trajectory Modification Using Dynamic Movement Primitives, Abstracts of the 37st Meeting of the Society of Neuroscience.
[Keywords: online movement correction, target switching, movement primitives, computational model, motor control]
[Detail] [BibTeX] [PDF]
Schaal, S; Mohajerian, P.; Ijspeert, A. (2007). Dynamics systems vs. optimal control — a unifying view, Progress in Brain Research, 165, pp.425-445.
[Keywords: discrete movement; rhythmic movement; movement primitives; dynamic systems; optimization; computational motor control]
[Detail] [BibTeX]
Nakanishi, J.;Morimoto, J.;Endo, G.;Cheng, G.;Schaal, S.;Kawato, M. (2004). Learning from demonstration and adaptation of biped locomotion, Robotics and Autonomous Systems, 47, 2-3, pp.79-91.
[Keywords: movement primitives, locomotion, phase resetting, learning from demonstration]
[Detail] [BibTeX] [PDF]
Nakanishi, J.;Morimoto, J.;Endo, G.;Cheng, G.;Schaal, S.;Kawato, M. (2004). A framework for learning biped locomotion with dynamic movement primitives, IEEE-RAS/RSJ International Conference on Humanoid Robots (Humanoids 2004), IEEE.
[Keywords: movement primitives, dynamic systems, locomotion, phase resetting, learning]
[Detail] [BibTeX] [PDF]
Schaal, S.;Ijspeert, A.;Billard, A. (2004). Computational approaches to motor learning by imitation, in: Frith, C. D.;Wolpert, D. (eds.), The Neuroscience of Social Interaction, 1431, pp.199-218, Oxford University Press.
[Keywords: imitation learning
computational
review
movement primitives
duality of movement generation and recognition
motor control]
[Detail] [BibTeX] [PDF]
Schaal, S.;Peters, J.;Nakanishi, J.;Ijspeert, A. (2004). Learning Movement Primitives, International Symposium on Robotics Research (ISRR2003), Springer.
[Keywords: movement primitives, supervised learning, reinforcment learning, locomotion, phase resetting, learning from demonstration]
[Detail] [BibTeX] [PDF]
Schaal, S.;Sternad, D.;Osu, R.;Kawato, M. (2004). Rhythmic movement is not discrete, Nature Neuroscience, 7, 10, pp.1137-1144.
[Keywords: fmri discrete rhythmic movement movement primitives]
[Detail] [BibTeX] [PDF]
Billard, A.;Epars, Y.;Schaal, S.;Cheng, G. (2003). Discovering imitation strategies through categorization of multi-cimensional data, IEEE International Conference on Intelligent Robots and Systems (IROS 2003).
[Keywords: movement primitives, sequencing]
[Detail] [BibTeX] [PDF]
Ijspeert, A.;Nakanishi, J.;Schaal, S. (2003). Learning attractor landscapes for learning motor primitives, in: Becker, S.;Thrun, S.;Obermayer, K. (eds.), Advances in Neural Information Processing Systems 15, pp.1547-1554, Cambridge, MA: MIT Press.
[Keywords: learning nonlinear attractor landscapes
movement primitives
humanoid robotics
statistical learning]
[Detail] [BibTeX] [PDF]
Nakanishi, J.;Morimoto, J.;Endo, G.;Schaal, S.;Kawato, M. (2003). Learning from demonstration and adaptation of biped locomotion with dynamical movement primitives, Workshop on Robot Learning by Demonstration, IEEE International Conference on Intelligent Robots and Systems (IROS 2003).
[Keywords: movement primitives, locomotion, phase resetting, learning from demonstration]
[Detail] [BibTeX] [PDF]
Peters, J.;Vijayakumar, S.;Schaal, S. (2003). Reinforcement learning for humanoid robotics, Humanoids2003, Third IEEE-RAS International Conference on Humanoid Robots.
[Keywords: reinforcement learning, policy gradients, movement primitives, behaviors, dynamic systems, humanoid robotics]
[Detail] [BibTeX] [PDF]
Schaal, S. (2003). Dynamic movement primitives - A framework for motor control in humans and humanoid robots, The International Symposium on Adaptive Motion of Animals and Machines.
[Keywords: movement primitives, behaviors, dynamic systems, computational motor control, attractor landscapes, fmri, behavioral evidence]
[Detail] [BibTeX] [PDF]
Schaal, S. (2003). Movement planning and imitation by shaping nonlinear attractors, Proceedings of the 12th Yale Workshop on Adaptive and Learning Systems.
[Keywords: movement primitives, behaviors, dynamic systems, computational motor control, attractor landscapes]
[Detail] [BibTeX] [PDF]
Schaal, S.;Ijspeert, A.;Billard, A. (2003). Computational approaches to motor learning by imitation, Philosophical Transaction of the Royal Society of London: Series B, Biological Sciences, 358, 1431, pp.537-547.
[Keywords: imitation learning, computational, review, movement primitives, duality of movement generation and recognition, motor control]
[Detail] [BibTeX] [PDF]
Schaal, S.;Peters, J.;Nakanishi, J.;Ijspeert, A. (2003). Control, planning, learning, and imitation with dynamic movement primitives, Workshop on Bilateral Paradigms on Humans and Humanoids, IEEE International Conference on Intelligent Robots and Systems (IROS 2003).
[Keywords: movement primitives, supervised learning, reinforcment learning, locomotion, phase resetting, learning from demonstration]
[Detail] [BibTeX] [PDF]
Ijspeert, J. A.;Nakanishi, J.;Schaal, S. (2002). Movement imitation with nonlinear dynamical systems in humanoid robots, International Conference on Robotics and Automation (ICRA2002).
[Keywords: movement primitives, behaviors, dynamic systems, computational motor control, attractor landscapes, humanoid robotics, overall best paper award]
[Detail] [BibTeX] [PDF]
Ijspeert, J. A.;Nakanishi, J.;Schaal, S. (2002). Learning rhythmic movements by demonstration using nonlinear oscillators, IEEE International Conference on Intelligent Robots and Systems (IROS 2002), pp.958-963, Piscataway, NJ: IEEE.
[Keywords: movement primitives, behaviors, dynamic systems, computational motor control, attractor landscapes, discrete, rhythmic]
[Detail] [BibTeX] [PDF]
Billard, A.;Schaal, S. (2001). Robust learning of arm trajectories through human demonstration, IEEE International Conference on Intelligent Robots and Systems (IROS 2001), Piscataway, NJ: IEEE.
[Keywords: imitation learning, movement primitives, humanoid robotics]
[Detail] [BibTeX] [PDF]
Ijspeert, A.;Nakanishi, J.;Schaal, S. (2001). Trajectory formation for imitation with nonlinear dynamical systems, IEEE International Conference on Intelligent Robots and Systems (IROS 2001), pp.752-757.
[Keywords: movement primitives
behaviors
dynamic systems
computational motor control
attractor landscapes]
[Detail] [BibTeX] [PDF]
Ijspeert, A. J.;Nakanishi, J.;Shibata, T.;Schaal, S. (2001). Nonlinear dynamical systems for imitation with humanoid robots, Humanoids2001, Second IEEE-RAS International Conference on Humanoid Robots.
[Keywords: movement primitives, behaviors, dynamic systems, computational motor control, attractor landscapes, humanoid robotics]
[Detail] [BibTeX] [PDF]
Kotosaka, S.;Schaal, S. (2001). Synchronized robot drumming by neural oscillator, Journal of the Robotics Society of Japan, 19, 1, pp.116-123.
[Keywords: movement primitives, behaviors, dynamic systems, computational motor control, attractor landscapes, humanoid robotics, drumming, synchronization, best annual paper award of the japanese robotics society]
[Detail] [BibTeX]
Schaal, S.;Vijayakumar, S.;D'Souza, A.;Ijspeert, A.;Nakanishi, J. (2001). Real-time statistical learning for robotics and human augmentation, in: Jarvis, R. A.;Zelinsky, A. (eds.), International Symposium on Robotics Research.
[Keywords: humanoid robotics, statistical learning, movement primitives, real-time learning]
[Detail] [BibTeX] [PDF]
Sternad, D.;Duarte, M.;Katsumata, H.;Schaal, S. (2001). Bouncing a ball: Tuning into dynamic stability, Journal of Experimental Psychology: Human Perception and Performance, 27, 5, pp.1163-1184.
[Keywords: dynamic systems approach, computational motor control, task dynamics, perception-action coupling, movement primitives]
[Detail] [BibTeX]
Sternad, D.;Dean, W. J.;Schaal, S. (2000). Interaction of rhythmic and discrete pattern generators in single joint movements, Human Movement Science, 19, 4, pp.627-665.
[Keywords: movement primitives, behaviors, dynamic systems, computational motor control, movement sequencing, discrete, rhythmic, superposition, phase resetting]
[Detail] [BibTeX] [PDF]
Sternad, D.;Duarte, M.;Katsumata, H.;Schaal, S. (2000). Dynamics of a bouncing ball in human performance, Physical Review E, 63, 011902, pp.1-8.
[Keywords: dynamic systems approach, computational motor control, task dynamics, perception-action coupling, movement primitives, special press comment]
[Detail] [BibTeX] [PDF]
Schaal, S. (1999). Is imitation learning the route to humanoid robots?, Trends in Cognitive Sciences, 3, 6, pp.233-242.
[Keywords: imitation learning, movement primitives, humanoid robotics, review, internal models]
[Detail] [BibTeX] [PDF]
Sternad, D.;Schaal, D. (1999). Segmentation of endpoint trajectories does not imply segmented control, Experimental Brain Research, 124, 1, pp.118-136.
[Keywords: trajectory formation, segementation, movement primitives, motor control, piecewise planarity]
[Detail] [BibTeX] [PDF]
Schaal, S.;Sternad, D. (1998). Programmable pattern generators, 3rd International Conference on Computational Intelligence in Neuroscience, pp.48-51.
[Keywords: movement primitives
behaviors
dynamic systems
computational motor control
attractor landscapes]
[Detail] [BibTeX] [PDF]
Schaal, S. (1997). Learning from demonstration, in: Mozer, M. C.;Jordan, M.;Petsche, T. (eds.), Advances in Neural Information Processing Systems 9, pp.1040-1046, MIT Press.
[Keywords: imitation learning, movement primitives, reinforcement learning, shaping, priming]
[Detail] [BibTeX] [PDF]
Schaal, S.;Sternad, D.;Atkeson, C. G. (1996). One-handed juggling: A dynamical approach to a rhythmic movement task, Journal of Motor Behavior, 28, 2, pp.165-183.
[Keywords: biological motor control, trajectory formation, nonlinear dynamics, task dynamics, movement primitives]
[Detail] [BibTeX] [PDF]