ICRA 2019 Workshop


Bringing perception-based manipulation to the real world: standardizing robot manipulation learning


Manipulation is the key skill to a ffect, interact with and learn from the environment. In recent years we have seen many successful learning approaches in a variety of manipulation tasks, relying on di fferent robotic arms, ende ffectors, perception systems and computational resources. However, the heterogeneity of these setups is hindering researchers from focusing their e fforts on the key open issues and is preventing them from performing objective comparisons. This workshop aims to specify a robot manipulation platform that is capable of addressing the most relevant open problems in robot manipulation learning. This field consists of countless subproblems and defining a common platform for them is a significant challenge in its own right. However, it is a first step towards having a basis for comparable research in the field of perception-based manipulation learning. It will involve participants from academia and industry in order to jointly defi ne these problems and the necessary specications that a prototypical robotic setup requires in order to solve them.


Invited Speakers


This workshop aims for the definition of a benchmark platform to evaluate perception-based robot manipulation learning in a standardized manner. As a community-driven effort, we will send an call for platforms open to any hardware provider or robotics developers who would like to contribute. The call will specify the requirements of the platform as defined by the organizers of the workshop. The tentative timeline until the workshop is the following:

  • 1st of March: Internal feedback and suggestions for general framework for benchmark platform
  • 10th of March: Internal feedback and suggestions for concrete experimental setups for benchmark tasks
  • 15th of March: Open call for platforms
  • 30th of April: Deadline for platform submissions
  • 7th of May: Submission reviews and notifications
  • 20th – 24th of May: ICRA, exact workshop day is TBD



Robot manipulation learning has become one of the key topics in robotics research. Researchers mostly focus on algorithmic aspects of robot learning, which are then evaluated in very specific, maybe even unique and nowhere else accessible robotic platforms. Although the dependency on specific robot hardware has been neglected, its influence on results should not be understated. At the same time, this hardware dependency prevents researchers from performing objective, unbiased and effortless comparisons between algorithms. This workshop aims for the definition of an \emph{accessible and high-performance robotic setup} capable of addressing the most relevant open problems in robot manipulation learning and that, at the same time, serves as a benchmarking platform. To this end, renowned experts in the field will share their experience with different hardware platforms, including manipulators, end-effectors, tactile/force sensors, vision system and computational resources, and their interplay and dependencies with robot learning algorithms. Instrumental for the workshop's goal is the final discussion, which will first jointly define the most relevant set of problems and their respective performance measures followed by a second session targeting specific hardware specifications. This session will be complemented with the opinion of invited hardware system manufacturers, that will provide their own view on the problem and the feasibility of potential platforms based on currently available products.

9:00-9:10Introduction by the organizers
9:10-9:30Control and skills (Oussamma Khatib)
9:30-9:50Control and learning (Sami Haddadin)
9:50-10:10Learning by demonstration (Aude Billard)
10:10-10:30Industrial poster spotlight session
10:30-11:00Coffee break and poster session
11:00-11:20Learning by demonstration (Tamim Asfour)
11:20-11:40Planning and learning (Marc Toussaint)
11:40-12:00Reinforcement learning (Jan Peters)
12:00-12:20Manipulation (Oliver Brock)
12:30-13:30Lunch break
13:30-13:50Deep learning (Patrick van der Smagt)
14:10-14:30Vision and learning (Dieter Fox)
14:30-15:00Vision and learning (Wolfram Burgard)
15:00-15:30Coffee break and poster session
15:30-16:00Vision and learning (Greg Hager)
16:00-17:00Discussion panel: Problems and benchmarks
17:00-18:00Discussion panel: Platform specifications

Plan to solicit participation

We will advertise the workshop in the robotics mailing lists the mailing lists of the mentioned technical committees. In addition, we will also contact key industrial players to join the workshop, participate in the discussion panel and expose their solutions in the industrial spotlight session and the poster session during the coffee breaks. Participants will be able to complement and support the discussion session with the presented results.

Plan to encourage interaction among participants

The workshop's main goal is the final discussion panel to figure out how a prototypical setup for robot manipulation learning should be, what should it do and how robot manipulation learning should be benchmarked. The impact of the workshop's outcome will only be significant with the input and support of the whole robot learning community. This includes not only the knowledge and experience of world-renowned experts but also the hands-on experience of early career researchers who spend days directly interacting with the hardware and hardware manufacturers. The planned poster sessions and poster preview session will be the first step to encourage the interaction between these two groups. The final discussion session will encourage an open and fruitful discussion among all participants. The two one hour sessions will specifically provide slots for experts, early career researchers and industrial speakers in order to consider all necessary opinions for a real community-driven outcome.


This workshop will publish, if speakers agree, all presented slides on the workshop's website. The outcome of the discussion session of the workshop will be summarized in a document and published on the workshop's website. Additionally, the workshop organizers will aim for a joint publication with all invited speakers where the resulting chosen problems and the platform specifications and benchmarks are formally described. This will serve as a reference sheet to deeply understand and also fully exploit the capabilities of the platform. After the workshop, the organizers will host a community-maintained website to keep track of all experiments, publicly available code, benchmarks and publications done in the chosen prototypical setup.