From picking up a cube to writing with a pen – learning dexterous manipulation skills on real-world robotic systems

Tübingen – On August 3, the Max Planck Institute for Intelligent Systems (MPI-IS) in Tübingen, Germany, together with Mila and New York University will launch the first global real-world robotics manipulation challenge in the cloud. The robotics challenge aims to democratize research by giving researchers around the globe access to advanced robotic systems. Until now, this access has generally been limited to top academic and industrial labs in the US, Europe, and Japan.

“We hope that this competition will lead to a more inclusive robotic learning community, foster exchange across academic and industry labs, and spark cooperation,” says Stefan Bauer, a research group leader in the Empirical Inference Department at the MPI-IS. “But we also want to create a benchmark to know what algorithm works best. We can collect much more data than what is possible in individual labs, as we have standardized tasks on a standardized platform.” Manuel Wüthrich, who is also part of the challenge’s organizing team, adds: “Since the robotic systems run around the clock, they generate enough data to actually use deep learning.”

The robotic platform resembles a hand with three fingers. It is surrounded by three cameras that provide images to measure the results. Several identical robotic platforms have been set up around the robotic lab at the MPI-IS in Tübingen to run the learning algorithms the participating teams submit. Participants can execute their algorithms remotely thanks to a convenient and simple interface. Each team will have a budget of more than 100 real robot hours on one of the high-performing robotic platforms, without any restrictions on the algorithms they execute.

The challenge comprises three stages. First, participants familiarize themselves with the overall setup and aim to solve a task in a physical simulator. Next, they solve simple real-world tasks: specifically, they are required to train the robot to pick up an object and perform in-hand manipulation with a cube. In the final stage, the robot has to pick up a pen and write a short phrase like “I won!”.

To achieve these tasks, participants need to design advanced control or reinforcement learning algorithms. “We want to see the extent to which the teams are able to tune their algorithms to fulfill these very difficult tasks. Using a learning-based algorithm for writing with a pen can probably be considered a breakthrough in robotic manipulation,” says Wüthrich.

“For the first time, we can benchmark all these algorithms on real robots, making them comparable,” says Bauer. “Before, this kind of benchmarking was done mainly in simulation. There is no noise or friction, and contacts are simplified, which makes very impressive results in simulation possible. However, these insights do not typically translate to real robots, in particular for tasks that involve contact and complex interactions with the environment.” Wüthrich adds: “Now, a relatively large number of people who previously didn’t have access to such an advanced robotic platform are able to participate. While some top-notch universities may have extraordinarily good and advanced robots, for most researchers, it is very difficult to gain access to real robotic systems. It is very costly in terms of time and money. Our goal is thus to give a greater number of people  access and an opportunity  to achieve impressive manipulation results.”

The Real Robotic Challenge will run for 16 weeks. Ultimately, it will help advance the state of the art in dexterous manipulation, which has myriad applications. These include, for instance, construction, cleaning, patient and elder care, agriculture, and product assembly.

Twitter: @robo_challenge #RoboChallenge

https://real-robot-challenge.com/

The robot hand lifts an object and twists it around.

Several objects make this task more difficult.

Grabbing a pen is one thing....

....but actually writing with it quite another.

Several completely identical setups are waiting for algorithms to run on them.