Topology based motion synthesis for dextrous manipulation.

The goal of TOMSY is to enable a generational leap in the techniques and scalability of motion synthesis systems. Motion synthesis is a key component of future robotic and cognitive systems to enable their physical interaction with humans and physical manipulation of their environment. Existing motion synthesis algorithms are severely limited in their ability to cope with real-world objects such as flexible objects or objects with many degrees of freedom. TOMSY propose to solve these problems by learning and exploiting appropriate topological representations and testing them on challenging domains of flexible, multi-object manipulation and close contact robot control and computer animation. The results of this project provide the necessary key technologies for future robots and computer vision systems to enable fluent interaction with their environment.

Project Schematics

The following figure shows the principal components of the system and how these are divided into different work packages.

TOMSY work packages

Project Details

  • Beginning date: 01 April 2011
  • Ending date: 31 March 2014
  • Funding: European Comission, FP7-ICT-2009-6

My Role 

I researched how depth maps (disparity maps) could be segmented in robotics so that a) object position could be retrieved b) segmented data would be driving object recognition and c) grasping vector could be calculated in order to manipulate objects of interest. During my stay at KTH I colaborated with Dr. Mårten Björkman. During this project I published the following scientific paper:

Segmentation and Object Recognition Results

Following figure shows results for the segmentation algorithm that I deviced while working on this project. The idea is to segment the image into objects after which another algorithm is used to detect the actual object based on both 2D and 3D features, after which the object can be manipulated by a robot arm. 

kth bk

Typically segmentation algorithms struggle to segment correctly objects that are very close or touching each other, as is the case in the above image. As it can be seen, the algorithm has correctly segmented the parts of the objects. Next step would be deducing relationships between various parts of the objects. 

Jarno Ralli
Author: Jarno Ralli
Jarno Ralli is a computer vision scientist and a programmer.

Let's get social!

FacebookTwitterGoogle BookmarksLinkedIn