iCLAP: shape recognition by combining proprioception and touch sensing

Shan Luo*, Wenxuan Mou, Kaspar Althoefer, Hongbin Liu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

25 Citations (Scopus)


For humans, both the proprioception and touch sensing are highly utilized when performing haptic perception. However, most approaches in robotics use only either proprioceptive data or touch data in haptic object recognition. In this paper, we present a novel method named Iterative Closest Labeled Point (iCLAP) to link the kinesthetic cues and tactile patterns fundamentally and also introduce its extensions to recognize object shapes. In the training phase, the iCLAP first clusters the features of tactile readings into a codebook and assigns these features with distinct label numbers. A 4D point cloud of the object is then formed by taking the label numbers of the tactile features as an additional dimension to the 3D sensor positions; hence, the two sensing modalities are merged to achieve a synthesized perception of the touched object. Furthermore, we developed and validated hybrid fusion strategies, product based and weighted sum based, to combine decisions obtained from iCLAP and single sensing modalities. Extensive experimentation demonstrates a dramatic improvement of object recognition using the proposed methods and it shows great potential to enhance robot perception ability.

Original languageEnglish
Number of pages12
JournalAutonomous Robots
Early online date23 Jun 2018
Publication statusE-pub ahead of print - 23 Jun 2018


  • Data fusion
  • Object recognition
  • Robot systems
  • Tactile sensing


Dive into the research topics of 'iCLAP: shape recognition by combining proprioception and touch sensing'. Together they form a unique fingerprint.

Cite this