Learning to Infer Kinematic Hierarchies for Novel Object Instances

Conference Paper

Manipulating an articulated object requires perceiving its kinematic hierarchy: its parts, how each can move, and how those motions are coupled. Previous work has explored perception for kinematics, but none infers a complete kinematic hierarchy on never-before-seen object instances, without relying on a schema or template. We present a novel perception system that achieves this goal. Our system infers the moving parts of an object and the kinematic couplings that relate them. To infer parts, it uses a point cloud instance segmentation neural network and to infer kinematic hierarchies, it uses a graph neural network to predict the existence, direction, and type of edges (i.e. joints) that relate the inferred parts. We train these networks using simulated scans of synthetic 3D models. We evaluate our system on simulated scans of 3D objects, and we demonstrate a proof-of-concept use of our system to drive real-world robotic manipulation.

Full Text

Duke Authors

Cited Authors

  • Abdul-Rashid, H; Freeman, M; Abbatematteo, B; Konidaris, G; Ritchie, D

Published Date

  • January 1, 2022

Published In

Start / End Page

  • 8461 - 8467

International Standard Serial Number (ISSN)

  • 1050-4729

International Standard Book Number 13 (ISBN-13)

  • 9781728196817

Digital Object Identifier (DOI)

  • 10.1109/ICRA46639.2022.9811968

Citation Source

  • Scopus