O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning

• DOI: 10.48550/arxiv.2106.15087
• Publication Date: 2021-01-01
• Authors: Mo, Kaichun; Qin, Yuzhe; Xiang, Fanbo; Su, Hao; Guibas, Leonidas
• Keywords: FOS: Computer and information sciences; Computer Science - Robotics; 0209 industrial biotechnology; Computer Vision and Pattern Recognition (cs.CV); Computer Science - Computer Vision and Pattern Recognition; 02 engineering and technology; Robotics (cs.RO)

Contrary to the vast literature in modeling, perceiving, and understanding agent-object (e.g., human-object, hand-object, robot-object) interaction in computer vision and robotics, very few past works have studied the task of object-object interaction, which also plays an important role in robotic manipulation and planning tasks. There is a rich space of object-object interaction scenarios in our daily life, such as placing an object on a messy tabletop, fitting an object inside a drawer, pushing an object using a tool, etc. In this paper, we propose a unified affordance learning framework to learn object-object interaction for various tasks. By constructing four object-object interaction task environments using physical simulation (SAPIEN) and thousands of ShapeNet models with rich geometric diversity, we are able to conduct large-scale object-object affordance learning without the need for human annotations or demonstrations. At the core of technical contribution, we propose an object-kernel point convolution network to reason about detailed interaction between two objects. Experiments on large-scale synthetic data and real-world data prove the effectiveness of the proposed approach. Please refer to the project webpage for code, data, video, and more materials: https://cs.stanford.edu/~kaichun/o2oafford<br/>to appear in CoRL 2021<br/>