STOPNet: Multiview-based 6-DoF Suction Detection for Transparent Objects on Production Lines

Yuxuan Kuang1,2*         Qin Han1*         Danshi Li2,3         Qiyu Dai1
Lian Ding4         Dong Sun4         Hanlin Zhao4         He Wang1†
1CFCS, School of Computer Science, Peking University          
2Galbot           3New York University          
4Huawei Cloud Computing Technologies Co., Ltd. ICRA 2024
Paper Video Code
pipeline

Overview of our production line real robot setup and our proposed STOPNet. Taking multi-timestep RGB input from two cameras, our framework conducts 3D scene reconstruction and suction detection in real time. Our model is trained on a large-scale synthetic dataset but it can also generalize to real-world objects and environments effectively.

Abstract

In this work, we present STOPNet, a framework for 6-DoF object suction detection on production lines, with a focus on but not limited to transparent objects, which is an important and challenging problem in robotic systems and modern industry. Current methods requiring depth input fail on transparent objects due to depth cameras' deficiency in sensing their geometry, while we proposed a novel framework to reconstruct the scene on the production line depending only on RGB input, based on multiview stereo. Compared to existing works, our method not only reconstructs the whole 3D scene in order to obtain high-quality 6-DoF suction poses in real time but also generalizes to novel environments, novel arrangements and novel objects, including challenging transparent objects, both in simulation and the real world. Extensive experiments in simulation and the real world show that our method significantly surpasses the baselines and has better generalizability, which caters to practical industrial needs.



Video

BibTeX

@inproceedings{kuang2023stopnet,
      title={STOPNet: Multiview-based 6-DoF Suction Detection for Transparent Objects on Production Lines},
      author={Kuang, Yuxuan and Han, Qin and Li, Danshi and Dai, Qiyu and Ding, Lian and Sun, Dong and Zhao, Hanlin and Wang, He},
      booktitle={2024 IEEE International Conference on Robotics and Automation (ICRA)},
      year={2024},
      organization={IEEE}
}