A critical capability for intelligent manufacturing is the ability for robotic systems to understand the spatial operation environment – the ability for robots to precisely recognize and estimate the spatial positions and orientations of objects in industrial scenes. Existing scene reconstruction methods are designed for general settings with low precision needs of objects and abundant data. However, manufacturing hinges on high object precision and operates with limited data. Addressing such challenges and limitations, we propose a novel Learning-based Scene Point-cloud Registration framework for automatic industrial scene reconstruction (iLSPR). The proposed iLSPR framework leverages point cloud representation and integrates three key innovations (i) a Multi-Feature Robust Point Matching Network (MF-RPMN) that learns from both raw data and deep features of the objects to accurately align point clouds, (ii) a Geometric-Primitive-based Data Generation (GPDG) method for efficient synthetic data generation, and (iii) a digital model library of industrial target objects. During operation, vision sensors capture point clouds in the scenes, and the iLSPR method registers high-fidelity object models in the scenes using MF-RPMN, pre-trained with GPDG-generated data. We introduce an Industrial Scene Object Point-cloud Registration (ISOPR) dataset in IsaacSim to benchmark performance. Experimental results demonstrate that iLSPR significantly outperforms existing methods in accuracy and robustness. We further validate the approach on a real-world robotic manufacturing system, demonstrating reliable digital reconstruction of industrial scenes.
Overview of the proposed iLSPR method for precision industrial scene reconstruction and robotic manufacturing. The robotic manufacturing system consists of a robotic manipulator, a workpiece (object), a platform, and an RGB-D camera. While working, the raw point cloud of the scene is captured by an RGB-D camera, and the partial point cloud of the object is segmented from it by a predefined bounding box. Next, iLSPR selects and registers the object’s ground-truth model in the scene. The scene is then reconstructed and can be used as a spatial guide for the robotic manufacturing system.
Process of the data collection process for the ISOPR dataset in IsaacSim.
Example CAD models in the proposed ISOPR digital model library, and example geometric primitives in the proposed iLSPR framework.
We invite you to explore the paper, dive into the dataset, and build on this work toward more intelligent, precise, and data-efficient robotic manufacturing systems!
