Research

14 pages, 5830 KiB

Open AccessArticle

Grasping of Solid Industrial Objects Using 3D Registration

by Monica Sileo, Domenico Daniele Bloisi and Francesco Pierri

Machines 2023, 11(3), 396; https://doi.org/10.3390/machines11030396 - 17 Mar 2023

Cited by 1 | Viewed by 1195

Robots allow industrial manufacturers to speed up production and to increase the product’s quality. This paper deals with the grasping of partially known industrial objects in an unstructured environment. The proposed approach consists of two main steps: (1) the generation of an object [...] Read more.

Robots allow industrial manufacturers to speed up production and to increase the product’s quality. This paper deals with the grasping of partially known industrial objects in an unstructured environment. The proposed approach consists of two main steps: (1) the generation of an object model, using multiple point clouds acquired by a depth camera from different points of view; (2) the alignment of the generated model with the current view of the object in order to detect the grasping pose. More specifically, the model is obtained by merging different point clouds with a registration procedure based on the iterative closest point (ICP) algorithm. Then, a grasping pose is placed on the model. Such a procedure only needs to be executed once, and it works even in the presence of objects only partially known or when a CAD model is not available. Finally, the current object view is aligned to the model and the final grasping pose is estimated. Quantitative experiments using a robot manipulator and three different real-world industrial objects were conducted to demonstrate the effectiveness of the proposed approach. Full article

(This article belongs to the Special Issue Computer/Machine Vision Applications in Automation, Robotic, Mechatronic Systems)

► Show Figures

Figure 1

19 pages, 3349 KiB

Open AccessArticle

Salient Preprocessing: Robotic ICP Pose Estimation Based on SIFT Features

by Lihe Hu, Yi Zhang, Yang Wang, Gengyu Ge and Wei Wang

Machines 2023, 11(2), 157; https://doi.org/10.3390/machines11020157 - 23 Jan 2023

Cited by 3 | Viewed by 1559

Abstract

The pose estimation can be effectively solved according to the feature point matching relationship in RGB-D. However, the extraction and matching process based on the whole image’s feature point is very computationally intensive and lacks robustness, which is the bottleneck of the traditional [...] Read more.

The pose estimation can be effectively solved according to the feature point matching relationship in RGB-D. However, the extraction and matching process based on the whole image’s feature point is very computationally intensive and lacks robustness, which is the bottleneck of the traditional ICP algorithm. This paper proposes representing the whole image’s feature points by the salient objects’ robustness SIFT feature points through the salient preprocessing, and further solving the pose estimation. The steps are as follows: (1) salient preprocessing; (2) salient object’s SIFT feature extraction and matching; (3) RANSAC removes mismatching salient feature points; (4) ICP pose estimation. This paper proposes salient preprocessing aided by RANSAC processing based on the SIFT feature for pose estimation for the first time, which is a coarse-to-fine method. The experimental results show that our salient preprocessing algorithm can coarsely reduce the feature points’ extractable range and interfere. Furthermore, the results are processed by RANSAC good optimization, reducing the calculation amount in the feature points’ extraction process and improving the matching quality of the point pairs. Finally, the calculation amount of solving R, t based on all the matching feature points is reduced and provides a new idea for related research. Full article

(This article belongs to the Special Issue Computer/Machine Vision Applications in Automation, Robotic, Mechatronic Systems)

► Show Figures

Figure 1

19 pages, 2893 KiB

Open AccessArticle

A Surface Defect Inspection Model via Rich Feature Extraction and Residual-Based Progressive Integration CNN

by Guizhong Fu, Wenwu Le, Zengguang Zhang, Jinbin Li, Qixin Zhu, Fuzhou Niu, Hao Chen, Fangyuan Sun and Yehu Shen

Machines 2023, 11(1), 124; https://doi.org/10.3390/machines11010124 - 16 Jan 2023

Cited by 3 | Viewed by 1943

Abstract

Surface defect inspection is vital for the quality control of products and the fault diagnosis of equipment. Defect inspection remains challenging due to the low level of automation in some manufacturing plants and the difficulty in identifying defects. To improve the automation and [...] Read more.

Surface defect inspection is vital for the quality control of products and the fault diagnosis of equipment. Defect inspection remains challenging due to the low level of automation in some manufacturing plants and the difficulty in identifying defects. To improve the automation and intelligence levels of defect inspection, a CNN model is proposed for the high-precision defect inspection of USB components in the actual demands of factories. First, the defect inspection system was built, and a dataset named USB-SG, which contained five types of defects—dents, scratches, spots, stains, and normal—was established. The pixel-level defect ground-truth annotations were manually marked. This paper puts forward a CNN model for solving the problem of defect inspection tasks, and three strategies are proposed to improve the model’s performance. The proposed model is built based on the lightweight SqueezeNet network, and a rich feature extraction block is designed to capture semantic and detailed information. Residual-based progressive feature integration is proposed to fuse the extracted features, which can reduce the difficulty of model fine-tuning and improve the generalization ability. Finally, a multi-step deep supervision scheme is proposed to supervise the feature integration process. The experiments on the USB-SG dataset prove that the model proposed in this paper has better performance than that of other methods, and the running speed can meet the real-time demand, which has broad application prospects in the industrial inspection scene. Full article

(This article belongs to the Special Issue Computer/Machine Vision Applications in Automation, Robotic, Mechatronic Systems)

► Show Figures

Figure 1

23 pages, 5071 KiB

Open AccessArticle

Loop Closure Detection for Mobile Robot based on Multidimensional Image Feature Fusion

by Jinming Li, Peng Wang, Cui Ni, Dong Zhang and Weilong Hao

Machines 2023, 11(1), 16; https://doi.org/10.3390/machines11010016 - 23 Dec 2022

Cited by 1 | Viewed by 2341

Abstract

Loop closure detection is a crucial part of VSLAM. However, the traditional loop closure detection algorithms are difficult to adapt to complex and changeable scenes. In this paper, we fuse Gist features, semantic features and appearance features of the image to detect the [...] Read more.

Loop closure detection is a crucial part of VSLAM. However, the traditional loop closure detection algorithms are difficult to adapt to complex and changeable scenes. In this paper, we fuse Gist features, semantic features and appearance features of the image to detect the loop closures quickly and accurately. Firstly, we take advantage of the fast extraction speed of the Gist feature by using it to screen the loop closure candidate frames. Then, the current frame and the candidate frame are semantically segmented to obtain the mask blocks of various types of objects, and the semantic nodes are constructed to calculate the semantic similarity between them. Next, the appearance similarity between the images is calculated according to the shape of the mask blocks. Finally, based on Gist similarity, semantic similarity and appearance similarity, the image similarity calculation model can be built as the basis for loop closure detection. Experiments are carried out on both public and self-filmed datasets. The results show that our proposed algorithm can detect the loop closure in the scene quickly and accurately when the illumination, viewpoint and object change. Full article

(This article belongs to the Special Issue Computer/Machine Vision Applications in Automation, Robotic, Mechatronic Systems)

► Show Figures

Figure 1

14 pages, 5876 KiB

Open AccessArticle

Semantic Segmentation for Point Clouds via Semantic-Based Local Aggregation and Multi-Scale Global Pyramid

by Shipeng Cao, Huaici Zhao and Pengfei Liu

Machines 2023, 11(1), 11; https://doi.org/10.3390/machines11010011 - 22 Dec 2022

Cited by 2 | Viewed by 1476

Abstract

Recently, point-based networks have begun to prevail because they retain more original geometric information from point clouds than other deep learning-based methods. However, we observe that: (1) the set abstraction design for local aggregation in point-based networks neglects that the points in a [...] Read more.

Recently, point-based networks have begun to prevail because they retain more original geometric information from point clouds than other deep learning-based methods. However, we observe that: (1) the set abstraction design for local aggregation in point-based networks neglects that the points in a local region may belong to different semantic categories, and (2) most works focus on single-scale local features while ignoring the importance of multi-scale global features. To tackle the above issues, we propose two novel strategies named semantic-based local aggregation (SLA) and multi-scale global pyramid (MGP). The key idea of SLA is to augment local features based on the semantic similarity of neighboring points in the local region. Additionally, we propose a hierarchical global aggregation (HGA) module to extend local feature aggregation to global feature aggregation. Based on HGA, we introduce MGP to obtain discriminative multi-scale global features from multi-resolution point cloud scenes. Extensive experiments on two prevailing benchmarks, S3DIS and Semantic3D, demonstrate the effectiveness of our method. Full article

(This article belongs to the Special Issue Computer/Machine Vision Applications in Automation, Robotic, Mechatronic Systems)

► Show Figures

Figure 1

13 pages, 7515 KiB

Open AccessArticle

Cross-Attention-Based Reflection-Aware 6D Pose Estimation Network for Non-Lambertian Objects from RGB Images

by Chenrui Wu, Long Chen and Shiqing Wu

Machines 2022, 10(12), 1107; https://doi.org/10.3390/machines10121107 - 22 Nov 2022

Cited by 2 | Viewed by 1199

Abstract

Six-dimensional pose estimation for non-Lambertian objects, such as metal parts, is essential in intelligent manufacturing. Current methods pay much less attention to the influence of the surface reflection problem in 6D pose estimation. In this paper, we propose a cross-attention-based reflection-aware 6D pose [...] Read more.

Six-dimensional pose estimation for non-Lambertian objects, such as metal parts, is essential in intelligent manufacturing. Current methods pay much less attention to the influence of the surface reflection problem in 6D pose estimation. In this paper, we propose a cross-attention-based reflection-aware 6D pose estimation network (CAR6D) for solving the surface reflection problem in 6D pose estimation. We use a pseudo-Siamese network structure to extract features from both an RGB image and a 3D model. The cross-attention layers are designed as a bi-directional filter for each of the inputs (the RGB image and 3D model) to focus on calculating the correspondences of the objects. The network is trained to segment the reflection area from the object area. Training images with ground-truth labels of the reflection area are generated with a physical-based rendering method. The experimental results on a 6D dataset of metal parts demonstrate the superiority of CAR6D in comparison with other state-of-the-art models. Full article

(This article belongs to the Special Issue Computer/Machine Vision Applications in Automation, Robotic, Mechatronic Systems)

► Show Figures

Figure 1

20 pages, 8493 KiB

Open AccessArticle

Robust Object Positioning for Visual Robotics in Automatic Assembly Line under Data-Scarce Environments

by Yigong Zhang, Huadong Song, Xiaoting Guo and Chaoqing Tang

Machines 2022, 10(11), 1079; https://doi.org/10.3390/machines10111079 - 16 Nov 2022

Cited by 1 | Viewed by 1362

Abstract

Object positioning is a basic need for visual robotics in automatic assembly lines. An assembly line requires fast transfer to new object positioning tasks with few or no training data for deep learning algorithms, and the captured visual images usually suffer from partial [...] Read more.

Object positioning is a basic need for visual robotics in automatic assembly lines. An assembly line requires fast transfer to new object positioning tasks with few or no training data for deep learning algorithms, and the captured visual images usually suffer from partial missing and cropping and environmental lighting interference. These features call for efficient and robust arbitrary shape positioning algorithms under data-scarce and shape distortion cases. To this end, this paper proposes the Random Verify Generalised Hough Transform (RV-GHT). The RV-GHT builds a much more concise shape dictionary than traditional GHT methods with just a single training image. The location, orientation, and scaling of multiple target objects are given simultaneously during positioning. Experiments were carried out on a dataset in an automatic assembly line with real shape distortions, and the performance was improved greatly compared to the state-of-the art methods. Although the RV-GHT was initially designed for vision robotics in an automatic assembly line, it works for other object positioning mechatronics systems, which can be modelled as shape distortion on a standard reference object. Full article

(This article belongs to the Special Issue Computer/Machine Vision Applications in Automation, Robotic, Mechatronic Systems)

► Show Figures

Figure 1

13 pages, 2861 KiB

Open AccessArticle

A Lightweight Aerial Power Line Segmentation Algorithm Based on Attention Mechanism

by Gujing Han, Min Zhang, Qiang Li, Xia Liu, Tao Li, Liu Zhao, Kaipei Liu and Liang Qin

Machines 2022, 10(10), 881; https://doi.org/10.3390/machines10100881 - 01 Oct 2022

Cited by 5 | Viewed by 1522

Abstract

Power line segmentation is very important to ensure the safe and stable operation of unmanned aerial vehicles in intelligent power line inspection. Although the power line segmentation algorithm based on deep learning has made some progress, it is still quite difficult to achieve [...] Read more.

Power line segmentation is very important to ensure the safe and stable operation of unmanned aerial vehicles in intelligent power line inspection. Although the power line segmentation algorithm based on deep learning has made some progress, it is still quite difficult to achieve accurate power line segmentation due to the complex and changeable background of aerial power line images and the small power line targets, and the existing segmentation models is too large and not suitable for edge deployment. This paper proposes a lightweight power line segmentation algorithm—G-UNets. The algorithm uses the improved U-Net of Lei Yang et al. (2022) as the basic network (Y-UNet). The encoder part combines traditional convolution with Ghost bottleneck to extract features and adopts a multi-scale input fusion strategy to reduce information loss. While ensuring the segmentation accuracy, the amount of Y-UNet parameters is significantly reduced; Shuffle Attention (SA) with fewer parameters is introduced in the decoding stage to improve the model segmentation accuracy; at the same time, in order to further alleviate the impact of the imbalanced distribution of positive and negative samples on the segmentation accuracy, a weighted hybrid loss function fused with Focal loss and Dice loss is constructed. The experimental results show that the number of parameters of the G-UNets algorithm is only about 26.55% of that of Y-UNet, and the F1-Score and IoU values both surpass those of Y-UNet, reaching 89.24% and 82.98%, respectively. G-UNets can greatly reduce the number of network parameters while ensuring the accuracy of the model, providing an effective way for the power line segmentation algorithm to be applied to resource-constrained edge devices such as drones. Full article

(This article belongs to the Special Issue Computer/Machine Vision Applications in Automation, Robotic, Mechatronic Systems)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Computer/Machine Vision Applications in Automation, Robotic, Mechatronic Systems

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (8 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI