Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
6.8
3.9
Journals
Sensors
Special Issues
Sensing and Processing for 3D Computer Vision: 2nd Edition
sensors-logo
Submit to Sensors Review for Sensors Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Sensing and Processing for 3D Computer Vision: 2nd Edition

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Sensing and Imaging".

Deadline for manuscript submissions: closed (30 November 2023) | Viewed by 26996

Special Issue Editor

Prof. Dr. Denis Laurendeau

E-Mail Website
Guest Editor
Computer Vision and Systems Laboratory, Laval University, 1665 Rue de l’Universite, Universite Laval, Quebec City, QC G1V 0A6, Canada
Interests: 3D sensors; active vision; 3D image processing and understanding; modelling; geometry; 3D sensing and modelling for augmented and virtual reality; applications of 3D computer vision
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

This Special Issue is targeting the submission of research articles on 3D sensing technology in addition to the use of advanced 3D sensors in computer vision. Original contributions on novel active 3D sensors, stereo reconstruction approaches and sensor calibration techniques are solicited. Articles on 3D point cloud/mesh processing, geometric modeling, shape representation and recognition are also of interest for this Special Issue. Articles on the application of 3D sensing and modeling to metrology, industrial inspection and quality control, augmented/virtual reality, heritage preservation, arts and other fields are also welcome.

Prof. Dr. Denis Laurendeau
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • active/passive 3D sensors
  • sensor calibration
  • stereo reconstruction
  • point cloud/mesh processing
  • geometry
  • modeling and representation
  • shape analysis and recognition
  • applications of 3D vision

Related Special Issues

Published Papers (26 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

Jump to: Other

16 pages, 9434 KiB  
Article
Omnidirectional-Sensor-System-Based Texture Noise Correction in Large-Scale 3D Reconstruction
by Wenya Xie and Xiaoping Hong
Sensors 2024, 24(1), 78; https://doi.org/10.3390/s24010078 - 22 Dec 2023
Viewed by 581
Abstract
The evolution of cameras and LiDAR has propelled the techniques and applications of three-dimensional (3D) reconstruction. However, due to inherent sensor limitations and environmental interference, the reconstruction process often entails significant texture noise, such as specular highlight, color inconsistency, and object occlusion. Traditional [...] Read more.
The evolution of cameras and LiDAR has propelled the techniques and applications of three-dimensional (3D) reconstruction. However, due to inherent sensor limitations and environmental interference, the reconstruction process often entails significant texture noise, such as specular highlight, color inconsistency, and object occlusion. Traditional methodologies grapple to mitigate such noise, particularly in large-scale scenes, due to the voluminous data produced by imaging sensors. In response, this paper introduces an omnidirectional-sensor-system-based texture noise correction framework for large-scale scenes, which consists of three parts. Initially, we obtain a colored point cloud with luminance value through LiDAR points and RGB images organization. Next, we apply a voxel hashing algorithm during the geometry reconstruction to accelerate the computation speed and save the computer memory. Finally, we propose the key innovation of our paper, the frame-voting rendering and the neighbor-aided rendering mechanisms, which effectively eliminates the aforementioned texture noise. From the experimental results, the processing rate of one million points per second shows its real-time applicability, and the output figures of texture optimization exhibit a significant reduction in texture noise. These results indicate that our framework has advanced performance in correcting multiple texture noise in large-scale 3D reconstruction. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

18 pages, 24207 KiB  
Article
Unsupervised Stereo Matching with Surface Normal Assistance for Indoor Depth Estimation
by Xiule Fan, Ali Jahani Amiri, Baris Fidan and Soo Jeon
Sensors 2023, 23(24), 9850; https://doi.org/10.3390/s23249850 - 15 Dec 2023
Viewed by 780
Abstract
To obtain more accurate depth information with stereo cameras, various learning-based stereo-matching algorithms have been developed recently. These algorithms, however, are significantly affected by textureless regions in indoor applications. To address this problem, we propose a new deep-neural-network-based data-driven stereo-matching scheme that utilizes [...] Read more.
To obtain more accurate depth information with stereo cameras, various learning-based stereo-matching algorithms have been developed recently. These algorithms, however, are significantly affected by textureless regions in indoor applications. To address this problem, we propose a new deep-neural-network-based data-driven stereo-matching scheme that utilizes the surface normal. The proposed scheme includes a neural network and a two-stage training strategy. The neural network involves a feature-extraction module, a normal-estimation branch, and a disparity-estimation branch. The training processes of the feature-extraction module and the normal-estimation branch are supervised while the training of the disparity-estimation branch is performed unsupervised. Experimental results indicate that the proposed scheme is capable of estimating the surface normal accurately in textureless regions, leading to improvement in the disparity-estimation accuracy and stereo-matching quality in indoor applications involving such textureless regions. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

23 pages, 23651 KiB  
Article
Autonomous Trajectory Planning for Spray Painting on Complex Surfaces Based on a Point Cloud Model
by Saul Nieto Bastida and Chyi-Yeu Lin
Sensors 2023, 23(24), 9634; https://doi.org/10.3390/s23249634 - 05 Dec 2023
Cited by 2 | Viewed by 1073
Abstract
Using teach pendants or offline programming methods can generate tool paths for robot manipulators to carry out production activities, such as spray painting on objects of different geometries. This task, in which the complexity of painting the surface is one of the main [...] Read more.
Using teach pendants or offline programming methods can generate tool paths for robot manipulators to carry out production activities, such as spray painting on objects of different geometries. This task, in which the complexity of painting the surface is one of the main challenges, requires highly skilled operators. In addition, the time spent setting up a robot task can be justified for the mass production of the same workpiece. However, it is inconvenient for low-production and high-variation production lines. In order to overcome these challenges, this study presents an algorithm to autonomously generate robot trajectories for a spray-painting process applied to objects with complex surfaces based on input 3D point cloud data. A predefined spherical mesh wraps the object, organizing the geometrical attributes into a structured data set. Subsequently, the region of interest is extracted and isolated from the model, which serves as the basis for the automatic path-planning operation. A user-friendly graphical user interface (GUI) is developed to define input parameters, visualize the point cloud model and the generated trajectory, simulate paint quality using a color map, and ultimately generate the robot’s code. A 3D sensor is used to localize the pose of the workpiece ahead of the robot and adjust the robot’s trajectory. The efficacy of the proposed approach is validated first by using various workpieces within a simulated environment and second by employing a real robot to execute the motion task. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

20 pages, 7391 KiB  
Article
FGCN: Image-Fused Point Cloud Semantic Segmentation with Fusion Graph Convolutional Network
by Kun Zhang, Rui Chen, Zidong Peng, Yawei Zhu and Xiaohong Wang
Sensors 2023, 23(19), 8338; https://doi.org/10.3390/s23198338 - 09 Oct 2023
Viewed by 1143
Abstract
In interpreting a scene for numerous applications, including autonomous driving and robotic navigation, semantic segmentation is crucial. Compared to single-modal data, multi-modal data allow us to extract a richer set of features, which is the benefit of improving segmentation accuracy and effect. We [...] Read more.
In interpreting a scene for numerous applications, including autonomous driving and robotic navigation, semantic segmentation is crucial. Compared to single-modal data, multi-modal data allow us to extract a richer set of features, which is the benefit of improving segmentation accuracy and effect. We propose a point cloud semantic segmentation method, and a fusion graph convolutional network (FGCN) which extracts the semantic information of each point involved in the two-modal data of images and point clouds. The two-channel k-nearest neighbors (KNN) module of the FGCN was created to address the issue of the feature extraction’s poor efficiency by utilizing picture data. Notably, the FGCN utilizes the spatial attention mechanism to better distinguish more important features and fuses multi-scale features to enhance the generalization capability of the network and increase the accuracy of the semantic segmentation. In the experiment, a self-made semantic segmentation KITTI (SSKIT) dataset was made for the fusion effect. The mean intersection over union (MIoU) of the SSKIT can reach 88.06%. As well as the public datasets, the S3DIS showed that our method can enhance data features and outperform other methods: the MIoU of the S3DIS can reach up to 78.55%. The segmentation accuracy is significantly improved compared with the existing methods, which verifies the effectiveness of the improved algorithms. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

24 pages, 8093 KiB  
Article
Probabilistic Modeling of Multicamera Interference for Time-of-Flight Sensors
by Bryan Rodriguez, Xinxiang Zhang and Dinesh Rajan
Sensors 2023, 23(19), 8047; https://doi.org/10.3390/s23198047 - 23 Sep 2023
Cited by 1 | Viewed by 785
Abstract
The behavior of multicamera interference in 3D images (e.g., depth maps), which is based on infrared (IR) light, is not well understood. In 3D images, when multicamera interference is present, there is an increase in the amount of zero-value pixels, resulting in a [...] Read more.
The behavior of multicamera interference in 3D images (e.g., depth maps), which is based on infrared (IR) light, is not well understood. In 3D images, when multicamera interference is present, there is an increase in the amount of zero-value pixels, resulting in a loss of depth information. In this work, we demonstrate a framework for synthetically generating direct and indirect multicamera interference using a combination of a probabilistic model and ray tracing. Our mathematical model predicts the locations and probabilities of zero-value pixels in depth maps that contain multicamera interference. Our model accurately predicts where depth information may be lost in a depth map when multicamera interference is present. We compare the proposed synthetic 3D interference images with controlled 3D interference images captured in our laboratory. The proposed framework achieves an average root mean square error (RMSE) of 0.0625, an average peak signal-to-noise ratio (PSNR) of 24.1277 dB, and an average structural similarity index measure (SSIM) of 0.9007 for predicting direct multicamera interference, and an average RMSE of 0.0312, an average PSNR of 26.2280 dB, and an average SSIM of 0.9064 for predicting indirect multicamera interference. The proposed framework can be used to develop and test interference mitigation techniques that will be crucial for the successful proliferation of these devices. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

20 pages, 12568 KiB  
Article
A Virtual Multi-Ocular 3D Reconstruction System Using a Galvanometer Scanner and a Camera
by Zidong Han and Liyan Zhang
Sensors 2023, 23(7), 3499; https://doi.org/10.3390/s23073499 - 27 Mar 2023
Cited by 4 | Viewed by 1791
Abstract
A novel visual 3D reconstruction system, composed of a two-axis galvanometer scanner, a camera with a lens, and a set of control units, is introduced in this paper. By changing the mirror angles of the galvanometer scanner fixed in front of the camera, [...] Read more.
A novel visual 3D reconstruction system, composed of a two-axis galvanometer scanner, a camera with a lens, and a set of control units, is introduced in this paper. By changing the mirror angles of the galvanometer scanner fixed in front of the camera, the boresight of the camera can be quickly adjusted. With the variable boresight, the camera can serve as a virtual multi-ocular system (VMOS), which captures the object at different perspectives. The working mechanism with a definite physical meaning is presented. A simple and efficient method for calibrating the intrinsic and extrinsic parameters of the VMOS is presented. The applicability of the proposed system for 3D reconstruction is investigated. Owing to the multiple virtual poses of the camera, the VMOS can provide stronger constraints in the object pose estimation than an ordinary perspective camera does. The experimental results demonstrate that the proposed VMOS is able to achieve 3D reconstruction performance competitive with that of a conventional stereovision system with a much more concise hardware configuration. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

22 pages, 7836 KiB  
Article
SliceLRF: A Local Reference Frame Sliced along the Height on the 3D Surface
by Bin Zhong and Dong Li
Sensors 2023, 23(7), 3483; https://doi.org/10.3390/s23073483 - 27 Mar 2023
Viewed by 1083
Abstract
The local reference frame (LRF) plays a vital role in local 3D shape description and matching. Numerous LRF methods have been proposed in recent decades. However, few LRFs can achieve a balance between repeatability and robustness under exposure to a variety of nuisances, [...] Read more.
The local reference frame (LRF) plays a vital role in local 3D shape description and matching. Numerous LRF methods have been proposed in recent decades. However, few LRFs can achieve a balance between repeatability and robustness under exposure to a variety of nuisances, including Gaussian noise, mesh resolution variation, clutter, and occlusion. Additionally, most LRFs are heuristic and lack generalizability to different applications and data modalities. In this paper, we first define the degree of distinction to describe the distribution of 2D point clouds and explore the relationship between the relative deviation of the distinction degree and the LRF error through experiments. Based on Gaussian noise and a random sampling analysis, several factors that affect the relative deviation of the distinction degree and result in the LRF error are identified. A scoring criterion is proposed to evaluate the robustness of the point cloud distribution. On this basis, we propose an LRF method (SliceLRF) based on slicing along the Z-axis, which selects the most robust adjacent slices in the point cloud region by scoring criteria for X-axis estimation to improve the repeatability and robustness. SliceLRF is rigorously tested on four public benchmark datasets which have different applications and involve different data modalities. It is also compared with the state-of-the-art LRFs. The experimental results show that the SliceLRF has more comprehensive repeatability and robustness than the other LRFs under exposure to Gaussian noise and random sampling. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

18 pages, 15975 KiB  
Article
Automatic 3D Building Reconstruction from OpenStreetMap and LiDAR Using Convolutional Neural Networks
by Marcos Barranquero, Alvaro Olmedo, Josefa Gómez, Abdelhamid Tayebi, Carlos Javier Hellín and Francisco Saez de Adana
Sensors 2023, 23(5), 2444; https://doi.org/10.3390/s23052444 - 22 Feb 2023
Cited by 7 | Viewed by 2874
Abstract
This paper presents the implementation of an automatic method for the reconstruction of 3D building maps. The core innovation of the proposed method is the supplementation of OpenStreetMap data with LiDAR data to reconstruct 3D urban environments automatically. The only input of the [...] Read more.
This paper presents the implementation of an automatic method for the reconstruction of 3D building maps. The core innovation of the proposed method is the supplementation of OpenStreetMap data with LiDAR data to reconstruct 3D urban environments automatically. The only input of the method is the area that needs to be reconstructed, defined by the enclosing points in terms of the latitude and longitude. First, area data are requested in OpenStreetMap format. However, there are certain buildings and geometries that are not fully received in OpenStreetMap files, such as information on roof types or the heights of buildings. To complete the information that is missing in the OpenStreetMap data, LiDAR data are read directly and analyzed using a convolutional neural network. The proposed approach shows that a model can be obtained with only a few samples of roof images from an urban area in Spain, and is capable of inferring roofs in other urban areas of Spain as well as other countries that were not used to train the model. The results allow us to identify a mean of 75.57% for height data and a mean of 38.81% for roof data. The finally inferred data are added to the 3D urban model, resulting in detailed and accurate 3D building maps. This work shows that the neural network is able to detect buildings that are not present in OpenStreetMap for which in LiDAR data are available. In future work, it would be interesting to compare the results of the proposed method with other approaches for generating 3D models from OSM and LiDAR data, such as point cloud segmentation or voxel-based approaches. Another area for future research could be the use of data augmentation techniques to increase the size and robustness of the training dataset. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

20 pages, 8623 KiB  
Article
Feature-Based Characterisation of Turned Surface Topography with Suppression of High-Frequency Measurement Errors
by Przemysław Podulka
Sensors 2022, 22(24), 9622; https://doi.org/10.3390/s22249622 - 08 Dec 2022
Cited by 3 | Viewed by 1243
Abstract
Errors that occur when surface topography is measured and analysed can be classified depending on the type of surface studied. Many types of surface topographies are considered when frequency-based errors are studied. However, turned surface topography is not comprehensively studied when data processing [...] Read more.
Errors that occur when surface topography is measured and analysed can be classified depending on the type of surface studied. Many types of surface topographies are considered when frequency-based errors are studied. However, turned surface topography is not comprehensively studied when data processing errors caused by false estimation (definition and suppression) of selected surface features (form or noise) are analysed. In the present work, the effects of the application of various methods (regular Gaussian regression, robust Gaussian regression, and spline and fast Fourier Transform filters) for the suppression of high-frequency measurement noise from the raw measured data of turned surface topography are presented and compared. The influence and usage of commonly used available commercial software, e.g., autocorrelation function, power spectral density, and texture direction, which function on the values of areal surface topography parameters from selected (ISO 25178) standards, are also introduced. Analysed surfaces were measured with a stylus or via non-contact (optical–white light interferometry) methods. It was found that the characterisation of surface topography, based on the analysis of selected features, can be crucial in reducing measurement and data analysis errors when various filters are applied. Moreover, the application of common functions can be advantageous when feature-based studies are proposed for both profile and areal data processing. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

14 pages, 1202 KiB  
Article
GC-MLP: Graph Convolution MLP for Point Cloud Analysis
by Yong Wang, Guohua Geng, Pengbo Zhou, Qi Zhang, Zhan Li and Ruihang Feng
Sensors 2022, 22(23), 9488; https://doi.org/10.3390/s22239488 - 05 Dec 2022
Cited by 2 | Viewed by 1557
Abstract
With the objective of addressing the problem of the fixed convolutional kernel of a standard convolution neural network and the isotropy of features making 3D point cloud data ineffective in feature learning, this paper proposes a point cloud processing method based on graph [...] Read more.
With the objective of addressing the problem of the fixed convolutional kernel of a standard convolution neural network and the isotropy of features making 3D point cloud data ineffective in feature learning, this paper proposes a point cloud processing method based on graph convolution multilayer perceptron, named GC-MLP. Unlike traditional local aggregation operations, the algorithm generates an adaptive kernel through the dynamic learning features of points, so that it can dynamically adapt to the structure of the object, i.e., the algorithm first adaptively assigns different weights to adjacent points according to the different relationships between the different points captured. Furthermore, local information interaction is then performed with the convolutional layers through a weight-sharing multilayer perceptron. Experimental results show that, under different task benchmark datasets (including ModelNet40 dataset, ShapeNet Part dataset, S3DIS dataset), our proposed algorithm achieves state-of-the-art for both point cloud classification and segmentation tasks. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

17 pages, 76872 KiB  
Article
Plane Fitting in 3D Reconstruction to Preserve Smooth Homogeneous Surfaces
by Yanan Xu, Yohwan So and Sanghyuk Woo
Sensors 2022, 22(23), 9391; https://doi.org/10.3390/s22239391 - 01 Dec 2022
Cited by 1 | Viewed by 1703
Abstract
Photogrammetric techniques for weakly-textured surfaces without sufficient information about the R (red), G (green) and B (blue) primary colors of light are challenging. Considering that most urban or indoor object surfaces follow simple geometric shapes, a novel method for reconstructing smooth homogeneous planar [...] Read more.
Photogrammetric techniques for weakly-textured surfaces without sufficient information about the R (red), G (green) and B (blue) primary colors of light are challenging. Considering that most urban or indoor object surfaces follow simple geometric shapes, a novel method for reconstructing smooth homogeneous planar surfaces based on MVS (Multi-View Stereo) is proposed. The idea behind it is to extract enough features for the image description, and to refine the dense points generated by the depth values of pixels with plane fitting, to favor the alignment of the surface to the detected planes. The SIFT (Scale Invariant Feature Transform) and AKAZE (Accelerated-KAZE) feature extraction algorithms are combined to ensure robustness and help retrieve connections in small samples. The smoothness of the enclosed watertight Poisson surface can be enhanced by enforcing the 3D points to be projected onto the absolute planes detected by a RANSAC (Random Sample Consensus)-based approach. Experimental evaluations of both cloud-to-mesh comparisons in the per-vertex distances with the ground truth models and visual comparisons with a popular mesh filtering based post-processing method indicate that the proposed method can considerably retain the integrity and smoothness of the reconstruction results. Combined with other primitive fittings, the reconstruction extent of homogeneous surfaces can be further extended, serving as primitive models for 3D building reconstruction, and providing guidance for future works in photogrammetry and 3D surface reconstruction. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

16 pages, 2153 KiB  
Article
Facial Expression Recognition with Geometric Scattering on 3D Point Clouds
by Yi He, Keren Fu, Peng Cheng and Jianwei Zhang
Sensors 2022, 22(21), 8293; https://doi.org/10.3390/s22218293 - 29 Oct 2022
Viewed by 1280
Abstract
As one of the pioneering data representations, the point cloud has shown its straightforward capacity to depict fine geometry in many applications, including computer graphics, molecular structurology, modern sensing signal processing, and more. However, unlike computer graphs obtained with auxiliary regularization techniques or [...] Read more.
As one of the pioneering data representations, the point cloud has shown its straightforward capacity to depict fine geometry in many applications, including computer graphics, molecular structurology, modern sensing signal processing, and more. However, unlike computer graphs obtained with auxiliary regularization techniques or from syntheses, raw sensor/scanner (metric) data often contain natural random noise caused by multiple extrinsic factors, especially in the case of high-speed imaging scenarios. On the other hand, grid-like imaging techniques (e.g., RGB images or video frames) tend to entangle interesting aspects with environmental variations such as pose/illuminations with Euclidean sampling/processing pipelines. As one such typical problem, 3D Facial Expression Recognition (3D FER) has been developed into a new stage, with remaining difficulties involving the implementation of efficient feature abstraction methods for high dimensional observations and of stabilizing methods to obtain adequate robustness in cases of random exterior variations. In this paper, a localized and smoothed overlapping kernel is proposed to extract discriminative inherent geometric features. By association between the induced deformation stability and certain types of exterior perturbations through manifold scattering transform, we provide a novel framework that directly consumes point cloud coordinates for FER while requiring no predefined meshes or other features/signals. As a result, our compact framework achieves 78.33% accuracy on the Bosphorus dataset for expression recognition challenge and 77.55% on 3D-BUFE. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

13 pages, 2470 KiB  
Article
Improvement of AD-Census Algorithm Based on Stereo Vision
by Yina Wang, Mengjiao Gu, Yufeng Zhu, Gang Chen, Zhaodong Xu and Yingqing Guo
Sensors 2022, 22(18), 6933; https://doi.org/10.3390/s22186933 - 13 Sep 2022
Cited by 9 | Viewed by 1996
Abstract
Problems such as low light, similar background colors, and noisy image acquisition often occur when collecting images of lunar surface obstacles. Given these problems, this study focuses on the AD-Census algorithm. In the original Census algorithm, in the bit string calculated with the [...] Read more.
Problems such as low light, similar background colors, and noisy image acquisition often occur when collecting images of lunar surface obstacles. Given these problems, this study focuses on the AD-Census algorithm. In the original Census algorithm, in the bit string calculated with the central pixel point, the bit string will be affected by the noise that the central point is subjected to. The effect of noise results in errors and mismatching. We introduce an improved algorithm to calculate the average window pixel for solving the problem of being susceptible to the central pixel value and improve the accuracy of the algorithm. Experiments have proven that the object contour in the grayscale map of disparity obtained by the improved algorithm is more apparent, and the edge part of the image is significantly improved, which is more in line with the real scene. In addition, because the traditional Census algorithm matches the window size in a fixed rectangle, it is difficult to obtain a suitable window in the image range of different textures, affecting the timeliness of the algorithm. An improvement idea of area growth adaptive window matching is proposed. The improved Census algorithm is applied to the AD-Census algorithm. The results show that the improved AD-Census algorithm has been shown to have an average run time of 5.3% and better matching compared to the traditional AD-Census algorithm for all tested image sets. Finally, the improved algorithm is applied to the simulation environment, and the experimental results show that the obstacles in the image can be effectively detected. The improved algorithm has important practical application value and is important to improve the feasibility and reliability of obstacle detection in lunar exploration projects. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

13 pages, 2076 KiB  
Article
TIMo—A Dataset for Indoor Building Monitoring with a Time-of-Flight Camera
by Pascal Schneider, Yuriy Anisimov, Raisul Islam, Bruno Mirbach, Jason Rambach, Didier Stricker and Frédéric Grandidier
Sensors 2022, 22(11), 3992; https://doi.org/10.3390/s22113992 - 25 May 2022
Cited by 4 | Viewed by 4403
Abstract
We present TIMo (Time-of-flight Indoor Monitoring), a dataset for video-based monitoring of indoor spaces captured using a time-of-flight (ToF) camera. The resulting depth videos feature people performing a set of different predefined actions, for which we provide detailed annotations. [...] Read more.
We present TIMo (Time-of-flight Indoor Monitoring), a dataset for video-based monitoring of indoor spaces captured using a time-of-flight (ToF) camera. The resulting depth videos feature people performing a set of different predefined actions, for which we provide detailed annotations. Person detection for people counting and anomaly detection are the two targeted applications. Most existing surveillance video datasets provide either grayscale or RGB videos. Depth information, on the other hand, is still a rarity in this class of datasets in spite of being popular and much more common in other research fields within computer vision. Our dataset addresses this gap in the landscape of surveillance video datasets. The recordings took place at two different locations with the ToF camera set up either in a top-down or a tilted perspective on the scene. Moreover, we provide experimental evaluation results from baseline algorithms. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

31 pages, 10789 KiB  
Article
Optimal Sensor Placement for Modal-Based Health Monitoring of a Composite Structure
by Sandris Ručevskis, Tomasz Rogala and Andrzej Katunin
Sensors 2022, 22(10), 3867; https://doi.org/10.3390/s22103867 - 19 May 2022
Cited by 17 | Viewed by 3188
Abstract
Optimal sensor placement is one of the important issues in monitoring the condition of structures, which has a major influence on monitoring system performance and cost. Due to this, it is still an open problem to find a compromise between these two parameters. [...] Read more.
Optimal sensor placement is one of the important issues in monitoring the condition of structures, which has a major influence on monitoring system performance and cost. Due to this, it is still an open problem to find a compromise between these two parameters. In this study, the problem of optimal sensor placement was investigated for a composite plate with simulated internal damage. To solve this problem, different sensor placement methods with different constraint variants were applied. The advantage of the proposed approach is that information for sensor placement was used only from the structure’s healthy state. The results of the calculations according to sensor placement methods were subsets of possible sensor network candidates, which were evaluated using the aggregation of different metrics. The evaluation of selected sensor networks was performed and validated using machine learning techniques and visualized appropriately. Using the proposed approach, it was possible to precisely detect damage based on a limited number of strain sensors and mode shapes taken into consideration, which leads to efficient structural health monitoring with resource savings both in costs and computational time and complexity. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision: 2nd Edition)
►▼ Show Figures

Figure 1

45 pages, 9677 KiB  
Article
Calibration of Stereo Pairs Using Speckle Metrology
by Éric Samson, Denis Laurendeau and Marc Parizeau
Sensors 2022, 22(5), 1784; https://doi.org/10.3390/s22051784 - 24 Feb 2022
Viewed by 1563
Abstract
The accuracy of 3D reconstruction for metrology applications using active stereo pairs depends on the quality of the calibration of the system. Active stereo pairs are generally composed of cameras mounted on tilt/pan mechanisms separated by a constant or variable baseline. This paper [...] Read more.
The accuracy of 3D reconstruction for metrology applications using active stereo pairs depends on the quality of the calibration of the system. Active stereo pairs are generally composed of cameras mounted on tilt/pan mechanisms separated by a constant or variable baseline. This paper presents a calibration approach based on speckle metrology that allows the separation of translation and rotation in the estimation of extrinsic parameters. To achieve speckle-based calibration, a device called an Almost Punctual Speckle Source (APSS) is introduced. Using the APSS, a thorough method for the calibration of extrinsic parameters of stereo pairs is described. Experimental results obtained with a stereo system called the Agile Stereo Pair (ASP) demonstrate that speckle-based calibration achieves better reconstruction performance than methods using standard calibration procedures. Although the experiments were performed with a specific stereo pair, such as the ASP, which is described in the paper, the speckle-based calibration approach using the APSS can be transposed to other stereo setups. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision)
►▼ Show Figures

Figure 1

22 pages, 5577 KiB  
Article
Probabilistic Modeling of Motion Blur for Time-of-Flight Sensors
by Bryan Rodriguez, Xinxiang Zhang and Dinesh Rajan
Sensors 2022, 22(3), 1182; https://doi.org/10.3390/s22031182 - 04 Feb 2022
Cited by 5 | Viewed by 2122
Abstract
Synthetically creating motion blur in two-dimensional (2D) images is a well-understood process and has been used in image processing for developing deblurring systems. There are no well-established techniques for synthetically generating arbitrary motion blur within three-dimensional (3D) images, such as depth maps and [...] Read more.
Synthetically creating motion blur in two-dimensional (2D) images is a well-understood process and has been used in image processing for developing deblurring systems. There are no well-established techniques for synthetically generating arbitrary motion blur within three-dimensional (3D) images, such as depth maps and point clouds since their behavior is not as well understood. As a prerequisite, we have previously developed a method for generating synthetic motion blur in a plane that is parallel to the sensor detector plane. In this work, as a major extension, we generalize our previously developed framework for synthetically generating linear and radial motion blur along planes that are at arbitrary angles with respect to the sensor detector plane. Our framework accurately captures the behavior of the real motion blur that is encountered using a Time-of-Flight (ToF) sensor. This work uses a probabilistic model that predicts the location of invalid pixels that are typically present within depth maps that contain real motion blur. More specifically, the probabilistic model considers different angles of motion paths and the velocity of an object with respect to the image plane of a ToF sensor. Extensive experimental results are shown that demonstrate how our framework can be applied to synthetically create radial, linear, and combined radial-linear motion blur. We quantify the accuracy of the synthetic generation method by comparing the resulting synthetic depth map to the experimentally captured depth map with motion. Our results indicate that our framework achieves an average Boundary F1 (BF) score of 0.7192 for invalid pixels for synthetic radial motion blur, an average BF score of 0.8778 for synthetic linear motion blur, and an average BF score of 0.62 for synthetic combined radial-linear motion blur. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision)
►▼ Show Figures

Figure 1

15 pages, 3891 KiB  
Article
Nonlinear Optimization of Light Field Point Cloud
by Yuriy Anisimov, Jason Raphael Rambach and Didier Stricker
Sensors 2022, 22(3), 814; https://doi.org/10.3390/s22030814 - 21 Jan 2022
Cited by 1 | Viewed by 2066
Abstract
The problem of accurate three-dimensional reconstruction is important for many research and industrial applications. Light field depth estimation utilizes many observations of the scene and hence can provide accurate reconstruction. We present a method, which enhances existing reconstruction algorithm with per-layer disparity filtering [...] Read more.
The problem of accurate three-dimensional reconstruction is important for many research and industrial applications. Light field depth estimation utilizes many observations of the scene and hence can provide accurate reconstruction. We present a method, which enhances existing reconstruction algorithm with per-layer disparity filtering and consistency-based holes filling. Together with that we reformulate the reconstruction result to a form of point cloud from different light field viewpoints and propose a non-linear optimization of it. The capability of our method to reconstruct scenes with acceptable quality was verified by evaluation on a publicly available dataset. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision)
►▼ Show Figures

Figure 1

20 pages, 3348 KiB  
Article
Classification of Cracks in Composite Structures Subjected to Low-Velocity Impact Using Distribution-Based Segmentation and Wavelet Analysis of X-ray Tomograms
by Angelika Wronkowicz-Katunin, Andrzej Katunin, Marko Nagode and Jernej Klemenc
Sensors 2021, 21(24), 8342; https://doi.org/10.3390/s21248342 - 14 Dec 2021
Cited by 4 | Viewed by 2145
Abstract
The problem of characterizing the structural residual life is one of the most challenging issues of the damage tolerance concept currently applied in modern aviation. Considering the complexity of the internal architecture of composite structures widely applied for aircraft components nowadays, as well [...] Read more.
The problem of characterizing the structural residual life is one of the most challenging issues of the damage tolerance concept currently applied in modern aviation. Considering the complexity of the internal architecture of composite structures widely applied for aircraft components nowadays, as well as the additional complexity related to the appearance of barely visible impact damage, prediction of the structural residual life is a demanding task. In this paper, the authors proposed a method based on detection of structural damage after low-velocity impact loading and its classification with respect to types of acting stress on constituents of composite structures using the developed processing algorithm based on segmentation of 3D X-ray computed tomograms using the rebmix package, real-oriented dual-tree wavelet transform and supporting image processing procedures. The presented algorithm allowed for accurate distinguishing of defined types of damage from X-ray computed tomograms with strong robustness to noise and measurement artifacts. The processing was performed on experimental data obtained from X-ray computed tomography of a composite structure with barely visible impact damage, which allowed better understanding of fracture mechanisms in such conditions. The gained knowledge will allow for a more accurate simulation of structural damage in composite structures, which will provide higher accuracy in predicting structural residual life. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision)
►▼ Show Figures

Figure 1

17 pages, 12998 KiB  
Article
Multiple Cylinder Extraction from Organized Point Clouds
by Saed Moradi, Denis Laurendeau and Clement Gosselin
Sensors 2021, 21(22), 7630; https://doi.org/10.3390/s21227630 - 17 Nov 2021
Cited by 1 | Viewed by 2668
Abstract
Most man-made objects are composed of a few basic geometric primitives (GPs) such as spheres, cylinders, planes, ellipsoids, or cones. Thus, the object recognition problem can be considered as one of geometric primitives extraction. Among the different geometric primitives, cylinders are the most [...] Read more.
Most man-made objects are composed of a few basic geometric primitives (GPs) such as spheres, cylinders, planes, ellipsoids, or cones. Thus, the object recognition problem can be considered as one of geometric primitives extraction. Among the different geometric primitives, cylinders are the most frequently used GPs in real-world scenes. Therefore, cylinder detection and extraction are of great importance in 3D computer vision. Despite the rapid progress of cylinder detection algorithms, there are still two open problems in this area. First, a robust strategy is needed for the initial sample selection component of the cylinder extraction module. Second, detecting multiple cylinders simultaneously has not yet been investigated in depth. In this paper, a robust solution is provided to address these problems. The proposed solution is divided into three sub-modules. The first sub-module is a fast and accurate normal vector estimation algorithm from raw depth images. With the estimation method, a closed-form solution is provided for computing the normal vector at each point. The second sub-module benefits from the maximally stable extremal regions (MSER) feature detector to simultaneously detect cylinders present in the scene. Finally, the detected cylinders are extracted using the proposed cylinder extraction algorithm. Quantitative and qualitative results show that the proposed algorithm outperforms the baseline algorithms in each of the following areas: normal estimation, cylinder detection, and cylinder extraction. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision)
►▼ Show Figures

Figure 1

13 pages, 3112 KiB  
Article
SAP-Net: A Simple and Robust 3D Point Cloud Registration Network Based on Local Shape Features
by Jinlong Li, Yuntao Li, Jiang Long, Yu Zhang and Xiaorong Gao
Sensors 2021, 21(21), 7177; https://doi.org/10.3390/s21217177 - 28 Oct 2021
Cited by 5 | Viewed by 1907
Abstract
Point cloud registration is a key step in the reconstruction of 3D data models. The traditional ICP registration algorithm depends on the initial position of the point cloud. Otherwise, it may get trapped into local optima. In addition, the registration method based on [...] Read more.
Point cloud registration is a key step in the reconstruction of 3D data models. The traditional ICP registration algorithm depends on the initial position of the point cloud. Otherwise, it may get trapped into local optima. In addition, the registration method based on the feature learning of PointNet cannot directly or effectively extract local features. To solve these two problems, this paper proposes SAP-Net, inspired by CorsNet and PointNet++, as an optimized CorsNet. To be more specific, SAP-Net firstly uses the set abstraction layer in PointNet++ as the feature extraction layer and then combines the global features with the initial template point cloud. Finally, PointNet is used as the transform prediction layer to obtain the six parameters required for point cloud registration directly, namely the rotation matrix and the translation vector. Experiments on the ModelNet40 dataset and real data show that SAP-Net not only outperforms ICP and CorsNet on both seen and unseen categories of the point cloud but also has stronger robustness. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision)
►▼ Show Figures

Figure 1

24 pages, 14291 KiB  
Article
Enhanced Soft 3D Reconstruction Method with an Iterative Matching Cost Update Using Object Surface Consensus
by Min-Jae Lee, Gi-Mun Um, Joungil Yun, Won-Sik Cheong and Soon-Yong Park
Sensors 2021, 21(19), 6680; https://doi.org/10.3390/s21196680 - 08 Oct 2021
Cited by 9 | Viewed by 2597
Abstract
In this paper, we propose a multi-view stereo matching method, EnSoft3D (Enhanced Soft 3D Reconstruction) to obtain dense and high-quality depth images. Multi-view stereo is one of the high-interest research areas and has wide applications. Motivated by the Soft3D reconstruction method, we introduce [...] Read more.
In this paper, we propose a multi-view stereo matching method, EnSoft3D (Enhanced Soft 3D Reconstruction) to obtain dense and high-quality depth images. Multi-view stereo is one of the high-interest research areas and has wide applications. Motivated by the Soft3D reconstruction method, we introduce a new multi-view stereo matching scheme. The original Soft3D method is introduced for novel view synthesis, while occlusion-aware depth is also reconstructed by integrating the matching costs of the Plane Sweep Stereo (PSS) and soft visibility volumes. However, the Soft3D method has an inherent limitation because the erroneous PSS matching costs are not updated. To overcome this limitation, the proposed scheme introduces an update process of the PSS matching costs. From the object surface consensus volume, an inverse consensus kernel is derived, and the PSS matching costs are iteratively updated using the kernel. The proposed EnSoft3D method reconstructs a highly accurate 3D depth image because both the multi-view matching cost and soft visibility are updated simultaneously. The performance of the proposed method is evaluated by using structured and unstructured benchmark datasets. Disparity error is measured to verify 3D reconstruction accuracy, and both PSNR and SSIM are measured to verify the simultaneous enhancement of view synthesis. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision)
►▼ Show Figures

Figure 1

27 pages, 25484 KiB  
Article
Visual Browse and Exploration in Motion Capture Data with Phylogenetic Tree of Context-Aware Poses
by Songle Chen, Xuejian Zhao, Bingqing Luo and Zhixin Sun
Sensors 2020, 20(18), 5224; https://doi.org/10.3390/s20185224 - 13 Sep 2020
Cited by 1 | Viewed by 2269
Abstract
Visual browse and exploration in motion capture data take resource acquisition as a human–computer interaction problem, and it is an essential approach for target motion search. This paper presents a progressive schema which starts from pose browse, then locates the interesting region and [...] Read more.
Visual browse and exploration in motion capture data take resource acquisition as a human–computer interaction problem, and it is an essential approach for target motion search. This paper presents a progressive schema which starts from pose browse, then locates the interesting region and then switches to online relevant motion exploration. It mainly addresses three core issues. First, to alleviate the contradiction between the limited visual space and ever-increasing size of real-world database, it applies affinity propagation to numerical similarity measure of pose to perform data abstraction and obtains representative poses of clusters. Second, to construct a meaningful neighborhood for user browsing, it further merges logical similarity measures of pose with the weight quartets and casts the isolated representative poses into a structure of phylogenetic tree. Third, to support online motion exploration including motion ranking and clustering, a biLSTM-based auto-encoder is proposed to encode the high-dimensional pose context into compact latent space. Experimental results on CMU’s motion capture data verify the effectiveness of the proposed method. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision)
►▼ Show Figures

Figure 1

13 pages, 2809 KiB  
Article
3D Transparent Object Detection and Reconstruction Based on Passive Mode Single-Pixel Imaging
by Anumol Mathai, Ningqun Guo, Dong Liu and Xin Wang
Sensors 2020, 20(15), 4211; https://doi.org/10.3390/s20154211 - 29 Jul 2020
Cited by 8 | Viewed by 3370
Abstract
Transparent object detection and reconstruction are significant, due to their practical applications. The appearance and characteristics of light in these objects make reconstruction methods tailored for Lambertian surfaces fail disgracefully. In this paper, we introduce a fixed multi-viewpoint approach to ascertain the shape [...] Read more.
Transparent object detection and reconstruction are significant, due to their practical applications. The appearance and characteristics of light in these objects make reconstruction methods tailored for Lambertian surfaces fail disgracefully. In this paper, we introduce a fixed multi-viewpoint approach to ascertain the shape of transparent objects, thereby avoiding the rotation or movement of the object during imaging. In addition, a simple and cost-effective experimental setup is presented, which employs two single-pixel detectors and a digital micromirror device, for imaging transparent objects by projecting binary patterns. In the system setup, a dark framework is implemented around the object, to create shades at the boundaries of the object. By triangulating the light path from the object, the surface shape is recovered, neither considering the reflections nor the number of refractions. It can, therefore, handle transparent objects with a relatively complex shape with the unknown refractive index. The implementation of compressive sensing in this technique further simplifies the acquisition process, by reducing the number of measurements. The experimental results show that 2D images obtained from the single-pixel detectors are better in quality with a resolution of 32×32. Additionally, the obtained disparity and error map indicate the feasibility and accuracy of the proposed method. This work provides a new insight into 3D transparent object detection and reconstruction, based on single-pixel imaging at an affordable cost, with the implementation of a few numbers of detectors. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision)
►▼ Show Figures

Figure 1

26 pages, 52976 KiB  
Article
Quantitative 3D Reconstruction from Scanning Electron Microscope Images Based on Affine Camera Models
by Stefan Töberg and Eduard Reithmeier
Sensors 2020, 20(12), 3598; https://doi.org/10.3390/s20123598 - 26 Jun 2020
Cited by 1 | Viewed by 4233
Abstract
Scanning electron microscopes (SEMs) are versatile imaging devices for the micro- and nanoscale that find application in various disciplines such as the characterization of biological, mineral or mechanical specimen. Even though the specimen’s two-dimensional (2D) properties are provided by the acquired images, detailed [...] Read more.
Scanning electron microscopes (SEMs) are versatile imaging devices for the micro- and nanoscale that find application in various disciplines such as the characterization of biological, mineral or mechanical specimen. Even though the specimen’s two-dimensional (2D) properties are provided by the acquired images, detailed morphological characterizations require knowledge about the three-dimensional (3D) surface structure. To overcome this limitation, a reconstruction routine is presented that allows the quantitative depth reconstruction from SEM image sequences. Based on the SEM’s imaging properties that can be well described by an affine camera, the proposed algorithms rely on the use of affine epipolar geometry, self-calibration via factorization and triangulation from dense correspondences. To yield the highest robustness and accuracy, different sub-models of the affine camera are applied to the SEM images and the obtained results are directly compared to confocal laser scanning microscope (CLSM) measurements to identify the ideal parametrization and underlying algorithms. To solve the rectification problem for stereo-pair images of an affine camera so that dense matching algorithms can be applied, existing approaches are adapted and extended to further enhance the yielded results. The evaluations of this study allow to specify the applicability of the affine camera models to SEM images and what accuracies can be expected for reconstruction routines based on self-calibration and dense matching algorithms. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision)
►▼ Show Figures

Figure 1

Other

Jump to: Research

14 pages, 4044 KiB  
Letter
High-Speed Measurement of Shape and Vibration: Whole-Field Systems for Motion Capture and Vibration Modal Analysis by OPPA Method
by Yoshiharu Morimoto
Sensors 2020, 20(15), 4263; https://doi.org/10.3390/s20154263 - 30 Jul 2020
Cited by 8 | Viewed by 3385
Abstract
In shape measurement systems using a grating projection method, the phase analysis of a projected grating provides accurate results. The most popular phase analysis method is the phase shifting method, which requires several images for one shape analysis. Therefore, the object must not [...] Read more.
In shape measurement systems using a grating projection method, the phase analysis of a projected grating provides accurate results. The most popular phase analysis method is the phase shifting method, which requires several images for one shape analysis. Therefore, the object must not move during the measurement. The authors previously proposed a new accurate and high-speed shape measurement method, i.e., the one-pitch phase analysis (OPPA) method, which can determine the phase at every point of a single image of an object with a grating projected onto it. In the OPPA optical system, regardless of the distance of the object from the camera, the one-pitch length (number of pixels) on the imaging surface of the camera sensor is always constant. Therefore, brightness data for one pitch at any point of the image can be easily analyzed to determine phase distribution, or shape. This technology will apply to the measurement of objects in motion, including automobiles, robot arms, products on a conveyor belt, and vibrating objects. This paper describes the principle of the OPPA method and example applications for real-time human motion capture and modal analysis of free vibration of a flat cantilever plate after hammering. The results show the usefulness of the OPPA method. Full article
(This article belongs to the Special Issue Sensing and Processing for 3D Computer Vision)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-26
Back to TopTop