Submit to Special Issue Submit Abstract to Special Issue Review for Sensors Propose a Special Issue

Journal Menu

Journal Browser

► Journal Browser

3D Reconstruction with RGB-D Cameras and Multi-sensors

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Published Papers

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Sensing and Imaging".

Deadline for manuscript submissions: 31 July 2024 | Viewed by 9585

Share This Special Issue

Special Issue Editors

Dr. Pengpeng Hu

E-Mail Website
Guest Editor

Department of Electronics and Informatics, Vrije Universiteit Brussel, Brussels, Belgium
Interests: biometrics; measurement; point cloud processing; deep learning; 3D body scanning
Special Issues, Collections and Topics in MDPI journals

Prof. Dr. Adrian Munteanu

E-Mail Website
Guest Editor

Electronics and Informatics Department, Vrije Universiteit Brussel, 1050 Brussels, Belgium
Interests: machine learning; computer vision; 3D graphics; anthropometry
Special Issues, Collections and Topics in MDPI journals

Dr. He Wang

E-Mail Website
Guest Editor

School of Computing, University of Leeds, Leeds, UK
Interests: computer graphics; computer animation; computer vision; machine learning and robotics
Special Issues, Collections and Topics in MDPI journals

Dr. Walid Darwish

E-Mail Website
Guest Editor

Civil Engineering Department, Geomatics Engineering Lab, Faculty of Eningeering, Cairo University, Cairo, Egypt
Interests: RGB-D sensors; SLAM; indoor navigation; plenoptic cameras

Special Issue Information

Dear Colleagues,

Multi-sensor systems are widely used in 3D reconstruction tasks, such as 3D shape reconstruction, 4D body scanning, and human activity monitoring, to name a few. Compared to single-sensor systems, multi-sensor systems can simultaneously capture data from different viewports, which enables real-time complete shape capture. However, multi-sensor systems are usually expensive and require professional knowledge for operation. With the advancement of commodity RGB-D cameras, there have been countless attempts to build low-cost 3D reconstruction systems. During these attempts, additional challenges were encountered (e.g., calibration of multiple RGB-D sensors, human joint detection from point clouds, low-resolution of the scanned images, and compression of large-scale point clouds), which have encouraged researchers to explore more advanced algorithms.

In this context, the objective of this Special Issue is to connect researchers in the field of camera calibration of multiple sensors, RGB-D sensors, machine learning, 3D scanning, 4D capture, and other related fields. This issue will provide a state-of-the-art representation of methods that have led to progress in the research and application of multiple sensors.

We are soliciting original, full-length, unpublished research articles and reviews focused on this research topic. Topics of interest include, but are not limited to, the following:

Point cloud processing
Multi-sensor shape capture
Multi-sensor human activity understanding
RGB-D 3D reconscturtion
RGB-D human activity understanding
RGB-D calibration
RGB-D SLAM
RGB-D data processing

Dr. Pengpeng Hu
Prof. Dr. Adrian Munteanu
Dr. He Wang
Dr. Walid Darwish
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Published Papers (5 papers)

Download All Papers

Order results

Result details

Show export options Show export options

Select all

Export citation of selected articles as:

Research

19 pages, 10016 KiB

Open AccessArticle

LNMVSNet: A Low-Noise Multi-View Stereo Depth Inference Method for 3D Reconstruction

by Weiming Luo, Zongqing Lu and Qingmin Liao

Sensors 2024, 24(8), 2400; https://doi.org/10.3390/s24082400 - 09 Apr 2024

Viewed by 354

Abstract

With the widespread adoption of modern RGB cameras, an abundance of RGB images is available everywhere. Therefore, multi-view stereo (MVS) 3D reconstruction has been extensively applied across various fields because of its cost-effectiveness and accessibility, which involves multi-view depth estimation and stereo matching algorithms. However, MVS tasks face noise challenges because of natural multiplicative noise and negative gain in algorithms, which reduce the quality and accuracy of the generated models and depth maps. Traditional MVS methods often struggle with noise, relying on assumptions that do not always hold true under real-world conditions, while deep learning-based MVS approaches tend to suffer from high noise sensitivity. To overcome these challenges, we introduce LNMVSNet, a deep learning network designed to enhance local feature attention and fuse features across different scales, aiming for low-noise, high-precision MVS 3D reconstruction. Through extensive evaluation of multiple benchmark datasets, LNMVSNet has demonstrated its superior performance, showcasing its ability to improve reconstruction accuracy and completeness, especially in the recovery of fine details and clear feature delineation. This advancement brings hope for the widespread application of MVS, ranging from precise industrial part inspection to the creation of immersive virtual environments. Full article

(This article belongs to the Special Issue 3D Reconstruction with RGB-D Cameras and Multi-sensors)

► Show Figures

Figure 1

26 pages, 11619 KiB

Open AccessArticle

Neural Radiance Fields-Based 3D Reconstruction of Power Transmission Lines Using Progressive Motion Sequence Images

by Yujie Zeng, Jin Lei, Tianming Feng, Xinyan Qin, Bo Li, Yanqi Wang, Dexin Wang and Jie Song

Sensors 2023, 23(23), 9537; https://doi.org/10.3390/s23239537 - 30 Nov 2023

Viewed by 1093

Abstract

To address the fuzzy reconstruction effect on distant objects in unbounded scenes and the difficulty in feature matching caused by the thin structure of power lines in images, this paper proposes a novel image-based method for the reconstruction of power transmission lines (PTLs). The dataset used in this paper comprises PTL progressive motion sequence datasets, constructed by a visual acquisition system carried by a developed Flying–walking Power Line Inspection Robot (FPLIR). This system captures close-distance and continuous images of power lines. The study introduces PL-NeRF, that is, an enhanced method based on the Neural Radiance Fields (NeRF) method for reconstructing PTLs. The highlights of PL-NeRF include (1) compressing the unbounded scene of PTLs by exploiting the spatial compression of normal

L^{\infty}

; (2) encoding the direction and position of the sample points through Integrated Position Encoding (IPE) and Hash Encoding (HE), respectively. Compared to existing methods, the proposed method demonstrates good performance in 3D reconstruction, with fidelity indicators of PSNR = 29, SSIM = 0.871, and LPIPS = 0.087. Experimental results highlight that the combination of PL-NeRF with progressive motion sequence images ensures the integrity and continuity of PTLs, improving the efficiency and accuracy of image-based reconstructions. In the future, this method could be widely applied for efficient and accurate 3D reconstruction and inspection of PTLs, providing a strong foundation for automated monitoring of transmission corridors and digital power engineering. Full article

(This article belongs to the Special Issue 3D Reconstruction with RGB-D Cameras and Multi-sensors)

► Show Figures

Figure 1

21 pages, 13877 KiB

Open AccessArticle

Recognition and Counting of Apples in a Dynamic State Using a 3D Camera and Deep Learning Algorithms for Robotic Harvesting Systems

by R. M. Rasika D. Abeyrathna, Victor Massaki Nakaguchi, Arkar Minn and Tofael Ahamed

Sensors 2023, 23(8), 3810; https://doi.org/10.3390/s23083810 - 07 Apr 2023

Cited by 10 | Viewed by 2953

Abstract

Recognition and 3D positional estimation of apples during harvesting from a robotic platform in a moving vehicle are still challenging. Fruit clusters, branches, foliage, low resolution, and different illuminations are unavoidable and cause errors in different environmental conditions. Therefore, this research aimed to develop a recognition system based on training datasets from an augmented, complex apple orchard. The recognition system was evaluated using deep learning algorithms established from a convolutional neural network (CNN). The dynamic accuracy of the modern artificial neural networks involving 3D coordinates for deploying robotic arms at different forward-moving speeds from an experimental vehicle was investigated to compare the recognition and tracking localization accuracy. In this study, a Realsense D455 RGB-D camera was selected to acquire 3D coordinates of each detected and counted apple attached to artificial trees placed in the field to propose a specially designed structure for ease of robotic harvesting. A 3D camera, YOLO (You Only Look Once), YOLOv4, YOLOv5, YOLOv7, and EfficienDet state-of-the-art models were utilized for object detection. The Deep SORT algorithm was employed for tracking and counting detected apples using perpendicular, 15°, and 30° orientations. The 3D coordinates were obtained for each tracked apple when the on-board camera in the vehicle passed the reference line and was set in the middle of the image frame. To optimize harvesting at three different speeds (0.052 ms⁻¹, 0.069 ms⁻¹, and 0.098 ms⁻¹), the accuracy of 3D coordinates was compared for three forward-moving speeds and three camera angles (15°, 30°, and 90°). The mean average precision (mAP@0.5) values of YOLOv4, YOLOv5, YOLOv7, and EfficientDet were 0.84, 0.86, 0.905, and 0.775, respectively. The lowest root mean square error (RMSE) was 1.54 cm for the apples detected by EfficientDet at a 15° orientation and a speed of 0.098 ms⁻¹. In terms of counting apples, YOLOv5 and YOLOv7 showed a higher number of detections in outdoor dynamic conditions, achieving a counting accuracy of 86.6%. We concluded that the EfficientDet deep learning algorithm at a 15° orientation in 3D coordinates can be employed for further robotic arm development while harvesting apples in a specially designed orchard. Full article

(This article belongs to the Special Issue 3D Reconstruction with RGB-D Cameras and Multi-sensors)

► Show Figures

Figure 1

25 pages, 20118 KiB

Open AccessArticle

Light Field View Synthesis Using the Focal Stack and All-in-Focus Image

by Rishabh Sharma, Stuart Perry and Eva Cheng

Sensors 2023, 23(4), 2119; https://doi.org/10.3390/s23042119 - 13 Feb 2023

Viewed by 1633

Abstract

Light field reconstruction and synthesis algorithms are essential for improving the lower spatial resolution for hand-held plenoptic cameras. Previous light field synthesis algorithms produce blurred regions around depth discontinuities, especially for stereo-based algorithms, where no information is available to fill the occluded areas in the light field image. In this paper, we propose a light field synthesis algorithm that uses the focal stack images and the all-in-focus image to synthesize a 9 × 9 sub-aperture view light field image. Our approach uses depth from defocus to estimate a depth map. Then, we use the depth map and the all-in-focus image to synthesize the sub-aperture views, and their corresponding depth maps by mimicking the apparent shifting of the central image according to the depth values. We handle the occluded regions in the synthesized sub-aperture views by filling them with the information recovered from the focal stack images. We also show that, if the depth levels in the image are known, we can synthesize a high-accuracy light field image with just five focal stack images. The accuracy of our approach is compared with three state-of-the-art algorithms: one non-learning and two CNN-based approaches, and the results show that our algorithm outperforms all three in terms of PSNR and SSIM metrics. Full article

(This article belongs to the Special Issue 3D Reconstruction with RGB-D Cameras and Multi-sensors)

► Show Figures

Figure 1

10 pages, 4347 KiB

Open AccessArticle

3D Reconstruction Using 3D Registration-Based ToF-Stereo Fusion

by Sukwoo Jung, Youn-Sung Lee, Yunju Lee and KyungTaek Lee

Sensors 2022, 22(21), 8369; https://doi.org/10.3390/s22218369 - 01 Nov 2022

Cited by 7 | Viewed by 2304

Abstract

Depth sensing is an important issue in many applications, such as Augmented Reality (AR), eXtended Reality (XR), and Metaverse. For 3D reconstruction, a depth map can be acquired by a stereo camera and a Time-of-Flight (ToF) sensor. We used both sensors complementarily to improve the accuracy of 3D information of the data. First, we applied a generalized multi-camera calibration method that uses both color and depth information. Next, depth maps of two sensors were fused by 3D registration and reprojection approach. Then, hole-filling was applied to refine the new depth map from the ToF-stereo fused data. Finally, the surface reconstruction technique was used to generate mesh data from the ToF-stereo fused pointcloud data. The proposed procedure was implemented and tested with real-world data and compared with various algorithms to validate its efficiency. Full article

(This article belongs to the Special Issue 3D Reconstruction with RGB-D Cameras and Multi-sensors)

► Show Figures

Journal Menu

Journal Browser

3D Reconstruction with RGB-D Cameras and Multi-sensors

Share This Special Issue

Special Issue Editors

Special Issue Information

Published Papers (5 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI