Application, Optimization and Architecture of Deep Learning Neural Network

A special issue of Applied Sciences (ISSN 2076-3417). This special issue belongs to the section "Computing and Artificial Intelligence".

Deadline for manuscript submissions: 20 May 2024 | Viewed by 10913

Special Issue Editor

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
Interests: artificial intelligence; deep learning; medical image processing

Special Issue Information

Dear Colleagues,

With the surge in images and videos that need to be processed, deep learning neural networks are becoming a popular and essential tool for solving various problems, such as classification, detection, and regression. The new deep architectures have led to advances in less computational resources and to more reliable results.

Therefore, in this Special Issue, we aim to present novel deep learning algorithms and their applications in various research fields. Submitted articles can cover various topics, including but not limited to deep learning, artificial intelligence, and the processing of large datasets from medical instruments, ground-based datasets, scientific experiments, and many other sources.

Dr. Haigang Gong
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Applied Sciences is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • deep learning
  • artificial intelligence
  • reinforcement learning
  • image detection and segmentation
  • forecasting using deep networks
  • predications using deep networks
  • applications of deep learning neural networks

Published Papers (11 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

12 pages, 1385 KiB  
Article
Heterogeneous Graph-Convolution-Network-Based Short-Text Classification
by Jiwei Hua, Debing Sun, Yanxiang Hu, Jiayu Wang, Shuquan Feng and Zhaoyang Wang
Appl. Sci. 2024, 14(6), 2279; https://doi.org/10.3390/app14062279 - 08 Mar 2024
Viewed by 355
Abstract
With the development of online interactive media platforms, a large amount of short text has appeared on the internet. Determining how to classify these short texts efficiently and accurately is of great significance. Graph neural networks can capture information dependencies in the entire [...] Read more.
With the development of online interactive media platforms, a large amount of short text has appeared on the internet. Determining how to classify these short texts efficiently and accurately is of great significance. Graph neural networks can capture information dependencies in the entire short-text corpus, thereby enhancing feature expression and improving classification accuracy. However, existing works have overlooked the role of entities in these short texts. In this paper, we propose a heterogeneous graph-convolution-network-based short-text classification (SHGCN) method that integrates heterogeneous graph convolutional neural networks of text, entities, and words. Firstly, the model constructs a graph network of the text and extracts entity nodes and word nodes. Secondly, the relationship of the graph nodes in the heterogeneous graphs is determined by the mutual information between the words, the relationship between the documents and words, and the confidence between the words and entities. Then, the feature is represented through a word graph and combined with its BERT embedding, and the word feature is strengthened through BiLstm. Finally, the enhanced word features are combined with the document graph representation features to predict the document categories. To verify the performance of the model, experiments were conducted on the public datasets AGNews, R52, and MR. The classification accuracy of SHGCN reached 88.38%, 93.87%, and 82.87%, respectively, which is superior to that of some existing advanced classification methods. Full article
Show Figures

Figure 1

14 pages, 4362 KiB  
Article
Unraveling Convolution Neural Networks: A Topological Exploration of Kernel Evolution
by Lei Yang, Mengxue Xu and Yunan He
Appl. Sci. 2024, 14(5), 2197; https://doi.org/10.3390/app14052197 - 06 Mar 2024
Viewed by 298
Abstract
Convolutional Neural Networks (CNNs) have become essential in deep learning applications, especially in computer vision, yet their complex internal mechanisms pose significant challenges to interpretability, crucial for ethical applications. Addressing this, our paper explores CNNs by examining their topological changes throughout the learning [...] Read more.
Convolutional Neural Networks (CNNs) have become essential in deep learning applications, especially in computer vision, yet their complex internal mechanisms pose significant challenges to interpretability, crucial for ethical applications. Addressing this, our paper explores CNNs by examining their topological changes throughout the learning process, specifically employing persistent homology, a core method within Topological Data Analysis (TDA), to observe the dynamic evolution of their structure. This approach allows us to identify consistent patterns in the topological features of CNN kernels, particularly through shifts in Betti curves, which is a key concept in TDA. Our analysis of these Betti curves, initially focusing on the zeroth and first Betti numbers (respectively referred to as Betti-0 and Betti-1, which denote the number of connected components and loops), reveals insights into the learning dynamics of CNNs and potentially indicates the effectiveness of the learning process. We also discover notable differences in topological structures when CNNs are trained on grayscale versus color datasets, indicating the need for more extensive parameter space adjustments in color image processing. This study not only enhances the understanding of the intricate workings of CNNs but also contributes to bridging the gap between their complex operations and practical, interpretable applications. Full article
Show Figures

Figure 1

13 pages, 3072 KiB  
Article
A Lithology Recognition Network Based on Attention and Feature Brownian Distance Covariance
by Dake Zheng, Shudong Liu, Yidan Chen and Boyu Gu
Appl. Sci. 2024, 14(4), 1501; https://doi.org/10.3390/app14041501 - 12 Feb 2024
Viewed by 441
Abstract
In the context of mountain tunnel mining through the drilling and blasting method, the recognition of lithology from palm face images is crucial for the comprehensive analysis of geological conditions and the prevention of geological risks. However, the complexity of the background in [...] Read more.
In the context of mountain tunnel mining through the drilling and blasting method, the recognition of lithology from palm face images is crucial for the comprehensive analysis of geological conditions and the prevention of geological risks. However, the complexity of the background in the acquired palm face images, coupled with an insufficient data sample size, poses challenges. While the incorporation of deep learning technology has enhanced lithology recognition accuracy, issues persist, including inadequate feature extraction and suboptimal recognition accuracy. To address these challenges, this paper proposes a lithology recognition network integrating attention mechanisms and a feature Brownian distance covariance approach. Drawing inspiration from the Brownian distance covariance concept, a feature Brownian distance covariance module is devised to enhance the network’s attention to rock sample features and improve classification accuracy. Furthermore, an enhanced lightweight Convolutional Block Attention Module is introduced, with upgrades to the multilayer perceptron in the channel attention module. These improvements emphasize attention to lithological features while mitigating interference from background information. The proposed method is evaluated on a palm face image dataset collected in the field. The proposed method was evaluated on a dataset comprising field-collected images of a tunnel rock face. The results illustrate a significant enhancement in the improved model’s ability to recognize rock images, as evidenced by improvements across all objective evaluation metrics. The achieved accuracy rate of 97.60% surpasses that of the current mainstream lithology recognition neural network. Full article
Show Figures

Figure 1

13 pages, 2928 KiB  
Article
Biogas Production Prediction Based on Feature Selection and Ensemble Learning
by Shurong Peng, Lijuan Guo, Yuanshu Li, Haoyu Huang, Jiayi Peng and Xiaoxu Liu
Appl. Sci. 2024, 14(2), 901; https://doi.org/10.3390/app14020901 - 20 Jan 2024
Viewed by 638
Abstract
The allocation of biogas between power generation and heat supply in traditional kitchen waste power generation system is unreasonable; for this reason, a biogas prediction method based on feature selection and heterogeneous model integration learning is proposed for biogas production predictions. Firstly, the [...] Read more.
The allocation of biogas between power generation and heat supply in traditional kitchen waste power generation system is unreasonable; for this reason, a biogas prediction method based on feature selection and heterogeneous model integration learning is proposed for biogas production predictions. Firstly, the working principle of the biogas generation system based on kitchen waste is analyzed, the relationship between system features and biogas production is mined, and the important features are extracted. Secondly, the prediction performance of different individual learner models is comprehensively analyzed, and the training set is divided to reduce the risk of overfitting by combining K-fold cross-validation. Finally, different primary learners and meta learners are selected according to the prediction error and diversity index, and different learners are fused to construct the stacking ensemble learning model with a two-layer structure. The experimental results show that the research method has a higher prediction accuracy in predicting biogas production, which provides supporting data for the economic planning of kitchen waste power generation systems. Full article
Show Figures

Figure 1

24 pages, 5037 KiB  
Article
Soft Generative Adversarial Network: Combating Mode Collapse in Generative Adversarial Network Training via Dynamic Borderline Softening Mechanism
by Wei Li and Yongchuan Tang
Appl. Sci. 2024, 14(2), 579; https://doi.org/10.3390/app14020579 - 09 Jan 2024
Viewed by 564
Abstract
In this paper, we propose the Soft Generative Adversarial Network (SoftGAN), a strategy that utilizes a dynamic borderline softening mechanism to train Generative Adversarial Networks. This mechanism aims to solve the mode collapse problem and enhance the training stability of the generated outputs. [...] Read more.
In this paper, we propose the Soft Generative Adversarial Network (SoftGAN), a strategy that utilizes a dynamic borderline softening mechanism to train Generative Adversarial Networks. This mechanism aims to solve the mode collapse problem and enhance the training stability of the generated outputs. Within the SoftGAN, the objective of the discriminator is to learn a fuzzy concept of real data with a soft borderline between real and generated data. This objective is achieved by balancing the principles of maximum concept coverage and maximum expected entropy of fuzzy concepts. During the early training stage of the SoftGAN, the principle of maximum expected entropy of fuzzy concepts guides the learning process due to the significant divergence between the generated and real data. However, in the final stage of training, the principle of maximum concept coverage dominates as the divergence between the two distributions decreases. The dynamic borderline softening mechanism of the SoftGAN can be likened to a student (the generator) striving to create realistic images, with the tutor (the discriminator) dynamically guiding the student towards the right direction and motivating effective learning. The tutor gives appropriate encouragement or requirements according to abilities of the student at different stages, so as to promote the student to improve themselves better. Our approach offers both theoretical and practical benefits for improving GAN training. We empirically demonstrate the superiority of our SoftGAN approach in addressing mode collapse issues and generating high-quality outputs compared to existing approaches. Full article
Show Figures

Figure 1

16 pages, 8580 KiB  
Article
Enhanced YOLOv8 with BiFPN-SimAM for Precise Defect Detection in Miniature Capacitors
by Ning Li, Tianrun Ye, Zhihua Zhou, Chunming Gao and Ping Zhang
Appl. Sci. 2024, 14(1), 429; https://doi.org/10.3390/app14010429 - 03 Jan 2024
Viewed by 1524
Abstract
In the domain of automatic visual inspection for miniature capacitor quality control, the task of accurately detecting defects presents a formidable challenge. This challenge stems primarily from the small size and limited sample availability of defective micro-capacitors, which leads to issues such as [...] Read more.
In the domain of automatic visual inspection for miniature capacitor quality control, the task of accurately detecting defects presents a formidable challenge. This challenge stems primarily from the small size and limited sample availability of defective micro-capacitors, which leads to issues such as reduced detection accuracy and increased false-negative rates in existing inspection methods. To address these challenges, this paper proposes an innovative approach employing an enhanced ‘you only look once’ version 8 (YOLOv8) architecture specifically tailored for the intricate task of micro-capacitor defect inspection. The merging of the bidirectional feature pyramid network (BiFPN) architecture and the simplified attention module (SimAM), which greatly improves the model’s capacity to recognize fine features and feature representation, is at the heart of this methodology. Furthermore, the model’s capacity for generalization was significantly improved by the addition of the weighted intersection over union (WISE-IOU) loss function. A micro-capacitor surface defect (MCSD) dataset comprising 1358 images representing four distinct types of micro-capacitor defects was constructed. The experimental results showed that our approach achieved 95.8% effectiveness in the mean average precision (mAP) at a threshold of 0.5. This indicates a notable 9.5% enhancement over the original YOLOv8 architecture and underscores the effectiveness of our approach in the automatic visual inspection of miniature capacitors. Full article
Show Figures

Figure 1

24 pages, 4332 KiB  
Article
Multi-Path Routing Algorithm Based on Deep Reinforcement Learning for SDN
by Yi Zhang, Lanxin Qiu, Yangzhou Xu, Xinjia Wang, Shengjie Wang, Agyemang Paul and Zhefu Wu
Appl. Sci. 2023, 13(22), 12520; https://doi.org/10.3390/app132212520 - 20 Nov 2023
Viewed by 998
Abstract
Software-Defined Networking (SDN) enhances network control but faces Distributed Denial of Service (DDoS) attacks due to centralized control and flow-table constraints in network devices. To overcome this limitation, we introduce a multi-path routing algorithm for SDN called Trust-Based Proximal Policy Optimization (TBPPO). TBPPO [...] Read more.
Software-Defined Networking (SDN) enhances network control but faces Distributed Denial of Service (DDoS) attacks due to centralized control and flow-table constraints in network devices. To overcome this limitation, we introduce a multi-path routing algorithm for SDN called Trust-Based Proximal Policy Optimization (TBPPO). TBPPO incorporates a Kullback–Leibler divergence (KL divergence) trust value and a node diversity mechanism as the security assessment criterion, aiming to mitigate issues such as network fluctuations, low robustness, and congestion, with a particular emphasis on countering DDoS attacks. To avoid routing loops, differently from conventional ‘Next Hop’ routing decision methodology, we implemented an enhanced Depth-First Search (DFS) approach involving the pre-computation of path sets, from which we select the best path. To optimize the routing efficiency, we introduced an improved Proximal Policy Optimization (PPO) algorithm based on deep reinforcement learning. This enhanced PPO algorithm focuses on optimizing multi-path routing, considering security, network delay, and variations in multi-path delays. The TBPPO outperforms traditional methods in the Germany-50 evaluation, reducing average delay by 20%, cutting delay variation by 50%, and leading in trust value by 0.5, improving security and routing efficiency in SDN. TBPPO provides a practical and effective solution to enhance SDN security and routing efficiency. Full article
Show Figures

Figure 1

12 pages, 393 KiB  
Article
Pipelined Stochastic Gradient Descent with Taylor Expansion
by Bongwon Jang, Inchul Yoo and Dongsuk Yook
Appl. Sci. 2023, 13(21), 11730; https://doi.org/10.3390/app132111730 - 26 Oct 2023
Viewed by 625
Abstract
Stochastic gradient descent (SGD) is an optimization method typically used in deep learning to train deep neural network (DNN) models. In recent studies for DNN training, pipeline parallelism, a type of model parallelism, is proposed to accelerate SGD training. However, since SGD is [...] Read more.
Stochastic gradient descent (SGD) is an optimization method typically used in deep learning to train deep neural network (DNN) models. In recent studies for DNN training, pipeline parallelism, a type of model parallelism, is proposed to accelerate SGD training. However, since SGD is inherently sequential, naively implemented pipeline parallelism introduces the weight inconsistency and the delayed gradient problems, resulting in reduced training efficiency. In this study, we propose a novel method called TaylorPipe to alleviate these problems. The proposed method generates multiple model replicas to solve the weight inconsistency problem, and adopts a Taylor expansion-based gradient prediction algorithm to mitigate the delayed gradient problem. We verified the efficiency of the proposed method using the VGG-16 and the ResNet-34 on the CIFAR-10 and CIFAR-100 datasets. The experimental results show that not only the training time is reduced by up to 2.7 times but also the accuracy of TaylorPipe is comparable with that of SGD. Full article
Show Figures

Figure 1

19 pages, 4711 KiB  
Article
Prediction of Wind Power with Machine Learning Models
by Ömer Ali Karaman
Appl. Sci. 2023, 13(20), 11455; https://doi.org/10.3390/app132011455 - 19 Oct 2023
Cited by 4 | Viewed by 2672
Abstract
Wind power is a vital power grid component, and wind power forecasting represents a challenging task. In this study, a series of multiobjective predictive models were created utilising a range of cutting-edge machine learning (ML) methodologies, namely, artificial neural networks (ANNs), recurrent neural [...] Read more.
Wind power is a vital power grid component, and wind power forecasting represents a challenging task. In this study, a series of multiobjective predictive models were created utilising a range of cutting-edge machine learning (ML) methodologies, namely, artificial neural networks (ANNs), recurrent neural networks (RNNs), convolutional neural networks, and long short-term memory (LSTM) networks. In this study, two independent data sets were combined and used to predict wind power. The first data set contained internal values such as wind speed (m/s), wind direction (°), theoretical power (kW), and active power (kW). The second data set was external values that contained the meteorological data set, which can affect the wind power forecast. The k-nearest neighbours (kNN) algorithm completed the missing data in the data set. The results showed that the LSTM, RNN, CNN, and ANN algorithms were powerful in forecasting wind power. Furthermore, the performance of these models was evaluated by incorporating statistical indicators of performance deviation to demonstrate the efficacy of the employed methodology effectively. Moreover, the performance of these models was evaluated by incorporating statistical indicators of performance deviation, including the coefficient of determination (R2), root mean square error (RMSE), mean absolute error (MAE), and mean square error (MSE) metrics to effectively demonstrate the efficacy of the employed methodology. When the metrics are examined, it can be said that ANN, RNN, CNN, and LSTM methods effectively forecast wind power. However, it can be said that the LSTM model is more successful in estimating the wind power with an R2 value of 0.9574, MAE of 0.0209, MSE of 0.0038, and RMSE of 0.0614. Full article
Show Figures

Figure 1

18 pages, 12281 KiB  
Article
Lane Line Type Recognition Based on Improved YOLOv5
by Boyu Liu, Hao Wang, Yongqiang Wang, Congling Zhou and Lei Cai
Appl. Sci. 2023, 13(18), 10537; https://doi.org/10.3390/app131810537 - 21 Sep 2023
Cited by 1 | Viewed by 1157
Abstract
The recognition of lane line type plays an important role in the perception of advanced driver assistance systems (ADAS). In actual vehicle driving on roads, there are a variety of lane line type and complex road conditions which present significant challenges to ADAS. [...] Read more.
The recognition of lane line type plays an important role in the perception of advanced driver assistance systems (ADAS). In actual vehicle driving on roads, there are a variety of lane line type and complex road conditions which present significant challenges to ADAS. To address this problem, this paper proposes an improved YOLOv5 method for recognising lane line type. This method can accurately and quickly identify the types of lane lines and can show good recognition results in harsh environments. The main strategy of this method includes the following steps: first, the FasterNet lightweight network is introduced into all the concentrated-comprehensive convolution (C3) modules in the network to accelerate the inference speed and reduce the number of parameters. Then, the efficient channel attention (ECA) mechanism is integrated into the backbone network to extract image feature information and improve the model’s detection accuracy. Finally, the sigmoid intersection over union (SIoU) loss function is used to replace the original generalised intersection over union (GIoU) loss function to further enhance the robustness of the model. Through experiments, the improved YOLOv5s algorithm achieves 95.1% of mAP@0.5 and 95.2 frame·s−1 of FPS, which can satisfy the demand of ADAS for accuracy and real-time performance. And the number of model parameters are only 6M, and the volume is only 11.7 MB, which will be easily embedded into ADAS and does not require huge computing power to support it. Meanwhile, the improved algorithms increase the accuracy and speed of YOLOv5m, YOLOv5l, and YOLOv5x models to different degrees. The appropriate model can be selected according to the actual situation. This plays a practical role in improving the safety of ADAS. Full article
Show Figures

Figure 1

14 pages, 1028 KiB  
Article
CDF-LS: Contrastive Network for Emphasizing Feature Differences with Fusing Long- and Short-Term Interest Features
by Kejian Liu, Wei Wang, Rongju Wang, Xuran Cui, Liying Zhang, Xianzhi Yuan and Xianyong Li
Appl. Sci. 2023, 13(13), 7627; https://doi.org/10.3390/app13137627 - 28 Jun 2023
Viewed by 703
Abstract
Modelling both long- and short-term user interests from historical data is crucial for generating accurate recommendations. However, unifying these metrics across multiple application domains can be challenging, and existing approaches often rely on complex, intertwined models which can be difficult to interpret. To [...] Read more.
Modelling both long- and short-term user interests from historical data is crucial for generating accurate recommendations. However, unifying these metrics across multiple application domains can be challenging, and existing approaches often rely on complex, intertwined models which can be difficult to interpret. To address this issue, we propose a lightweight, plug-and-play interest enhancement module that fuses interest vectors from two independent models. After analyzing the dataset, we identify deviations in the recommendation performance of long- and short-term interest models. To compensate for these differences, we use feature enhancement and loss correction during training. In the fusion process, we explicitly split long-term interest features with longer duration into multiple local features. We then use a shared attention mechanism to fuse multiple local features with short-term interest features to obtain interaction features. To correct for bias between models, we introduce a comparison learning task that monitors the similarity between local features, short-term features, and interaction features. This adaptively reduces the distance between similar features. Our proposed module combines and compares multiple independent long-term and short-term interest models on multiple domain datasets. As a result, it not only accelerates the convergence of the models but also achieves outstanding performance in challenging recommendation scenarios. Full article
Show Figures

Figure 1

Back to TopTop