Recent Advances in Artificial Intelligence, Machine Learning, and Deep Learning

A special issue of Applied Sciences (ISSN 2076-3417). This special issue belongs to the section "Computing and Artificial Intelligence".

Deadline for manuscript submissions: closed (30 January 2024) | Viewed by 16923

Special Issue Editor

School of Electronics Engineering, Kyungpook National University, Daegu 41566, Korea
Interests: artificial Intelligence; brain–computer interface
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Over the last few decades, artificial intelligence (AI) techniques have significantly advanced; thus, they have great potential for future technologies. Traditionally, in the AI field, various machine learning algorithms have been used to extract significant features. Recently, researchers have focused on applying deep learning models to various domains such as computer vision, natural language processing, medical imaging, signal processing, and the brain–computer interface. However, there are still some issues in the field arising from a lack of large-scale data, data imbalance, opaqueness, etc.

Therefore, sophisticated approaches based on novel machine or deep learning models are needed to overcome these issues and further advance research in this field. The goal of this Special Issue is to inspire collaboration among researchers in the AI field as well as other related research fields, sharing approaches, opinions, and comments to increase our understanding of AI techniques.

Prof. Dr. Sangtae Ahn
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Applied Sciences is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • computer vision
  • natural language processing
  • data augmentation
  • continual learning, lifelong learning, transfer learning
  • explainable artificial intelligence
  • advances and challenges on artificial
  • brain–computer interface
  • diagnosis and prediction of diseases
  • intelligent signal processing

Published Papers (7 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

Jump to: Review

21 pages, 20331 KiB  
Article
Deep Fusion Prediction Method for Nonstationary Time Series Based on Feature Augmentation and Extraction
by Yu-Lei Zhang, Yu-Ting Bai, Xue-Bo Jin, Ting-Li Su, Jian-Lei Kong and Wei-Zhen Zheng
Appl. Sci. 2023, 13(8), 5088; https://doi.org/10.3390/app13085088 - 19 Apr 2023
Viewed by 1063
Abstract
Deep learning effectively identifies and predicts modes but faces performance reduction under few-shot learning conditions. In this paper, a time series prediction framework for small samples is proposed, including a data augmentation algorithm, time series trend decomposition, multi-model prediction, and error-based fusion. First, [...] Read more.
Deep learning effectively identifies and predicts modes but faces performance reduction under few-shot learning conditions. In this paper, a time series prediction framework for small samples is proposed, including a data augmentation algorithm, time series trend decomposition, multi-model prediction, and error-based fusion. First, data samples are augmented by retaining and extracting time series features. Second, the expanded data are decomposed based on data trends, and then, multiple deep models are used for prediction. Third, the models’ predictive outputs are combined with an error estimate from the intersection of covariances. Finally, the method is verified using natural systems and classic small-scale simulation datasets. The results show that the proposed method can improve the prediction accuracy of small sample sets with data augmentation and multi-model fusion. Full article
Show Figures

Figure 1

17 pages, 1216 KiB  
Article
Stochastic Weight Averaging Revisited
by Hao Guo, Jiyong Jin and Bin Liu
Appl. Sci. 2023, 13(5), 2935; https://doi.org/10.3390/app13052935 - 24 Feb 2023
Cited by 3 | Viewed by 2093
Abstract
Averaging neural network weights sampled by a backbone stochastic gradient descent (SGD) is a simple-yet-effective approach to assist the backbone SGD in finding better optima, in terms of generalization. From a statistical perspective, weight-averaging contributes to variance reduction. Recently, a well-established stochastic weight-averaging [...] Read more.
Averaging neural network weights sampled by a backbone stochastic gradient descent (SGD) is a simple-yet-effective approach to assist the backbone SGD in finding better optima, in terms of generalization. From a statistical perspective, weight-averaging contributes to variance reduction. Recently, a well-established stochastic weight-averaging (SWA) method was proposed, which featured the application of a cyclical or high-constant (CHC) learning-rate schedule for generating weight samples for weight-averaging. Then, a new insight on weight-averaging was introduced, which stated that weight average assisted in discovering a wider optima and resulted in better generalization. We conducted extensive experimental studies concerning SWA, involving 12 modern deep neural network model architectures and 12 open-source image, graph, and text datasets as benchmarks. We disentangled the contributions of the weight-averaging operation and the CHC learning-rate schedule for SWA, showing that the weight-averaging operation in SWA still contributed to variance reduction, and the CHC learning-rate schedule assisted in exploring the parameter space more widely than the backbone SGD, which could be be under-fitted due to a lack of training budget. We then presented an algorithm termed periodic SWA (PSWA) that comprised a series of weight-averaging operations to exploit such wide parameter space structures as explored by the CHC learning-rate schedule, and we empirically demonstrated that PSWA outperformed its backbone SGD remarkably. Full article
Show Figures

Figure 1

17 pages, 3995 KiB  
Article
RSMDA: Random Slices Mixing Data Augmentation
by Teerath Kumar, Alessandra Mileo, Rob Brennan and Malika Bendechache
Appl. Sci. 2023, 13(3), 1711; https://doi.org/10.3390/app13031711 - 29 Jan 2023
Viewed by 2487
Abstract
Advanced data augmentation techniques have demonstrated great success in deep learning algorithms. Among these techniques, single-image-based data augmentation (SIBDA), in which a single image’s regions are randomly erased in different ways, has shown promising results. However, randomly erasing image regions in SIBDA can [...] Read more.
Advanced data augmentation techniques have demonstrated great success in deep learning algorithms. Among these techniques, single-image-based data augmentation (SIBDA), in which a single image’s regions are randomly erased in different ways, has shown promising results. However, randomly erasing image regions in SIBDA can cause a loss of the key discriminating features, consequently misleading neural networks and lowering their performance. To alleviate this issue, in this paper, we propose the random slices mixing data augmentation (RSMDA) technique, in which slices of one image are placed onto another image to create a third image that enriches the diversity of the data. RSMDA also mixes the labels of the original images to create an augmented label for the new image to exploit label smoothing. Furthermore, we propose and investigate three strategies for RSMDA: (i) the vertical slices mixing strategy, (ii) the horizontal slices mixing strategy, and (iii) a random mix of both strategies. Of these strategies, the horizontal slice mixing strategy shows the best performance. To validate the proposed technique, we perform several experiments using different neural networks across four datasets: fashion-MNIST, CIFAR10, CIFAR100, and STL10. The experimental results of the image classification with RSMDA showed better accuracy and robustness than the state-of-the-art (SOTA) single-image-based and multi-image-based methods. Finally, class activation maps are employed to visualize the focus of the neural network and compare maps using the SOTA data augmentation methods. Full article
Show Figures

Figure 1

13 pages, 3435 KiB  
Article
A Data-Driven Model of Cable Insulation Defect Based on Convolutional Neural Networks
by Weixing Han, Guang Yang, Chunsheng Hao, Zhengqi Wang, Dejing Kong and Yu Dong
Appl. Sci. 2022, 12(16), 8374; https://doi.org/10.3390/app12168374 - 22 Aug 2022
Cited by 2 | Viewed by 1281
Abstract
The insulation condition of cables has been the focus of research in power systems. To address the problem that the electric field is not easily measured under the operating condition of 10 kV transmission cables with insulation defects, this paper proposes a data-driven [...] Read more.
The insulation condition of cables has been the focus of research in power systems. To address the problem that the electric field is not easily measured under the operating condition of 10 kV transmission cables with insulation defects, this paper proposes a data-driven cable insulation defect model based on a convolutional neural network approach. The electric field data during cable operation is obtained by finite element calculation, and a multi-dimensional input feature quantity and a data set with the electric field strength as the output feature quantity are constructed. A convolutional neural network algorithm is applied to construct a cable data-driven model. The model is used to construct a cloud map of the electric field distribution during cable operation. Comparing the results with the finite element method, the overall accuracy of the data-driven model is 94.3% and the calculation time of the data-driven model is 0.025 s, which is 360 times faster than the finite element calculation. The results show that the data-driven model can quickly construct the electric field distribution under cable insulation defects, laying the foundation for a digital twin structure for cables. Full article
Show Figures

Figure 1

15 pages, 12349 KiB  
Article
Intelligent Measurement of Coal Moisture Based on Microwave Spectrum via Distance-Weighted kNN
by Ming Li, Jun Tian, Yuliang Wang, Haiyang Zhang, Dongping Yang and Meng Lei
Appl. Sci. 2022, 12(12), 6199; https://doi.org/10.3390/app12126199 - 18 Jun 2022
Viewed by 1451
Abstract
Realizing the rapid measurement of coal moisture content (MC) is of great significance. However, existing measurement methods are time-consuming and damage the original properties of the samples. To address these concerns, a coal MC intelligent measurement system is designed in this study that [...] Read more.
Realizing the rapid measurement of coal moisture content (MC) is of great significance. However, existing measurement methods are time-consuming and damage the original properties of the samples. To address these concerns, a coal MC intelligent measurement system is designed in this study that integrates microwave spectrum analysis and the distance-weighted k-nearest neighbor (DW-kNN) algorithm to realize rapid and non-destructive measurement of coal MC. Specifically, the measurement system is built using portable microwave analysis equipment, which can efficiently collect the microwave signals of coal. To improve the cleanliness of modeling data, an iterative clipping method based on Mahalanobis distance (MD-ICM) is used to detect and eliminate outliers. Based on multiple microwave frequency bands, various machine learning methods are evaluated, and it is found that coal MC measurement using broad frequency signals of 8.05–12.01 GHz yields the best results. Experiments are also carried out on coals from different regions to examine the regional robustness of the proposed method. The results of on-site testing with 27 additional samples show that the method based on the combination of microwave spectrum analysis and DW-kNN has a potential application prospect in the rapid measurement of coal MC. Full article
Show Figures

Figure 1

11 pages, 1100 KiB  
Article
Comparative Study of Multiclass Text Classification in Research Proposals Using Pretrained Language Models
by Eunchan Lee, Changhyeon Lee and Sangtae Ahn
Appl. Sci. 2022, 12(9), 4522; https://doi.org/10.3390/app12094522 - 29 Apr 2022
Cited by 3 | Viewed by 3083
Abstract
Recently, transformer-based pretrained language models have demonstrated stellar performance in natural language understanding (NLU) tasks. For example, bidirectional encoder representations from transformers (BERT) have achieved outstanding performance through masked self-supervised pretraining and transformer-based modeling. However, the original BERT may only be effective for [...] Read more.
Recently, transformer-based pretrained language models have demonstrated stellar performance in natural language understanding (NLU) tasks. For example, bidirectional encoder representations from transformers (BERT) have achieved outstanding performance through masked self-supervised pretraining and transformer-based modeling. However, the original BERT may only be effective for English-based NLU tasks, whereas its effectiveness for other languages such as Korean is limited. Thus, the applicability of BERT-based language models pretrained in languages other than English to NLU tasks based on those languages must be investigated. In this study, we comparatively evaluated seven BERT-based pretrained language models and their expected applicability to Korean NLU tasks. We used the climate technology dataset, which is a Korean-based large text classification dataset, in research proposals involving 45 classes. We found that the BERT-based model pretrained on the most recent Korean corpus performed the best in terms of Korean-based multiclass text classification. This suggests the necessity of optimal pretraining for specific NLU tasks, particularly those in languages other than English. Full article
Show Figures

Figure 1

Review

Jump to: Research

29 pages, 8179 KiB  
Review
A Review of Deep Learning Applications for Railway Safety
by Kyuetaek Oh, Mintaek Yoo, Nayoung Jin, Jisu Ko, Jeonguk Seo, Hyojin Joo and Minsam Ko
Appl. Sci. 2022, 12(20), 10572; https://doi.org/10.3390/app122010572 - 19 Oct 2022
Cited by 8 | Viewed by 4001
Abstract
Railways speedily transport many people and goods nationwide, so railway accidents can pose immense damage. However, the infrastructure of railways is so complex that its maintenance is challenging and expensive. Therefore, using artificial intelligence for railway safety has attracted many researchers. This paper [...] Read more.
Railways speedily transport many people and goods nationwide, so railway accidents can pose immense damage. However, the infrastructure of railways is so complex that its maintenance is challenging and expensive. Therefore, using artificial intelligence for railway safety has attracted many researchers. This paper examines artificial intelligence applications for railway safety, mainly focusing on deep learning approaches. This paper first introduces deep learning methods widely used for railway safety. Then, we investigated and classified earlier studies into four representative application areas: (1) railway infrastructure (catenary, surface, components, and geometry), (2) train body and bogie (door, wheel, suspension, bearing, etc.), (3) operation (railway detection, railroad trespassing, wind risk, train running safety, etc.), and (4) station (air quality control, accident prevention, etc.). We present fundamental problems and popular approaches for each application area. Finally, based on the literature reviews, we discuss the opportunities and challenges of artificial intelligence for railway safety. Full article
Show Figures

Figure 1

Back to TopTop