entropy-logo

Journal Browser

Journal Browser

Entropy in Soft Computing and Machine Learning Algorithms II

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Multidisciplinary Applications".

Deadline for manuscript submissions: closed (20 January 2023) | Viewed by 17503

Special Issue Editors

Departamento de Innovación Basada en la Información y el Conocimiento, Universidad de Guadalajara, CUCEI, Guadalajara 44430, Mexico
Interests: metaheuristic algorithms; bioinspired computation; image processing; machine learning; optimization
Special Issues, Collections and Topics in MDPI journals
Department of Mechanical Engineering, Bursa Uludağ University, 16059 Bursa, Turkey
Interests: optimum design; evolutionary algorithm; additive manufacturing; intelligent manufacturing; RBDO

Special Issue Information

Dear Colleagues,

Soft computing and machine learning algorithms are used in different fields of science and technology. They are important tools designed to solve complex real-life problems under uncertainty.

Entropy is a powerful tool that has changed the analysis of information. The use of entropy has been extended in soft computing and machine learning methodologies, from measuring uncertainty to exploring and exploiting search spaces in optimization. Different kinds of entropy are used depending on what is required. Moreover, it is necessary to use soft computing and machine learning methods to provide accurate solutions to complex problems in the information era. Hybrid algorithms are also important; they merge skills from different approaches and make decisions based on different rules to accurately explore the possible solutions.

Since the fields of soft computing and machine algorithms are constantly growing, following all the different branches in which entropy is used is complicated. Considering the above, this Special Issue aims to present the latest advances in soft computing and machine learning algorithms that employ or solve problems where entropy is included. It also seeks to include literature reviews and surveys on related topics.

Dr. Diego Oliva
Prof. Dr. Ali Rıza Yıldız
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • metaheuristics
  • bioinspired algorithms
  • cross entropy
  • Shannon entropy
  • fuzzy entropy
  • machine learning
  • neural networks
  • swarm optimization
  • evolutionary computation
  • fuzzy logic
  • genetic algorithms
  • deep learning

Related Special Issues

Published Papers (11 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

29 pages, 4759 KiB  
Article
A Hybrid Parallel Balanced Phasmatodea Population Evolution Algorithm and Its Application in Workshop Material Scheduling
by Song Han, Shanshan Chen, Fengting Yan, Jengshyang Pan and Yunxiang Zhu
Entropy 2023, 25(6), 848; https://doi.org/10.3390/e25060848 - 25 May 2023
Viewed by 821
Abstract
The phasmatodea population evolution algorithm (PPE) is a recently proposed meta-heuristic algorithm based on the evolutionary characteristics of the stick insect population. The algorithm simulates the features of convergent evolution, population competition, and population growth in the evolution process of the stick insect [...] Read more.
The phasmatodea population evolution algorithm (PPE) is a recently proposed meta-heuristic algorithm based on the evolutionary characteristics of the stick insect population. The algorithm simulates the features of convergent evolution, population competition, and population growth in the evolution process of the stick insect population in nature and realizes the above process through the population competition and growth model. Since the algorithm has a slow convergence speed and falls easily into local optimality, in this paper, it is mixed with the equilibrium optimization algorithm to make it easier to avoid the local optimum. Based on the hybrid algorithm, the population is grouped and processed in parallel to accelerate the algorithm’s convergence speed and achieve better convergence accuracy. On this basis, we propose the hybrid parallel balanced phasmatodea population evolution algorithm (HP_PPE), and this algorithm is compared and tested on the CEC2017, a novel benchmark function suite. The results show that the performance of HP_PPE is better than that of similar algorithms. Finally, this paper applies HP_PPE to solve the AGV workshop material scheduling problem. Experimental results show that HP_PPE can achieve better scheduling results than other algorithms. Full article
(This article belongs to the Special Issue Entropy in Soft Computing and Machine Learning Algorithms II)
Show Figures

Figure 1

19 pages, 7368 KiB  
Article
MSIA-Net: A Lightweight Infrared Target Detection Network with Efficient Information Fusion
by Jimin Yu, Shun Li, Shangbo Zhou and Hui Wang
Entropy 2023, 25(5), 808; https://doi.org/10.3390/e25050808 - 17 May 2023
Cited by 2 | Viewed by 1143
Abstract
In order to solve the problems of infrared target detection (i.e., the large models and numerous parameters), a lightweight detection network, MSIA-Net, is proposed. Firstly, a feature extraction module named MSIA, which is based on asymmetric convolution, is proposed, and it can greatly [...] Read more.
In order to solve the problems of infrared target detection (i.e., the large models and numerous parameters), a lightweight detection network, MSIA-Net, is proposed. Firstly, a feature extraction module named MSIA, which is based on asymmetric convolution, is proposed, and it can greatly reduce the number of parameters and improve the detection performance by reusing information. In addition, we propose a down-sampling module named DPP to reduce the information loss caused by pooling down-sampling. Finally, we propose a feature fusion structure named LIR-FPN that can shorten the information transmission path and effectively reduce the noise in the process of feature fusion. In order to improve the ability of the network to focus on the target, we introduce coordinate attention (CA) into the LIR-FPN; this integrates the location information of the target into the channel so as to obtain more expressive feature information. Finally, a comparative experiment with other SOTA methods was completed on the FLIR on-board infrared image dataset, which proved the powerful detection performance of MSIA-Net. Full article
(This article belongs to the Special Issue Entropy in Soft Computing and Machine Learning Algorithms II)
Show Figures

Figure 1

32 pages, 3832 KiB  
Article
Quantile-Adaptive Sufficient Variable Screening by Controlling False Discovery
by Zihao Yuan, Jiaqing Chen, Han Qiu and Yangxin Huang
Entropy 2023, 25(3), 524; https://doi.org/10.3390/e25030524 - 17 Mar 2023
Viewed by 990
Abstract
Sufficient variable screening rapidly reduces dimensionality with high probability in ultra-high dimensional modeling. To rapidly screen out the null predictors, a quantile-adaptive sufficient variable screening framework is developed by controlling the false discovery. Without any specification of an actual model, we first introduce [...] Read more.
Sufficient variable screening rapidly reduces dimensionality with high probability in ultra-high dimensional modeling. To rapidly screen out the null predictors, a quantile-adaptive sufficient variable screening framework is developed by controlling the false discovery. Without any specification of an actual model, we first introduce a compound testing procedure based on the conditionally imputing marginal rank correlation at different quantile levels of response to select active predictors in high dimensionality. The testing statistic can capture sufficient dependence through two paths: one is to control false discovery adaptively and the other is to control the false discovery rate by giving a prespecified threshold. It is computationally efficient and easy to implement. We establish the theoretical properties under mild conditions. Numerical studies including simulation studies and real data analysis contain supporting evidence that the proposal performs reasonably well in practical settings. Full article
(This article belongs to the Special Issue Entropy in Soft Computing and Machine Learning Algorithms II)
Show Figures

Figure A1

14 pages, 2976 KiB  
Article
A Pedestrian Detection Network Model Based on Improved YOLOv5
by Ming-Lun Li, Guo-Bing Sun and Jia-Xiang Yu
Entropy 2023, 25(2), 381; https://doi.org/10.3390/e25020381 - 19 Feb 2023
Cited by 8 | Viewed by 3066
Abstract
Advanced object detection methods always face high algorithmic complexity or low accuracy when used in pedestrian target detection for the autonomous driving system. This paper proposes a lightweight pedestrian detection approach called the YOLOv5s-G2 network to address these issues. We apply [...] Read more.
Advanced object detection methods always face high algorithmic complexity or low accuracy when used in pedestrian target detection for the autonomous driving system. This paper proposes a lightweight pedestrian detection approach called the YOLOv5s-G2 network to address these issues. We apply Ghost and GhostC3 modules in the YOLOv5s-G2 network to minimize computational cost during feature extraction while keeping the network’s capability of extracting features intact. The YOLOv5s-G2 network improves feature extraction accuracy by incorporating the Global Attention Mechanism (GAM) module. This application can extract relevant information for pedestrian target identification tasks and suppress irrelevant information, improving the unidentified problem of occluded and small targets by replacing the GIoU loss function used in the bounding box regression with the α-CIoU loss function. The YOLOv5s-G2 network is evaluated on the WiderPerson dataset to ensure its efficacy. Our proposed YOLOv5s-G2 network offers a 1.0% increase in detection accuracy and a 13.2% decrease in Floating Point Operations (FLOPs) compared to the existing YOLOv5s network. As a result, the YOLOv5s-G2 network is preferable for pedestrian identification as it is both more lightweight and more accurate. Full article
(This article belongs to the Special Issue Entropy in Soft Computing and Machine Learning Algorithms II)
Show Figures

Figure 1

13 pages, 294 KiB  
Article
On the Fitness Functions Involved in Genetic Algorithms and the Cryptanalysis of Block Ciphers
by Osmani Tito-Corrioso, Mijail Borges-Quintana, Miguel A. Borges-Trenard, Omar Rojas and Guillermo Sosa-Gómez
Entropy 2023, 25(2), 261; https://doi.org/10.3390/e25020261 - 31 Jan 2023
Cited by 2 | Viewed by 1548
Abstract
There are many algorithms used with different purposes in the area of cryptography. Amongst these, Genetic Algorithms have been used, particularly in the cryptanalysis of block ciphers. Interest in the use of and research on such algorithms has increased lately, with a special [...] Read more.
There are many algorithms used with different purposes in the area of cryptography. Amongst these, Genetic Algorithms have been used, particularly in the cryptanalysis of block ciphers. Interest in the use of and research on such algorithms has increased lately, with a special focus on the analysis and improvement of the properties and characteristics of these algorithms. In this way, the present work focuses on studying the fitness functions involved in Genetic Algorithms. First, a methodology was proposed to verify that the closeness to 1 of some fitness functions’ values that use decimal distance implies decimal closeness to the key. On the other hand, the foundation of a theory is developed in order to characterize such fitness functions and determine, a priori, if one method is more effective than another in the attack to block ciphers using Genetic Algorithms. Full article
(This article belongs to the Special Issue Entropy in Soft Computing and Machine Learning Algorithms II)
Show Figures

Figure 1

29 pages, 7443 KiB  
Article
Multi-Level Thresholding Image Segmentation Based on Improved Slime Mould Algorithm and Symmetric Cross-Entropy
by Yuanyuan Jiang, Dong Zhang, Wenchang Zhu and Li Wang
Entropy 2023, 25(1), 178; https://doi.org/10.3390/e25010178 - 16 Jan 2023
Cited by 4 | Viewed by 2141
Abstract
Multi-level thresholding image segmentation divides an image into multiple regions of interest and is a key step in image processing and image analysis. Aiming toward the problems of the low segmentation accuracy and slow convergence speed of traditional multi-level threshold image segmentation methods, [...] Read more.
Multi-level thresholding image segmentation divides an image into multiple regions of interest and is a key step in image processing and image analysis. Aiming toward the problems of the low segmentation accuracy and slow convergence speed of traditional multi-level threshold image segmentation methods, in this paper, we present multi-level thresholding image segmentation based on an improved slime mould algorithm (ISMA) and symmetric cross-entropy for global optimization and image segmentation tasks. First, elite opposition-based learning (EOBL) was used to improve the quality and diversity of the initial population and accelerate the convergence speed. The adaptive probability threshold was used to adjust the selection probability of the slime mould to enhance the ability of the algorithm to jump out of the local optimum. The historical leader strategy, which selects the optimal historical information as the leader for the position update, was found to improve the convergence accuracy. Subsequently, 14 benchmark functions were used to evaluate the performance of ISMA, comparing it with other well-known algorithms in terms of the optimization accuracy, convergence speed, and significant differences. Subsequently, we tested the segmentation quality of the method proposed in this paper on eight grayscale images and compared it with other image segmentation criteria and well-known algorithms. The experimental metrics include the average fitness (mean), standard deviation (std), peak signal to noise ratio (PSNR), structure similarity index (SSIM), and feature similarity index (FSIM), which we utilized to evaluate the quality of the segmentation. The experimental results demonstrated that the improved slime mould algorithm is superior to the other compared algorithms, and multi-level thresholding image segmentation based on the improved slime mould algorithm and symmetric cross-entropy can be effectively applied to the task of multi-level threshold image segmentation. Full article
(This article belongs to the Special Issue Entropy in Soft Computing and Machine Learning Algorithms II)
Show Figures

Figure 1

22 pages, 2133 KiB  
Article
Quantitative Study of Non-Linear Convection Diffusion Equations for a Rotating-Disc Electrode
by Fahad Sameer Alshammari, Hamad Jan, Muhammad Sulaiman, Din Prathumwan and Ghaylen Laouini
Entropy 2023, 25(1), 134; https://doi.org/10.3390/e25010134 - 09 Jan 2023
Cited by 1 | Viewed by 1411
Abstract
Rotating-disc electrodes (RDEs) are favored technologies for analyzing electrochemical processes in electrically charged cells and other revolving machines, such as engines, compressors, gearboxes, and generators. The model is based on the concept of the nonlinear entropy convection-diffusion equations, which are constructed using semi-boundaries [...] Read more.
Rotating-disc electrodes (RDEs) are favored technologies for analyzing electrochemical processes in electrically charged cells and other revolving machines, such as engines, compressors, gearboxes, and generators. The model is based on the concept of the nonlinear entropy convection-diffusion equations, which are constructed using semi-boundaries as an infinite notion. In this model, the surrogate solutions with different parameter values for the mathematical characterization of non-dimensional OH and H+ ion concentrations at a rotating-disc electrode (RDE) are investigated using an intelligent hybrid technique by utilizing neural networks (NN) and the Levenberg–Marquardt algorithm (LMA). Reference solutions were calculated using the RK-4 numerical method. Through the training, validation, and testing sampling of reference solutions, the NN-BLMA approximations were recorded. Error histograms, absolute error, curve fitting graphs, and regression graphs validated the NN-BLMA’s resilience and accuracy for the problem. Additionally, the comparison graphs between the reference solution and the NN-BLMA procedure established that our paradigm is reliable and accurate. Full article
(This article belongs to the Special Issue Entropy in Soft Computing and Machine Learning Algorithms II)
Show Figures

Figure 1

18 pages, 2944 KiB  
Article
A Dual Adaptive Interaction Click-Through Rate Prediction Based on Attention Logarithmic Interaction Network
by Shiqi Li, Zhendong Cui and Yongquan Pei
Entropy 2022, 24(12), 1831; https://doi.org/10.3390/e24121831 - 15 Dec 2022
Cited by 1 | Viewed by 1224
Abstract
Click-through rate (CTR) prediction is crucial for computing advertisement and recommender systems. The key challenge of CTR prediction is to accurately capture user interests and deliver suitable advertisements to the right people. However, there are an immense number of features in CTR prediction [...] Read more.
Click-through rate (CTR) prediction is crucial for computing advertisement and recommender systems. The key challenge of CTR prediction is to accurately capture user interests and deliver suitable advertisements to the right people. However, there are an immense number of features in CTR prediction datasets, which hardly fit when only using an individual feature. To solve this problem, feature interaction that combines several features via an operation is introduced to enhance prediction performance. Many factorizations machine-based models and deep learning methods have been proposed to capture feature interaction for CTR prediction. They follow an enumeration-filter pattern that could not determine the appropriate order of feature interaction and useful feature interaction. The attention logarithmic network (ALN) is presented in this paper, which uses logarithmic neural networks (LNN) to model feature interactions, and the squeeze excitation (SE) mechanism to adaptively model the importance of higher-order feature interactions. At first, the embedding vector of the input was absolutized and a very small positive number was added to the zeros of the embedding vector, which made the LNN input positive. Then, the adaptive-order feature interactions were learned by logarithmic transformation and exponential transformation in the LNN. Finally, SE was applied to model the importance of high-order feature interactions adaptively for enhancing CTR performance. Based on this, the attention logarithmic interaction network (ALIN) was proposed for the effectiveness and accuracy of CTR, which integrated Newton’s identity into ALN. ALIN supplements the loss of information, which is caused by the operation becoming positive and by adding a small positive value to the embedding vector. Experiments are conducted on two datasets, and the results prove that ALIN is efficient and effective. Full article
(This article belongs to the Special Issue Entropy in Soft Computing and Machine Learning Algorithms II)
Show Figures

Figure 1

23 pages, 4297 KiB  
Article
Optimized LightGBM Power Fingerprint Identification Based on Entropy Features
by Lin Lin, Jie Zhang, Na Zhang, Jiancheng Shi and Cheng Chen
Entropy 2022, 24(11), 1558; https://doi.org/10.3390/e24111558 - 29 Oct 2022
Cited by 3 | Viewed by 1416
Abstract
The huge amount of power fingerprint data often has the problem of unbalanced categories and is difficult to upload by the limited data transmission rate for IoT communications. An optimized LightGBM power fingerprint extraction and identification method based on entropy features is proposed. [...] Read more.
The huge amount of power fingerprint data often has the problem of unbalanced categories and is difficult to upload by the limited data transmission rate for IoT communications. An optimized LightGBM power fingerprint extraction and identification method based on entropy features is proposed. First, the voltage and current signals were extracted on the basis of the time-domain features and V-I trajectory features, and a 56-dimensional original feature set containing six entropy features was constructed. Then, the Boruta algorithm with a light gradient boosting machine (LightGBM) as the base learner was used for feature selection of the original feature set, and a 23-dimensional optimal feature subset containing five entropy features was determined. Finally, the Optuna algorithm was used to optimize the hyperparameters of the LightGBM classifier. The classification performance of the power fingerprint identification model on imbalanced datasets was further improved by improving the loss function of the LightGBM model. The experimental results prove that the method can effectively reduce the computational complexity of feature extraction and reduce the amount of power fingerprint data transmission. It meets the recognition accuracy and efficiency requirements of a massive power fingerprint identification system. Full article
(This article belongs to the Special Issue Entropy in Soft Computing and Machine Learning Algorithms II)
Show Figures

Figure 1

18 pages, 850 KiB  
Article
Utility–Privacy Trade-Off in Distributed Machine Learning Systems
by Xia Zeng, Chuanchuan Yang and Bin Dai
Entropy 2022, 24(9), 1299; https://doi.org/10.3390/e24091299 - 14 Sep 2022
Cited by 1 | Viewed by 1315
Abstract
In distributed machine learning (DML), though clients’ data are not directly transmitted to the server for model training, attackers can obtain the sensitive information of clients by analyzing the local gradient parameters uploaded by clients. For this case, we use the differential privacy [...] Read more.
In distributed machine learning (DML), though clients’ data are not directly transmitted to the server for model training, attackers can obtain the sensitive information of clients by analyzing the local gradient parameters uploaded by clients. For this case, we use the differential privacy (DP) mechanism to protect the clients’ local parameters. In this paper, from an information-theoretic point of view, we study the utility–privacy trade-off in DML with the help of the DP mechanism. Specifically, three cases including independent clients’ local parameters with independent DP noise, dependent clients’ local parameters with independent/dependent DP noise are considered. Mutual information and conditional mutual information are used to characterize utility and privacy, respectively. First, we show the relationship between utility and privacy for the three cases. Then, we show the optimal noise variance that achieves the maximal utility under a certain level of privacy. Finally, the results of this paper are further illustrated by numerical results. Full article
(This article belongs to the Special Issue Entropy in Soft Computing and Machine Learning Algorithms II)
Show Figures

Figure 1

11 pages, 292 KiB  
Article
Analytic Function Approximation by Path-Norm-Regularized Deep Neural Networks
by Aleksandr Beknazaryan
Entropy 2022, 24(8), 1136; https://doi.org/10.3390/e24081136 - 16 Aug 2022
Cited by 2 | Viewed by 947
Abstract
We show that neural networks with an absolute value activation function and with network path norm, network sizes and network weights having logarithmic dependence on 1/ε can ε-approximate functions that are analytic on certain regions of Cd. [...] Read more.
We show that neural networks with an absolute value activation function and with network path norm, network sizes and network weights having logarithmic dependence on 1/ε can ε-approximate functions that are analytic on certain regions of Cd. Full article
(This article belongs to the Special Issue Entropy in Soft Computing and Machine Learning Algorithms II)
Back to TopTop