# Escaping Stagnation through Improved Orca Predator Algorithm with Deep Reinforcement Learning for Feature Selection

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

## 3. Preliminaries

#### 3.1. Biomimetic Orca Predator Algorithm

#### 3.1.1. Chase Phase

#### 3.1.2. Attack Phase

Algorithm 1: Pseudocode for the orca predator method. |

#### 3.2. Deep Reinforcement Learning

#### 3.3. Feature Selection

## 4. Developed Solution

Algorithm 2: Pseudocode for the improved orca predator method. |

## 5. Experimental Setup

#### 5.1. Methodology

- Preparation and planning: Define specific multi-objective goals for feature selection effectiveness, aiming to minimize the number of selected features while simultaneously maximizing accuracy and the ${F}_{1}$ score. Design experiments to systematically evaluate the enhanced technique under controlled conditions, ensuring a balanced optimization of these criteria.
- Execution and assessment: Perform a multi-faceted evaluation of the technique, assessing not only the quality of the solutions generated but also the computational efficiency and convergence properties. Employ rigorous statistical tests to compare the performance with baseline methods. Here, to evaluate data independence and statistical significance, we use the Kolmogorov–Smirnov–Lilliefors test for assessing sample autonomy and the Mann–Whitney–Wilcoxon test for comparative analysis. This approach involves calculating the fitness from each one executions per instance.
- Analysis and validation: Conduct thorough in-depth analysis to understand the deep Q-learning’s parameter influence and the orca predator algorithm’s behavior on the feature selection task. This involves iterating over a range of hyperparameters to fine-tune the model, using the dataset to validate the consistency and stability of the selected features. To ensure the validity of the results generated by our proposal, we conducted tests to evaluate the final outcomes. We can assure that all simulated experiments were carried out with reliability.

#### 5.2. Dataset

#### 5.3. Implementation Aspects

## 6. Results and Discussion

#### 6.1. Statistical Test

#### 6.2. Comparing OPADQL vs. State-of-the-Art Algorithms

## 7. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Gad, A.G. Particle swarm optimization algorithm and its applications: A systematic review. Arch. Comput. Methods Eng.
**2022**, 29, 2531–2561. [Google Scholar] [CrossRef] - Salhi, S.; Thompson, J. An overview of heuristics and metaheuristics. In The Palgrave Handbook of Operations Research; Springer: Berlin/Heidelberg, Germany, 2022; pp. 353–403. [Google Scholar]
- Abualigah, L.; Yousri, D.; Abd Elaziz, M.; Ewees, A.A.; Al-Qaness, M.A.; Gandomi, A.H. Aquila optimizer: A novel meta-heuristic optimization algorithm. Comput. Ind. Eng.
**2021**, 157, 107250. [Google Scholar] [CrossRef] - Prabha, S.; Yadav, R. Differential evolution with biological-based mutation operator. Eng. Sci. Technol. Int. J.
**2020**, 23, 253–263. [Google Scholar] [CrossRef] - Olivares, R.; Soto, R.; Crawford, B.; Riquelme, F.; Munoz, R.; Ríos, V.; Cabrera, R.; Castro, C. Entropy-based diversification approach for bio-computing methods. Entropy
**2022**, 24, 1293. [Google Scholar] [CrossRef] [PubMed] - Molina, D.; Poyatos, J.; Ser, J.D.; García, S.; Hussain, A.; Herrera, F. Comprehensive taxonomies of nature-and bio-inspired optimization: Inspiration versus algorithmic behavior, critical analysis recommendations. Cogn. Comput.
**2020**, 12, 897–939. [Google Scholar] [CrossRef] - Ahmed, H.R. An efficient fitness-based stagnation detection method for particle swarm optimization. In Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation, Vancouver, BC, Canada, 12–16 July 2014; pp. 1029–1032. [Google Scholar]
- Worasucheep, C. A particle swarm optimization with stagnation detection and dispersion. In Proceedings of the 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence), Hong Kong, China, 1–6 June 2008; IEEE: Piscataway, NJ, USA, 2008; pp. 424–429. [Google Scholar]
- Zaman, H.R.R.; Gharehchopogh, F.S. An improved particle swarm optimization with backtracking search optimization algorithm for solving continuous optimization problems. Eng. Comput.
**2022**, 38, 2797–2831. [Google Scholar] [CrossRef] - Dokeroglu, T.; Kucukyilmaz, T.; Talbi, E.G. Hyper-heuristics: A survey and taxonomy. Comput. Ind. Eng.
**2024**, 187, 109815. [Google Scholar] [CrossRef] - Chen, X.; Zhang, K.; Ji, Z.; Shen, X.; Liu, P.; Zhang, L.; Wang, J.; Yao, J. Progress and Challenges of Integrated Machine Learning and Traditional Numerical Algorithms: Taking Reservoir Numerical Simulation as an Example. Mathematics
**2023**, 11, 4418. [Google Scholar] [CrossRef] - Peres, F.; Castelli, M. Combinatorial Optimization Problems and Metaheuristics: Review, Challenges, Design, and Development. Appl. Sci.
**2021**, 11, 6449. [Google Scholar] [CrossRef] - Remeseiro, B.; Bolon-Canedo, V. A review of feature selection methods in medical applications. Comput. Biol. Med.
**2019**, 112, 103375. [Google Scholar] [CrossRef] - Colaco, S.; Kumar, S.; Tamang, A.; Biju, V.G. A review on feature selection algorithms. In Emerging Research in Computing, Information, Communication and Applications: ERCICA 2018; Springer: Berlin/Heidelberg, Germany, 2019; Volume 2, pp. 133–153. [Google Scholar]
- Fazeli, S. ECG Heartbeat Categorization Dataset. 2018. Available online: https://www.kaggle.com/datasets/shayanfazeli/heartbeat (accessed on 17 April 2024).
- Jakšić, Z.; Devi, S.; Jakšić, O.; Guha, K. A comprehensive review of bio-inspired optimization algorithms including applications in microelectronics and nanophotonics. Biomimetics
**2023**, 8, 278. [Google Scholar] [CrossRef] [PubMed] - Liang, Y.C.; Cuevas Juarez, J.R. A self-adaptive virus optimization algorithm for continuous optimization problems. Soft Comput.
**2020**, 24, 13147–13166. [Google Scholar] [CrossRef] - Olamaei, J.; Moradi, M.; Kaboodi, T. A new adaptive modified firefly algorithm to solve optimal capacitor placement problem. In Proceedings of the 18th Electric Power Distribution Conference, Turin, Italy, 6–9 June 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 1–6. [Google Scholar]
- Li, X.; Yin, M. Self-adaptive constrained artificial bee colony for constrained numerical optimization. Neural Comput. Appl.
**2014**, 24, 723–734. [Google Scholar] [CrossRef] - Li, X.; Yin, M. Modified cuckoo search algorithm with self adaptive parameter method. Inf. Sci.
**2015**, 298, 80–97. [Google Scholar] [CrossRef] - Cui, L.; Li, G.; Zhu, Z.; Wen, Z.; Lu, N.; Lu, J. A novel differential evolution algorithm with a self-adaptation parameter control method by differential evolution. Soft Comput.
**2018**, 22, 6171–6190. [Google Scholar] [CrossRef] - De Barros, J.B.; Sampaio, R.C.; Llanos, C.H. An adaptive discrete particle swarm optimization for mapping real-time applications onto network-on-a-chip based MPSoCs. In Proceedings of the 32nd Symposium on Integrated Circuits and Systems Design, São Paulo, Brazil, 26–30 August 2019; pp. 1–6. [Google Scholar]
- Zhang, L.; Chen, H.; Wang, W.; Liu, S. Improved Wolf Pack Algorithm for Solving Traveling Salesman Problem. In FSDM; IOS Press: Amsterdam, The Netherlands, 2018; pp. 131–140. [Google Scholar]
- Nasser, A.B.; Zamli, K.Z. Parameter free flower algorithm based strategy for pairwise testing. In Proceedings of the 2018 7th International Conference on Software and Computer Applications, Kuantan, Malaysia, 8–10 February 2018; pp. 46–50. [Google Scholar]
- Cruz-Salinas, A.F.; Perdomo, J.G. Self-adaptation of genetic operators through genetic programming techniques. In Proceedings of the Genetic and Evolutionary Computation Conference, Berlin, Germany, 15–19 July 2017; pp. 913–920. [Google Scholar]
- Kavoosi, M.; Dulebenets, M.A.; Abioye, O.F.; Pasha, J.; Wang, H.; Chi, H. An augmented self-adaptive parameter control in evolutionary computation: A case study for the berth scheduling problem. Adv. Eng. Inform.
**2019**, 42, 100972. [Google Scholar] [CrossRef] - Bacanin, N.; Stoean, C.; Zivkovic, M.; Jovanovic, D.; Antonijevic, M.; Mladenovic, D. Multi-swarm algorithm for extreme learning machine optimization. Sensors
**2022**, 22, 4204. [Google Scholar] [CrossRef] [PubMed] - Wong, L.H.; Looi, C.K. Swarm intelligence: New techniques for adaptive systems to provide learning support. Interact. Learn. Environ.
**2012**, 20, 19–40. [Google Scholar] [CrossRef] - Kicska, G.; Kiss, A. Comparing swarm intelligence algorithms for dimension reduction in machine learning. Big Data Cogn. Comput.
**2021**, 5, 36. [Google Scholar] [CrossRef] - Mavrovouniotis, M.; Li, C.; Yang, S. A survey of swarm intelligence for dynamic optimization: Algorithms and applications. Swarm Evol. Comput.
**2017**, 33, 1–17. [Google Scholar] [CrossRef] - Seyyedabbasi, A.; Aliyev, R.; Kiani, F.; Gulle, M.U.; Basyildiz, H.; Shah, M.A. Hybrid algorithms based on combining reinforcement learning and metaheuristic methods to solve global optimization problems. Knowl.-Based Syst.
**2021**, 223, 107044. [Google Scholar] [CrossRef] - Sadeg, S.; Hamdad, L.; Remache, A.R.; Karech, M.N.; Benatchba, K.; Habbas, Z. Qbso-fs: A reinforcement learning based bee swarm optimization metaheuristic for feature selection. In Proceedings of the Advances in Computational Intelligence: 15th International Work-Conference on Artificial Neural Networks, IWANN 2019, Gran Canaria, Spain, 12–14 June 2019; Part II 15. Springer: Berlin/Heidelberg, Germany, 2019; pp. 785–796. [Google Scholar]
- Sagban, R.; Ku-Mahamud, K.R.; Bakar, M.S.A. Nature-inspired parameter controllers for ACO-based reactive search. Res. J. Appl. Sci. Eng. Technol.
**2015**, 11, 109–117. [Google Scholar] [CrossRef] - Nijimbere, D.; Zhao, S.; Gu, X.; Esangbedo, M.O.; Dominique, N. Tabu search guided by reinforcement learning for the max-mean dispersion problem. J. Ind. Manag. Optim.
**2020**, 17, 3223–3246. [Google Scholar] [CrossRef] - Reyes-Rubiano, L.; Juan, A.; Bayliss, C.; Panadero, J.; Faulin, J.; Copado, P. A biased-randomized learnheuristic for solving the team orienteering problem with dynamic rewards. Transp. Res. Procedia
**2020**, 47, 680–687. [Google Scholar] [CrossRef] - Kusy, M.; Zajdel, R. Stateless Q-learning algorithm for training of radial basis function based neural networks in medical data classification. In Intelligent Systems in Technical and Medical Diagnostics; Springer: Berlin/Heidelberg, Germany, 2014; pp. 267–278. [Google Scholar]
- Kalaiselvi, B.; Pushparani, M. A novel impulsive genetic fuzzy C-means for task scheduling and hybridization of improved Fire Hawk optimizer and enhanced Deep Q-Learning algorithm for load balancing in cloud computing. J. Data Acquis. Processing
**2023**, 38, 1091. [Google Scholar] - Agrawal, P.; Abutarboush, H.F.; Ganesh, T.; Mohamed, A.W. Metaheuristic algorithms on feature selection: A survey of one decade of research (2009–2019). IEEE Access
**2021**, 9, 26766–26791. [Google Scholar] [CrossRef] - Dokeroglu, T.; Deniz, A.; Kiziloz, H.E. A comprehensive survey on recent metaheuristics for feature selection. Neurocomputing
**2022**, 494, 269–296. [Google Scholar] [CrossRef] - Ren, K.; Zeng, Y.; Cao, Z.; Zhang, Y. ID-RDRL: A deep reinforcement learning-based feature selection intrusion detection model. Sci. Rep.
**2022**, 12, 15370. [Google Scholar] [CrossRef] [PubMed] - Priya, S.; Kumar, K. Feature Selection with Deep Reinforcement Learning for Intrusion Detection System. Comput. Syst. Sci. Eng.
**2023**, 46, 3340–3353. [Google Scholar] [CrossRef] - Barrera-García, J.; Cisternas-Caneo, F.; Crawford, B.; Gómez Sánchez, M.; Soto, R. Feature Selection Problem and Metaheuristics: A Systematic Literature Review about Its Formulation, Evaluation and Applications. Biomimetics
**2024**, 9, 9. [Google Scholar] [CrossRef] - Mengash, H.A.; Alruwais, N.; Kouki, F.; Singla, C.; Abd Elhameed, E.S.; Mahmud, A. Archimedes Optimization Algorithm-Based Feature Selection with Hybrid Deep-Learning-Based Churn Prediction in Telecom Industries. Biomimetics
**2023**, 9, 1. [Google Scholar] [CrossRef] [PubMed] - Jiang, Y.; Wu, Q.; Zhu, S.; Zhang, L. Orca predation algorithm: A novel bio-inspired algorithm for global optimization problems. Expert Syst. Appl.
**2022**, 188, 116026. [Google Scholar] [CrossRef] - Sutton, R.S.; Barto, A.G. Reinforcement Learning: An Introduction; MIT Press: Cambridge, MA, USA, 2018. [Google Scholar]
- Wang, L.; Pan, Z.; Wang, J. A review of reinforcement learning based intelligent optimization for manufacturing scheduling. Complex Syst. Model. Simul.
**2021**, 1, 257–270. [Google Scholar] [CrossRef] - Sun, H.; Yang, L.; Gu, Y.; Pan, J.; Wan, F.; Song, C. Bridging locomotion and manipulation using reconfigurable robotic limbs via reinforcement learning. Biomimetics
**2023**, 8, 364. [Google Scholar] [CrossRef] [PubMed] - Zhu, K.; Zhang, T. Deep reinforcement learning based mobile robot navigation: A review. Tsinghua Sci. Technol.
**2021**, 26, 674–691. [Google Scholar] [CrossRef] - Azar, A.T.; Koubaa, A.; Ali Mohamed, N.; Ibrahim, H.A.; Ibrahim, Z.F.; Kazim, M.; Ammar, A.; Benjdira, B.; Khamis, A.M.; Hameed, I.A.; et al. Drone deep reinforcement learning: A review. Electronics
**2021**, 10, 999. [Google Scholar] [CrossRef] - Alavizadeh, H.; Alavizadeh, H.; Jang-Jaccard, J. Deep Q-learning based reinforcement learning approach for network intrusion detection. Computers
**2022**, 11, 41. [Google Scholar] [CrossRef] - Zhang, L.; Tang, L.; Zhang, S.; Wang, Z.; Shen, X.; Zhang, Z. A Self-Adaptive Reinforcement-Exploration Q-Learning Algorithm. Symmetry
**2021**, 13, 1057. [Google Scholar] [CrossRef] - Jang, B.; Kim, M.; Harerimana, G.; Kim, J.W. Q-learning algorithms: A comprehensive classification and applications. IEEE Access
**2019**, 7, 133653–133667. [Google Scholar] [CrossRef] - Wang, H.n.; Liu, N.; Zhang, Y.y.; Feng, D.w.; Huang, F.; Li, D.s.; Zhang, Y.m. Deep reinforcement learning: A survey. Front. Inf. Technol. Electron. Eng.
**2020**, 21, 1726–1744. [Google Scholar] [CrossRef] - Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A.A.; Veness, J.; Bellemare, M.G.; Graves, A.; Riedmiller, M.; Fidjeland, A.K.; Ostrovski, G.; et al. Human-level control through deep reinforcement learning. Nature
**2015**, 518, 529–533. [Google Scholar] [CrossRef] [PubMed] - Diekmann, N.; Walther, T.; Vijayabaskaran, S.; Cheng, S. Deep reinforcement learning in a spatial navigation task: Multiple contexts and their representation. In Proceedings of the 2019 Conference on Cognitive Computational Neuroscience, Berlin, Germany, 13–16 September 2019. [Google Scholar] [CrossRef]
- Schaul, T.; Quan, J.; Antonoglou, I.; Silver, D. Prioritized Experience Replay. arXiv
**2015**, arXiv:1511.05952. [Google Scholar] - Ramicic, M.; Bonarini, A. Correlation minimizing replay memory in temporal-difference reinforcement learning. Neurocomputing
**2020**, 393, 91–100. [Google Scholar] [CrossRef] - Liu, F.; Viano, L.; Cevher, V. Understanding Deep Neural Function Approximation in Reinforcement Learning via ϵ-Greedy Exploration. In Proceedings of the NeurIPS 2022, New Orleans, LA, USA, 28 November–9 December 2022. [Google Scholar]
- Nguyen, B.H.; Xue, B.; Zhang, M. A survey on swarm intelligence approaches to feature selection in data mining. Swarm Evol. Comput.
**2020**, 54, 100663. [Google Scholar] [CrossRef] - Pudjihartono, N.; Fadason, T.; Kempa-Liehr, A.W.; O’Sullivan, J.M. A Review of Feature Selection Methods for Machine Learning-Based Disease Risk Prediction. Front. Bioinform.
**2022**, 2, 927312. [Google Scholar] [CrossRef] - Kaur, S.; Kumar, Y.; Koul, A.; Kumar Kamboj, S. A Systematic Review on Metaheuristic Optimization Techniques for Feature Selections in Disease Diagnosis: Open Issues and Challenges. Arch. Comput. Methods Eng.
**2022**, 30, 1863–1895. [Google Scholar] [CrossRef] [PubMed] - Li, J.; Cheng, K.; Wang, S.; Morstatter, F.; Trevino, R.P.; Tang, J.; Liu, H. Feature selection: A data perspective. ACM Comput. Surv. (CSUR)
**2017**, 50, 1–45. [Google Scholar] [CrossRef] - Hoque, N.; Bhattacharyya, D.K.; Kalita, J.K. MIFS-ND: A mutual information-based feature selection method. Expert Syst. Appl.
**2014**, 41, 6371–6385. [Google Scholar] [CrossRef] - Peng, Y.; Wu, Z.; Jiang, J. A novel feature selection approach for biomedical data classification. J. Biomed. Inform.
**2010**, 43, 15–23. [Google Scholar] [CrossRef] [PubMed] - Ma, L.; Li, M.; Gao, Y.; Chen, T.; Ma, X.; Qu, L. A novel wrapper approach for feature selection in object-based image classification using polygon-based cross-validation. IEEE Geosci. Remote Sens. Lett.
**2017**, 14, 409–413. [Google Scholar] [CrossRef] - Tan, F.; Yan, P.; Guan, X. Deep reinforcement learning: From Q-learning to deep Q-learning. In Proceedings of the Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, 14–18 November 2017; Part IV 24. Springer: Berlin/Heidelberg, Germany, 2017; pp. 475–483. [Google Scholar]
- Ramaswamy, A. Theory of Deep Q-Learning: A Dynamical Systems Perspective. arXiv
**2020**, arXiv:2008.10870. [Google Scholar] - Hong, Z.W.; Su, S.Y.; Shann, T.Y.; Chang, Y.H.; Lee, C.Y. A Deep Policy Inference Q-Network for Multi-Agent Systems. arXiv
**2017**, arXiv:1712.07893. [Google Scholar] - Hu, X.; Chu, L.; Pei, J.; Liu, W.; Bian, J. Model complexity of deep learning: A survey. Knowl. Inf. Syst.
**2021**, 63, 2585–2619. [Google Scholar] [CrossRef] - Fan, J.; Wang, Z.; Xie, Y.; Yang, Z. A theoretical analysis of deep Q-learning. Learning for dynamics and control. In Proceedings of the PMLR, Virtual, 13–18 July 2020; pp. 486–489. [Google Scholar]
- Bartz-Beielstein, T.; Preuss, M. Experimental research in evolutionary computation. In Proceedings of the 9th Annual Conference Companion on Genetic and Evolutionary Computation, ACM, GECCO07, London, UK, 7–11 July 2007. [Google Scholar] [CrossRef]
- Arcuri, A.; Fraser, G. Parameter tuning or default values? An empirical investigation in search-based software engineering. Empir. Softw. Eng.
**2013**, 18, 594–623. [Google Scholar] [CrossRef] - Ravelo, C.; Olivares, R. Biomimetic Orca Predator Algorithm Improved by Deep Reinforcement Leaning for Feature Selection. 2024. Available online: https://figshare.com/articles/online_resource/Biomimetic_Orca_Predator_Algorithm_improved_by_Deep_Reinforcement_Leaning_for_Feature_Selection/25126043 (accessed on 17 April 2024).
- Katoch, S.; Chauhan, S.S.; Kumar, V. A review on genetic algorithm: Past, present, and future. Multimed. Tools Appl.
**2020**, 80, 8091–8126. [Google Scholar] [CrossRef] [PubMed] - Shami, T.M.; El-Saleh, A.A.; Alswaitti, M.; Al-Tashi, Q.; Summakieh, M.A.; Mirjalili, S. Particle Swarm Optimization: A Comprehensive Survey. IEEE Access
**2022**, 10, 10031–10061. [Google Scholar] [CrossRef] - Shehab, M.; Abu-Hashem, M.A.; Shambour, M.K.Y.; Alsalibi, A.I.; Alomari, O.A.; Gupta, J.N.D.; Alsoud, A.R.; Abuhaija, B.; Abualigah, L. A Comprehensive Review of Bat Inspired Algorithm: Variants, Applications, and Hybridization. Arch. Comput. Methods Eng.
**2022**, 30, 765–797. [Google Scholar] [CrossRef] [PubMed] - Abualigah, L.; Elaziz, M.A.; Sumari, P.; Khasawneh, A.M.; Alshinwan, M.; Mirjalili, S.; Shehab, M.; Abuaddous, H.Y.; Gandomi, A.H. Black hole algorithm: A comprehensive survey. Appl. Intell.
**2022**, 52, 11892–11915. [Google Scholar] [CrossRef] - Faris, H.; Aljarah, I.; Al-Betar, M.A.; Mirjalili, S. Grey wolf optimizer: A review of recent variants and applications. Neural Comput. Appl.
**2018**, 30, 413–435. [Google Scholar] [CrossRef] - Mohammadi-Balani, A.; Nayeri, M.D.; Azar, A.; Taghizadeh-Yazdi, M. Golden eagle optimizer: A nature-inspired metaheuristic algorithm. Comput. Ind. Eng.
**2021**, 152, 107050. [Google Scholar] [CrossRef] - Abualigah, L.; Abd Elaziz, M.; Sumari, P.; Geem, Z.W.; Gandomi, A.H. Reptile Search Algorithm (RSA): A nature-inspired meta-heuristic optimizer. Expert Syst. Appl.
**2022**, 191, 116158. [Google Scholar] [CrossRef]

**Figure 2.**Distributions for all metrics employed to analyze the effectiveness of OPA and OPADQL on machine learning methods. (

**a**) Distribution of ${F}_{1}$ score values obtained by OPA (left side) and OPADQL (right side). (

**b**) Distribution of accuracy values obtained by OPA (left side) and OPADQL (right side). (

**c**) Distribution of diversity values obtained by OPA (left side) and OPADQL (right side). (

**d**) Distribution of reduction values obtained by OPA (left side) and OPADQL (right side). (

**e**) Distribution of FEI values obtained by OPA (left side) and OPADQL (right side).

**Figure 3.**Radar analysis comparing OPA and OPADQL performance across all machine learning techniques. (

**a**) DT radar analysis: performance comparison between OPA and OPADQL. (

**b**) RF radar analysis: performance comparison between OPA and OPADQL. (

**c**) SVM radar analysis: performance comparison between OPA and OPADQL. (

**d**) ERT radar analysis: performance comparison between OPA and OPADQL.

**Figure 4.**Convergence analysis of OPADQL versus state-of-the-art algorithms across various machine learning models. (

**a**) DT: c analysis. (

**b**) RF: convergence analysis. (

**c**) SVM: convergence analysis. (

**d**) ERT: convergence analysis.

Parameter | Value or Description |
---|---|

q | 0.9 |

a | Closer to 1 |

b | Random |

d | Closer to 1 |

F | Close to 2 |

S | 10 |

Maximum iterations | 100 |

State size | 5 |

Action size | 40 |

Neurons per layer | 24 |

Activation | ReLU (layers), Linear (final layer) |

Loss function | Huber |

Optimizer | RMSprop with a learning rate of $0.001$ |

Epsilon | Starts at 1.0, decays to $0.01$ |

Network update | Every 50 training steps |

Initial parameters $a,b,d,e$ | Randomly generated in range [0, 1] |

Binarization threshold ($\varphi $) | Random value uniformly distributed [0, 1] |

Metrics | DT | RF | |||||
---|---|---|---|---|---|---|---|

n/o | OPA | OPADQL | n/o | OPA | OPADQL | ||

$max$ ${\mathbf{F}}_{\mathbf{1}}$ score | $best$ | 0.7911 | 0.8242 | 0.8250 | 0.9154 | 0.9165 | 0.9186 |

$worst$ | 0.7753 | 0.7601 | 0.7936 | 0.8671 | 0.8921 | 0.9087 | |

$mean$ | 0.7829 | 0.7845 | 0.8093 | 0.8956 | 0.9110 | 0.9124 | |

$std$ | 0.0037 | 0.0062 | 0.0133 | 0.0016 | 0.0045 | 0.0137 | |

$median$ | 0.7835 | 0.7851 | 0.8087 | 0.9002 | 0.9107 | 0.9120 | |

$iqr$ | 0.0045 | 0.0070 | 0.0209 | 0.0023 | 0.0062 | 0.0194 | |

$max$ Accuracy | $best$ | 0.7408 | 0.7886 | 0.9035 | 0.8978 | 0.8989 | 0.9035 |

$worst$ | 0.7207 | 0.7225 | 0.7432 | 0.8897 | 0.8342 | 0.8673 | |

$mean$ | 0.7299 | 0.7307 | 0.8921 | 0.8731 | 0.8921 | 0.8943 | |

$std$ | 0.0047 | 0.0071 | 0.0159 | 0.0021 | 0.0071 | 0.0176 | |

$median$ | 0.7304 | 0.7328 | 0.8922 | 0.8782 | 0.8922 | 0.8940 | |

$iqr$ | 0.0065 | 0.0091 | 0.0240 | 0.0035 | 0.0108 | 0.0257 | |

$min$ Diversity | $best$ | n/a | 0.9647 | 0.9691 | n/a | 0.9666 | 0.9692 |

$worst$ | n/a | 0.9576 | 0.9116 | n/a | 0.9420 | 0.9405 | |

$mean$ | n/a | 0.9611 | 0.9659 | n/a | 0.9647 | 0.9616 | |

$std$ | n/a | 0.0096 | 0.0019 | n/a | 0.0096 | 0.0053 | |

$median$ | n/a | 0.9636 | 0.9663 | n/a | 0.9663 | 0.9637 | |

$iqr$ | n/a | 0.0003 | 0.0003 | n/a | 0.0014 | 0.0003 | |

$max$ Reduction | $best$ | n/a | 0.6043 | 0.9645 | n/a | 0.6043 | 0.8014 |

$worst$ | n/a | 0.5026 | 0.6099 | n/a | 0.5133 | 0.5390 | |

$mean$ | n/a | 0.5378 | 0.6811 | n/a | 0.5575 | 0.6622 | |

$std$ | n/a | 0.0234 | 0.7328 | n/a | 0.0259 | 0.8782 | |

$median$ | n/a | 0.5957 | 0.6738 | n/a | 0.5615 | 0.6773 | |

$iqr$ | n/a | 0.0321 | 0.0390 | n/a | 0.0401 | 0.1170 | |

$max$ FEI | $best$ | n/a | 0.7800 | 0.8991 | n/a | 0.8432 | 0.9259 |

$worst$ | n/a | 0.7646 | 0.7992 | n/a | 0.8154 | 0.8615 | |

$mean$ | n/a | 0.7710 | 0.8293 | n/a | 0.8300 | 0.8881 | |

$std$ | n/a | 0.0042 | 0.0117 | n/a | 0.0059 | 0.0160 | |

$median$ | n/a | 0.0770 | 0.8276 | n/a | 0.8311 | 0.8871 | |

$iqr$ | n/a | 0.0065 | 0.0178 | n/a | 0.0082 | 0.0265 |

Metrics | SVM | ERT | |||||
---|---|---|---|---|---|---|---|

n/o | OPA | OPADQL | n/o | OPA | OPADQL | ||

$max$ ${\mathbf{F}}_{\mathbf{1}}$
score | $best$ | 0.8841 | 0.8843 | 0.9155 | 0.9168 | 0.9180 | 0.9228 |

$worst$ | 0.8841 | 0.8644 | 0.8645 | 0.9108 | 0.8392 | 0.9048 | |

$mean$ | 0.8841 | 0.8766 | 0.8919 | 0.9145 | 0.8946 | 0.9153 | |

$std$ | 0.0000 | 0.0051 | 0.0129 | 0.0014 | 0.0048 | 0.0177 | |

$median$ | 0.8841 | 0.8771 | 0.8954 | 0.9147 | 0.8974 | 0.9155 | |

$iqr$ | 0.0000 | 0.0082 | 0.0175 | 0.0021 | 0.0066 | 0.0234 | |

$max$ Accuracy | $best$ | 0.8565 | 0.8594 | 0.8966 | 0.8975 | 0.9013 | 0.9078 |

$worst$ | 0.8565 | 0.8328 | 0.8344 | 0.8910 | 0.8023 | 0.8826 | |

$mean$ | 0.8565 | 0.8488 | 0.8682 | 0.8951 | 0.8710 | 0.8973 | |

$std$ | 0.0000 | 0.0069 | 0.0158 | 0.0016 | 0.0066 | 0.0218 | |

$median$ | 0.8565 | 0.8493 | 0.8713 | 0.8953 | 0.8755 | 0.8976 | |

$iqr$ | 0.0000 | 0.0106 | 0.0215 | 0.0026 | 0.0091 | 0.0276 | |

$min$ Diversity | $best$ | n/a | 0.9671 | 0.9638 | n/a | 0.9637 | 0.9661 |

$worst$ | n/a | 0.9525 | 0.9306 | n/a | 0.9326 | 0.9557 | |

$mean$ | n/a | 0.9650 | 0.9593 | n/a | 0.9629 | 0.9642 | |

$std$ | n/a | 0.0053 | 0.0088 | n/a | 0.0015 | 0.0061 | |

$median$ | n/a | 0.9658 | 0.9631 | n/a | 0.9634 | 0.9659 | |

$iqr$ | n/a | 0.0003 | 0.0041 | n/a | 0.0010 | 0.0010 | |

$max$ Reduction | $best$ | n/a | 0.5829 | 0.8014 | n/a | 0.6043 | 0.8014 |

$worst$ | n/a | 0.5133 | 0.5673 | n/a | 0.5133 | 0.6312 | |

$mean$ | n/a | 0.5519 | 0.7026 | n/a | 0.5565 | 0.7272 | |

$std$ | n/a | 0.0190 | 0.8713 | n/a | 0.0214 | 0.8755 | |

$median$ | n/a | 0.5508 | 0.7270 | n/a | 0.5615 | 0.7234 | |

$iqr$ | n/a | 0.0254 | 0.0975 | n/a | 0.0254 | 0.0408 | |

$max$ FEI | $best$ | n/a | 0.8220 | 0.9342 | n/a | 0.8434 | 0.9265 |

$worst$ | n/a | 0.8014 | 0.8650 | n/a | 0.8237 | 0.8798 | |

$mean$ | n/a | 0.8105 | 0.8955 | n/a | 0.8333 | 0.9039 | |

$std$ | n/a | 0.0048 | 0.0168 | n/a | 0.0048 | 0.0110 | |

$median$ | n/a | 0.8100 | 0.8981 | n/a | 0.8333 | 0.9043 | |

$iqr$ | n/a | 0.0046 | 0.0223 | n/a | 0.0059 | 0.0139 |

Metrics | OPADQL v/s OPA | |||
---|---|---|---|---|

DT | RF | SVM | ERT | |

${F}_{1}\phantom{\rule{3.33333pt}{0ex}}\mathrm{score}$ | $1.6564\times {10}^{-10}$ | $3.2788\times {10}^{-5}$ | $3.2529\times {10}^{-6}$ | $6.2779\times {10}^{-9}$ |

Accuracy | $1.1289\times {10}^{-10}$ | $9.7638\times {10}^{-6}$ | $1.9864\times {10}^{-6}$ | $7.1438\times {10}^{-9}$ |

Diversity | sws | $1.4360\times {10}^{-11}$ | $1.4360\times {10}^{-11}$ | sws |

Reduction | $1.3330\times {10}^{-11}$ | $9.7638\times {10}^{-6}$ | $3.3184\times {10}^{-11}$ | $1.3483\times {10}^{-11}$ |

FEI | $1.5876\times {10}^{-11}$ | $2.9094\times {10}^{-7}$ | $1.4360\times {10}^{-11}$ | $1.4360\times {10}^{-11}$ |

Metrics | Random | GA | PSO | BAT | BH | GWO | GEO | RSA | OPADQL | |
---|---|---|---|---|---|---|---|---|---|---|

Decision Tree | ||||||||||

$max$ ${\mathbf{F}}_{\mathbf{1}}$
score | $best$ | 0.8191 | 0.8085 | 0.8106 | 0.8126 | 0.8147 | 0.8168 | 0.8137 | 0.8151 | 0.8250 |

$worst$ | 0.7781 | 0.7777 | 0.7797 | 0.7817 | 0.7837 | 0.7857 | 0.7811 | 0.7827 | 0.7936 | |

$mean$ | 0.7929 | 0.7931 | 0.7951 | 0.7972 | 0.7992 | 0.8012 | 0.7965 | 0.7981 | 0.8093 | |

$std$ | 0.0011 | 0.0049 | 0.0069 | 0.0049 | 0.0059 | 0.0109 | 0.0058 | 0.0067 | 0.0133 | |

$median$ | 0.7935 | 0.7931 | 0.7951 | 0.7971 | 0.7991 | 0.8011 | 0.7965 | 0.7980 | 0.8087 | |

$iqr$ | 0.0011 | 0.0207 | 0.0177 | 0.0027 | 0.0201 | 0.0200 | 0.0137 | 0.0146 | 0.0209 | |

$max$ Accuracy | $best$ | 0.7720 | 0.8854 | 0.8877 | 0.8899 | 0.8922 | 0.8945 | 0.8703 | 0.8744 | 0.9035 |

$worst$ | 0.7107 | 0.7283 | 0.7302 | 0.7321 | 0.7339 | 0.7358 | 0.7285 | 0.7303 | 0.7432 | |

$mean$ | 0.7499 | 0.8743 | 0.8765 | 0.8787 | 0.8809 | 0.8832 | 0.8573 | 0.8616 | 0.8921 | |

$std$ | 0.0117 | 0.0049 | 0.0049 | 0.0049 | 0.0049 | 0.0049 | 0.0060 | 0.0073 | 0.0159 | |

$median$ | 0.7463 | 0.8741 | 0.8761 | 0.8791 | 0.8811 | 0.8830 | 0.8566 | 0.8611 | 0.8922 | |

$iqr$ | 0.0115 | 0.0215 | 0.0211 | 0.0225 | 0.0235 | 0.0217 | 0.0203 | 0.0208 | 0.0240 | |

$min$ Diversity | $best$ | n/a | 0.9645 | 0.9650 | 0.9701 | 0.9748 | 0.9698 | 0.9689 | 0.9699 | 0.9691 |

$worst$ | n/a | 0.9574 | 0.9575 | 0.9576 | 0.9577 | 0.9578 | 0.9499 | 0.9509 | 0.9116 | |

$mean$ | n/a | 0.9600 | 0.9611 | 0.9612 | 0.9613 | 0.9614 | 0.9618 | 0.9628 | 0.9659 | |

$std$ | n/a | 0.0048 | 0.0049 | 0.0052 | 0.0053 | 0.0052 | 0.0045 | 0.0055 | 0.0019 | |

$median$ | n/a | 0.9590 | 0.9600 | 0.9615 | 0.9611 | 0.9612 | 0.9615 | 0.9625 | 0.9663 | |

$iqr$ | n/a | 0.0048 | 0.0049 | 0.0050 | 0.0051 | 0.0052 | 0.0042 | 0.0052 | 0.0003 | |

$max$ Reduction | $best$ | 0.5634 | 0.6041 | 0.6042 | 0.6043 | 0.6044 | 0.7045 | 0.6142 | 0.6579 | 0.9645 |

$worst$ | 0.4564 | 0.5024 | 0.5025 | 0.5026 | 0.5027 | 0.5928 | 0.5099 | 0.5224 | 0.6099 | |

$mean$ | 0.4873 | 0.5376 | 0.5377 | 0.5378 | 0.5379 | 0.6380 | 0.5460 | 0.5629 | 0.6811 | |

$std$ | 0.0375 | 0.0276 | 0.0278 | 0.0277 | 0.0279 | 0.0279 | 0.0294 | 0.1173 | 0.7328 | |

$median$ | 0.4863 | 0.5370 | 0.5371 | 0.5371 | 0.5373 | 0.6373 | 0.5454 | 0.5614 | 0.6738 | |

$iqr$ | 0.0144 | 0.0276 | 0.0277 | 0.0278 | 0.0279 | 0.0280 | 0.0256 | 0.0272 | 0.0390 | |

$max$ FEI | $best$ | 0.7200 | 0.8570 | 0.7756 | 0.8100 | 0.7890 | 0.8700 | 0.8036 | 0.8155 | 0.8991 |

$worst$ | 0.6320 | 0.7900 | 0.7100 | 0.7110 | 0.6915 | 0.7950 | 0.7216 | 0.7313 | 0.7992 | |

$mean$ | 0.6767 | 0.8200 | 0.7400 | 0.7710 | 0.7215 | 0.8250 | 0.7590 | 0.7678 | 0.8293 | |

$std$ | 0.0204 | 0.0100 | 0.0102 | 0.0104 | 0.0106 | 0.0110 | 0.0121 | 0.0121 | 0.0117 | |

$median$ | 0.6747 | 0.8230 | 0.7354 | 0.7340 | 0.7545 | 0.8270 | 0.7581 | 0.7668 | 0.8276 | |

$iqr$ | 0.0276 | 0.0160 | 0.0162 | 0.0164 | 0.0166 | 0.0170 | 0.0183 | 0.0182 | 0.0178 | |

Random Forest | ||||||||||

$max$ ${\mathbf{F}}_{\mathbf{1}}$ score | $best$ | 0.8915 | 0.9002 | 0.9025 | 0.9048 | 0.9071 | 0.9094 | 0.9026 | 0.9046 | 0.9186 |

$worst$ | 0.8671 | 0.8905 | 0.8928 | 0.8951 | 0.8973 | 0.8996 | 0.8904 | 0.8927 | 0.9087 | |

$mean$ | 0.8956 | 0.8942 | 0.8964 | 0.8987 | 0.901 | 0.9033 | 0.8982 | 0.9000 | 0.9124 | |

$std$ | 0.0016 | 0.0213 | 0.0213 | 0.0213 | 0.0213 | 0.0213 | 0.0180 | 0.0175 | 0.0137 | |

$median$ | 0.8802 | 0.8940 | 0.8960 | 0.8990 | 0.9010 | 0.9030 | 0.8955 | 0.8976 | 0.9120 | |

$iqr$ | 0.0023 | 0.0173 | 0.0183 | 0.0131 | 0.0013 | 0.0113 | 0.0106 | 0.0117 | 0.0194 | |

$max$ Accuracy | $best$ | 0.9001 | 0.8854 | 0.8877 | 0.8899 | 0.8922 | 0.8945 | 0.8916 | 0.8931 | 0.9035 |

$worst$ | 0.8897 | 0.8556 | 0.8521 | 0.8543 | 0.8565 | 0.8586 | 0.8611 | 0.8619 | 0.8673 | |

$mean$ | 0.8731 | 0.8764 | 0.8786 | 0.8809 | 0.8831 | 0.8854 | 0.8796 | 0.8814 | 0.8943 | |

$std$ | 0.0021 | 0.0213 | 0.0213 | 0.0213 | 0.0213 | 0.0213 | 0.0181 | 0.0180 | 0.0176 | |

$median$ | 0.8782 | 0.8760 | 0.8790 | 0.8810 | 0.8830 | 0.8850 | 0.8804 | 0.8821 | 0.8940 | |

$iqr$ | 0.0015 | 0.0213 | 0.0225 | 0.0215 | 0.0213 | 0.0223 | 0.0184 | 0.0193 | 0.0257 | |

$min$ Diversity | $best$ | n/a | 0.9725 | 0.9701 | 0.9766 | 0.9699 | 0.9749 | 0.9728 | 0.9723 | 0.9692 |

$worst$ | n/a | 0.9574 | 0.9575 | 0.9576 | 0.9577 | 0.9578 | 0.9576 | 0.9552 | 0.9405 | |

$mean$ | n/a | 0.9710 | 0.9654 | 0.9681 | 0.9613 | 0.9740 | 0.9680 | 0.9671 | 0.9616 | |

$std$ | n/a | 0.0048 | 0.0049 | 0.0050 | 0.0051 | 0.0052 | 0.0050 | 0.0050 | 0.0053 | |

$median$ | n/a | 0.9711 | 0.9610 | 0.9610 | 0.9610 | 0.9610 | 0.9630 | 0.9631 | 0.9637 | |

$iqr$ | n/a | 0.0048 | 0.0059 | 0.0050 | 0.0051 | 0.0052 | 0.0052 | 0.0045 | 0.0003 | |

$max$ Reduction | $best$ | 0.5469 | 0.6041 | 0.6242 | 0.6143 | 0.6344 | 0.7045 | 0.6214 | 0.6439 | 0.8014 |

$worst$ | 0.4475 | 0.5024 | 0.5025 | 0.5026 | 0.5027 | 0.5028 | 0.4934 | 0.4991 | 0.5390 | |

$mean$ | 0.4948 | 0.5376 | 0.5377 | 0.5578 | 0.5479 | 0.6380 | 0.5523 | 0.5660 | 0.6622 | |

$std$ | 0.0269 | 0.0276 | 0.0277 | 0.0278 | 0.0289 | 0.0280 | 0.0278 | 0.1341 | 0.8782 | |

$median$ | 0.4917 | 0.5378 | 0.5485 | 0.5599 | 0.5364 | 0.6370 | 0.5519 | 0.5676 | 0.6773 | |

$iqr$ | 0.0469 | 0.0276 | 0.0277 | 0.0278 | 0.0279 | 0.0280 | 0.0310 | 0.0417 | 0.1170 | |

$max$ FEI | $best$ | 0.7823 | 0.8000 | 0.7892 | 0.7821 | 0.8021 | 0.8823 | 0.8063 | 0.8213 | 0.9259 |

$worst$ | 0.7257 | 0.6250 | 0.5355 | 0.6160 | 0.7065 | 0.8300 | 0.6731 | 0.6967 | 0.8615 | |

$mean$ | 0.7567 | 0.7010 | 0.6635 | 0.7040 | 0.7445 | 0.8660 | 0.7393 | 0.7579 | 0.8881 | |

$std$ | 0.0150 | 0.0140 | 0.0142 | 0.0144 | 0.0146 | 0.0150 | 0.0145 | 0.0147 | 0.0160 | |

$median$ | 0.7564 | 0.7050 | 0.6655 | 0.7060 | 0.7465 | 0.8670 | 0.7411 | 0.7593 | 0.8871 | |

$iqr$ | 0.0257 | 0.0257 | 0.0252 | 0.0254 | 0.0056 | 0.0260 | 0.0223 | 0.0228 | 0.0265 |

Metrics | Random | GA | PSO | BAT | BH | GWO | GEO | RSA | OPADQL | |
---|---|---|---|---|---|---|---|---|---|---|

Support Vector Machine | ||||||||||

$max$ ${\mathbf{F}}_{\mathbf{1}}$ score | $best$ | 0.8874 | 0.9052 | 0.9073 | 0.9094 | 0.9115 | 0.9170 | 0.9063 | 0.9075 | 0.9155 |

$worst$ | 0.8485 | 0.8550 | 0.8560 | 0.8570 | 0.8580 | 0.8660 | 0.8567 | 0.8577 | 0.8645 | |

$mean$ | 0.8666 | 0.8890 | 0.8895 | 0.8900 | 0.8905 | 0.8930 | 0.8864 | 0.8871 | 0.8919 | |

$std$ | 0.0087 | 0.0125 | 0.0126 | 0.0127 | 0.0128 | 0.0130 | 0.0121 | 0.0122 | 0.0131 | |

$median$ | 0.8664 | 0.8890 | 0.8895 | 0.8900 | 0.8905 | 0.8978 | 0.8872 | 0.8882 | 0.8954 | |

$iqr$ | 0.0098 | 0.0160 | 0.0162 | 0.0164 | 0.0166 | 0.0178 | 0.0155 | 0.0157 | 0.0175 | |

$max$ Accuracy | $best$ | 0.8615 | 0.8860 | 0.8865 | 0.8870 | 0.8875 | 0.8994 | 0.8847 | 0.8861 | 0.8966 |

$worst$ | 0.8144 | 0.8250 | 0.8260 | 0.8270 | 0.8280 | 0.8290 | 0.8249 | 0.8261 | 0.8344 | |

$mean$ | 0.8361 | 0.8610 | 0.8615 | 0.8620 | 0.8625 | 0.8692 | 0.8587 | 0.8599 | 0.8682 | |

$std$ | 0.0113 | 0.0145 | 0.0146 | 0.0147 | 0.0148 | 0.0149 | 0.0141 | 0.0143 | 0.0158 | |

$median$ | 0.8360 | 0.8610 | 0.8615 | 0.8620 | 0.8625 | 0.8630 | 0.8577 | 0.8594 | 0.8713 | |

$iqr$ | 0.0131 | 0.0200 | 0.0002 | 0.0204 | 0.0006 | 0.0308 | 0.0142 | 0.0151 | 0.0215 | |

$min$ Diversity | $best$ | n/a | 0.9636 | 0.9637 | 0.9638 | 0.9639 | 0.9640 | 0.9638 | 0.9638 | 0.9638 |

$worst$ | n/a | 0.9304 | 0.9305 | 0.9306 | 0.9307 | 0.9308 | 0.9306 | 0.9306 | 0.9306 | |

$mean$ | n/a | 0.9591 | 0.9592 | 0.9593 | 0.9594 | 0.9595 | 0.9593 | 0.9593 | 0.9593 | |

$std$ | n/a | 0.0086 | 0.0087 | 0.0088 | 0.0089 | 0.0090 | 0.0088 | 0.0088 | 0.0088 | |

$median$ | n/a | 0.9591 | 0.9592 | 0.9593 | 0.9594 | 0.9595 | 0.9593 | 0.9598 | 0.9631 | |

$iqr$ | n/a | 0.0038 | 0.0039 | 0.0040 | 0.0041 | 0.0042 | 0.0040 | 0.0040 | 0.0041 | |

$max$ Reduction | $best$ | 0.5524 | 0.6812 | 0.4813 | 0.5814 | 0.6715 | 0.7916 | 0.6266 | 0.6484 | 0.8014 |

$worst$ | 0.4088 | 0.5571 | 0.4072 | 0.5573 | 0.5574 | 0.5575 | 0.5076 | 0.5150 | 0.5673 | |

$mean$ | 0.4987 | 0.6024 | 0.4525 | 0.6926 | 0.5927 | 0.6928 | 0.5886 | 0.6029 | 0.7026 | |

$std$ | 0.0350 | 0.8608 | 0.8609 | 0.8610 | 0.8611 | 0.8612 | 0.7233 | 0.7418 | 0.8713 | |

$median$ | 0.5027 | 0.6024 | 0.4625 | 0.6926 | 0.5927 | 0.6928 | 0.5910 | 0.6080 | 0.7270 | |

$iqr$ | 0.0441 | 0.0950 | 0.0435 | 0.0252 | 0.0053 | 0.0954 | 0.0514 | 0.0572 | 0.0975 | |

$max$ FEI | $best$ | 0.5524 | 0.6812 | 0.4813 | 0.5814 | 0.6715 | 0.7916 | 0.6266 | 0.6484 | 0.8014 |

$worst$ | 0.4088 | 0.5571 | 0.4072 | 0.5573 | 0.5574 | 0.5575 | 0.5076 | 0.5150 | 0.5673 | |

$mean$ | 0.4987 | 0.6024 | 0.4525 | 0.6926 | 0.5927 | 0.6928 | 0.5886 | 0.6029 | 0.7026 | |

$std$ | 0.0350 | 0.8608 | 0.8609 | 0.8610 | 0.8611 | 0.8612 | 0.7233 | 0.7418 | 0.8713 | |

$median$ | 0.5027 | 0.6024 | 0.4625 | 0.6926 | 0.5927 | 0.6928 | 0.5910 | 0.6080 | 0.7270 | |

$iqr$ | 0.0441 | 0.0950 | 0.0435 | 0.0252 | 0.0053 | 0.0954 | 0.0514 | 0.0572 | 0.0975 | |

Extremely Randomized Trees | ||||||||||

$max$ ${\mathbf{F}}_{\mathbf{1}}$ score | $best$ | 0.9121 | 0.9201 | 0.9202 | 0.9203 | 0.9204 | 0.9240 | 0.9195 | 0.9199 | 0.9228 |

$worst$ | 0.8823 | 0.9020 | 0.9021 | 0.9022 | 0.9023 | 0.9060 | 0.8995 | 0.9001 | 0.9048 | |

$mean$ | 0.9023 | 0.9140 | 0.9141 | 0.9142 | 0.9143 | 0.9160 | 0.9125 | 0.9128 | 0.9153 | |

$std$ | 0.0014 | 0.0172 | 0.0173 | 0.0174 | 0.0175 | 0.0179 | 0.0148 | 0.0152 | 0.0177 | |

$median$ | 0.8947 | 0.9140 | 0.9141 | 0.9142 | 0.9143 | 0.9160 | 0.9112 | 0.9118 | 0.9155 | |

$iqr$ | 0.0021 | 0.0225 | 0.0226 | 0.0227 | 0.0228 | 0.0235 | 0.0194 | 0.0199 | 0.0234 | |

$max$ Accuracy | $best$ | 0.9052 | 0.9050 | 0.9055 | 0.9060 | 0.9065 | 0.9070 | 0.9059 | 0.9061 | 0.9078 |

$worst$ | 0.8620 | 0.8800 | 0.8805 | 0.8810 | 0.8815 | 0.8820 | 0.8778 | 0.8784 | 0.8826 | |

$mean$ | 0.8806 | 0.8950 | 0.8955 | 0.8960 | 0.8965 | 0.8970 | 0.8934 | 0.8962 | 0.8973 | |

$std$ | 0.0016 | 0.0205 | 0.0206 | 0.0207 | 0.0208 | 0.0209 | 0.0175 | 0.0181 | 0.0218 | |

$median$ | 0.8953 | 0.8950 | 0.8955 | 0.8960 | 0.8965 | 0.8970 | 0.8959 | 0.8961 | 0.8976 | |

$iqr$ | 0.0026 | 0.0260 | 0.0262 | 0.0264 | 0.0266 | 0.0268 | 0.0224 | 0.0231 | 0.0276 | |

$min$ Diversity | $best$ | n/a | 0.9659 | 0.9660 | 0.9661 | 0.9662 | 0.9663 | 0.9665 | 0.9664 | 0.9661 |

$worst$ | n/a | 0.9555 | 0.9556 | 0.9557 | 0.9558 | 0.9559 | 0.9561 | 0.9563 | 0.9557 | |

$mean$ | n/a | 0.9640 | 0.9641 | 0.9642 | 0.9643 | 0.9644 | 0.9646 | 0.9645 | 0.9642 | |

$std$ | n/a | 0.0062 | 0.0061 | 0.0062 | 0.0063 | 0.0064 | 0.0066 | 0.0065 | 0.0061 | |

$median$ | n/a | 0.9640 | 0.9641 | 0.9642 | 0.9643 | 0.9644 | 0.9647 | 0.9646 | 0.9659 | |

$iqr$ | n/a | 0.0008 | 0.0009 | 0.0010 | 0.0011 | 0.0012 | 0.0014 | 0.0013 | 0.0010 | |

$max$ Reduction | $best$ | 0.5414 | 0.7912 | 0.7913 | 0.7914 | 0.7915 | 0.7916 | 0.7497 | 0.7562 | 0.8014 |

$worst$ | 0.4530 | 0.6210 | 0.6211 | 0.6212 | 0.6213 | 0.6214 | 0.5932 | 0.5979 | 0.6312 | |

$mean$ | 0.4983 | 0.7170 | 0.7171 | 0.7172 | 0.7173 | 0.7174 | 0.6807 | 0.6865 | 0.7272 | |

$std$ | 0.0270 | 0.8650 | 0.8651 | 0.8652 | 0.8653 | 0.8654 | 0.7255 | 0.7443 | 0.8755 | |

$median$ | 0.5000 | 0.7170 | 0.7171 | 0.7172 | 0.7173 | 0.7174 | 0.681 | 0.6863 | 0.7234 | |

$iqr$ | 0.0359 | 0.0395 | 0.0396 | 0.0397 | 0.0398 | 0.0399 | 0.0391 | 0.0393 | 0.0408 | |

$max$ FEI | $best$ | 0.7894 | 0.8752 | 0.8053 | 0.8125 | 0.8257 | 0.9059 | 0.8357 | 0.8470 | 0.9265 |

$worst$ | 0.7334 | 0.8087 | 0.6888 | 0.7189 | 0.7590 | 0.8091 | 0.7530 | 0.7688 | 0.8798 | |

$mean$ | 0.7604 | 0.84027 | 0.7228 | 0.7630 | 0.7731 | 0.8433 | 0.7838 | 0.7988 | 0.9039 | |

$std$ | 0.0156 | 0.0106 | 0.0107 | 0.0108 | 0.0109 | 0.0110 | 0.0116 | 0.0115 | 0.0110 | |

$median$ | 0.7691 | 0.8427 | 0.7328 | 0.7630 | 0.7701 | 0.8333 | 0.7852 | 0.8001 | 0.9043 | |

$iqr$ | 0.0020 | 0.0035 | 0.0136 | 0.0107 | 0.0108 | 0.0039 | 0.0074 | 0.0082 | 0.0139 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Olivares, R.; Ravelo, C.; Soto, R.; Crawford, B.
Escaping Stagnation through Improved Orca Predator Algorithm with Deep Reinforcement Learning for Feature Selection. *Mathematics* **2024**, *12*, 1249.
https://doi.org/10.3390/math12081249

**AMA Style**

Olivares R, Ravelo C, Soto R, Crawford B.
Escaping Stagnation through Improved Orca Predator Algorithm with Deep Reinforcement Learning for Feature Selection. *Mathematics*. 2024; 12(8):1249.
https://doi.org/10.3390/math12081249

**Chicago/Turabian Style**

Olivares, Rodrigo, Camilo Ravelo, Ricardo Soto, and Broderick Crawford.
2024. "Escaping Stagnation through Improved Orca Predator Algorithm with Deep Reinforcement Learning for Feature Selection" *Mathematics* 12, no. 8: 1249.
https://doi.org/10.3390/math12081249