# When Optimization Meets AI: An Intelligent Approach for Network Disintegration with Discrete Resource Allocation

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

- We model the network disintegration problem with discrete entity allocation using nonlinear optimization programming. By looking into the characteristics of limited entity resources and the heterogeneity of the removal difficulty of different nodes, we reveal that existing solutions cannot balance effectiveness and efficiency well.
- We propose Net-Cracker, a deep reinforcement learning method to solve the network disintegration problem. This approach transforms the two-stage entity and network node selection task in the solution process into a new object selection form so as to simplify the solving process.
- We conduct extensive experiments in multiple settings. The results demonstrate that our method has significant advantages regarding solution quality, computation time and scalability compared to the traditional method.

## 2. Related Work

#### 2.1. Network Disintegration

#### 2.2. Deep Reinforcement Learning in Combinatorial Optimization Problems

#### 2.3. Deep Reinforcement Learning Method in Network Disintegration

## 3. Network Disintegration Model with Discrete Entity Resources

#### 3.1. Problem Illustration

**Definition**

**1.**

#### 3.2. Problem Model

#### 3.3. Complexity Analysis

**Definition**

**2.**

**Theorem**

**1.**

**Proof of Theorem**

**1.**

## 4. The Design of Net-Cracker

#### 4.1. Framework Overview

#### 4.1.1. Stage I: Combine

#### 4.1.2. Stage II: Selection

#### 4.1.3. Stage III: Mapping

#### 4.2. Detailed Design of the Neural Network Architecture

#### 4.2.1. The Encoder

#### 4.2.2. The Decoder

#### 4.2.3. The Attention

Algorithm 1: Solution Process of the Net-Cracker Model |

#### 4.3. Training Procedure

Algorithm 2: Training Algorithm Based on Actor–Critic. |

## 5. Performance Evaluation

#### 5.1. Experimental Settings

#### 5.1.1. Dataset

#### 5.1.2. Hyperparameter Setting

#### 5.1.3. Device Configuration

#### 5.2. Benchmarks

**Genetic Algorithm (GA):**Genetic algorithms search for optimal solutions by simulating the processes of natural selection, inheritance and evolution, and are characterized by simplicity, robustness and strong global search capabilities. By simulating genetic processes such as selection, crossover and mutation, it gradually evolves solutions that better adapt to the given problem. We first encode the entities and nodes in the network, and set the maximum number of iterations, and define the crossover and mutation operations of the GA. Next, we use the natural connectivity of the disintegrated network as the fitness function of the GA for individual evaluation. Finally, the optimal solution to the problem is output based on the fitness function.**Differential Evolutionary Algorithm (DE):**The differential evolution algorithm is an intelligent optimization search algorithm that emerges through cooperation and competition among individuals within a population. It has strengths such as strong adaptability, few control parameters, simple settings and robust optimization results. The solution process of DE is the same as that of GA, but the setting of the mutation scale factor and crossover probability of DE is different from that of GA.

#### 5.3. Performance Results

#### 5.3.1. Solving Quality

#### 5.3.2. Solving Speed

#### 5.3.3. Generalization Ability

## 6. Discussion

## 7. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Albert, R.; Barabási, A.L. Statistical mechanics of complex networks. Rev. Mod. Phys.
**2002**, 74, 47. [Google Scholar] [CrossRef] - Newman, M.E.J. The structure and function of complex networks. Siam Rev.
**2003**, 45, 167–256. [Google Scholar] [CrossRef] - Buldyrev, S.V.; Parshani, R.; Paul, G.; Stanley, H.E.; Havlin, S. Catastrophic cascade of failures in interdependent networks. Nature
**2010**, 464, 1025–1028. [Google Scholar] [CrossRef] [PubMed] - Wu, J.; Barahona, M.; Tan, Y.J.; Deng, H.Z. Spectral Measure of Structural Robustness in Complex Networks. IEEE Trans. Syst. Man Cybern. Part A
**2011**, 41, 1244–1252. [Google Scholar] [CrossRef] - Wu, Z.X.; Holme, P. Onion structure and network robustness. Phys. Rev. E
**2011**, 84, 026106. [Google Scholar] [CrossRef] [PubMed] - Arulselvan, A.; Commander, C.W.; Elefteriadou, L.; Pardalos, P.M. Detecting critical nodes in sparse graphs. Comput. Oper. Res.
**2009**, 36, 2193–2200. [Google Scholar] [CrossRef] - Raab, J.; ton Milward, H. Dark Networks as Problems. J. Public Adm. Res. Theory J-PART
**2003**, 13, 413–439. [Google Scholar] [CrossRef] - Tripathy, R.M.; Bagchi, A.; Mehta, S. A study of rumor control strategies on social networks. In Proceedings of the Acm Conference on Information & Knowledge Management, Toronto, ON, Canada, 26–30 October 2010. [Google Scholar]
- Zdeborová, L.; Zhang, P.; Zhou, H.J. Fast and simple decycling and dismantling of networks. Sci. Rep.
**2016**, 6, 37954. [Google Scholar] [CrossRef] [PubMed] - Wandelt, S.; Lin, W.; Sun, X.; Zanin, M. From random failures to targeted attacks in network dismantling. Reliab. Eng. Syst. Saf.
**2022**, 218, 108146. [Google Scholar] [CrossRef] - Li, J.; Zhao, D.; Ge, B.; Jiang, J.; Yang, K. Disintegration of Operational Capability of Heterogeneous Combat Networks Under Incomplete Information. IEEE Trans. Syst. Man Cybern. Syst.
**2020**, 50, 5172–5179. [Google Scholar] [CrossRef] - Li, Q.; Liu, S.Y.; Yang, X.S. Neighborhood information-based probabilistic algorithm for network disintegration. Expert Syst. Appl.
**2020**, 139, 112853. [Google Scholar] [CrossRef] - Deng, Y.; Deng, H.; Wu, J.; Qi, M. Optimal disintegration strategy in multiplex networks. Chaos
**2018**, 28, 121104. [Google Scholar] - Wang, T.; Fu, L.; Wei, Z.; Zhou, Y.; Gao, S. Unmanned ground weapon target assignment based on deep Q-learning network with an improved multi-objective artificial bee colony algorithm. Eng. Appl. Artif. Intell.
**2023**, 117, 105612. [Google Scholar] [CrossRef] - Quan, L.; Xu, S.; Xu, H.; Chen, F.; Wu, S.; Zhu, J.; Liu, S.; Su, T. Depression, anxiety, lower sleep quality and social support in square cabin hospitals during Shanghai’s COVID-19 lockdown, China. Front. Psychiatry
**2024**, 15, 1339774. [Google Scholar] [CrossRef] - Lalou, M.; Tahraoui, M.; Kheddouci, H. The Critical Node Detection Problem in networks: A survey. Comput. Sci. Rev.
**2018**, 28, 92–117. [Google Scholar] [CrossRef] - Shen, Y.; Nguyen, N.; Xuan, Y.; Thai, M. On the Discovery of Critical Links and Nodes for Assessing Network Vulnerability. IEEE-Acm Trans. Netw.
**2013**, 21, 963–973. [Google Scholar] [CrossRef] - Morone, F.; Makse, H.A. Influence maximization in complex networks through optimal percolation. Nature
**2015**, 524, 65–68. [Google Scholar] [CrossRef] - Lozano, M.; Garciá-Martińez, C.; Rodriǵuez, F.; Trujillo, H. Optimizing network attacks by artificial bee colony. Inf. Sci.
**2017**, 377, 30–50. [Google Scholar] [CrossRef] - Lin, M.; Chen, T.; Chen, H.; Ren, B.; Zhang, M. When architecture meets AI: A deep reinforcement learning approach for system of systems design. Adv. Eng. Inform.
**2023**, 56, 101965. [Google Scholar] [CrossRef] - Wang, Q.; Lai, K.H.; Tang, C. Solving combinatorial optimization problems over graphs with BERT-Based Deep Reinforcement Learning. Inf. Sci.
**2023**, 619, 930–946. [Google Scholar] [CrossRef] - Yu, J.J.Q.; Yu, W.; Gu, J. Online Vehicle Routing with Neural Combinatorial Optimization and Deep Reinforcement Learning. IEEE Trans. Intell. Transp. Syst.
**2019**, 20, 3806–3817. [Google Scholar] [CrossRef] - Pastor-Satorras, R.; Vespignani, A. Epidemic spreading in scale-free networks. Phys. Rev. Lett.
**2000**, 86, 3200–3203. [Google Scholar] [CrossRef] - Borgatti, S.P. Centrality and network flow. Soc. Netw.
**2005**, 27, 55–71. [Google Scholar] [CrossRef] - Holme, P.; Kim, B.J.; Yoon, C.N.; Han, S.K. Attack vulnerability of complex networks. Phys. Rev. E
**2002**, 65, 056109. [Google Scholar] [CrossRef] [PubMed] - Deng, Y.; Wu, J.; Xiao, Y.; Zhang, M.; Yu, Y.; Zhang, Y. Optimal Disintegration Strategy with Heterogeneous Costs in Complex Networks. IEEE Trans. Syst. Man Cybern. Syst.
**2018**, 50, 2905–2913. [Google Scholar] [CrossRef] - Wu, J.; Xiao, Y.; Li, Y.; Deng, Y. Efficient disintegration strategies with cost constraint in complex networks: The crucial role of nodes near average degree. Chaos
**2018**, 28, 061101. [Google Scholar] - Li, Q.; Liu, S.; Bai, Y.; He, X.; Yang, X.S. An elitism-based multi-objective evolutionary algorithm for min-cost network disintegration. Knowl.-Based Syst.
**2022**, 239, 107944. [Google Scholar] [CrossRef] - Koopman, B.O. The Optimum Distribution of Effort. J. Oper. Res. Soc. Am.
**1953**, 1, 52–63. [Google Scholar] [CrossRef] - Papadimitriou, C.H.; Steiglitz, K. Combinatorial Optimization: Algorithms and Complexity; Dover Publications, Inc.: Mineola, NY, USA, 1998. [Google Scholar]
- Dorigo, M.; Gambardella, L. Ant colony system: A cooperative learning approach to the traveling salesman problem. IEEE Trans. Evol. Comput.
**1997**, 1, 53–66. [Google Scholar] [CrossRef] - Schulz, A.; Suzuki, Y. An efficient heuristic for the fixed-route vehicle-refueling problem. Transp. Res. Part E Logist. Transp. Rev.
**2023**, 169, 102963. [Google Scholar] [CrossRef] - Li, R.; Hu, S.; Zhang, H.; Yin, M. An efficient local search framework for the minimum weighted vertex cover problem. Inf. Sci.
**2016**, 372, 428–445. [Google Scholar] [CrossRef] - Marinescu, R.; Dechter, R. AND/OR Branch-and-Bound search for combinatorial optimization in graphical models. Artif. Intell.
**2009**, 173, 1457–1491. [Google Scholar] [CrossRef] - Rabiner, L. Combinatorial optimization:Algorithms and complexity. IEEE Trans. Acoust. Speech Signal Process.
**2003**, 32, 1258–1259. [Google Scholar] [CrossRef] - Li, K.; Zhang, T.; Wang, R.; Qin, W.; He, H.; Huang, H. Research reviews of combinatorial optimization methods based on deep reinforcement learning. Acta Autom. Sin.
**2021**, 47, 2521–2537. [Google Scholar] - Hopfield, J.J.; Tank, D.W. Neural computation of decisions in optimization problems. Biol. Cybern.
**1985**, 52, 141–152. [Google Scholar] [CrossRef] - Sutskever, I.; Vinyals, O.; Le, Q.V. Sequence to sequence learning with neural networks. In Proceedings of the NIPS’14: Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, QC, Canada, 8–13 December 2014; Volume 2. [Google Scholar]
- Vinyals, O.; Fortunato, M.; Jaitly, N. Pointer Networks. Statistics
**2015**, 28, 1–9. [Google Scholar] - Bello, I.; Pham, H.; Le, Q.V.; Norouzi, M.; Bengio, S. Neural Combinatorial Optimization with Reinforcement Learning. arXiv
**2016**, arXiv:1611.09940. [Google Scholar] - Dai, H.; Khalil, E.B.; Zhang, Y.; Dilkina, B.; Song, L. Learning Combinatorial Optimization Algorithms over Graphs. Statistics
**2017**, 30, 1–11. [Google Scholar] - Li, Z.; Chen, Q.; Koltun, V. Combinatorial optimization with graph convolutional networks and guided tree search. In Proceedings of the NIPS’18: Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montréal, QC, Canada, 3–8 December 2018. [Google Scholar]
- Fan, C.; Sun, Y.; Liu, Y.Y.; Zeng, L. Finding key players in complex networks through deep reinforcement learning. Nat. Mach. Intell.
**2020**, 2, 317–324. [Google Scholar] [CrossRef] - Chen, L.; Wang, L.; Zeng, C.; Liu, H.; Chen, J. Searching High-value Edges Attack sequence through deep Reinforcement learning. Knowl.-Based Syst.
**2023**, 272, 110562. [Google Scholar] [CrossRef] - Zeng, C.; Liu, H.; Lu, L.; Chen, J.; Zhou, Z. SHATTER: Searching Heterogeneous Combat Network Attack Sequences Through Network Embedding and Reinforcement Learning. IEEE Syst. J.
**2023**, 17, 4497–4508. [Google Scholar] [CrossRef] - Yao, W.Z.J. DND: Deep learning-based directed network disintegrator. IEEE J. Emerg. Sel. Top. Circuits Syst.
**2023**, 13, 841–850. [Google Scholar] - Zeng, C.; Lu, L.; Liu, H.; Chen, J.; Zhou, Z. Multiplex network disintegration strategy inference based on deep network representation learning. Chaos
**2022**, 32, 053109. [Google Scholar] [CrossRef] [PubMed] - Jun, W.; Barahona, M.; Yue-Jin, T.; Hong-Zhong, D. Natural Connectivity of Complex Networks. Chin. Phys. Lett.
**2010**, 27, 078902. [Google Scholar] [CrossRef] - Ren, X.L.; Gleinig, N.; Tolić, D.; Antulov-Fantulin, N. Underestimated Cost of Targeted Attacks on Complex Networks. Complexity
**2018**, 2018, 9826243. [Google Scholar] [CrossRef] - Yiping, C.; Gerald, P.; Shlomo, H.; Fredrik, L.; H Eugene, S. Finding a better immunization strategy. Phys. Rev. Lett.
**2008**, 101, 058701. [Google Scholar] - Schoefield, X.; Tardos, E. Subset Sum Is NP-Complete. 2018. Available online: http://www.cs.cornell.edu/courses/cs4820/2018fa/lectures/subset_sum.pdf (accessed on 3 January 2024).
- Singh, R.; Gupta, A.; Shroff, N.B. Learning in Constrained Markov Decision Processes. IEEE Trans. Control. Netw. Syst.
**2023**, 10, 441–453. [Google Scholar] [CrossRef] - Siriborvornratanakul, T. Human behavior in image-based Road Health Inspection Systems despite the emerging AutoML. J. Big Data
**2022**, 9, 1–17. [Google Scholar] [CrossRef] - Bahdanau, D.; Brakel, P.; Xu, K.; Goyal, A.; Lowe, R.; Pineau, J.; Courville, A.; Bengio, Y. An actor-critic algorithm for sequence prediction. arXiv
**2016**, arXiv:1607.07086. [Google Scholar] - Cheng, Y.; Huang, L.; Chen, C.P.; Wang, X. Robust Actor-Critic with Relative Entropy Regulating Actor. IEEE Trans. Neural Netw. Learn. Syst.
**2023**, 34, 9054–9063. [Google Scholar] [CrossRef] - Williams, R.J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn.
**1992**, 8, 229–256. [Google Scholar] [CrossRef]

**Figure 2.**Two different disintegration strategies. (

**a**) Three entities attack three network nodes, only two of which can be removed. The network performance after disintegration is 2.12. (

**b**) Three entities attack and remove two network nodes. The network performance after disintegration is 1.77.

**Figure 3.**A simple example to show the network disintegration problem is NP-hard. Only ${v}_{1}$ and ${v}_{2}$ lie in the attack range of all entities.

**Figure 4.**The framework of Net-Cracker. In the combination stage, the entity set and node set are combined to form a new object set through the Cartesian product. In the selection stage, we select the object based on the neural network until a complete solution is constructed. The parameters of the neural network can be trained by actor and critic networks. In the mapping phase, by mapping the objects into the entity–node pairs, we can calculate the natural connectivity of the destroyed network.

**Figure 5.**Combine entities and nodes into new objects. Each object has the attributes of entities and nodes.

**Figure 6.**The proposed neural network architecture in AC framework. The encoder extracts the features of the input object through the embedding layer. The decoder is used to store the decoded sequence information. The attention module uses the attention mechanism to output the probability distribution of the following input according to the embedded information and the hidden layer state of the decoding network.

**Figure 8.**The solution time of various algorithms for the ER network disintegration problem at the same scale.

**Figure 9.**The solution time of various algorithms for the SF network disintegration problem at the same scale.

**Figure 10.**The correlation between the solving speed of the algorithm and problem scale and the number in each lattice represents the average solution time of the algorithm under a given problem scale.

Notation | Description |
---|---|

G | complex network |

V | network node set |

E | network edge set |

${v}_{j}$ | ${j}_{th}$ node |

${e}_{j}$ | ${j}_{th}$ edge |

$A\left(G\right)$ | the adjacency matrix of G |

${a}_{ij}$ | whether node i is connected with node j |

${k}_{j}$ | degree of node ${v}_{j}$ |

W | entity set |

${w}_{i}$ | ${i}_{th}$ entity |

${c}_{i}$ | attack ability of entity ${w}_{i}$ |

${r}_{i}$ | attack range of entity ${w}_{i}$ |

${q}_{j}$ | the removal threshold of node ${v}_{j}$ |

${d}_{ij}$ | the distance between ${i}_{th}$ entity and ${j}_{th}$ node |

${x}_{ij}$ | whether ${i}_{th}$ entity attacks ${j}_{th}$ node |

${u}_{j}$ | the sum of damage value to ${j}_{th}$ node |

${y}_{j}$ | whether ${j}_{th}$ node can be removed |

Y | disintegration strategy |

$\widehat{G}$ | the network after removing nodes |

Algorithm | Parameter | Value |
---|---|---|

GA | Population size | 100 |

Maximum number of iterations | 200 and 500 | |

Selection operator | Tournament selection | |

Crossover operator | Two-point crossover | |

Mutation operator | Breeder-GA mutation | |

Crossover probability | 0.5 | |

Mutation probability | 1/$Dim$ | |

DE | Population size | 100 |

Maximum number of iterations | 200 and 500 | |

Crossover probability | 0.5 | |

Mutation scaling factor | 0.5 |

Node Number | GA-200 | GA-500 | DE-200 | DE-500 | DRL-25 | DRL-40 | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|

$\mathbf{\Phi}$ | Time | $\mathbf{\Phi}$ | Time | $\mathbf{\Phi}$ | Time | $\mathbf{\Phi}$ | Time | $\mathbf{\Phi}$ | Time | $\mathbf{\Phi}$ | Time | |

ER Network | ||||||||||||

40 | 7.84 | 8.05 | 5.30 | 22.32 | 10.54 | 8.41 | 5.94 | 24.06 | 4.09 | 0.31 | 4.33 | 0.30 |

80 | 31.41 | 13.21 | 28.97 | 31.91 | 34.07 | 13.97 | 28.83 | 32.60 | 26.75 | 0.61 | 27.40 | 0.59 |

120 | 56.24 | 26.30 | 53.81 | 62.17 | 58.77 | 27.38 | 53.97 | 63.06 | 51.61 | 0.87 | 51.73 | 0.94 |

160 | 77.20 | 88.89 | 75.10 | 203.67 | 78.61 | 91.10 | 73.33 | 208.94 | 70.22 | 1.18 | 70.26 | 1.21 |

SF Network | ||||||||||||

40 | 1.17 | 7.97 | 0.62 | 19.72 | 2.80 | 8.38 | 0.75 | 20.07 | 0.38 | 0.30 | 0.32 | 0.30 |

80 | 3.83 | 13.04 | 3.02 | 32.65 | 5.27 | 13.52 | 3.65 | 31.99 | 1.47 | 0.63 | 1.46 | 0.59 |

120 | 5.96 | 24.52 | 5.34 | 60.33 | 6.65 | 25.39 | 5.54 | 61.12 | 2.56 | 0.89 | 4.09 | 0.86 |

160 | 5.12 | 84.06 | 3.36 | 191.37 | 6.26 | 87.00 | 3.74 | 193.36 | 1.56 | 1.18 | 1.53 | 1.19 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Li, R.; Yuan, H.; Ren, B.; Zhang, X.; Chen, T.; Luo, X.
When Optimization Meets AI: An Intelligent Approach for Network Disintegration with Discrete Resource Allocation. *Mathematics* **2024**, *12*, 1252.
https://doi.org/10.3390/math12081252

**AMA Style**

Li R, Yuan H, Ren B, Zhang X, Chen T, Luo X.
When Optimization Meets AI: An Intelligent Approach for Network Disintegration with Discrete Resource Allocation. *Mathematics*. 2024; 12(8):1252.
https://doi.org/10.3390/math12081252

**Chicago/Turabian Style**

Li, Ruozhe, Hao Yuan, Bangbang Ren, Xiaoxue Zhang, Tao Chen, and Xueshan Luo.
2024. "When Optimization Meets AI: An Intelligent Approach for Network Disintegration with Discrete Resource Allocation" *Mathematics* 12, no. 8: 1252.
https://doi.org/10.3390/math12081252