A DNN Architecture Generation Method for DDoS Detection via Genetic Alogrithm

Abstract

Nowdays, DNNs (Deep Neural Networks) are widely used in the field of DDoS attack detection. However, designing a good DNN architecture relies on the designer’s experience and requires considerable work. In this paper, a GA (genetic algorithm) is used to automatically generate the DNN architecture for DDoS detection to minimize human intervention in the design process. Furthermore, given the complexity of contemporary networks and the diversity of DDoS attacks, the objective of this paper is to generate a DNN model that boasts superior performance, real-time capability, and generalization ability to tackle intricate network scenarios. This paper presents a fitness function that guarantees the best model generated possesses a specific level of real-time capability. Additionally, the proposed method employs multiple datasets to joint models generated, thereby enhancing the model’s generalization performance. This paper conducts several experiments to validate the viability of the proposed method. Firstly, the best model generated with one dataset is compared with existing DNN models on the CICDDoS2019 dataset. The experimental results indicate that the model generated with one dataset has higher precision and F1-score than the existing DNN models. Secondly, model generation experiments are conducted on the CICIDS2017 and CICIDS2018 datasets, and the best model generated still performs well. Finally, this paper conducts comparative experiments on multiple datasets using the best model generated with six datasets and the best model generated by existing methods. The experimental results demonstrate that the best model generated with six datasets has better generalization ability and real-time capability.

1. Introduction

Recently, many researchers have used machine learning and deep-learning methods to detect DDoS attacks [1,2,3,4,5]. In research related to machine learning, researchers use data extracted by professionals to train and test the model and make it perform better. Whereas in deep learning based studies, models are typically allowed to select features automatically, some authors train and test models directly using the original network flow data, letting the model constantly update the feature selection method during training. Compared with machine learning, deep learning works better with large volumes of data and in complex situations [6,7,8]; deep learning can also fine-tune models according to the new data [9]. Although deep learning performs well in DDoS attack detection methods, the following problems remain:

• With the access of various devices to the network and the emergence of more network services [10,11,12], DDoS attacks are becoming more secretive and changeable and can cause more significant losses [13,14,15,16,17]. For some networks, it is necessary to select a particular model to prevent DDoS attacks [18,19] and constantly adjust the model to identify new DDoS attack types or even reconstruct the model to detect rapidly changing DDoS attack patterns [20]. In these scenarios, the model’s construction needs experienced researchers. Frequent model construcution will consume many resources [21].

In certain scenarios where resources and researchers are scarce, there is a need for a method to automatically generate DDoS detection models to minimize the cost of model construction. Therefore, this paper proposes a DNN model generation method based on GA, which automatically adapts to different scenarios, minimizes human intervention during the model generation process and the method generation model has a certain degree of generalization, making it a better choice for the current diverse network environment to defend DDoS attacks. Compared with the previous research, using GA to adjust the network hyperparameters [22,23,24,25,26], this paper pays more attention to the architecture of the DNN model and proposes a DNN model architecture generation method based on a GA. Although some existing methods use GAs to adjust the DNN structure to optimize the performance of the model in DDoS detection, these existing methods fix the number of layers of the DNN model and only use the full connection layer to build the DNN model, so the search space of the GA is small. In the model evaluation stage, the number of DNN model training rounds is not dynamically adjusted, and the resource consumption is large. In terms of the fitness function, only the DDoS detection rate of the DNN model is considered, and the model’s running time is not considered. In terms of generalization, previous methods only used one data set to generate the model. The generated DNN model does not have a generalization and is prone to over-fitting problems [27]. The methods proposed in this paper solve these problems. Firstly, this paper expands the search space of the GA, and the generated DNN model can cope with more DDoS attack scenarios. Secondly, this paper dynamically adjusts the number of model training rounds in the model evaluation stage to reduce resource consumption. Thirdly, the fitness function designed in this paper is to take into account the running time of the model and, to some extent, ensure the ability of the DNN model to detect DDoS in real time [28]. Fourthly, this paper ensures that the generated model has a certain generalization ability through a simultaneous model generation experiment on multiple datasets [29]. This paper finds that the model generated by the proposed method has good performance in terms of accuracy, precision, recall and F1-score. The main contributions of this article cover the following points:

• This paper uses the GA to generate a model architecture while minimizing human intervention. In some scenarios, the proposed method can find a model with good performance in a limited time to deal with the problem of DDoS detection.
• This paper uses the model generated, some existing advanced DNN models and some advanced machine learning models to experiment on the same dataset. This paper evaluates theses models in terms of precision, recall, F1-score and accuracy. The analysis results show that the models generated by the proposed method have better performance than the existing methods studied in this paper.
• In addition to the CICDDoS2019 dataset, this article also conducted experiments on two other datasets, and the models generated by the proposed method still perform well.
• Six datasets are used to evaluate the best model generated by the method proposed in this paper and the best model generated by existing methods. The results show that the model generated in this paper has stronger generalization.
• After generating about 200,000 models, this paper analyzed the architecture of models with good performance and found that some combinations of architecture patterns frequently appeared in models with good performance. This paper conducted a t-SNE dimension reduction analysis on the model architecture sequences and found that the architecture sequences of models with good performance would also produce aggregation in the low-dimensional space; this shows that the models with good performance have some commonalities, and these results can provide a certain reference for future researchers in model design.

The rest of the paper is structured as follows: Section 2 introduces the existing DDoS detection model optimization method; Section 3 presents the methods proposed in this paper; Section 4 presents the experiment and evaluation methods in this paper; and Section 5 concludes and suggests some future work.

2. Previous Works

This section will introduce some existing methods of applying the GA to model optimization. Most of these use the GA to select data features or optimize a model’s hyperparameters.

Ankit Agrawal et al. used M-DBNN to detect DDoS attacks and to continuously adjust the model’s weights through the chimpanzee optimization algorithm. The loss between the output of the model and the tag as the fitness was used to evaluate the weight selection method. After the evaluation of the model’s weight selection method, there is a 50% probability of updating the currently selected weights and a 50% probability of randomly selecting another group of weights. After several rounds of experiments, the accuracy of the model reached 87% [23].

Hasan Kamel et al. believe that selecting model hyperparameters is highly resource-intensive. The model may fall into a locally optimal solution when trained under the set hyperparameters. Because of the different performances of individuals within the population, the GA can make more choices for the model. Through continuous iteration, the individuals within the population can gradually tend to the optimal solution. The author uses GA to adjust the data division method and the depth of the decision tree. The author uses accuracy as a measure of fitness to evaluate the model’s performance. After multiple iterations, the model’s accuracy reached 99.46% [25].

Sriparna Saha et al. believe that selecting data features significantly impacts the model’s performance. The authors argue that the feature selection method is conducive to reducing the impact of the dimension curse, enhancing the model’s generalization ability, accelerating the model’s learning process and improving the model’s interpretability. Before that, some researchers selected data features through information gain sorting and PCA. Still, the authors believe both these methods have limitations and that the GA is better qualified for this job. The authors use binary coding to represent the selected features, update the feature selection method with the GA and finally use the selected features to train and test the model. The experimental results on the KDD CUP99 dataset show that GA performs well in data feature selection [24].

Amin Erfan et al. used GAs to select the features of the KDDCUP99 dataset. A binary string represented the feature selection method, and the accuracy was used as the fitness measure to evaluate the selected features. The authors compare the DNN and other models trained with all features with the DNN and others selected with the GA as input. The experimental results on DDoS detection show that the model trained with the features selected by the GA is superior to the model trained with all features regarding accuracy, precision and recall [26].

Yesi Novaria Kunang et al. adjusted the number of hidden layers, the number of hidden layer neurons, the learning rate and the epoch of the DNN training through grid search and random search and evaluated the quality of the selected relevant hyperparameters on the NSL-KDD and CICIDS2018 datasets. Because the search space selected by the author is too large, HPO was used to select excellent hyperparameters automatically. Through the experiment, the author found that the second and third hidden layers play a crucial role in the performance of the model, the epoch has almost no effect on the performance of the model and the lower the learning rate, the better the performance of the model, but the longer the training time of the model [30].

Zouhir Chiba et al. use an optimized GA to automatically select the data processing mode and a DNN architecture for cloud network intrusion detection. The authors believe that the main factors affecting the results are the data normalization method, the activation function in the DNN, the number of hidden layers in the DNN, the learning rate and the momentum term. The authors use binary coding to represent an instance of a solution and uses the AUC as the fitness measure to evaluate the instances; the CICIDS2017 dataset was used to evaluate the best performance instance, and the accuracy reached 99.86%. It can be seen that the GA can select instances with good performance in a limited time [27].

The limitation of the existing research is that only the GA is used to optimize the model’s hyperparameters or select the data features [31,32]. These are not the main factors that determine the performance of the model [33,34]. The main factor that determines the performance of the model is the architecture of the model [35]. Although, in previous works, some researchers have used GAs to adjust the architecture of the model, the search space of the GA needs to be bigger and the fitness function needs to consider the model’s running time. The method proposed in this paper solves these problems.

3. Methodology

3.1. Definition of Proposed Method

The method proposed in this paper is mainly divided into three parts: dataset preparation, model generation and model evaluation. The purpose of data preparation is to reduce the time consumed by the model and ensure the evaluation’s objectivity. The model generation part randomly generates a DNN population at the beginning and it improves the performance of the DNN population through continuous iteration of the GA until the qualified model appears or the number of iteration rounds reaches the set value. The model evaluation part decodes the corresponding sequence of the model architecture generated by the GA and trains the model after decoding. To reduce the resources consumed by the model evaluation, the model training is stopped early according to the model performance changes. Finally, the model is evaluated and the model’s fitness is recorded. A flowchart of the proposed method is shown in Figure 1. The flowchart has three parts from left to right: the first part shows the process of dataset preparation, the second part shows the process of the model generation method proposed in this paper and the third part shows the evaluation process of the generated model.

3.2. Dataset Preparation

The method proposed in this paper needs to generate and evaluate many models and the evaluation result of the generated model depends on the selected data, so how to choose data is always a difficult problem [36,37]. To objectively assess the performance of the models generated and reduce the consumption of resources in the model generation stage, this paper divides the dataset into several sub-datasets with the same label and containing the same amount of data and randomly selects the same amount of sub-datasets each time to train and test the generated model.

3.3. Model Generation

To prevent a single large-scale population from producing a large number of models with poor performance at a certain stage, resulting in poor performance of the final generated model, this paper uses several small-scale populations to conduct model generation experiments under the condition that the total number of generated models remains unchanged [38] so that the GA has more choices in the process of model generation. After the DNN models in all populations are trained, tested and evaluated, the next-generation populations are generated by the GA. The model architectures with good performance are recorded in this process, and the above operation is repeated until the number of iteration rounds reaches the set value.

3.3.1. Introduction of the GA

The GA is a simple version of Darwinian evolution [12] and simulates natural evolution to find the best solution. A simple flowchart of the GA is shown in Figure 2. Its main principles are as follows [39]:

• Selection: individuals in the population compete for survival resources, and the better individuals survive.
• Crossover: some characteristics of the previous generation can be passed on to the progeny so that the progeny and parents have a certain degree of similarity.
• Mutation: the newly generated individuals in the population will mutate, which leads to the introduction of some new characteristics in the population.

The individual in this paper is a sequence corresponding to the DNN architecture, or more accurately, a list containing strings and numbers, where the strings and numbers in the list represent the characteristics of the corresponding individual and multiple sequences constitute a population [40]. Selection operation refers to randomly selecting two individuals from the population to produce a new individual. The crossover operation between individuals is carried out to form a new sequence with the characteristics of two previous sequences. The individual mutation operation randomly replaces a segment in a new sequence. The above operations are based on the survival ability of individuals, which, in this paper, refers to individual fitness.

Table 1. Symbol table.

Symbol	Description
P	Set of solutions
$p_{id}$	A solution in the population, the subscript id represents the id number of the solution
$q_{p_{id}}$	The probability that a solution is selected to participate in generating a new solution
$l_{p_{id}}$	The length of a sequence corresponding to a solution
F	Set of fitness of solutions
${\alpha }_1$	Weight of accuracy in fitness function
${\alpha }_2$	Weight of single data processing time in fitness function
$\overline{T}$	Represents the average time of processing a single piece of data in the test phase of the model
$fitnes{s}_{p_{id}}$	The fitness value of a solution
[$lowe{r}_{p_{id}}$,$uppe{r}_{p_{id}}$]	The corresponding cumulative probability interval of the solution
${\tilde{q}}_{p_{id}}$	The probability that a solution is selected to provide a gene fragment to a new solution
$n_{random}$	A random number
${\beta }_1$	A value used to determine whether to use GEN LRU

shows some symbols that will be used when introducing the GA.

The following subsection introduces the GA used in this paper from five aspects: Coding Model Architecture, Fitness Function, Solution Selection, Crossover and Mutation. In the following, the solution will be used to refer to an individual in the population.

3.3.2. Coding Model Architecture

In this paper, strings are used to represent the architecture of a DNN, the activation function, the loss function and the optimizer [41] and numbers are used to represent the learning rate.

Table 2. Model Architecture Coding Method.

Description	Code	Parameter
Full Connected Layer	0	Neurons {2,4,6,8,16,32,64}
Conv2d	1	Kernel size {2,4,6,8} Stride {1,2,3,4}
Maxpool2d	2	Pooling {2,3,4,5}
Droupout	3	Dropout rate {0.1,0.2,0.3,0.4,0.5}
Residual layer	4	Kernel size {2,4,6,8} Padding {1,2,3,4}

shows the DNN layer used in this paper, the parameters related to the DNN layer and their coding methods.

Table 3. Codes of activation function, optimizer and loss function.

Activation Functions	Code	Optimizer	Code	Loss Function	Code
Relu	5	Adam	0	Logarithmic Cross Entropy Loss	0
Sigmoid	6	Sgd	1	Mse Loss	1
Tanh	7	Rmdgrop	2	Smooth Mse Loss	2
Softplus	8			L1 Loss	3

shows the activation function, loss function, optimizer and corresponding codes used in this article. In addition, this paper uses unequal-length coding to represent the architecture of the model. To ensure the performance of the model, the number of layers of the model is limited to 3–12 layers. To more clearly show the DNN architecture and its related parameters after coding,

Table 4. The Sequence Corresponding to a five-layer DNN.

Description	Code	Parameter
Full Connection Layer with 32 neurons	0	32
Full Connection Layer with 64 neurons	0	64
Conv2d with 4∗4 Convolutional filter and stride is 2	1	4,2
Relu activation	5
Full Connected Layer with 2 neurons	0	2
Logarithmic cross entropy loss	0
Adam optimizer	0
Learn Rate	0.01

shows the sequence corresponding to a five-layer DNN. It corresponds to a list containing strings and numbers.

3.3.3. Fitness Function

The performance of the model is evaluated from two aspects: one is the detection rate of the model for DDoS traffic and the other is the running time of the model. The accuracy of the model after the test and the average time of the model processing a single sample on the test set are therefore selected to calculate its fitness [39]. The specific calculation formulas are as follows:

$$ACC=\frac{TP+TN}{TP+FP+TN+FN}$$

(1)

$$FITNESS={\alpha }_1\times \frac{ACC}{1.01-ACC}+\frac{{\alpha }_2}{\overline{T}}$$

(2)

Here, ACC is the accuracy, while $\overline{T}$ refers to the average time (ms) for processing a piece of data in the model classification stage. ${\alpha }_1$ and ${\alpha }_2$ represent the weight and are taken as 0.8 and 0.001, respectively, in this paper. By setting the weight of accuracy and running time in this way, the impact of accuracy and running time on the fitness calculation can be the same order of magnitude, and the final generated model has better performance in accuracy and running.

There may be many poor performance solutions in the current population. Taking accuracy as the evaluation standard will make the probability of selecting excellent individuals very low. Therefore, fitness will be dynamically amplified to improve the probability of selecting an ideal solution. From the fitness calculation formula, the closer the accuracy is to 1.0, the faster the fitness increases, which can better highlight the probability of selecting an excellent solution.

3.3.4. Solution Selection

In the process of population iteration, the parent solution in the population needs to be selected randomly according to fitness to generate the child solution. The specific operation is as follows:

The set P of problem solutions is as follows:

$$P=\{p_1,p_2,\cdots ,p_n\}$$

(3)

Each solution in the set is trained and tested to calculate fitness. The fitness set F corresponding to the solution set P is as follows:

$$F=\{fitnes{s}_{p_1},fitnes{s}_{p_2},\cdots ,fitnes{s}_{p_n}\}$$

(4)

Then, the probability $q_{p_x}$ that a solution will be selected is as follows:

$$q_{p_x}=\frac{fitnes{s}_{p_x}}{{\sum }_{i=1}^{n}fitnes{s}_{p_i}}$$

(5)

where $fitnes{s}_{p_x}$ represents the fitness of solution $p_x$ and ${\sum }_{i=0}^{n}fitnes{s}_{p_i}$ represents the sum of fitness of solutions within the population.

A random value $n_{random}$ ∈ [0, 1] is generated each time a solution is selected. Assuming that $n_{random}$ falls within the cumulative probability interval [$lowe{r}_{p_x}$,$uppe{r}_{p_x}$] corresponding to solution $p_x$, $p_x$ is selected to crossover, and the cumulative probability interval is calculated as follows:

$$lowe{r}_{p_x}=\sum _{i=1}^{x-1}{q}_{p_i}$$

(6)

$$uppe{r}_{p_x}=\sum _{i=1}^x{q}_{p_i}$$

(7)

3.3.5. Crossover

The crossover operation in this paper randomly fills the gene fragments of two parent solutions into the child solution, the length of the child solution is a random number in the interval composed of two parent solutions length, and only one new solution is generated each time. The specific operation is as follows: Select two parent solutions, $p_x$ and $p_y$. The length of $p_x$ is $l_{p_x}$ and the length of $py$ is $l_{p_y}$. Then, randomly select a number $l_{p_z}$ from [min($l_{p_x}$, $l_{p_y}$), max($l_{p_x}$,$l_{p_y}$)] as the length of the child solution $p_z$ and calculate the probability that the gene fragments of parent solutions $p_x$ and $p_y$ fill the child solution $p_z$, where the probability that $p_x$ is selected is calculated as follows (the probability of $p_y$ being selected is calculated in the same way as $p_x$):

$${\tilde{q}}_{p_x}={\displaystyle \frac{fitnes{s}_{p_x}}{fitnes{s}_{p_x}+fitnes{s}_{p_y}}}$$

(8)

where $fitnes{s}_{p_x}$ refers to the fitness of the parent solution $p_x$ and $fitnes{s}_{p_y}$ refers to the fitness of the parent solution $p_y$. In Figure 3, the red sequence on the left represents the solution $p_x$, the green sequence on the right represents the solution $p_y$ and the middle sequence represents the new solution $p_z$ generated by the crossover of $p_x$ and $p_y$.

3.3.6. Mutation

Since the random mutation may cause the average fitness of the population to decrease, this paper sets up a GEN (Gene) LRU (Least Recently Used) to alleviate this situation. After each solution in the set calculates the fitness, the solution with the top 20% of the fitness is selected and its gene fragments are shuffled and stored in the GEN LRU. Before the start of each solution’s mutation operation, a random value $n_{random}$ ∈ [0,1] is generated. If the randomly generated number $n_{random}$ is lower than the preset threshold value ${\beta }_1$ = 0.5 (${\beta }_1$ can be set in this way to ensure the stability of the average performance of the population), the child solution randomly selects a fragment from the GEN LRU as a replacement and the child solution performs a random mutation. The GEN LRU is shown in Figure 4, where different colors represent different gene fragments. A mutation of one solution is shown in Figure 5.

3.4. Model Evaluation

In this paper, evaluating the generated model is a critical link. First, the model sequence is decoded, and the code is used to build a runnable model. Next, the same amount of data is randomly selected to train the model. Because model training is highly resource-consuming, this paper stops the model training early by evaluating the change in the model’s performance. The evaluation indicators of the change in model performance are as follows:

$$va{r}_{pc}={\mu }_1\times \frac{AC{C}_{var}}{AC{C}_{mean}}+{\mu }_2\times \frac{AC{C}_{max}-AC{C}_{min}}{AC{C}_{max}+AC{C}_{min}}$$

(9)

$va{r}_{pc}$ represents the value used to evaluate the change trend of the model’s performance; ${\mu }_1$ and ${\mu }_2$ represent the weights selected in this paper, which are 0.4 and 0.6, respectively; $AC{C}_{var}$ represents the variance of model accuracy in five consecutive rounds; $AC{C}_{mean}$ represents the mean of model accuracy in five consecutive rounds; $AC{C}_{max}$ represents the maximum value of model accuracy in five consecutive rounds and $AC{C}_{min}$ represents the minimum value of model accuracy in five consecutive rounds. The model stops training when $va{r}_{pc}$ is less than 0.01. Under the weight set in this paper, the model can stop training when it almost reaches convergence, significantly reducing resource use. Finally, the model is tested and the fitness of the model is calculated and recorded.

4. Experiment and Evaluation

4.1. Dataset Selection

The CICDDoS2019 dataset, released by the Canadian Web Security Institute, was used as the primary training and testing dataset in the experiments. Most previous datasets have often had attack flows lacking diversity, but the CICDDoS2019 dataset is generated using more advanced tools and contains more diverse DDoS attack traffic. A more realistic and comprehensive attack flow dataset has a significant impact on the implementation of deep learning networks as well as the examination of learning outcomes, so this paper chose to use the recently updated CICDDoS2019 dataset that utilizes the B_Profile and M_Profile methods during its generation to simulate the behavior of human use of the network in practice and generate more natural attack flow data and that covers 13 different types of DDoS attacks [10,42]. The CICDDoS2019 dataset simulates an attack network as well as a defense network. Method B_Profile is responsible for analyzing the abstraction behavior of human interactions and encapsulating user-generated network events using machine learning and statistical analysis techniques to simulate benign traffic in the real world. Method M_Profile is used to describe and execute attack scenarios, execute attack scenarios in the real world and generate malicious traffic. Therefore, this paper chose the CICDDOS2019 dataset as the primary dataset in our experiments. In addition, to show that the model generation method presented in this paper can perform well under different scenarios, this paper also used five other datasets for experiments.

4.2. Data Preprocessing

The CICDDoS2019 dataset provides the network traffic data of pcap. However, this paper did not directly apply these data to the DNN. The CIC flowmeter tool is available from the Canadian Web Security Institute and uses this tool to process flow data from pcap files, extract the feature information of 87 of these data points, apply the processed feature information to the DNN and improve the efficiency of DNN updates. The other five datasets in this paper used the same method to extract features as input to the DNN. In addition to applying the tools mentioned above to process the data, experiments in this paper normalize and widen the data to enhance performance.

• Remove invalid information from the input data: Some of the 87 features extracted by the CIC flowmeter tool should not be used as input features in the DNN. These are flow ID, time stamp, source IP, source port, target IP and target port. These features would all have different distributions in different application contexts and should not be taken as features to discern the abnormal flow of DDoS, which could otherwise cause overfitting problems in the DNN. Therefore, this paper removed these features in the data processing stage and chose other flow features as inputs to the DNN [43].
• Cleaning of datasets: This paper complemented the large number of NAN and INF in the raw data with the mean values of the other data.
• Transform labels in the manner of one-hot code: What was addressed in the experiment was a 2-Classification problem, so 1 × 2 one-hot coding was used to indicate the category of a piece of data [44].
• Data normalization processing: The normalization of the data can avoid numerically undesirable effects on the results because of the large difference in the orders of magnitude of the input data and is also convenient for the initialization of the DNN weights. Additionally, the data can be normalized to improve the speed of the gradient descent method for solving the optimal solution. This paper used the maximum–minimum Fmethod in the experiments to normalize the input data. The maximum–minimum method’s formula is as follows:

  $$X_{norm}=\frac{X-X_{min}}{X_{max}-X_{min}}$$

  (10)

  where $X_{norm}$ is the normalized data, X is the original data and $X_{max}$ and $X_{min}$ are the maximum and minimum values of the original dataset, respectively.
• Data expansion to solve the problem of uneven sample distribution: In the CICDDoS2019 dataset, there is a problem of an uneven distribution of sample data, in which the proportion of TFTP, SNMP and DNS traffic is significantly higher than other traffic. In the case of uneven data distribution in the dataset, the training and testing of the model are not objective [36,37,45]. In this paper, the dataset is expanded by the smote method. The smote method generates data through one piece of data and N pieces of data with the same category and the most similar features to avoid the over-fitting problem [46]. The formula generated by the smote method is as follows:

  $$S_{new}=S_{old}+RAMDOM(0,1)\times (S_{neighbor}-S_{old})$$

  (11)

  where $S_{new}$ represents the expanded data, $S_{old}$ represents the original data in the dataset and $S_{neighbor}$ represents a piece of data with the same category and similar features as $S_{old}$.

  Table 5. Percentage of dataset categories before and after the smote method is applied.

  Tag	Percentage before Smote Method	Percentage after Smote Method
  BENIGN	0.00114	0.05011
  DrDoS_DNS	0.10129	0.08262
  DrDoS_NetBIOS	0.08176	0.08309
  DrDoS_NTP	0.02402	0.07811
  UDP-lag	0.00732	0.08163
  WebDDoS	0.00001	0.04184
  DrDoS_UDP	0.06261	0.08296
  DrDoS_MSSQL	0.09034	0.08306
  Syn	0.03161	0.08333
  DrDoS_LDAP	0.04354	0.08332
  DrDoS_SSDP	0.05215	0.08329
  TFTP	0.40115	0.08333
  DrDoS_SNMP	0.10307	0.08332

  shows the proportion of each type before and after smote method processing.

4.3. Evaluation

In the model generation stage, this paper divides the dataset into several sub-datasets of equal size, and each sub-dataset contains the same category of data. Each sub-dataset includes 1000 pieces of data. During each evaluation of the generated model, 50 sub-datasets having normal traffic and 50 sub-datasets having DDoS traffic will be randomly selected. A total of 100,000 pieces of data will be used to test and train the generated model. The data will be divided into the training set and testing set at a ratio of 6:4, the epoch is 20 and the data batch size is 32.

In the subsequent comparison experiment, the model generated and the comparison model will be trained and tested using a complete dataset. To more objectively compare the performance of the model generated and the comparison model, 10-fold cross-validation will be used with an epoch of 20 and a data batch size of 32.

The evaluation indicators used in the experiment phase and their formulas are as follows.

$$PRE=\frac{TP}{TP+FP}$$

(12)

$$REC=\frac{TP}{TP+FN}$$

(13)

$$F1\_SCORE=2\times \frac{PRE\times REC}{PRE+REC}$$

(14)

where $TP$ = true positives; $TN$ = true negatives; $FP$ = false positives; and $FN$ = false negatives.

4.4. Hyperparameter Tuning

In this paper, a total of 10 populations are used for experiments. The crossovers of solutions within the population are independent of each other, but the same GEN LRU is used for mutation. In each experiment, the total number of solutions generated by each population is fixed at 20,000. This paper generates the model with the best performance by adjusting the number of solutions in the population and the number of iteration rounds. Through experiments, it is found that when the number of solutions in the population is 200 and the number of iteration rounds is 100, the model generated by this method has the best performance.

Table 6. The first number in the method field represents the number of solutions in the population and the second number represents the number of iteration rounds.

Method	ACC	PRE	REC	F1_SCORE
(50,400)	0.9913	0.9978	0.9848	0.9913
(100,200)	0.9913	0.9975	0.9850	0.9912
(200,100)	0.9937	0.9996	0.9878	0.9937
(400,50)	0.9909	0.9974	0.9844	0.9909

shows the performance of the best model generated under some representative parameters.

4.5. Model Generation

Table 7. The range of related complexity of the generated model.

Complexity Cost	Range
Flops (M)	[0.01,6.1]
Pamras (M)	[0.0,0.01]

and

Table 8. The average value of related complexity of the generated model.

Complexity Cost	Average Value
Flops (M)	0.1105
Pamras (M)	0.0020

show the complexity range and average value of the model generated in this paper. The total complexity of this method is equal to the total number of model generation times the average complexity of a single model. Flops represent the time complexity of DNN model and parms represent the space complexity of DNN model.

The abscissa represents the number of rounds of iteration and the ordinate represents the model’s average performance or average fitness changes with the number of rounds. Figure 6 shows the average performance of the population in the iteration process. Figure 7 shows that the normalized average fitness continues to improve in the iterative process, indicating that the model is making continuous progress in accuracy and running time. Since the growth speed of the fitness function set in this paper will increase with the increase in accuracy, the curve in Figure 7 does not show a trend of slower growth. Moreover, under the condition that the accuracy is almost constant in the later stage, the continuous reduction in the running time of the model also greatly affects the change in the average fitness of the population.

4.6. Evaluation of Models Generated

This paper selected the best model generated and used the KS (Kolmogorov–Smirnov) curve to evaluate it. The KS curve combines the TPR and FPR to comprehensively assess the model. The yellow curve represents the difference between the TPR and FPR under different thresholds, and the KS value is the maximum value of the difference. As shown in Figure 8, the KS value is 0.624, which means that the model’s performance is good. The abscissa represents the confidence level and the ordinate represents the TPR and FPR under different confidence levels. The green curve represents the changing trend of the TPR under different confidence levels, the red curve represents the changing trend of the FPR under different confidence levels and the yellow curve represents the changing trend of the difference between the TPR and FPR under different confidence levels.

To accurately describe the model’s classification performance, this paper uses a confusion matrix to show the classification results of the model. As shown in

Table 9. Confusion matrix of the best model selected.

	Normal	Attack
Normal	0.9994	0.0006
Attack	0.0119	0.9881

, the normal flow detection rate reached 0.9994 and the DDoS attack detection rate reached 0.9881, indicating that the model can distinguish between attack traffic and normal traffic well.

4.7. Comparative Experiment

Additionally, this paper compares the best model generated using the proposed method tp the models of Mahmoud Said Elsayed, Duy-Cat Can, Abdullah Emir Ci, Shalaka S. Mahadik and Aman Rangapur. This paper also selects some classical machine learning algorithms as the baseline, including decision tree, random forest, SVM, logistic regression and naive Bayes. The evaluation results of each model are obtained through 10-fold cross-validation. To objectively evaluate the model’s performance, this paper uses the processed data to train, test and evaluate the model.

Table 10. The comparative Results with related works.

Techniques	ACC	PRE	REC	F1_SCORE
Best model generated	0.9937	0.9993	0.9881	0.9937
Mahmoud Said Elsayed et al. [47]	0.9913	0.9990	0.9837	0.9913
Andrés Chartuni et al. [48]	0.9927	0.9999	0.9855	0.9927
Abdullah Emir Cil et al. [49]	0.9914	0.9985	0.9846	0.9915
Shalaka S. Mahadik et al. [50]	0.9866	0.9964	0.9771	0.9867
Aman Rangapur et al. [51]	0.9921	0.9987	0.9857	0.9921
SVM	0.8024	0.9646	0.6243	0.7580
C4.5	0.9817	0.9795	0.9840	0.9817
RF	0.7862	0.9931	0.5800	0.7323
LR	0.5071	0.5071	1.0000	0.6729

shows that the generated model is superior to the other models in terms of accuracy and F1-score.

To evaluate the availability of our proposed generation method, this paper generates models on the CICIDS2017 and CICIDS2018 datasets.

Table 11. Performce of best model generated.

Dataset	ACC	PRE	REC	F1_SCORE
CICIDS2017	0.9906	0.9896	0.9915	0.9906
CICIDS2018	0.9993	0.9986	1.0000	0.9993

shows that the proposed method can generate models with good performance in different datasets.

A DNN model generated on only one dataset may have over-fitting problems, so this paper conducts model generation experiments on six data sets at the same time to avoid this issue. (The six datasets selected in this paper are the CICDDoS2019 dataset, the CICIDS2017 dataset, the CICIDS2018 dataset, the KDD_CUP99 dataset, the NSL-KDD dataset and the UNSW dataset.) All generation experiments share a GEN LRU, and there is a 10% probability to select individuals generated on different datasets for crossover operation; only individuals within the top 20% in terms of fitness are used for the crossover operation.

Table 12. Average complexity of different methods.

Method	Flops (M)	Params (M)
Model generation using six datasets	0.2173	0.0038
Model generation using one dataset	0.1105	0.0020
Chiba Z et al. [27]	0.9507	0.0305

represents the average complexity of the method proposed in this paper and the existing methods. The second column represents the time complexity, and the third column represents the space complexity. It can be seen that the method proposed in this paper has lower complexity in terms of time and space.

This paper uses the best model generated with six datasets, the best model generated with one dataset and the best model generated by Chiba Z et al. to carry out comparative experiments on the CICDDoS2019 dataset. According to

Table 13. Performce of best model generated of different methods.

Techniques	ACC	PRE	REC	F1_SCORE	T (ms)	KS
Best model generated with six datasets	0.9902	0.9903	0.9903	0.9902	0.001477	0.607
Best model generated with one dataset	0.9937	0.9993	0.9881	0.9937	0.001340	0.624
Best model generated by Chiba Z et al. [27]	0.9777	0.9785	0.9778	0.9777	0.001371	0.605

, the best model generated with one dataset has the best performance, and the best model generated with six datasets has a slightly lower performance than best model generated with one dataset. The sixth column in Table 13 shows the time it takes for the generated model processes a single piece of data in the test phase, and the seventh column in the table represents the KS value of the generated model. However, considering that best model generated with one dataset may have an over-fitting problem, this paper uses the best model generated with six datasets, the best model generated with one dataset and the best model generated by Chiba Z et al. to carry out comparative experiments on the six datasets selected in this paper.

Table 14. Performance of models generated by different generation methods under different datasets.

Techniques	Dataset	ACC	PRE	REC	F1_SCORE
Best model generated with six datasets	CICDDoS2019	0.9902	0.9903	0.9903	0.9902
	CICIDS2017	0.9660	0.9664	0.9660	0.9660
	CICIDS2018	0.9984	0.9984	0.9984	0.9984
	KDD_CUP99	1.0000	1.0000	1.0000	1.00000
	NSL-KDD	0.9352	0.9354	0.9352	0.9352
	UNSW	1.0000	1.0000	1.0000	1.0000
Best model generated with one dataset	CICDDoS2019	0.9937	0.9993	0.9881	0.9937
	CICIDS2017	0.7460	0.7500	0.7458	0.7449
	CICIDS2018	0.5029	0.7514	0.5000	0.3346
	KDD_CUP99	0.9079	0.9225	0.9076	0.9071
	NSL-KDD	0.7675	0.7795	0.7663	0.7644
	UNSW	0.4817	0.4649	0.4835	0.4039
Best model generated by Chiba Z et al. [27]	CICDDoS2019	0.9777	0.9785	0.9778	0.9777
	CICIDS2017	0.8121	0.8202	0.8117	0.8108
	CICIDS2018	0.6591	0.6879	0.6583	0.6448
	KDD_CUP99	0.9995	0.9995	0.9995	0.9995
	NSL-KDD	0.7010	0.7018	0.7012	0.7010
	UNSW	0.4937	0.4968	0.4999	0.3308

shows the performance of best model generated with six datasets, the best model generated with one dataset and the best model generated by Chiba Z et al. on the six datasets selected in this paper. The best model generated with one dataset has the best performance on the CICDDoS2019 dataset but has poor performance on other datasets. The best model generated with six dataset shows good generalization, and it can be seen that using multiple datasets to conduct model generation experiments at the same time can alleviate the over-fitting problem to a certain extent.

5. Discussion

This paper selected the models within the top 20% in terms of fitness and analyzed their architecture.

Table 15. Percentage of layer occurrences.

Topology Layer	Percentage of Occurrence
Full Connected Layer	0.2597
Conv2d	0.0175
Maxpool2d	0.0957
Droupout	0.0087
Residual layer	0.1693
Relu	0.0896
Sigmoid	0.0412
Tanh	0.0773
Softplus	0.2402

shows that the Full Connected Layer and the Softplus activation function appear most frequently, followed by the Residual Layer and finally the Maxpool2d and Relu activation function. This shows that compared with other activation functions, choosing to use the Softplus activation function can provide a better nonlinear fitting ability to a DNN in data analysis. The wide use of the Residual Layer means that the dataset selected makes the general model converge faster, while the Residual Layer allows the model to have more choices and evolve in a better direction.

Table 16. Layer occurrence frequency.

Number of Occurrences	1	2	3	4
Full Connected Layer	0.5050	0.3110	0.1220	0.0470
Conv2d	0.8100	0.1800	0.0000	0.0000
Maxpool2d	0.2380	0.6875	0.0588	0.0147
Droupout	0.9705	0.0294	0.0000	0.0000
Residual layer	0.3375	0.4936	0.1278	0.0358
Relu	0.6000	0.2923	0.0769	0.0307
Sigmoid	0.6825	0.0950	0.0222	0.0000
Tanh	0.2463	0.6811	0.0724	0.0000
Softplus	0.3913	0.4434	0.1217	0.0347

shows that this paper has limited the depth of the model. Most of the same topology layers in the model only appear 1–2 times, which indicates that too many repeated topology layers may not improve the model performance, but may increase the model’s running cost. Through the fitness function set in this paper, the GA adjusts the model architecture well, making the model better adapted to the dataset selected.

The innovation in this paper reflects the potential of the GA in model generation. This paper proposes a DNN generation method based on a GA. The method is aimed at generating a DDoS detection model, which classifies input traffic into normal traffic and DDoS traffic. This generation method generated models through population iteration. Compared with existing models, the models generated achieved better results in accuracy, precision, recall and F1-score. This paper also generated models on multiple datasets, and the best model selected also showed better performance. Therefore, the method can generate a model with better performance in a limited time under the condition of changeable DDoS attacks. The results of this paper can provide certain reference significance for researchers, minimize the consumption of resources, realize the automation of model construction and facilitate the related work of DDoS detection to a certain extent.

This paper also found that models with similar performance have a certain architectural similarity. Figure 9 shows that the coding corresponding to the DNN architecture is reduced to two dimensions by t-SNE. The abscissa and ordinate correspond to the distribution of the coding corresponding to the DNN architecture in two-dimensional space. In this paper, the model’s fitness is normalized and the legend in the upper left corner shows the colors corresponding to the low-dimensional mapping of DNN architecture coding in different fitness ranges. It can be found that the low-dimensional maps of the DNN architecture’s coding have a fitness between 0.9 and 1.0 and are clustered together, indicating that DNN models with good performance may have similar architectures.

6. Conclusions and Future Work

Through observing the generated models, this paper can summarize the paradigms of several models so that future researchers can easily find a suitable model for the DDoS detection problem. This paper observed that models were most commonly generated with a depth of nine layers, and the best models were those containing multiple MaxPool2d and Residual layers. DDoS traffic has some characteristics that need to be amplified compared with ordinary traffic and some key information is hidden under the noise. The Relu and Tanh activation functions are widely used in models with good performance as these two activation functions can sufficiently express the nonlinear parts of the data. Moreover, the Relu activation function always appears after the maximum pool, while the Tanh activation function and the Residual Layer have a high probability of appearing together. The combination of this activation function and the specific structure makes it easier for the DNN to reach the convergence state in the iterative process.

The limitations of this paper lie in the following points. Firstly, there are too few layer types used when building the DNNs, and more kinds of layers need to be introduced to build DNN models for specific problems. Secondly, this paper only carries out model generation for binary classification problems. For model generation for multi-classification problems, more operators will need to be introduced to improve the search ability of the GA. Thirdly, the datasets used in the model generation in this paper contain similar attack types. In the future, we will need to conduct model generation experiments on multiple datasets with different similarities at the same time and summarize which dataset combination method is more conducive to model generation.