Membrane System-Based Improved Neural Networks for Time-Series Anomaly Detection

Abstract

Anomaly detection in time series has attracted much attention recently and is quite a challenging task. In this paper, a novel deep-learning approach (AL-CNN) that classifies the time series as normal or abnormal with less domain knowledge is proposed. The proposed algorithm combines Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) to effectively model the spatial and temporal information contained in time-series data, the techniques of Squeeze-and-Excitation are applied to implement the feature recalibration. However, the difficulty of selecting multiple parameters and the long training time of a single model make AL-CNN less effective. To alleviate these challenges, a hybrid dynamic membrane system (HM-AL-CNN) is designed which is a new distributed and parallel computing model. We have performed a detailed evaluation of this proposed approach on three well-known benchmarks including the Yahoo S5 datasets. Experiments show that the proposed method possessed a robust and superior performance than the state-of-the-art methods and improved the average on three used indicators significantly.

1. Introduction

Anomaly detection aims to find abnormal behavior of data and is widely studied in many fields, like fault detection or predicted maintenance in industrial systems [1]. The reason anomaly detection is important is because anomalies usually contain useful and critical message. To cope with the increasing data collected by research institutions and industries through the Internet of Things (IoT), it is important to have automated procedures that separate the anomalies from normal data.

However, anomaly detection is considered a hard problem [2]. The extremely unbalanced data distribution is the biggest difficulty, and the negative class rate is extremely low. One detection algorithm which works very well on a certain benchmark might get surprisingly bad performance on another. Moreover, anomaly detection for time series is much more difficult due to the issue inherent in time series. For these reasons, this paper tries to find an effective and robust detection algorithm. Many scholars have studied the methods of detecting abnormal patterns by extracting data features in the field of anomaly detection. Anomaly-detection methods mainly consist of three types: statistical modeling [3,4,5,6], such as the k-means clustering and Random forest methods, temporal feature modeling [7,8,9,10] which is mainly based on the LSTM, and spatial feature modeling [11,12,13] which takes the advantages of CNN. Traditionally, time-series anomaly detection has been tackled using distance-based methods, such as the dynamic time wrapping algorithm (DTW) [14], meanwhile, artificial neural networks have become powerful tools for time-series anomaly detection due to the large amount of data.

Membrane computing (P system) [15], a novel branch of natural computing, has gained popularity in recent years due to its promising features such as the distribution, uncertainty, and especially, parallelism. The P system were inspired by the structure and function of biological cells and communication in tissues, organs and cell populations [16,17]. Many variants of P system has been proposed and combined with many optimization approaches [18,19] which shows great performance of convergence and robustness [20,21,22,23,24]. Furthermore, with the development of GPU and to make most use of the parallelism of membrane systems, P system has been simulated in GPU [25] recently. However, the common P systems use the simplified membrane structures to deal with problems due to computation purposes; therefore, it is necessary to use complex structures of membranes to solve real applications.

Deep learning has become a popular machine learning approach due to its ability to learn high-level representations related to the data, such as the periodicity and seasonality of the time series. These representations are learned automatically from data with little or no need manual feature engineering and domain expertise [26]. For time-series data, LSTM has become the most widely used model for its ability to learn long-range patterns. LSTM works well in handling the variable-length sequences, but it lacks the ability to extract local contextual information and cannot use the contextual information; therefore CNN is integrated in this paper. Due to these considerations, the main intention of this work is to combine P systems and deep-learning approach to develop a novel framework for time-series anomaly detection. We proposed a hybrid dynamic membrane system (HM-AL-CNN) which reduces the time and takes advantage of ensemble learning in deep P system. In the novel membrane structure, we carry out multiple AL-CNNs for time-series anomaly detection which predicts the label of next timestamp using a window of time-series.

2. Main Contributions

The objective of HM-AL-CNN is to robustly detect time-series point anomalies and discord. As far as we know, this is the first attempt to solve temporal data anomaly-detection tasks via a membrane system-based approach. Profiting from its parallelism, the proposed P system can handle several AL-CNN models with different initialization to get effective features simultaneously. For comparison, we evaluate our methods on three well-known benchmarks that have been employed by many previous approaches. Experimental results show that the proposed methods possess a robust and superior performance compared to the state-of-the-art methods. The following are the main contributions of this paper.

• A hybrid dynamic P system is proposed to solve complex tasks, which integrate the tree-based and graph-based P system; two types of membrane evolutionary rules are introduced as well.
• This paper intends to take advantage of the outstanding performance of P systems and deep-learning methods. CNN and LSTM were integrated into the proposed membrane systems for time-series anomaly-detection tasks.
• The proposed method employs LSTM with attention mechanism; squeeze-and-excitation networks are extended and added to further improve the performance.
• The proposed approach is evaluated on three well-known benchmarks from different domains and shows better performance than other detection algorithms.

The rest of this paper is arranged as follows. Section 3 gives an overview of the background works and gives the architecture framework of HM-AL-CNN. In Section 4, experimental settings and datasets description are presented. Section 5 provides a detailed evaluation of the proposed algorithm on three well-known benchmarks along with other popular anomaly-detection methods. Finally, conclusions and direction for future work are laid out in Section 6.

3. Related Works and Method

3.1. Background Works

3.1.1. Long Short-Term Memory and Convolutional Neural Networks

Long short-term memory is a mainstream kind of RNN [27] and is much more complex, capable of learning long-term dependencies. LSTM relieves the problem of vanishing gradient by replacing the self-connected hidden units with memory blocks [28]. LSTM has been adopted widely for machine translation and time-series forecasting. The architecture of an LSTM is shown in Figure 1.

The formula of LSTM is given below, the $W^{\ast }s$, $R^{\ast }s$ and $b^{\ast }s$ are the input weights, the recurrent weights and the biases, respectively, $z_t$, $h_t$ indicates input and output of the LSTM unit, $s_t$ is the current cell state:

$$z_t=tanh(W^z{x}_t+R^z{h}_{t-1}+b^z)$$

(1)

$$i_t=\sigma (W^i{x}_t+R^i{h}_{t-1}+b^i)$$

(2)

$$f_t=\sigma (W^f{x}_t+R^f{h}_{t-1}+b^f)$$

(3)

$$o_t=\sigma (W^o{x}_t+R^o{h}_{t-1}+b^0)$$

(4)

$$c_t=z_t(W^o\odot i_t+c_t-1\odot f_t)$$

(5)

$$h_t=tanh\left(s_t\right)\odot o_t$$

(6)

Convolutional neural network (CNN) is also a type of ANN and was developed for image classification problems. CNNs can be applied to one-dimensional sequences of data as well, such as human activity recognition; the model can learn an internal representation of the time-series data and achieve comparable performance. The CNN employs a convolution operation and is defined as:

$$\left(t\right)=(x\ast w)\left(t\right)$$

(7)

This formula can be regarded as a weighted average of $x\left(\tau \right)$ at the time stamp t, where weight is calculated by $w(-\tau )$ shifted by amount t. One-dimensional convolutional is defined as:

$$s\left(t\right)=\sum _{\tau =-\infty }^{\infty }x\left(\tau \right)w(t-\tau )$$

(8)

3.1.2. Attention Mechanism

The attention mechanism was proposed by Bahdanau [29] and is used in various deep-learning models. As the function given by the following equation shows, the context vector $c_i$ for the output is calculated using a weighted sum of the annotations $h_i$ which means that the context vector $c_i$ depends on a sequence of annotations $(h_1,...,h_{T_x})$. Each annotation $h_i$ contains specific information and drops the irrelevant information about the whole input.

$$c_i=\sum _{j=1}^{T_x}{\alpha }_{ij}{h}_j$$

(9)

where weight ${\alpha }_{ij}$ is the attention score of each annotation. It can be calculated as follows:

$${\alpha }_{ij}=\frac{exp\left(e_{ij}\right)}{{\sum }_{k=1}^{T_x}exp\left(e_{ik}\right)}$$

(10)

where $e_{ij}$ is the output score of a neural networks given by $a(v_{i-1},h_j)$, $v_{i-1}$ is the hidden state, $h_j$ indicates the j-th annotation, $e_{ij}$ attempts to capture the alignments of the input at j and output at i.

3.1.3. Tissue-Like and Cell-Like Membrane Systems

In this section, we briefly introduce some concepts related to P systems which are distributed computational parallel models. Membrane computing was inspired by the structure and functions of cells, tissues and organs. In recent years, researchers have turned to the application of membrane computing models. Generally, there are three main families: cell-like P system [30], tissue-like P system [31,32] and neural-like P system [33]. The structure of the tissue-like P system can be viewed as a net, a tissue-like membrane system of degree $m>0$ is constructed as follows [34]:

$$\mathrm{\Pi }=(O,{\omega }_1,...,{\omega }_q,R_1,...,R_q,R^{\prime },i_0)$$

(11)

where O represents finite non-empty alphabets of objects; ${\omega }_i(1\le i\le q)$ are initial multisets of objects present in cell i; $R_i$ are finite sets of evolution rules in cell $i(1\le i\le q)$, $R^{\prime }$ is a finite set of communication rules; $i_0\in \{0,1,...,q\}$ indicates the output region where the computation results are placed.

A cell-like P system has a hierarchical arrangement of membranes inside a skin membrane. Each membrane delimits a region where multisets of objects and rules are placed and a set of evolution rules take the form $[\omega \to {\omega }^{\prime }]$ [34].

3.2. Deep Neural Network with Squeeze-and-Excitation and Attention Mechanism (AL-CNN)

Squeeze-and-Excitation

Hu [35] proposes a squeeze-and-excitation Network (SENet) for CNNs to improve the channel interdependencies. SENet is an architecture for transformation $F_{tr}:X\to U,X\in R^{W^{{}^{\prime }}\times H^{{}^{\prime }}\times C^{{}^{\prime }}},U\in R^{W\times H\times C}$. We can then represent output of $F_{tr}$ as $U=(u_1,u_2,...,u_c)$, $u_c$ is defined as follows, ∗ represents the convolution operation and $v_c^s$ is the kernel.

$$u_c=v_c\ast X=\sum _{s=1}^{C^{\prime }}{v}_c^s\ast x^s$$

(12)

Hu improves the channel interdependencies through squeeze-and-excitation operation. The squeeze process uses a global average pool to get a global understanding of each channel. In our case, similar to the image data, the U is generated by shrinking through the temporal dimension T to achieve the channel-wise statistics, $z\in R^C$, the cth element of z is calculated by $F_{sq}\left(u_c\right)$ which is defined as follows:

$$z_c=F_{sq}\left(u_c\right)=\frac{1}{T}\sum _{t=1}^T{u}_c\left(t\right)$$

(13)

To use the aggregated information obtained from the squeeze stage, excite operation that uses two fully connected layers is employed to get the channel-wise dependencies. We employ an equation given below:

$$s=F_{ex}(z,W)=\sigma \left(g(z,W)\right)=\sigma \left(W_2\delta \left(W_{1}z\right)\right)$$

(14)

where $F_{ex}$ is a neural network, $\sigma$ and $\delta$ indicates the sigmoid function and ReLU function respectively, $W_1$ and $W_2$ are learnable parameters of $F_{ex}$. Finally, the output of the block is gained by rescaling U as follows:

$$\widetilde{x_c}=F_{scale}(u_c,s_c)=s_c·u_c$$

(15)

where $\tilde{X}=({\tilde{x}}_1,{\tilde{x}}_2,...,{\tilde{x}}_C)$ and $F_{scale}(u_c,s_c)$ represents the channel-wise multiplication between the feature map $u_c$ and the scalar $s_c$.

Normally, time-series anomaly detection can be transformed into a binary classification problem but is much more complex, for the data is extremely imbalanced. LSTM and CNN have rarely been combined to realize time-series anomaly detection. Similar to the LSTM-FCN proposed by Fazle Karim [36], the model we proposed (AL-CNN) combines both and extends the LSTM with attention mechanism; furthermore, one-dimensional convolution (1DCNN) is added before the attention LSTM to improve the efficiency of the model. In particular, we extend the Squeeze-and-Excitation block to the case of 1D sequence models to enhance the anomaly-detection accuracy. The model can handle both the point anomaly and discords no matter how univariate or multivariate time series. The procedure of the proposed AL-CNN is shown in the Figure 2.

3.3. Hybrid Dynamic Membrane Systems Based Al-Cnn (Hm-Al-Cnn) for Time-Series Anomaly Detection

3.3.1. Architecture Summary of Hm-Al-Cnn

Generally, both tissue-like and cell-like P systems are predigestion and they are not applied to deal with hard problems in the real world. In this work, we intend to use the strengths of both tissue-like and cell-like membrane structure to develop a hybrid dynamic membrane structure as shown in Figure 3. The graph-based and tree-based membrane structure is depicted via rounded rectangle and squares, respectively. A hybrid dynamic membrane system is constructed as the form:

$$\mathrm{\Pi }=(O,E,u,u_T,u_G,{\omega }_1,...,{\omega }_q,R_1,...,R_q,i_0)$$

(16)

where O represents a finite set of objects; $E\subseteq O$ is the set of objects in the environment; $\mu$ is a membrane structure which include ${\mu }_T$ and ${\mu }_G$, here, ${\mu }_T$ are Tree-based membranes and ${\mu }_G$ represent Graph-based membranes; the symbols ${\omega }_1,...,{\omega }_q$ are finite sets of strings over O of q membranes; the $i_0$ represents the output membrane of $\mathrm{\Pi }$ and $R_1,...,R_q$ are finite sets of rules including two types described below:

The G-rule is used in the HM-AL-CNN to establish a synchronous communication channel within the computation cells, exchange the multiset $a_i$ of cell x with multiset $b_i$ of cell y, x and y are membrane labels:

$$\mathrm{\Pi }=(a_i,out,x;b_i,in,y)$$

(17)

The C-rule compares the output of each AL-CNN and picks the best one as the final result:

$$\{pre-Output1,pre-Output2,...,pre-Outputm\}\to output$$

(18)

3.3.2. Initialization

The P system yields the whole initial objects in Input cells; each object represents a 1-dimensional or 2-dimensional vector, denoting the original time series with a size of $w\times h$. Then, the Input cells communicate objects to m computation cells to carry out each AL-CNN.

$$O=({\beta }_{11},{\beta }_{12},...{\beta }_{1h};{\beta }_{i1},{\beta }_{i2},...{\beta }_{ih};{\beta }_{w1},{\beta }_{w2},...{\beta }_{wh})$$

(19)

where ${\beta }_{i1},{\beta }_{i2},...{\beta }_{ih}$ indicates the ith row of the time series.

3.3.3. Computation Mechanism

In this paper, the G-rule is introduced to implement several AL-CNNs and Pre-Output cell gives the result of every AL-CNN. Objects in the P system evolve according to the step of AL-CNN described in Section 3.2 during the computation phase. Then, the C-rule is applied to choose the best objects among the Pre-Outputs as the final result of the HM-AL-CNN.

3.3.4. Termination and Output

The above computing procedures are processed iteratively, and the maximum computation iteration is used as the halting condition. The membrane system halts when the maximum number of iterations is reached and all the objects in the output cell are considered to be the final results of the P system.

4. Experiments

4.1. Experiments Settings

To evaluate the proposed method, HM-AL-CNN has been tested on three benchmarks which are described in Section 4.4. The model was optimized using Adam with an initial learning rate of $1\times {10}^{-6}$ and the convolution kernels are initialized by the He initialization scheme [37], ReLU was used as the activation function for the hidden layers. The number of training epochs was determined based on the length of the input; for the Yahoo Webscope S5, the model was trained for 500 epochs using batches of 128. The Classic Anomaly Datasets and Space Shuttle Valve Dataset were trained for 700 epochs using batches of 256.

Time-series data need to be transformed into sequences of overlapping windows of size w so that the system makes sense. For $x_t$ at time step t, its condition (normal or abnormal) is used as the label of the former w elements; w is the time window size which is also called a history window. Then, we can define the data as a form of ($N,Q,M$), where N is the number of samples in the time series, Q indicates the maximum time steps and M represents the number of variables; we define the M to 1 if the time series is univariate.

In addition, both the train and test datasets are normalized using Equation (20). x and $x^{\prime }$ represents the value of the actual time-series data and the normalized value, respectively. Moreover, we define fixed-sized anomaly windows with each window centered around an anomaly; points in the anomaly window are labeled abnormal. For instance, if the anomaly window size is set to 10, indicating the former 5 points and latter 5 points are labeled abnormal. Only the training sets are operated as such; this up-sampling operation can relieve the extremely imbalance of the data and enhance the performance significantly especially the recall rate.

$$x^{\prime }=\frac{x-x_{min}}{x_{max}-x_{min}}$$

(20)

4.2. Loss Function and Output

Cross-Entropy Loss given in Equation (21) has been employed to measure the difference between the actual value $y_j$ and predicted value ${\widehat{y}}_j$.

$$L=(ylog\widehat{y_j}+(1-y)log(1-\widehat{y_j}))$$

(21)

In our case, the SoftMax layer classifies the output into two classes either normal or abnormal as described in Equation (22). C indicates the class, d is the output of the fully connected layer, w is the weight, L represents the last layer and $N_c$ is the total number of classes.

$$P\left(c\right|d)=argma{x}_{c\in C}\frac{exp\left(d^{L-1}{w}^L\right)}{{\sum }_{k=1}^{N_c}exp\left(d^{L-1}{w}_k\right)}$$

(22)

4.3. Evaluation Metrics

The proposed approach is evaluated using Precision, Recall, F-score and AUC. If an abnormal case is classified as a normal, this type of error is considered to be false negative (FN). True positive (TP), true negative (TN) and false positive (FP) is defined similarly; each algorithm was evaluated through TP, TN, FP and FN rates. In addition, AUC is also the most commonly used metric for evaluating anomaly-detection methods.

4.4. Datasets Description

In this section, we describe three well-known benchmarks from different domains, including real-world and the synthetic datasets which have been applied in previous works on anomaly detection, including the Yahoo Webscope S5, Classic Anomaly-Detection Datasets and Space Shuttle Valve Dataset.

• Yahoo Webscope S5 Datasets

Yahoo Webscope S5 consists of four classes. Class A1 contains the real Yahoo membership login data, and A2, A3 and A4 contain synthetic anomaly data (https://research.yahoo.com).

Table 1. Characteristics of each class for the Yahoo Webscope S5.

	A1	A2	A3	A4
Real	√	X	X	X
Synthetic	X	√	√	√
Number of instance	94,866	142,100	168,000	168,000
Number of anomalies	1669	466	943	837

shows the characteristics of each sub-benchmark. This dataset contains 367 time series. Each time series consists of almost 1500 data including 0.02% abnormal values. Figure 4a,b show the statistical graphs for the class A1. We can see from the two figures that the data distribution of each file is significantly different; it is not easy to carry out anomaly detection using statistical analysis techniques. Figure 5a shows a real-world time series of the A1 class.

• Classic Anomaly-Detection Datasets

Six commonly used natural datasets have been adopted in this section, which can be found at the UCIRepository [38] and OpenML; anomaly cases have already been marked as ground truth, including the Pima, Covertype, Ionosphere, Mammography, Shuttle and Kddcup99. We have removed all non-continuous attributes as done in [39,40]. Properties of each dataset are shown in

Table 2. Characteristics of the six classic datasets.

	Number of Instances	Number of Features	Anomaly Class	Anomaly Rate
Pima	768	8	pos	34.90%
CoverType	286,048	10	class = 4	0.96%
Mammography	11,183	6	class = 1	2%
Ionosphere	351	32	bad	36%
Shuttle	49,097	9	classes ≠ 1(class 4 removed)	7%
Kddcup99	494,021	41	class ≠ normal	80%

.

• NASA Space Shuttle Valve Dataset

This dataset collects values which control the flow of fuel on the space shuttle. Some subsequences are normal and few subsequences are abnormal. Figure 5b shows this time series, and the time series is segmented to several subsequences with an orange dotted line; some subsequence are considered abnormal or, in other words, discord subsequences.

4.5. Comparison to State-of-the-Art

Experiments on the Yahoo Webscope S5 are compared to several deep-learning approaches, including the CNN, LSTM, CNN + LSTM, DeepAnt [41] and two popular tools, Yahoo EGADS which was released by Yahoo Labs to detect anomalies in large scale time-series data and Twitter Anomaly-Detection method which aims to detect anomalies of social network data [38]. There are also many different previous works related to the classic anomaly benchmarks mentioned in Section 4.4, for the sake of brevity, we select the popular anomaly-detection techniques for comparison including the Isolation Forest (iForest), OCSVM, LOF [42].

5. Results and Discussion

5.1. Results

• YAHOO Webscope S5 Datasets

The experimental results of the proposed compared to the other detection algorithms are shown in

Table 3. Performance comparison of the proposed with other methods on class A1.

Methods	Accuracy	Precision	Recall	F1 Score
Proposed method	0.94	0.59	0.79	0.66
LSTM	0.72	0.14	0.24	0.18
CNN	0.73	0.25	0.50	0.23
CNN + LSTM	0.86	0.38	0.44	0.35
YAHOO EGADS	-	-	-	0.47
Twitter Anomaly Detection	-	-	-	0.48

which demonstrates that the proposed improves the detection performance compared to other algorithms, including the deep-learning and classic anomaly-detection algorithms. Figure 6a,b indicate the experimental result of an example time series in the A1 class; HM-AL-CNN detects five out of six anomalies and has only one false positive. Furthermore, HM-AL-CNN does the detection before the true anomalies occur, which is vital to the real application scenery, especially in the industry field. We compared our results with previous methods using the t-test as shown in

Table 4. The p-values of our method compared to the other methods on class A1.

Methods	Dataset	p-Value (F-Score)
LSTM	A1	0.0050 (p < 0.05)
CNN	A1	0.0021 (p < 0.05)
CNN + LSTM	A1	0.0013 (p < 0.05)

. The p-values for F-score of A1 are all < 0.05, and the proposed approach achieves a statistically significant improvement over other methods.

Table 5. Performance comparison of the proposed models with other methods on the whole Yahoo Webscope S5.

	A1	A2	A3	A4
Proposed methods	0.66	0.99	0.87	0.59
Yahoo EGADS	0.47	0.58	0.48	0.29
Twitter Anomaly Detection Alpha = 0.0	0.48	0	0.26	0.31
Twitter Anomaly Detection Alpha = 0.1	0.48	0	0.27	0.33
DeepAnt	0.46	0.97	0.87	0.68

shows a comparison of the proposed with other algorithms on the whole Yahoo Webscope S5. This table gives average F-score of the comparison algorithms along with the proposed data of each sub-benchmark. HM-AL-CNN outperforms other methods in three sub-benchmarks and works slightly worse for sub-benchmark A4. We compared our results with previous methods using a Wilcoxon signed-rank test as shown in

Table 6. The p-values of our method compared to the other methods on the whole Yahoo Webscope S5.

Methods	Dataset	p-Value (F-Score)
Yahoo EGADS	Yahoo S5	0.015 (p < 0.05)
Twitter Anomaly Detection Alpha = 0.0	Yahoo S5	0.015 (p < 0.05)
Twitter Anomaly Detection Alpha = 0.1	Yahoo S5	0.015 (p < 0.05)
DeepAnt	Yahoo S5	0.500 (p > 0.5)

; the proposed approach achieves a statistically significant improvement over other methods except DeepAnt. Even though HM-AL-CNN is not always the best, it achieves better means than DeepAnt and performs better in the whole dataset.

• Classic Anomaly-Detection Datasets

To evaluate different anomaly-detection algorithms along with the proposed on the Classic Anomaly-Detection Datasets, AUC has been used. AUC is used commonly for evaluating the detection approach on the mentioned datasets. We compare the results of three state-of-the-art anomaly-detection methods with HM-AL-CNN and the results are shown in

Table 7. Performance comparison of the proposed models with the rest methods on Classic Anomaly-Detection Datasets.

	iForest	OCSVM	LOF	DeepAnt	HM-AL-CNN
Pima	0.4	0.26	0.48	0.31	0.65
ForestType	0.78	0.70	0.57	0.85	0.92
Ionosphere	0.82	0.84	0.83	0.85	0.86
Mammography	0.85	0.89	0.72	0.99	0.97
Shuttle	0.98	0.99	0.56	0.99	0.94
kddcup99	0.98	0.99	0.42	0.99	0.99

. For iForest, OCSVM, and HM-AL-CNN, 40% of the actual data are used for training and rest for testing. We have used the default parameters suggested in [39] for iForest; RBF kernel for OCSVM and k = 10 is applied for LOF. Figure 7 shows that HM-AL-CNN has an arithmetic rank of 1.66 and performs better than the existing methods via a critical-difference comparison of the average arithmetic ranks.

• NASA Space Shuttle Valve Dataset

The above experiments have already shown that HM-AL-CNN is able to detect point anomalies in time-series data. In this section, HM-AL-CNN is proved to be suitable to time-series discord detection as well. Discords are subsequences that are different from the rest of a longer time series [43]. In this experiment, this proposed algorithm can label most of the points in an abnormal discord cycle and label the points in normal cycles normally. If the abnormal number is over the threshold we set, we classify the sequence discord. Figure 8a,b show the experiment results, there are four normal sequences and one discord in the test set; these experimental results demonstrate that the proposed algorithms work well.

5.2. Discussion

The proposed method achieves better results than the other methods in most cases. Due to the different distribution of the time series, the proposed works slightly worse in some cases. From Table 3, we found that the proposed performs better than CNN and LSTM-based algorithms, which indicates that squeeze-and-excitation and attention mechanism could be used in this case to improve the detection performance; in addition, the proposed achieves a recall value of 0.79, which improves the previous methods significantly. Table 5 shows that the proposed works slightly worse in A4. Whether or not the additional attributions such as the change-point and noise caused the slightly bad performance still need to be explored, generally, the proposed works better on the whole Yahoo S5. Table 7 indicates that HM-AL-CNN has an arithmetic rank of 1.66 and can find anomalies in a multi-variant dataset as well.

6. Conclusions

In this paper, we propose a novel hybrid dynamic membrane system which takes advantages of tissue-like and cell-like P system for a time-series anomaly-detection task. To get more accurate detection results, CNN and LSTM with attention mechanism are combined, and 1D Squeeze-and-Excitation mechanism is introduced to better learn effective features. Two types of rules are introduced in the designed membrane system, profiting from the parallelism of P system; this proposed HM-AL-CNN can process several AL-CNN models individually, which consumes less time. Experiments show that the proposed possesses better performance than other time-series anomaly-detection algorithms in different benchmarks. However, there are still many important parameters that need to be chosen manually in our system, which remains to be addressed. Evolutionary algorithms such as the particle swarm optimization could be used in the future. Moreover, the design of a more effective membrane system to solve complex problems is also meaningful.