Community Detection Based on Graph Representation Learning in Evolutionary Networks

Abstract

Aiming at analyzing the temporal structures in evolutionary networks, we propose a community detection algorithm based on graph representation learning. The proposed algorithm employs a Laplacian matrix to obtain the node relationship information of the directly connected edges of the network structure at the previous time slice, the deep sparse autoencoder learns to represent the network structure under the current time slice, and the K-means clustering algorithm is used to partition the low-dimensional feature matrix of the network structure under the current time slice into communities. Experiments on three real datasets show that the proposed algorithm outperformed the baselines regarding effectiveness and feasibility.

1. Introduction

The evolutionary network refers to a network in which the nodes and edges change over time. As a typical evolutionary network, social networks are very common in modern life, such as Flickr, Facebook, Twitter, and so on. In the above-mentioned social networks, an individual or a group of users may join and exit the network, and the intimacy with friends is increased through obtaining to know friends of friends. Community detection in an evolutionary network is a complex and challenging problem. The development and change of the network structure under the adjacent time slices are slow, however, the statistical characteristics of the network will change significantly over a long period.

Therefore, it is necessary to re-identify the community structure. As the organization of a dynamic community changes constantly, a dynamic community tends to be more complex. Network representation has gradually become the main supporting technology of dynamic community detection, which provides great convenience for dynamic community detection. The application of network representation technology can greatly improve the detection performance [1,2].

Traditional community detection algorithms are mostly based on static networks, and the influence of time on the network structure is ignored. For example, network communication operators hope to reduce the communication costs in the same social circle in different periods, proposing a flexible billing activity for their customers. The static community is uncertain about the actual network connectivity, and this marketing policy is obviously not effective. If considering the change of time for community detection, the real-time community structure can be dynamically represented, which is beneficial to providing individualized services for users.

Thus, it can be seen that the evolution of the network structure has an important impact on community detection, which is likely to bring commercial value [3]. Early research on the division of dynamic network communities [4,5,6,7,8] viewed the dynamic network as a series of snapshot networks in chronological order, and obtained the community structure of each snapshot by using the static network community detection algorithm; however, this kind of thinking does not make full use of the evolution information of the network structure, and a fundamental drawback rooted in such schemes is that most of traditional algorithms are sensitive to tiny changes in the network structure [9].

The DeepWalk [10] algorithm uses a random walk method to obtain the neighbors of nodes in the network to generate a fixed-length node sequence, and the node sequence is fed into the Skip-Gram model for learning representation to apply to other tasks. The SDNE [11] algorithm uses a deep sparse autoencoder [12] to simultaneously optimize the first-order and second-order similarity objective functions, conducting semi-supervised learning representation, and it is applied to visualization, multi-label classification, link prediction, etc. In machine learning research, scholars have proposed learning methods for graph embedding [13,14,15]; however, these methods usually only work well on certain networks.

To overcome these limitations, we propose a novel community detection algorithm based on evolutionary network representation learning (Learning Community Detection Based on Evolutionary Network, LCDEN). The LCDEN algorithm uses a Laplacian matrix to obtain the node relationship information of the directly connected edges in the network structure of the previous time slice, learns the network structure under the current time slice through the deep sparse autoencoder, and uses a K-means clustering algorithm to divide the community of the obtained low-dimensional feature matrix of the network structure under the current time slice. Our contributions can be summarized as follows:

(1)

We propose a novel algorithm for community detection in evolutionary networks, which solves the limitation that traditional community detection algorithms were unable to handle the temporal information of a network structure.

(2)

The proposed algorithm can effectively use the historical temporal information of a network structure and apply a deep learning model to the research of evolutionary network representation learning.

(3)

The proposed algorithm has advantages in different datasets and has higher detection performance and computational efficiency.

2. Related Work

Community detection is one of the most active topics in the field of graph mining and network science. With the continuous expansion of the real-world network scale and the introduction of temporal information, the research on community detection in evolutionary networks can explore community detection algorithms based on the network structure and network information conforming to the real world.

Network representation learning. In the research of network representation learning, one of the foremost requirements of network representation learning is preserving network topology properties in learned low-dimension representations [16]. Researchers often use methods based on matrix decomposition or random walking to learn the graph representation, such as LLE [17], Laplacian Eigenmaps [14], Deepwalk [10], and LINE [18]. However, these methods typically have high computational complexity and poor generalization ability. In addition, they lack the use of temporal information in evolutionary networks.

Evolutionary network. The research on the dynamic evolution of networks has been an ongoing hotspot. The dynamic information in networks has proven to be crucial for understanding networks. Many works have attempted to combine the research of evolutionary network representation learning with the basic research of complex networks. To reveal the dynamic mechanism of information dissemination networks, Rodriguez et al. [19] modeled the information propagation process as several discrete networks with continuous-time slices occurring at different rates.

Fathy et al. [20] realized the joint learning of the graph structure information and time dimension information by combining the graph attention mechanism with the convolution of the time dimension, and processing the dynamic graph structure data by using the self-attention layer as time passes. It is of great significance to learn and fuse a network structure, multi-modal information, and temporal information based on effective methods, which can obtain more accurate representations of evolutionary networks.

Community detection. In recent years, community structure detection has received extensive attention, and communities play an important role in complex systems [21]. However, most of the traditional community detection algorithms for static networks are not suitable for research on evolutionary network structures. Chen et al. [22] proposed a community detection algorithm based on the minimum change granularity for evolutionary networks (DWGD), which can complete dynamic community detection with time slices as the minimum granularity. Wang et al. [23] proposed that Markov chain-based community detection algorithm for evolutionary networks had better computational efficiency and detection performance. In addition, many researchers also studied community detection in evolutionary networks by the Coherent Neighborhood Propinquity of dynamic networks [24], building compressed graphs [25], etc.

3. Preliminaries

3.1. Definitions

Given a network $G=\left(V,E\right)$, where $V=\left\{v_1,v_2,\dots ,v_n\right\}$ is the set of nodes, $E=\left\{e_1,e_2,\dots ,e_n\right\}$ is the set of edges. The adjacency matrix $A={\left. [a_{ij}\right]}_{n\times n}$ represents the connection relationship between nodes, the value of the corresponding element in the matrix indicates whether the edge exists, if there is an edge between node v_i and node v_j, then a_ij = 1; otherwise a_ij = 0. In this paper, we take the adjacency matrix A as the closest relation matrix in the network to describe the similarity between nodes in the graph and their proximity relation.

(1) Node proximity

  Given a network $G=\left(V,E\right)$, $u, v\in V$, the proximity $Nei\left(u,v\right)$ between node $u$ and node $v$ is defined as

$$Nei\left(u,v\right)=e^{\alpha \left(2-l\right)},$$

(1)

where $l$ is the shortest path length from node $u$ to node $v$, $l \ge 2$, $\alpha$ is the degree of attenuation, and the value range is $\left(0, 1\right)$. If the path length l increases, the proximity decreases, and $\alpha$ controls the degree of proximity attenuation. The larger the value of $\alpha$, the faster the proximity relationship between the nodes decays.

(2) Proximity matrix

  Given a network $G=\left(V,E\right)$, $M={\left. [m_{ij}\right]}_{n\times n}$ is a matrix corresponding to the network $G$, where we use the proximity formula to calculate the proximity between the two corresponding nodes v_i and v_j in M, $m_{ij}=Nei\left(v_i,v_j\right)$, with $v_i,v_j\in V;$ then, we call $M$ the proximity matrix of $G$.

3.2. Data Preprocessing

The data preprocessing mainly converts the adjacency matrix into a proximity matrix. The breadth-first traversal algorithm is employed to obtain the path length, then we calculate the neighbor relationship between nodes that are not directly connected by the attenuation, and the topology of the community can be well exhibited. However, when the path length is greater than a certain threshold, node pairs that are not in the same community may also have a certain proximity value, which will lead to an ambiguous partition of the community. Therefore, we set the threshold Le of the path length. By adjusting the parameters empirically, we only calculate the proximity value between nodes that are mutually reachable within the length of Le.

At the same time slice, we construct a loss function for directly connected nodes in the network, this is formulated as

$$L_1={\displaystyle \sum }_{i,j=1}^n{A}_{i,j}|\left|\left(y_i-y_j\right)\right|{|}_2^2=2tr\left(Y^{T}LY\right)$$

(2)

where $i, j$ represent two adjacent nodes, $n$ is the number of nodes, $A_{ij}$ is the adjacency matrix, $y_i$ and $y_j$ are the representations of two adjacent nodes corresponding to the presentation layer, and $L$ is the Laplacian matrix of the network, obtained from the corresponding diagonal matrix and adjacency matrix. The Laplacian matrix is calculated as

$$L=D-A,$$

(3)

where $D$ is a diagonal matrix, $A$ is the adjacency matrix of the network. To ensure that the evolutionary network can smoothly evolve with time, the relational information of the directly connected nodes of the network structure of the last time slice are obtained by using the Laplacian matrix. By employing the Laplacian matrix, the node information loss function of directly connected nodes is constructed as

$${L^{\prime }}_1=2tr\left(Y_t^T{L}_{t-1}{Y}_t\right),$$

(4)

where $Y_t$ is the representation of the network structure of time slice $t$, $L_{t-1}$ is the Laplacian matrix of the network structure of time slice $t-1$.

Under different time slices, the scale of the evolutionary network varies. A zero-filling operation is usually performed for training the data calculations in neural networks. However, there may be such cases where non-zero elements are far less than the zero elements in the data matrix. In this way, the increase of zero elements will affect the sparsity of the matrix, and it is easy to increase the error of zero elements due to the reconstruction of the deep autoencoder. Therefore, a sparsity constraint based on the deep automatic encoder is beneficial to control the reconstruction difference of the deep autoencoder. In the proposed algorithm, the sparsity parameter is added to the depth autoencoder during the encoding process, and we obtain a deep sparse autoencoder, which is helpful to express the information of the original data without loss.

The first layer to the second layer in the deep sparse autoencoder is an encoding process as well as a dimensionality reduction process. According to the size of the data, we set the number of data nodes in each layer, which is also the data dimension. By setting the data dimension size in each layer, the deep sparse autoencoder will obtain the low-dimensional vector corresponding to the input data after encoding. From the second layer to the third layer, this is a decoding process. The deep sparse autoencoder decodes the low-dimensional vector obtained from the input data and then outputs the vector of the same dimension as the input data.

This process employs the backpropagation algorithm to train the data by adjusting the parameters related to the encoder and decoder—for example, weights, deviations, etc., to minimize the reconstruction difference. Consequently, the output expression vector approximates the input data information. Finally, the resulting representation of the coding layer is exactly the low-dimensional Eigenmatrix that needs to be output.

In the proposed algorithm, a deep sparse autoencoder is used to represent and learn the network structure under the current time slice of the evolutionary network. We adopt a similar idea to calculate the reconstruction error of building deep sparse autoencoders [11], and a backpropagation algorithm is employed to obtain the node information expression of the network at the next time slice. The reconstruction error of the deep sparse autoencoder is established, which is formulated as

$$L_2={{\displaystyle \sum }}_{i=1}^n\left|\right|\left({x^{\prime }}_i-x_i\right)\ast b_i|{|}_2^2,$$

(5)

where ${x^{\prime }}_i$ and $x_i$ are the input data and reconstructed data, respectively. $b_i=x_i\ast \left(\beta -1\right)+1$, $b_i$ is a sparsely constrained parameter, and $\beta$ is a tunable parameter.

To satisfy the temporal smooth transition of the evolutionary network structure, the node information of the directly connected edges in the network of the last time slice is employed to construct the loss function, and the reconstruction error of the deep sparse autoencoder is incorporated to construct the overall loss function.

In the proposed algorithm, to preserve part of the historical node information of the network structure of the last time slice and, simultaneously, effectively present the feature of the network structure of the next time slice, the overall loss functions are jointly optimized to effectively train the data. The loss function on constructing the learning model of the overall graph presentation is defined as

$$L_{min}=\theta {L^{\prime }}_1+\gamma L_2,$$

(6)

where $\theta$ and $\gamma$ are two tunable parameters.

4. Methods

The proposed algorithm employs a Laplacian matrix to map the nodes with directly connected edges in the network structure of the last time slice to the deep sparse autoencoder representation layer. It is beneficial to retain the node information of the network of the last time slice, and it is not easy to lose the historical evolution information of the network. Meanwhile, the temporal smoothness affects the structure of the evolutionary network and, thus, is taken into account in the graph representation learning algorithm.

Furthermore, the deep sparse autoencoder is used to represent the network structure under the current time slice of the evolution network, to obtain a better expression of the characteristics of the network structure, and then the representation vector of the node is achieved. Finally, the K-means clustering algorithm is used to cluster the low-dimensional feature matrix of the network structure under the current time slice to obtain the community structure.

For the network structure of continuous-time slices, we successively utilize the information between the node pairs of the network structure under the last time slice. The network structure of the current time slice is learned and presented through the deep sparse autoencoder, and then the low-dimensional representation matrix of the network structure of the current time slice is obtained.

The following shows the process of the proposed algorithm:

(1)

Initialize the relevant parameters and load the dataset.

(2)

Based on the breadth-first traversal algorithm to traverse the network structure adjacency matrix of the current time slice.

(3)

Calculate the degree by using the path length to obtain the proximity matrix.

(4)

Use a Laplacian Matrix to obtain the information of directly connected nodes in the network of a time slice, and the features of the proximity matrix of the network under the current time slice are extracted by employing a deep sparse autoencoder.

(5)

Use the K-means algorithm to obtain the network community structure under the current time slice.

The proposed algorithm (Algorithm 1) is based on the TensorFlow framework, and its pseudocode is as follows.

Algorithm 1. LCDEN algorithm

Input: Evolution graph $G$, adjacency matrix $A_{t-1}$, adjacency matrix $A_t$, number of    communities $k$, path length threshold $L_e$, degree of attenuation $\alpha$, deep sparse    autoencoder with 3 layers, number of nodes in each layer of the deep autoencoder $d^{\left(i\right)}$,    i = 1, 2, 3. Output: Community results $Co{m}_t=\left\{c_1, c_2,\dots ,c_k,\right\}$.

for$i$in$G.nodes$do   Initialize each node in graph $G$ to an unvisited state   Initialize the queue, $que$   Initialize traversal path length set, $length\_dict$   Mark $i$ as the accessed state, initialize the $path\_length\left(i\right)=0$   Record $path\_length$ to $length\_dict$     Add $i$ to the $que$   while $que !=None$ do      Remove node $j$ from $que$       for $u$ from $0$ to $Length\left(v\right)$ do        if $v\le L$ and $u$ is unvisited state then            Mark $u$ as the accessed state           $path\_length\left(i\right)={{\displaystyle \sum }}_{l=i}^{v+1}path\_length\left(l\right)$            Record $path\_length\left(u\right)$ and ${{\displaystyle \sum }}_{l=i}^{v}path\_length\left(l\right)$ to $length\_dict$         Add $u$ to $que$        end if      end for    Set $v$ to be accessed end state    end while    for $u$ in $G.nodes$ do       Calculate $Nei\left(i,u\right)=e^{\alpha \left(2-l\right)}$    end for  end for  Obtain neighbor degree matrix $M_t$  Build deep sparse autoencoder  Input neighbor degree matrix $M_t$ to deep autoencoder Calculate loss function $L_2$ Obtain hidden layer representation y Calculate Laplacian matrix $L_{t-1}=D_{t-1}-A_{t-1}$ Calculate local node information loss function ${L^{\prime }}_1$ Calculate $L_{min}=\theta {L^{\prime }}_1+\gamma L_2$  repeat   Joint optimization $L_{min}$  until converge Obtain a low dimensional feature matrix $Y_t$  Run K-means clustering  Return community results $Co{m}_t=\left\{c_1, c_2,\dots ,c_k,\right\}$

In this pseudocode, the Breadth-first traversal algorithm is used to convert the adjacency matrix $A_t$ into the proximity matrix $M_t$, and then to input it into the deep sparse autoencoder for encoding. Simultaneously, we use the adjacency matrix $A_{t-1}$ to calculate the Laplacian matrix, to map the node information with directly connected edges to the presentation layer of the deep sparse autoencoder, and to obtain the result of the presentation layer—that is, the required low-dimensional matrix representation vector $Y_t$. Finally, the K-means clustering algorithm is used to cluster the low-dimensional matrix representation vector $Y_t$, and the community results are obtained.

5. Experiments

In this section, we evaluate the computational efficiency and detection performance of our proposed algorithm on three different datasets. We use the Silhouette Coefficient [26] and time-consumption as evaluation metrics to evaluate the performance of different models.

5.1. Datasets

The following three network datasets were employed for the experiments.

(1) Superuser temporal network [27]

  The stack exchange website super user network dataset is an interactive temporal network on the stack exchange website. The relevant metadata stored are users and the interactions between users. Here, nodes represent users, and edges indicate interactions between users.

(2) Wiki-Talk-Temporal dataset [27]

  The Wikipedia network is a temporal network, and Wikipedia users edit each other’s “conversation” pages. The directed edge $\left(u,v,t\right)$ indicates that a user $u$ edited the conversation page of user $v$ at time $t$.

(3) Twitter network dataset [28]

  The data were purchased by The Numerical Analysis and Scientific Computing research group in the Department of Mathematics and Statistics at the University of Strathclyde, using a grant made available by the University of Strathclyde through their EPSRC-funded “Developing Leaders” program. In the dataset, node $i$ and node $j$ represent the node numbers.

5.2. Evaluation Metrics

We employed the Silhouette Coefficient and time-consumption to evaluate the experimental results. The Silhouette Coefficient effectively combines the cohesion and separation of the clustering for evaluation. The advantage of employing the Silhouette Coefficient as a cluster result evaluation is that no real data information is needed for comparison. The range of the Silhouette Coefficient is [–1, 1], and the larger the value is, the better the clustering effect.

The calculation formula of the Silhouette Coefficient of each sample point is as follows:

$$s\left(i\right)=\frac{b\left(i\right)-a\left(i\right)}{\max \left\{a\left(i\right),\left.b\left(i\right)\right\}\right.}$$

(7)

where $i$ represents a sample point, $a\left(i\right)$ represents the average distance between each sample $i$ and all other sample points in a cluster. It is used to quantify the cohesion of the cluster. We select $a$ cluster $b$, which does not contain sample point $i$, and then calculate the average distance between sample point $i$ and all sample points in cluster $b$. Similarly, we traverse all other clusters until the nearest average distance is found. This is represented as $b\left(i\right)$, which is used to quantify the degree of separation between clusters.

We calculate the Silhouette Coefficient of the overall sample—that is, we find the average value as the overall Silhouette Coefficient of the data cluster, to denote the closeness of the data cluster.

5.3. Baselines

To verify the effectiveness of the proposed algorithm, we chose the DWGD algorithm [22] for our comparative experiments. The DWGD algorithm can partition communities for dynamic weighted networks. The weight of the edges may increase, decrease, or remain unchanged with the change of time. The DWGD algorithm can well deal with the increase and decrease of the edge weight. Experiments on certain datasets verified the effectiveness of the algorithm. The DWGD algorithm can perform community detection for evolutionary networks and weighted networks. Therefore, the algorithm proposed in this paper was compared with the DWGD algorithm on the problem of community detection in dynamic weighted networks.

5.4. Experimental Analysis

The proposed algorithm (abbreviated as LCDEN) and the DWGD algorithm were used to divide the network structure of the data set containing a time stamp under different time slices, and the obtained Silhouette Coefficient and running time were compared. The experimental results and analysis are as follows.

(1)

The comparative experimental results on the Superuser temporal network are shown in Figure 1. As shown in Figure 1a, we can conclude that, in the same time slice, the Silhouette Coefficient of the proposed algorithm is higher than that of the DWGD algorithm, that is, the clustering effect is better. Figure 1b demonstrates that, in the same time slice, the running time of the proposed algorithm is less than the DWGD algorithm.

(2)

The comparative experimental results based on the Wiki-Talk-Temporal network (Wikipedia network dataset) are shown in Figure 2.

Figure 2a also shows that the Silhouette Coefficient of the proposed algorithm is higher than the DWGD algorithm at the same time slice, indicating a better clustering effect of the proposed algorithm. From Figure 2b, we can see that in the time slices of S0 and S4, the running time of the proposed algorithm and the DWGD algorithm are close to each other, and, in other time slices, the running time of the proposed algorithm is less than that of the DWGD algorithm.

(3)

The comparative experimental results based on the Twitter dataset are shown in Figure 3.

Figure 3a illustrates that the Silhouette Coefficient of the proposed algorithm is higher than that of the DWGD algorithm, indicating a better clustering effect of the proposed algorithm. Figure 3b discloses that the running time required for the proposed algorithm is less than that of the DWGD algorithm.

6. Results and Discussion

The community structures are groups of nodes that are more strongly or frequently connected among themselves than with others. Community detection is proposed to find the most reasonable partitions of a network via the observed topological structures [29]. However, traditional community detection methods are usually oriented to static networks and cannot deal with large-scale evolutionary networks, while the most common ones in the real world are complex systems that change with time.

The proposed algorithm makes full use of the historical structure of the network and uses the deep learning model to process the temporal information of the evolutionary network, thus, obtaining the most reasonable communities in different time slices of the given evolutionary network. By comparing with the baselines, the proposed algorithm demonstrated advantages in computational efficiency and detection performance.

This is because the use of the deep learning model uses the historical structure of the network as the prior knowledge of the community detection at the current time, and the deep learning model obtains more input data when compared with the baselines. The research in community detection will further promote the development of the optimization of transportation network structures [30,31], the analysis of social networks [32,33], and research of biological systems [3,34].

7. Conclusions

In this paper, we propose an evolutionary network community detection algorithm that embeds nodes into vectors and uses the K-means algorithm for community member clustering. To process the temporal data, the proposed algorithm employs a Laplacian matrix to represent the historical network structural information and uses a deep sparse autoencoder to encode the prior and the current information. The experimental results on representative datasets showed that the proposed algorithm outperformed the baselines in computational efficiency and detection performance.