Smart Contract Vulnerability Detection Based on Multi-Scale Encoders

Abstract

Vulnerabilities in smart contracts may trigger serious security events, and the detection of smart contract vulnerabilities has become a significant problem. In this paper, to solve the limitations of current deep learning-based vulnerability detection methods in extracting various code critical features, using the multi-scale cascade encoder architecture as the backbone, we propose a novel Multi-Scale Encoder Vulnerability Detection (MEVD) approach to hit well-known high-risk vulnerabilities in smart contracts. Firstly, we use the gating mechanism to design a unique Surface Feature Encoder (SFE) to enrich the semantic information of code features. Then, by combining a Base Transformer Encoder (BTE) and a Detail CNN Encoder (DCE), we introduce a dual-branch encoder to capture the global structure and local detail features of the smart contract code, respectively. Finally, to focus the model’s attention on vulnerability-related characteristics, we employ the Deep Residual Shrinkage Network (DRSN). Experimental results on three types of high-risk vulnerability datasets demonstrate performance compared to state-of-the-art methods, and our method achieves an average detection accuracy of 90%.

1. Introduction

Blockchain is a distributed ledger that records the financial transactions that take place within decentralized networks [1]. With the emergence of Ethereum smart contracts, blockchain has entered the era of programmable finance [2]. In recent years, blockchain technology has surpassed its original scope in finance and achieved success in a variety of industries and sectors. For instance, within the energy sector, blockchain is transforming conventional approaches to energy management and distribution [3]. In the field of agriculture, blockchain technology offers novel solutions for ensuring data integrity and traceability in supply chains, thereby enhancing food safety and traceability of agricultural products [4]. In healthcare, it is contributing to better patient care and efficient medical record management by securely storing and sharing patient data [5]. Furthermore, in the field of e-government, blockchain serves as a means to improve the transparency and efficiency of public services, which leads to a more effective and secure management of these services [6]. The success of blockchain technology is closely linked to smart contracts, which were initially proposed for the dissemination, validation, and enforcement of contracts in an informational manner [7]. Compared to traditional contracts, smart contracts enabled users to codify their agreements and trust relations by providing automated transactions without the supervision of a central authority [8].

With the widespread application of smart contracts in finance, industry, commerce, and other fields, the security of smart contracts is becoming increasingly important [9]. If there are vulnerabilities in the smart contract, attackers can exploit these smart contract vulnerabilities to maliciously penetrate blockchain networks and steal tens of millions of dollars in cryptocurrency. This not only has implications for the reliability of the blockchain system, but also serious for the interests of smart contract holders. There is a typical event that an attacker exploited a reentrancy vulnerability to successfully abscond with millions of Ether called the DAO attack [10]. This event seriously undermined the credibility of the blockchain system. Events based on smart contract vulnerability attacks are still occurring continuously [11]. Undoubtedly, these attacks highlight the urgent need for smart contract vulnerability detection. By detecting potential vulnerabilities in a smart contract, the security and stability of the blockchain system can be effectively improved [12]. Therefore, how to design efficient vulnerability detection models is an important problem and challenge that needs to be solved.

Traditional methods to detect vulnerabilities in smart contracts suffer from the problem of path explosion, which leads to a large increase in the execution time and memory overhead. Moreover, these methods cannot effectively parse complex data structures of program, which may result in higher false positive and false negative rates. In recent years, researchers have used deep learning techniques to improve the accuracy of detecting vulnerabilities in smart contracts [13]. By segmenting the source code slice of smart contracts into code snippets and using the BLSTM model, Qian et al. designed ReChecker to detect reentrancy vulnerabilities [14]. However, the defined slice criteria are not comprehensive, and the generated code snippets may implicitly ignore critical segments. In addition, for longer code snippets, BLSTM may not efficiently capture global semantic information [15], and bring about high false positives. Based on Convolutional Neural Network (CNN) and Bidirectional Gated Recurrent Unit (BiGRU) network models, Zhang et al. devised a CBGRU hybrid deep learning model to detect vulnerabilities [16]. However, due to CNN’s inability to fully extract local syntax and semantic detail features and BiGRU’s limited perception of global structural features, the detection results can lead to false negatives. To detect reentrancy vulnerabilities, Wu et al. proposed a peculiar approach for extracting critical data flow graphs from the smart contract code [17]. However, this method cannot detect other types of vulnerabilities and has a lack of generality.

To overcome the above shortcomings, we proposed a novel Multi-Scale Encoder Vulnerability Detection (MEVD) approach. Firstly, due to excessive semantic information that is irrelevant to vulnerability features in the set of the original ones [18], it may affect the correlation of the global semantic information and the coherence of the contextual structure. To address this problem, we have designed a Surface Feature Encoder (SFE) based on the gating mechanism. The unique mechanism can control the flow of critical semantic information, suppress noise data, and enhance the global semantic information of the features. Second, by combining a Base Transformer Encoder (BTE) based on Transformer blocks and Detail CNN Encoder (DCE), we devise a dual-branch structured encoder to comprehensively explore vulnerability patterns from global structure and local detail features, where the BTE is used to extract long-range dependencies and global structural features in the code and the DCE focuses more on the analysis and exploration of the syntax and semantics within the code to extract local detail features. Third, to improve the quality of the code vector representation, we introduce Deep Residual Shrinkage Networks (DRSNs) to resist the noise generated by vulnerability-unrelated variables. Using deep learning to automatically determine the threshold, the model can focus more on the vulnerability-related characteristics. Due to the large amount of irrelevant information may exist in the set of contract codes [19], in the data preprocessing phase, we intend to use the Vulnerability Syntax-driven Slice (VSS) to eliminate redundant information. VSS provides a more comprehensive set of slice criteria for identifying different vulnerability characteristics, which can maintain data dependency and control relevance of code statements. We conducted extensive experiments on three different types of vulnerability datasets: reentrancy, timestamp dependency, and infinite loop vulnerabilities. The experimental results show that our approach outperforms state-of-the-art methods in terms of detection performance. The major contributions of this paper are as follows:

• We present a vulnerability syntax-driven slice method, which can simplify the contract code by removing statements unrelated to vulnerability characteristics and preserving the data and controls dependencies in statements.
• We propose a novel MEVD approach to detect vulnerabilities in smart contracts. The global structure and local detail features are captured by multi-scale encoders, respectively to compensate for the lack of feature extraction capability of a single model.
• We have compared MEVD with state-of-the-art vulnerability detection methods. Experimental results show that the proposed MEVD outperforms existing methods for detecting reentrancy, timestamp dependency, and infinite loop vulnerabilities.

This paper is organized as follows. First, in Section 2, we discuss the background knowledge of smart contract vulnerabilities. Next, in Section 3, we introduce the related work. In Section 4, we provide a detailed explanation of our proposed method. Section 5 presents our experimental results. Finally, we conclude the paper by discussing future directions in Section 6.

2. Background

In this section, we introduce background about the smart contract and common vulnerabilities in the smart contract.

2.1. Smart Contract Basics

Smart Contracts: Smart contracts are typically written in the Solidity programming language. When smart contracts are deployed on the blockchain, transactions can be executed automatically and irreversibly. Smart contracts ensure the security and reliability of transactions without the need for third party verification and execution. This effectively reduces transaction costs and risks.

Fallback Function: In a smart contract, when an unmatched function is called by an external user, the fallback function is automatically executed to handle unknown transactions or calls [20]. It can be used to automatically receive transfers of Ether or other digital assets when performing fund transfer operations, enabling smart contracts to accept funds and execute related business logic.

2.2. Smart Contract Vulnerabilities

Vulnerabilities in smart contracts exist at three levels: Solidity language, EVM virtual machine, and blockchain [20]. For example, at the Solidity language level, types of vulnerabilities include reentrancy, integer overflow, and underflow. Types of vulnerabilities include short address attack and storage overlap attack at the EVM virtual machine level. At the blockchain level, types of vulnerabilities include timestamp dependency, transaction order dependency, etc. Our work is mainly focused on reentrancy, timestamp dependency, and infinite loop vulnerabilities. By checking 40,932 Ethereum smart contracts, Liu et al. discover that out of 307,396 functions, approximately 5013 functions may lead to reentrancy vulnerabilities, and about 4833 functions may result in timestamp dependency vulnerabilities [21]. In addition, around 56,800 functions contain for or while loop statements, which can lead to infinite loop vulnerabilities. These scenes have led us to investigate on these three types of vulnerabilities.

Reentrancy: The reentrancy vulnerability is a commonly exploited vulnerability in blockchain networks. With the well-known DAO attack incident being an attack that exploited this vulnerability, attackers can make a program execute malicious code designed by repeating a transaction until the victim’s account balance is 0 or gas is exhausted [13], thus causing huge financial losses. This allows attackers to illegally acquire a significant amount of funds within the contract.

Timestamp Dependency: As the system determines the timestamp during the mining process, it allows for a deviation of 900 s so that the miners can control the timestamp to a certain extent [22]. However, malicious miners can set arbitrary values for timestamps within a short time frame (<900 s). Additionally, they can gain foreknowledge of the timestamp of the next block, and thus manipulate the smart contract to gain illegal profits.

Infinite Loop: An infinite loop vulnerability is where the code within a contract function contains an iterative or looping structure (such as a for loop, while loop, etc.) but lacks a termination condition or the termination condition cannot be met, resulting in the loop not being able to terminate normally [21]. This can trap a smart contract in an infinite loop state, consuming the contract’s computational resources and potentially causing the contract to fail to execute.

The damage caused by these three vulnerabilities is very serious, and the logic of the contract code becomes more complex due to the numerous call relationships between contracts. Additionally, they are not easily discovered by developers. Therefore, detection of these three vulnerabilities is essential.

3. Related Work

In this section, we will discuss the current smart contract vulnerability detection methods, including traditional detection methods and deep learning-based methods. Then, we will introduce some deep learning models.

Traditional Detection Methods: Traditional methods for detecting vulnerabilities in smart contracts primarily include methods such as symbolic execution, formal verification [23], and fuzzing [13]. Oyente leverages the static symbolic execution technique to detect potential vulnerabilities in smart contracts [24]. Mythril combines static symbolic execution, taint analysis, and control flow checking to further improve the accuracy of vulnerability detection [25]. Slither can detect vulnerabilities by transforming contract code into an intermediate representation [26]. ContractFuzzer is the first vulnerability detection framework to apply fuzz testing techniques to smart contracts [22].

Deep Learning Detection Methods: In recent years, the deep learning technology has achieved remarkable success in various fields. Currently, there are a number of deep learning methods for vulnerability detection. ReChecker [14] employs code slicing of smart contract source code and uses a BLSTM-ATT sequence model to detect reentrancy vulnerabilities. To fully capture key statements associated with the vulnerability, Yu et al. had made improvements to the slicing technique by introducing the concept of Vulnerability Candidate Slices (VCSs), which feed into various temporal neural network models for vulnerability detection [27]. Cai et al. constructed a novel contract graph, incorporating Abstract Syntax Trees (ASTs), Control Flow Graphs (CFGs), and Program Dependency Graphs (PDGs) and employed graph neural network models for vulnerability detection [15]. Liu et al. introduced a new method for representing code graphs, AME, which classifies nodes in the graph into core and normal nodes, then extracts deep graph features and fuses global graph features with local expert patterns to improve accuracy [28]. VulnSense combines three types of features from smart contracts and uses a multi-modal learning approach for a comprehensive vulnerability detection method [29].

Our proposed MEVD approach is based on multi-scale encoders. Compared with ReChecker’s single feature extraction, MEVD can extract the semantic and syntactic features of the code more comprehensively, effectively combining global structures with local details. In addition, compared with the VCS method proposed by Yu et al., we have introduced the VSS, which contains more comprehensive vulnerability slicing criteria. The VSS can take into consideration the characteristics of different types of vulnerabilities in depth, and the extracted code snippets contain richer semantic information, which effectively reduces false positives caused by insufficient crucial information in the vulnerability code snippets. Finally, MEVD has demonstrated excellent detection performance compared to current state-of-the-art methods through extensive experimental validation.

Deep Learning Models: In recent years, many outstanding network models have emerged, such as Long Short-Term Memory (LSTM [30]), Graph Convolutional Networks (GCNs [31]), and CNNs. These models have achieved significant success in their respective domains. In addition to these basic models, more complex model structures have emerged in recent years, such as the Transformer model [32]. The core components of the Transformer model are encoders and decoders that use self-attention mechanisms to establish global correlations within input sequences, thereby better capturing contextual structural information. The Transformer has made significant breakthroughs in natural language processing, particularly machine translation. By refining the Transformer’s encoder and decoder modules, Restormer can better capture interactions between distant pixels [33]. RepVGG introduces a multi-branch structure to extract richer features [34]. DRSN is a network structure to improve the feature learning capability for highly noisy vibration signal analysis, enabling more accurate fault diagnosis [35]. CDDFuse designs a novel autoencoder to address the problem of multi-modal image fusion [36].

In the field of smart contract vulnerability detection, we have summarized traditional methods (such as symbolic execution, formal verification, and fuzz testing) and deep learning methods (such as ReChecker and AME). Traditional methods emphasize static symbolic execution techniques, while deep learning methods include various models such as BLSTM-ATT, graph neural networks, and self-attention mechanisms. Finally, we introduce outstanding deep learning models, such as LSTM, Transformer, and Restormer, which have made significant progress in various domains. These methods play an important role in enhancing the security of smart contracts.

4. Method

In this section, we introduce a novel method for slicing smart contract code and present the design of the MEVD framework.

4.1. Overview

As shown in Figure 1, the MEVD proposed in this paper consists of three phases: data preprocessing phase, model training phase, and vulnerability detection phase. Firstly, in the data preprocessing phase, we remove redundant information from the smart contract code, such as blank lines, comments, and non-ASCII characters. Then, to simplify the contract, the source code is sliced into VSS and uniform naming rules for normalization. Next, we use the Word2Vec word embedding technique [37] to convert code slices into vector forms suitable for input to the model in the model training phase. The model consists of multi-scale encoders that are used to enrich the semantic information of the feature vectors and capture global structural and local detail features that are used to improve the performance of vulnerability detection. Finally, in the vulnerability detection phase, to obtain a vector containing global structure and local detail features, the two feature vectors processed by the multiple encoders are fed into a fusion layer. These feature vectors are then fed into a fully connected layer and a SoftMax activation function is used to output vulnerability detection probabilities.

4.2. Data Preprocessing

4.2.1. Code Slice Criteria

It has been discovered that there are a number of code statements in the contract source code that are unrelated to vulnerabilities [38]. These irrelevant code statements can have an impact on vulnerability detection performance. Therefore, it is important to extract key code statements accurately. However, existing methods for the definition of slice criteria have certain limitations when dealing with smart contract code, which can result in the loss of implicit information about vulnerabilities. Therefore, as shown in

Table 1. The different vulnerability slice criteria.

Vulnerability	Slice Criteria
Reentrancy	fallback () call.value () a function involving call.value variable: correspond to user balance
Timestamp Dependency	block.timestamp block.number now
Infinite Loop	for while loop self-call function

, we propose an entirely new set of slice criteria that covers three types of vulnerabilities.

Taking the reentrancy vulnerability as an example, we have defined several slice criteria. Firstly, code statements that include call.value () and fallback () are considered to be critical code statements that are susceptible to reentrancy vulnerabilities, because they allow smart contracts to undergo state transitions as they interact with other contracts. This state transition could result in the contract continuing to execute after an external call, which may result in potential vulnerabilities. In addition, fund transfer functions that include call.value calls require special attention. These functions are widely used in the transfer and invocation processes of smart contracts and serve as conditions for launching reentrancy attacks. Then, we have also focused on variables related to user balances, as these variables are equally important in fund transfers. Similarly, we have also defined slice criteria for the other two types of vulnerabilities. This novel set of slice criteria enables more comprehensive vulnerability-related information to be captured within the contract code, avoiding the omission of critical implicit details.

4.2.2. Code Slice Generation

To address the impact of redundant information in smart contract source code on the accuracy of vulnerability detection, we have proposed a novel vulnerability syntax-driven slice method, as shown in Figure 2.

Taking a smart contract containing call.value () as an example, in step 2, we first remove comments, non-ASCII characters, and blank lines from the smart contract. Removing this irrelevant information does not affect the results. Next, in step 3, based on the vulnerability type associated with the code, we identify the key statements of the code according to the slice criteria in Table 1. For example, the ninth line of code contains the keyword call.value (), and the sixth line operates on the user’s balance. Therefore, we consider these to be key sensitive statements. The code statements relevant to the key statements are then extracted and organized into code snippets by analyzing the control and data dependencies between statements and variables. Then, in order to normalize the code snippets and avoid the potential impact of different naming rules, in step 4, we map user-defined variables to symbolic names (e.g., “VAR1”, “VAR2”) and user-defined functions to symbolic names (e.g., “FUN1”, “FUN2”). This is the final generated VSS. Finally, to convert code snippets into a format that can be accepted by neural networks, we use Word2Vec technology to convert code snippets into vector representations.

4.3. MEVD Model

In this section, we will introduce the model structure of MEVD. The MEVD model consists of three modules: the surface feature encoder, the dual-branch Transformer-CNN encoder, and the deep residual shrinkage network.

4.3.1. Surface Feature Encoder

We used the Word2Vec algorithm, which can convert tokens to vectors, but by converting code directly to vectors as input, semantic features within the code may be ignored. To solve this problem, we designed SFE, which can enhance the relevance of global semantics within the features and suppress the influence of irrelevant information, thus preserving richer syntactic and semantic information. The detailed design of the SFE module is shown in Figure 3.

SFE leverages three design strategies to enhance semantic features: a gating mechanism, dilated convolutions, and depth-wise convolutions. Gating mechanisms are typically used to control the flow of information by enhancing attention to critical information. Specifically, the gating mechanism is formulated as the element-wise product of two parallel paths of linear transformation layers. One path is activated by the non-linear activation function GELU, while the other path applies the reverse operation to the vector. The outputs of these two paths are then subjected to element-wise multiplication, allowing for information from one path to modulate the strength of the other, thereby enhancing focus on important semantic details and reducing the impact of irrelevant variables. On the other hand, the SFE employs dilated convolutions and depth-wise convolutions. Dilated convolutions extend the receptive field to cover a wider range of input information, helping to capture different scales and structural features within the code. Meanwhile, depth-wise convolutions focus on the local context and detailed information in the input data. They efficiently capture local interdependencies between different channels in the input data, helping the SFE to enrich the semantic information within the features. Given the input vector (feature matrix) $X\in {\mathbb{R}}^{L\times D}$, L represents the length of the token sequence and D represents the dimension of the token embedding. The SFE equation is as follows:

$$\mathrm{G}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{n}\mathrm{g}\left(X\right)=\varphi \left(W_{de}^1{W}_{di}^1\right(X\left)\right)\odot Reverse\left(W_{de}^2{W}_{di}^2\right(X\left)\right)$$

(1)

$$\widehat{X}=LN\left(\mathrm{Gating}\left(X\right)\right)+X$$

(2)

where $W_{di}^{(\cdot )}$ is the 3 × 3 dilated convolution, $W_{de}^{(\cdot )}$ is the 3 × 3 depth-wise convolution, ⊙ denotes element-wise multiplication, $\varphi$ represents the GELU non-linearity, and LN is the layer normalization. Reverse denotes the reversal of rows in a feature matrix, e.g., swapping the first row with the last. In summary, the design of the SFE allows each level to be concerned with details that are complementary to other levels, focusing on enriching features with code semantic context information.

4.3.2. Dual-Branch Encoder

The dual-branch encoder consists of two parallel encoders: the base Transformer encoder and the detail CNN encoder. These two modules are used to further extract vector features after the SFE. The BTE is used to effectively capture global structural information, allowing it to capture dependencies between code statements and contextual logical structural features. The DCE, on the other hand, focuses on exposing crucial latent features within the code, while preserving more syntactic and semantic details. This dual-branch design strategy allows the model to simultaneously consider global structure and fine-grained features, leading to a more comprehensive understanding of smart contract code and significantly improving the accuracy and performance of vulnerability detection.

• Base Transformer Encoder (BTE)

The BTE consists of multiple self-attention layers, with each self-attention layer containing multi-head self-attention and feedforward networks. It uses the multi-head self-attention feature of the Transformer encoder to capture contextual information and long-range dependencies within code sequences, thereby extracting richer global structural features. The multi-head self-attention mechanism serves as the core of BTE, allowing it to establish correlations between different positions in the input sequence and effectively capture contextual information. The structure of the BTE is shown in Figure 4.

BTE achieves parallel computation of multiple attentional heads, which allows the model to simultaneously focus on information from different positions within the input sequence. It then combines the outputs of the different heads using weighted fusion to obtain a more comprehensive feature representation. The calculation process of the BTE encoder is as follows:

$$X_{pos}=\mathrm{P}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{a}\mathrm{l}\text{\_}\mathrm{E}\mathrm{n}\mathrm{c}\mathrm{o}\mathrm{d}\mathrm{i}\mathrm{n}\mathrm{g} \left(X\right)$$

(3)

$$Q=\mathrm{Linear} \left(X_{pos}\right)=X_{pos}\ast W_Q$$

(4)

$$K=\mathrm{Linear} \left(X_{pos}\right)=X_{pos}\ast W_K$$

(5)

$$V=\mathrm{Linear} \left(X_{pos}\right)=X_{pos}\ast W_V$$

(6)

$$X_{\mathrm{attention}}=\mathrm{S}\mathrm{e}\mathrm{l}\mathrm{f}\text{\_}\mathrm{Attention}\left(Q,K,V\right)$$

(7)

Here, $X_{pos }$ is obtained using positional encoding operations, Q represents the query vector, K represents the key vector, and V represents the value vector. Each vector undergoes a linear transformation, with $W_Q$, $W_K$, and $W_V$ representing the weight matrices for these transformations. Finally, $X_{\mathrm{attention}}$ is obtained by Equation (7):

$$X_{\text{attention1}}=X_{pos}+X_{\mathrm{attention}}$$

(8)

$$X_{\text{attention2}}=\mathrm{LayerNorm}\left(X_{\text{attention1}}\right)$$

(9)

$$X_{\mathrm{output}}=Activate\left(Linear\right(Linear\left(X_{\text{attention2}}\right)\left)\right)$$

(10)

$$X_{\text{output1}}=X_{\text{attention2}}+X_{\mathrm{output}}$$

(11)

$$X_{\text{output2}}= \mathrm{LayerNorm} \left(X_{\text{output1}}\right)$$

(12)

Then, the $X_{\text{attention2}}$ is obtained by residual connection and layer normalization. Finally, $X_{\text{attention2}}$ is processed through a feedforward neural network and residual connection to obtain $X_{\text{output2}}$ feature vectors with global structure and semantic information. This design enables the model to better understand the overall structure and global semantic relationships of smart contract code and effectively handle complex smart contracts.

b.

Detail CNN Encoder (DCE)

BTE focuses primarily on capturing global semantic information and structural features, while DCE complements BTE by focusing on extracting more intricate and complex features. DCE has the ability to delve deep into the latent syntactic, semantic, and local features of the code, further enriching the feature representation of the code vectors. DCE consists of a multi-branch convolutional structure as shown in Figure 5, consisting of three branches: a 3 × 3 dilated convolution layer with Batch Normalization (BN), a 1 × 1 dilated convolution layer with BN, and an independent BN layer. Dilated convolution allows different dilation rates to be set, providing a larger receptive field without increasing the number of parameters. This helps the model to better understand the structure and relationships within the code, thus capturing more intricate details. In addition, the different kernel sizes in these convolutions allow the DCE to capture information at different scales. Finally, by fusing outputs from different branches, it integrates information from different scales and levels of detail. This helps to improve the model’s perception of fine-grained details and to combine features from different hierarchical levels for a better understanding of the detailed features of the code.

In this research, we utilize symbols to represent key concepts. ${\mathrm{W}}^{\left(3\right)}\in {\mathbb{R}}^{3\times 3}$ represents the convolution kernel for a 3 × 3 dilated convolution layer, while ${\mathrm{W}}^{\left(1\right)}\in {\mathbb{R}}^{1\times 1}$ represents the convolution kernel for a 1 × 1 dilated convolution layer. Additionally, we use ${\mu }^{(\cdot )}$, ${\sigma }^{(\cdot )}$, ${\gamma }^{(\cdot )}$, and ${\beta }^{(\cdot )}$ to respectively denote the accumulated mean, standard deviation, learned scaling factor, and bias for each branch in the BN layer. We have also defined symbolic representations for the input and output. ${\mathrm{X}}^{\left(1\right)}$ represents the input, and ${\mathrm{X}}^{\left(2\right)}$ represents the output. The DCE equation is as follows:

$$\begin{array}{cc}{\mathrm{X}}^{\left(2\right)}\hfill & ={\mathrm{bn}(\mathrm{X}}^{\left(1\right)}\ast {\mathrm{W}}^{\left(3\right)},{\mathsf{\mu }}^{\left(3\right)},{\mathsf{\sigma }}^{\left(3\right)},{\mathsf{\gamma }}^{\left(3\right)},{\mathsf{\beta }}^{\left(3\right)})\hfill \\ & +\mathrm{bn}({\mathrm{X}}^{\left(1\right)}\ast {\mathrm{W}}^{\left(1\right)},{\mathsf{\mu }}^{\left(1\right)},{\mathsf{\sigma }}^{\left(1\right)},{\mathsf{\gamma }}^{\left(1\right)},{\mathsf{\beta }}^{\left(1\right)})\hfill \\ & +\mathrm{bn}({\mathrm{X}}^{\left(1\right)},{\mathsf{\mu }}^{\left(0\right)},{\mathsf{\sigma }}^{\left(0\right)},{\mathsf{\gamma }}^{\left(0\right)},{\mathsf{\beta }}^{\left(0\right)})\hfill \end{array}$$

(13)

In summary, the use of DCE enhances the model’s ability to perceive and capture these intricate and complex features, thereby helping the model to better distinguish between vulnerable code and normal code.

4.3.3. Deep Residual Shrinkage Network

The DRSN module integrates soft thresholding and deep learning, introducing a method for adaptive thresholding [35]. Through DRSN, the model can effectively reduce irrelevant information during feature learning. The calculation of the soft threshold is as follows:

$$y=\left\{\begin{array}{cc}x-\tau & x>\tau \hfill \\ 0& -\tau \le x\le \tau \hfill \\ x+\tau & x<-\tau \hfill \end{array}\right.$$

(14)

The soft thresholding calculation is shown in Figure 6b, where $x$ represents the input features, $y$ represents the output features, and $\tau$ is the threshold, which is a positive parameter. As can be seen from Equation (14), the essence of soft thresholding is to discard features with smaller absolute values and shrink features with larger absolute values, thereby reducing information unrelated to the current task.

The structure of DRSN is shown in Figure 6c. First, the input vector passes two convolution layers, each accompanied by normalization layers and Relu activation functions. The output of the convolution layer is a matrix and is denoted as $X\in R^{T\times K}$, where $T$ is the number of token sets, and K is the number of convolution kernels. The resulting matrix undergoes an absolute value operation, followed by global average pooling to generate a one-dimensional vector, denoted as $X_{avg}\in R^K$:

$$X_{avg}=\frac{1}{T}{\sum }_{i=1}^T\mid X_i\mid$$

(15)

where $\mid X_i\mid$ denotes the absolute value of the $i^{th}$ row feature in $X$. Subsequently, $X_{avg}$ is fed through a fully connected layer, followed by batch normalization, a Relu activation function, and a second fully connected layer. Furthermore, a sigmoid activation function is used to scale the threshold parameter for each channel in the range (0, 1), resulting in the scaling parameter $sc\in R^K$, and the threshold $\tau \in R^K$ can be calculated using Equation (17):

$$sc=\frac{1}{1+e^{-Z}}$$

(16)

$$\tau =sc\ast X_{avg}$$

(17)

After DRSN processing, this results in eliminating residual redundant information and ensuring that the feature vectors retain only crucial information related to the vulnerability characteristics. Additionally, this helps to improve the performance of the model, enabling it to better distinguish between vulnerable and non-vulnerable samples, thereby increasing the accuracy and effectiveness of vulnerability detection.

4.4. Vulnerability Detection

In the model training phase, we obtain two feature vectors: one from the BTE, denoted as vector B and another from the DCE, denoted as vector D. These two vectors are individually subjected to soft thresholding processing through DRSN, resulting in processed feature vectors $B^{\prime }$ and $D^{\prime }$. To concatenate these two vectors, we flatten these two vectors as 1D vectors and we concatenate them to $B^{\prime }{D}^{\prime }$, which is then fed into a fully connected layer and a Relu activation function for non-linear transformation. Finally, the transformed vector is fed into a SoftMax layer to determine whether the code snippet contains vulnerabilities. The equations are as follows:

$$B^{\prime }{D}^{\prime }=\mathrm{concat}\left({Flatten(B}^{\prime }),Flatten(D^{\prime })\right)$$

(18)

$${\widehat{y}}_c=softmax(FC(B^{\prime }{D}^{\prime }))$$

(19)

Here, ${\widehat{y}}_c$ represents the predicted probability, FC represents the fully connected layer, concat signifies vector concatenation operation, and $softmax$ denotes the SoftMax function, used to transform real-valued vectors into probability vectors, indicating the probabilities of each category in a classification problem.

The MEVD model proposed in this paper is a framework that integrates several different functional encoders. In addition, we introduce a novel code-slicing method for data preprocessing. Through an extensive series of experiments, the method proposed in this paper demonstrates outstanding performance in the smart contract vulnerability detection. In the following sections, we present the experimental results to comprehensively substantiate the effectiveness of our proposed approach.

5. Experiments

In this section, we introduced the datasets and configuration parameters used in the experiments. We provided a detailed description of the experimental procedures and presented the experimental results. To evaluate the performance of our proposed method, we answered the following questions:

• RQ1: Can the proposed method effectively detect reentrancy, infinite loop, and timestamp dependency vulnerabilities? How does its accuracy, precision, recall, and F1-score compare to state-of-the-art vulnerability detection methods?

To answer this question, we compared state-of-the-art detection tools. Through these comparisons, we were able to assess the performance of our approach within the existing technology.

• RQ2: Is the proposed code slicing method helpful for vulnerability detection?

To answer this question, we divided the dataset into two parts: the original dataset and the Vulnerability Syntax-Driven Slice (VSS) dataset. We conducted experiments using these two datasets as model inputs and compared their performance separately.

• RQ3: What is the impact of the different components designed in the model on detection performance?

To answer this question, we conducted experiments to compare the performance of models with different combinations of components in terms of metrics such as accuracy, precision, recall, and F1-score.

5.1. Datasets

Our research dataset covers three types of vulnerabilities: reentrancy vulnerabilities, timestamp dependency vulnerabilities, and infinite loop vulnerabilities. The primary sources of these datasets include: (i) The SmartBugs Wild dataset consists of 47,398 Solidity language files containing a total of approximately 203,716 contracts with known vulnerabilities [39]. (ii) The ESC (Ethereum Smart Contracts) dataset consists of 307,396 smart contract functions extracted from 40,932 smart contracts. (iii) In addition, we use the datasets published by Qian et al. [40].

5.2. Experimental Settings

All experiments were performed on a computer equipped with 32 GB RAM and a GPU at 1080Ti. We implemented our method using the Keras and TensorFlow frameworks. In our experiment, 80% of the contracts form the training set, and the remaining 20% form the test set. The parameters used in the model are detailed in

Table 2. Parameters of the MEVD model.

Model Parameter	Value
Optimizer	Adam
Learning rate	[0.0001, 0.005]
Dropout rate	0.5
Batch size	8
Epochs	50

.

5.3. Evaluation Metrics

To measure the performance of our approach, we use the following four widely used evaluation metrics:

$Accuracy$: It represents the proportion of correctly predicted samples out of the total number of samples and measures the overall accuracy of the model’s predictions.

$$Accuracy=\frac{TP+TN}{TP+FP+TN+FN}$$

(20)

$Precision$: It represents the proportion of true positives among those predicted to be positive by the model and measures the accuracy of the model in predicting positive samples.

$$Precision=\frac{TP}{TP+FP}$$

(21)

$Recall$: It represents the proportion of true positives among the actual positive samples and measures the ability of the model to detect positive samples.

$$Recall=\frac{TP}{TP+FN}$$

(22)

$F1-Score$: This is a composite score that combines precision and recall to assess the overall performance of the model.

$$F1-Score=2\ast \frac{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{o}\mathrm{n}\ast \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{o}\mathrm{n}+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}$$

(23)

5.4. Results Analysis

Result for Answering RQ1: To evaluate the performance of MEVD in detecting these three types of vulnerabilities, we compared it with deep learning-based approaches (ReChecker [14], DR-GCN [40], TMP [40], CGE [21]). In

Table 3. Comparison with deep learning-based methods.

Vulnerability	Method	Acc (%)	Pre (%)	Rec (%)	F1 (%)
Reentrancy	ReChecker	74.65	69.47	76.93	75.62
	DR-GCN	82.58	75.39	80.83	79.57
	TMP	86.28	78.25	84.96	80.28
	CGE	88.32	84.15	86.77	85.64
	MEVD	92.13	89.49	90.15	89.28
Timestamp Dependency	ReChecker	72.68	70.86	69.24	73.21
	DR-GCN	80.21	76.54	78.86	79.04
	TMP	84.37	85.21	80.79	83.08
	CGE	87.76	86.15	82.77	84.56
	MEVD	90.85	89.17	92.98	91.04
Infinite Loop	ReChecker	68.79	71.63	69.41	71.33
	DR-GCN	76.55	72.25	74.21	78.62
	TMP	80.86	78.05	84.86	81.77
	CGE	85.89	82.62	83.19	83.25
	MEVD	86.94	84.15	85.63	86.21

, we show the comparison with state-of-the-art deep learning-based methods. These comparisons allow us to assess the performance of our proposed MEVD in vulnerability detection.

Based on the results presented in Table 3 and Figure 7, we can compare the performance of different models in terms of detection. The following conclusions can be drawn from these data: Firstly, the ReChecker model shows relatively poor detection performance compared to the other models, particularly in the context of identifying infinite loop vulnerabilities, where its accuracy is only 68.79%. However, when we use the MEVD model for detection, the accuracy improves significantly by 18.15%. It is worth noting that our approach not only outperforms in detecting infinite loop vulnerabilities, but also provides satisfactory performance improvements in other metrics. For example, in detecting timestamp dependency vulnerabilities, the accuracy of the MEVD model is 3.09% higher than that of the graph neural network-based CGE. In addition, the MEVD model achieves an accuracy of 92.13% for the detection of reentrancy vulnerabilities, which is well above the performance of other methods.

By comparing the experimental results, we come to a clear conclusion: the MEVD shows outstanding performance in different types of vulnerability detection. The MEVD shows significant performance advantages over state-of-the-art deep learning-based detection approaches.

Result for Answering RQ2: To evaluate the effectiveness of our proposed code slicing method, we divided the dataset into two parts: the unprocessed original dataset and the VSS dataset (VSS + Dataset). We then performed experiments on these two datasets separately, as shown in

Table 4. Performance comparison of sliced and original data.

Dataset	Acc (%)	Pre (%)	Rec (%)	F1 (%)
Reentrancy	86.32	82.94	85.61	84.49
Timestamp Dependency	85.21	81.54	83.86	86.24
Infinite Loop	83.05	77.63	78.67	81.09
VSS + Reentrancy	92.13	89.49	90.15	89.28
VSS + Timestamp Dependency	90.85	89.17	92.98	91.04
VSS + Infinite Loop	86.94	84.15	85.63	86.21

. This approach allows us to evaluate the impact of code slicing on model performance.

By analyzing the results in Table 4, we observe that running experiments on the sliced dataset, as opposed to the unprocessed dataset, significantly improves the performance of the models. For example, in the case of reentrancy vulnerability detection, experiments conducted on the sliced dataset yielded an accuracy of 92.13%, which is a significant improvement of 5.81% compared to the unprocessed dataset. At the same time, in the detection of vulnerabilities related to timestamps, the F1-score reaches 91.04%, which represents an improvement of 4.8%. By comparing the experimental results, we arrive at a definitive conclusion: The slice criteria defined by our approach are significant in covering crucial statements within the contract code. Furthermore, by extracting sliced code using control flow and data flow methods, we can expose critical code structure information and deepen the relationships between semantics. This helps models learn vulnerability patterns more effectively.

Result for Answering RQ3: In order to evaluate the effectiveness of different modules in the MEVD, we performed substitution experiments between modules. Specifically, we performed the following experiments individually: (1). Removal of the SFE module only, known as the MEVD-S model. (2). Replacement of the Transformer-CNN dual-branch encoder module with a CNN model, called the MEVD-T model. (3). Removal of the DRSN module only, called the MEVD-D model.

Table 5. Model comparison with different components.

Method	Vulnerability	Acc (%)	Pre (%)	Rec (%)	F1 (%)
MEVD-S	Reentrancy	88.71	84.41	83.59	85.94
	Timestamp Dependency	85.52	83.01	89.32	86.04
	Infinite Loop	82.15	80.37	81.86	82.18
MEVD-T	Reentrancy	84.31	81.93	82.29	80.07
	Timestamp Dependency	82.24	80.46	85.17	82.75
	Infinite Loop	76.18	75.42	78.25	77.92
MEVD-D	Reentrancy	89.31	86.24	85.13	84.27
	Timestamp Dependency	85.34	83.82	87.58	85.67
	Infinite Loop	81.28	78.32	83.53	82.30
MEVD	Reentrancy	92.13	89.49	90.15	89.28
	Timestamp Dependency	90.85	89.17	92.98	91.04
	Infinite Loop	86.94	84.15	85.63	86.21

shows the experimental results for these different modules. By comparing these results, we can determine the impact of the different modules on the model performance.

From an analysis of the experimental results in Table 5, we obtain a clear conclusion: when various modules are removed, the experimental performance of all models generally experience a decrease. Particularly noteworthy is the MEVD-T model, which shows a significant decrease in F1-score of 8.29% compared to the MEVD model for timestamp dependency vulnerability detection. In the case of reentrancy vulnerability detection, the accuracy of the MEVD-T model is only 84.31%, a decrease of 7.82% compared to the MEVD model, with other evaluation metrics also falling short of the MEVD model. This phenomenon is due to the limitation of the CNN models in adequately capturing the code relationships, thus failing to extract vulnerability features effectively. In contrast, our constructed Transformer CNN dual-branch encoder module comprehensively extracts the code global structure and fine-grained syntax semantic features, and fuses them into multi-scale feature vectors. This enables the model to learn vulnerability patterns, significantly improving detection performance. Further analysis of the MEVD-S and MEVD-D experimental results shows that the SFE and DRSN modules enhance semantic contextual details and effectively eliminate redundant information from feature vectors. For example, in the detection of reentrancy vulnerabilities, the SFE module improves accuracy by 3.42%, this improvement is attributed to SFE’s unique gating mechanism, which allows it to learn bidirectional semantic features from the code. By controlling the transmission of important semantic information, SFE increases the focus on critical semantic details. In addition, unlike traditional convolutional layers, SFE uses dilated convolutional layers for feature extraction, which can flexibly adjust dilation rates to capture a wider range of contextual semantic information, ensuring the integrity of the code’s semantics.

While the DRSN module improves the F1-score by 3.91% in the infinite loop vulnerability, this improvement is attributed to DRSN’s unique adaptive soft threshold setting strategy. This adaptability allows the model to flexibly capture vulnerability features at different positions in the code structure, enabling the network model to better adapt and learn different vulnerability patterns. By reducing information unrelated to the current task, DRSN effectively reduces the impact of features irrelevant to vulnerabilities on the model, resulting in improved vulnerability detection performance.

6. Conclusions and Future Work

6.1. Conclusions

In this paper, we propose a novel MEVD approach. In contrast to existing approaches, the proposed method can enrich the semantic information of code features, fully extract both global structure and local detail features of smart contract code, and effectively eliminate redundant elements for code features. In addition, we have introduced the concept of VSS, which removes code statements unrelated to vulnerability characteristics, contains rich syntactic and semantic information, and preserves the data and control dependencies in statements. It helps the model focus on the critical characteristics of the vulnerability, and capture more features about the vulnerability. A large number of experiments have shown that MEVD outperforms traditional methods and state-of-the-art deep learning approaches in detecting vulnerabilities in smart contracts.

This paper introduces the MEVD approach, which shows excellent accuracy in detecting vulnerabilities in smart contracts. In the field of blockchain technology, improving the security of smart contracts will increase the reliability and credibility of the blockchain system. In Decentralized Finance (DeFi), reducing financial fraud and losses caused by smart contract vulnerabilities may not only increase the credibility of blockchain financial products, but also attract more traditional financial institutions and investors to the blockchain sector. However, with the continuous development of smart contract platforms and various programming languages, new vulnerabilities may appear, making it difficult for MEVD to effectively detect them. To overcome this challenge, MEVD will need to become more scalable and flexible in the future.

6.2. Future Work

This paper proposes a novel multi-scale encoder approach for vulnerability detection and explores how to address the problem of inadequate code feature extraction ability. Our future work and research will include the following aspects:

• Construct a unified and standardized dataset of smart contract vulnerabilities. The dataset we are currently using is not comprehensive, as it does not include all types of vulnerabilities and is limited to smart contracts written in Solidity. Therefore, to fully exploit the potential of deep learning, we need to expand our research to include a wider range of programming languages and construct a dataset that covers a variety of vulnerability types.
• Develop a combined static and dynamic analysis model. Our current deep learning approach is primarily based on the static analysis of source code, which may ignore certain execution paths, leading to false positives. To address this issue, we plan to integrate detection tools at different levels to more effectively identify vulnerabilities in contracts. For example, the combination of deep learning models with dynamic execution techniques will be a valuable direction for research.
• Construction of an easy-to-use and highly scalable detection tool. Currently, the deployment and use of many vulnerability detection tools require complex environment configurations, which can be a barrier for non-expert users. When deploying MEVD in real-world scenarios, we need to consider issues such as computing resources and system compatibility. In addition, MEVD builds its model primarily on known types of vulnerabilities and may not be able to quickly adapt to newly emerging unknown vulnerabilities. Therefore, the development of a detection tool that is both user-friendly and capable of flexibly addressing new vulnerabilities will be an important part of our future research.