DpGuard: A Lightweight Attack Detection Method for an Industrial Bus Network

Abstract

In industrial control systems (ICSs), the PROFIBUS-DP (decentralized peripherals) protocol is widely used for communication between devices. Because PROFIBUS-DP is an unencrypted and insecure bus protocol, attackers can connect to the PROFIBUS-DP system and arbitrarily manipulate I/O process values, which may interrupt the normal operation of industrial equipment, or have more serious consequences. However, due to the complex structures of bus networks and the large number of attack areas, the existing scheme does not monitor all the messages in the industrial head office network and cannot effectively detect semantic attacks. To solve this problem, we propose a novel attack detection system DpGuard. DpGuard automatically builds a finite-state machine model of normal ICS behavior through a large number of historical ICS traffic data. The model includes state events, state transitions, state transition probability, and other normal behavior information. In addition, DpGuard records the execution status of the context data package, uses the real-time captured data package as the input of the model, and judges whether the state event and state transition probability conform to the constraints of the finite-state machine model, so as to identify the legitimate normal behavior of the ICS. Our proposal was evaluated using two Siemens PLCs (programmable logic controllers) deployed on the PROFIBUS-DP system. The experimental results demonstrated that the scheme could accurately detect fault injection and semantic attacks. Compared with other detection models, our scheme presented an improved detection performance, with a detection accuracy of 99.80%.

1. Introduction

Industrial control systems include SCADA (supervisory control and data acquisition), DCSs (distributed control systems), and industrial sensing devices [1]. With the development of information and communication, automation, and computer technologies, industrial facilities have gradually adopted TCP/IP as a communication system, but many industrial facilities still use bus networks as the underlying communication system [2] including, for example, the fieldbus standard PROFIBUS-DP [3] and the military standard MIL-STD-1553 for avionics bus network infrastructure [4]. Considering the perspective of network security, bus networks have proven to be vulnerable to denial-of-service, replay, and injection attacks due to the lack of cryptographic authentication mechanisms in bus protocols [5,6,7]. Consequently, the security issues and threats faced by ICS bus networks have received increasing attention.

To detect ICS attacks, statistical analysis [6,8,9] based on a “five-tuple” (i.e., <packet length, port, response time, destination IP, and source IP>) of ICS protocols is effective for noisy attacks (e.g., probing scans and erroneous packets), but fails to detect advanced attackers using advanced semantic attacks, i.e., legitimate protocols and ICS parameters [10]. As a result, semantic attacks can bypass this detection method and compromise the security of the physical world of the ICS. In general, existing ICS detection tools are circumvented by semantic attacks and are prone to false alarms due to the isolated analysis of traffic and sensor data, as they cannot relate their analysis to the attack execution context of the ICS [10]. Detecting bus network attacks on ICSs is difficult because of the complex bus network structure, the large attack surface, and the fact that attackers use packets that differ from those conforming to the protocol specification. For example, a study [11] of the avionics bus network infrastructure MIL-STD-1553 noted that all devices inside military warplanes are physically connected to the same wire and may suffer from denial-of-service and flooding attacks, and semantic attacks can have more serious consequences of cyber attacks.

We found that although these semantic attack behaviors of industrial bus networks are normal ICS activities, they are anomalous when executed in typical execution phases in an ICS. Therefore, in this work, instead of considering ICS bus network traffic behavior as a separate execution process, we specialized its behavior into unique execution phases, i.e., initialization and data exchange phases. We observed that to launch a semantic attack, the attacker must perform operations such as diagnostics, configuration, and parameter setting in the initialization phase, which do not occur in the data exchange phase but are necessary for a semantic attack. For example, to tamper with the point position of the DO module, the attacker needs to send the diagnostic, configuration, and parameter setting packets of the initialization phase, which is abnormal for the data exchange phase because no further initialization operations are required to enter the data exchange phase. These attacks are inconsistent with the behavior of an ICS in the data exchange phase and lead to incorrect state transitions. Therefore, if one identifies a limited set of legitimate data exchange phase behaviors, one can effectively monitor and detect malicious attacks in which the attacker violates these behaviors.

To overcome the limitations of existing solutions, we propose a novel attack detection scheme called DpGuard that detects fault injection and semantic attacks by correlating the specific behaviors of the ICS data exchange phases with the traffic context. DpGuard identifies the limited state transitions specific to each data exchange phase, which is possible because they are distinct from the attacker’s activity in those phases. Since industrial bus network traffic and protocols conform to the state machine model, there are strict state transition constraints. Therefore, DpGuard requires a finite-state machine model to identify disruptive control packets sent to the ICS. DpGuard automatically constructs a finite-state machine model of normal ICS behavior from a large amount of historical ICS traffic data, which contains information on normal behavior such as state events, state transitions, and state transition probabilities. In addition, DpGuard records the state of contextual packet execution and uses the real-time captured packets as the input for the model to determine whether the state events and state transfer probabilities conform to the finite-state machine model constraints, thus identifying legitimate normal ICS behavior.

DpGuard detects semantic attacks that are not detectable using existing tools. By correlating the initialization phase and data exchange phase through the characteristics of the ICS traffic, DpGuard provides contextual alerts to ICS operators in response to attacks on ICS devices through real-time attack detection and the detection of attackers entering abnormal initialization phases during typical data exchange phases. We evaluated DpGuard using a PROFIBUS-DP protocol communication system designed with Siemens S7-300 and ET200 PLCs. The experimental results showed that the solution could detect fault injection and semantic attacks accurately in real time without affecting the normal operation of industrial serial network systems. Compared with four other representative detection methods, our scheme presented an improved detection performance. In particular, the detection accuracy for semantic attacks reached 99.80%.

In short, our contribution:

(1)

We proposed a new scheme, DpGuard, that uses the characteristics of ICS traffic to correlate the initialization phase and the data exchange phase to automatically build a finite-state machine model to detect fault injection and semantic attacks in real-time using industrial serial protocol messages;

(2)

We developed a prototype system for attack detection based on a PROFIBUS-DP protocol communication system. Although the system was developed for a specific serial protocol, based on the idea it could be modified to extend and adapt it to other serial protocols such as CAN and MODBUS RTU;

(3)

We evaluated the scheme using two Siemens PLCs deployed on a PROFIBUS-DP system. The experiments showed that the scheme could detect fault injection and semantic attacks accurately in real-time without affecting the normal operation of the industrial serial network system. Our scheme outperformed the four other representative detection methods in terms of detection accuracy.

We structured the remainder of this paper as follows. Section 2 briefly introduces the technical background related to the bus protocol PROFIBUS, and Section 3 presents the current work on bus network attack detection methods. Section 4 presents the threat model and attack model. Section 5 details our proposed detection scheme. Section 6 details the experimental evaluation. Finally, Section 7 summarizes our detection scheme.

2. Background

2.1. PROFIBUS-DP System

Industrial bus networks are characterized by a high speed and periodicity and are widely used in industrial control systems, such as in the petroleum, chemical, and power industries [12,13]. PROFIBUS-DP is the most popular bus network system. PROFIBUS-DP is a standard fieldbus protocol, and the PROFIBUS-DP system is a bus-based industrial network for communication between device-level control systems and decentralized I/O for the cyclic exchange of process data and acyclic data exchange for configuration diagnosis [14]. The PROFIBUS-DP system consists of a DP master, a DP slave, and an actuator [15]. The master and slaves communicate using the PROFIBUS-DP protocol at rates ranging from 9200 kbps to 152,000 kbps. Figure 1 shows a typical PROFIBUS-DP system.

2.2. PROFIBUS-DP Protocol

Frame formats. There are five main message formats in the ROFIBUS-DP bus protocol [14] and, as shown in Figure 2, its most important function is to realize data exchange between the DP master and DP slave, communicating data values and control information, etc. The function of the SD1 (SD = start delimiter) message is the diagnosis of the DP slave by the DP master; that the SD2 message is the transmission of variable data; that the SD3 message is the transmission of fixed data; that the SD4 message is the transmission of token frames; and that the SC (short confirmation) message is the answer frame. We found that to launch a fault injection attack, the attacker usually injects non-normal-format messages into the bus network, i.e., the injected messages do not conform to the message format determined by the PROFIBUS-DP standard. Based on this, we judged the legitimacy of the messages in the industrial bus network and thus detected the presence of fault injection attacks.

Protocol state machine. According to the standard PROFIBUS definition, the PROFIBUS-DP protocol state machine is a communication task realized by a class-one DP master, a class-two DP master, and a DP slave. The state machine mainly comprises an initialization phase and a cyclic phase. In the initialization phase, DP slave diagnostics, configuration, and parameterization are performed [13]. The cyclic data exchange phase only starts after all verifications have been completed. Each state has strict state transition constraints and follows the protocol state machine model. Hence, for a semantic attack, one must re-initiate the initialization phase in the data exchange phase. However, the data exchange phase does not appear to transfer to the initialization phase, which violates this protocol state machine model. Therefore, we propose an attack detection method based on a finite-state machine model.

3. Related Work

With the frequent occurrence of security incidents in industrial control systems, more and more people have studied intrusion detection solutions for industrial control systems—for example, the well-known open-source intrusion detection tools from Snort [16] and Zeek [17]. However, existing solutions [9,18] apply to industrial Ethernet networks, and these detection tools rely on network IP addresses. Industrial bus network devices do not have IP addresses and use link broadcast communication. This makes the traditional TCP/IP detection methods ineffective, and so there is an urgent need to study attack detection methods and tools for industrial bus networks. Several researchers have studied industrial bus network protection schemes. Their work has focused on the security protection offered by MODBUS RTU and CAN protocols.

MODBUS RTU: Thomas et al. [7] discussed the need for such a system by describing four classes of intrusion vulnerabilities (denial of service, command injection, response injection, and system reconnaissance) that can be exploited in the MODBUS RTU/ASCII industry. H. Morris et al. [19] introduced an intrusion detection system rule set in MODBUS/TCP and MODBUS serial line systems. However, this work lacked validation experiments and was not convincing. Tylman et al. [8] proposed a new method for handling non-IP protocols in the Snort intrusion detection system based on the Snort data acquisition module (DAQ). This work did not require modifications to the Snort code, but without deep packet inspection PDUs (protocol data units), semantic attack packets on altered PDUs may bypass detection. Tomlin et al. [14] proposed an unsupervised machine learning approach for implementing network IDS in power system applications. However, the non-IP industrial bus network attack detection problem was not considered.

CAN: Song et al. [6] proposed a lightweight algorithm for in-vehicle network intrusion detection based on the analysis of CAN message intervals. This algorithm was unable to detect irregular messages. Cho et al. [20] proposed a novel scheme that identified the attacker’s ECU by measuring and exploiting the voltage in the in-vehicle network. Rohit et al. [21] proposed a lightweight defense called RAID, which enabled each ECU to make protocol-compatible modifications in its frame format in the VIDS retraining mode, thus generating a unique dialect (spoken by the ECU). Marcel et al. [8] proposed Scission, an intrusion detection system (IDS) that used fingerprints extracted from CAN frames and could identify the sender’s ECU. In the above four works [8,21,22,23,24], intrusion detection was performed for the CAN protocol and compatibility with the PROFIBUS- DP protocol was not considered.

The above mentioned literature mainly studied MODBUS RTU and CAN protocols. However, the PROFIBUS-DP protocol is widely used in many industrial control systems [22,23,24] because is characterized by fast data transfer and high stability. Moura et al. [2] proposed an expert system combining knowledge and unsupervised techniques to improve the protection of PROFIBUS-DP industrial bus networks. The limitations of this work were the use of data from simulators and the lack of testing and validation based on actual data collected in industrial networks. Furthermore, the authors did not consider whether the deployment of this test solution in the real world would affect the proper functioning of the system, and their research scheme could not be used to analyze the detection of semantic attacks. Because attackers launch semantic attacks and construct malicious packets that are protocol compliant, there is nothing wrong with the packets themselves. Therefore, developing a detection scheme for this type of attack is extremely challenging.

Our work solves the above challenges: first, DpGuard automatically constructs a finite-state machine model of normal ICS behavior from a large amount of historical ICS traffic data, which contains normal behavior information such as state events, state transfers, and state transfer probabilities. Secondly, DpGuard records the state of the contextual packet execution and uses the real-time captured packets as the input of the model to judge whether the state events and state transfer probabilities conform to the finite-state machine model constraints, thus identifying legitimate normal ICS behavior. Finally, our attack detection system, DpGuard, is lightweight and was deployed on a real PROFIBUS-DP system built with Siemens S7-300 and ET200 PLCs, having little impact on bus bandwidth constraints and bus network in real time. Hence, our work is more applicable to the network security protection of industrial bus networks.

4. Threat Model

We assumed a threat model similar to those in existing work on ICS attacks launched against bus networks [4], wherein the attacker can establish communication with the bus system by means of physical access. In contrast to an industrial Ethernet attack, an attacker can launch an attack on all ICS devices on the bus network by simply accessing any node on the bus, which makes the attack less expensive. DpGuard learns in a trusted environment without attacks. We also assumed that DpGuard would not be attacked directly. There are limitations to this assumption, because advanced attackers may bypass our detection model, resulting in false positives. We made the following practical assumptions: we did not consider attacks originating from outside the ICS, such as those originating from side channels [25,26]. Existing work [27,28] and practice have addressed the PLC man-in-the-middle (MITM) problem via non-PLC diode gateways [29], which are therefore out of the scope of this work. Moreover, DpGuard relies on the window host, and its own security was not considered in this work. Meanwhile, we assumed that the attacker’s intention may be to bypass the current mainstream industrial bus network IDS and implement an I/O register tampering semantic attack to accomplish the task of disconnecting relay closure and affecting the whole system operation. In addition, it was assumed that the attacker is fully aware of the slave’s configuration, including addresses, GSD files (which contain information about the basic capabilities of the slave device), and DO (digital output) modules. This does not require a substantial effort from the attacker, who can obtain information about these parameters from the industrial bus network messages through expert knowledge. Based on this knowledge, the attacker can carefully design and construct protocol-compliant messages to launch semantic attacks on the industrial bus network.

In our hypothetical attack model, as shown in

Table 1. Attack model details.

Type of Attack	Attack Description	Attack Test Cases
Fault attack	Replay of non-normal-format messages	0x68100202327D16
	Replay of function code message	0x100802005316
	Replay of SC frames	0xE5
	Replay of SD1 frame	0x100802495316
	New function code	0x100802xx5316 *
	New destination address	0x10xx02495316 *
	New length of main site SD2 news	0x68xxxx6888826D3C3EF116 *
	New length of slave SD2 message	0xA28288083E3C000400FF00008F16
Semantic attack	Register tampering	0x6805056808027DxxxxED16 *
	Diagnosis message tampering	0x68xx68xxxxxx3E3Cx…xx16 *
	Input data tampering	0x68xx68xxxxxx3E38x…xx16 *
	Slave address tampering	0x68090968xxxxxx373Exxxxxx16 *
	Output data tampering	0x68xx68xxxxxx3E39xxx16 *
	Token frame tampering	0xDCxx02 *

* “x” stands for the variable field.

, the attacker launched two types of attacks, namely, fault injection and semantic attacks. Fault injection aims to cause failures in industrial bus networks by injecting attack traffic such as spoofing, DoS (denial of service), bus shutdown, and fuzzing attacks, which do not require an adversary to have much knowledge of the bus communication protocol. We assumed that the DoS attack could be launched by injecting two types of messages, i.e., messages that do not conform to the protocol specification and messages that do not conform to the correct protocol state transitions. If messages do not conform to the protocol specification, our proposed scheme detects them by analyzing the format of the message. If the attack messages conform to the protocol specification, the scheme identifies the attack messages by verifying whether they conform to the protocol state transitions. For example, by exploiting the fragile error-handling scheme of the PROFIBUS-DP bus, an adversary can inject specific messages to disconnect or reconnect slaves. An attacker can use a fault injection attack to sniff the packets that establish a connection between a master and a slave, relying on expert empirical knowledge as the basis for the next stage of implementing the semantic attack. Semantic attacks aim to cause abnormal system states such as I/O register failures, the loss of control of the master, and data tampering. Such attacks always require an adversary with extensive knowledge of the target protocol, as the attacker needs to inject special information into the industrial bus network to control both masters and slaves. For example, by hacking the local physical access bus network, an adversary can tamper with the slave I/O by injecting prepared information into the industrial bus network, making it impossible for the master to collect data from the slave. Compared to fault injection attacks, semantic attacks impact and harm the system more severely. DpGuard is tasked with detecting both types of attacks, especially semantic attacks.

5. Methods

Due to the excellent performance and scalability of finite-state machines, protocols are usually formally described and analyzed using finite-state machines. In particular, industrial bus network protocols follow the finite-state machine model and have strict state transition constraints. Attackers launch fault injection and semantic attacks that precisely violate the finite-state machine of industrial bus network protocols. Therefore, based on the concept of the finite-state machines, we proposed a new industrial bus network attack detection scheme, DpGuard, and applied it to attack detection in industrial bus networks. In this section, we introduce DpGuard in detail, providing the FSM definitions and presenting and describing the model training and detection algorithm.

5.1. FSM

Finite-state machines are widely used in the field of protocol description analysis and attack detection [5,30,31]. The definitions related to the application of finite-state machine models for attack detection in industrial bus networks are provided below.

Definition 1.

A finite-state machine M is defined as a five-tuple: M = (Q, S, θ, の, Y).

Q: a finite set of state events, containing all state events of the protocol.

S: node state, where S₀ is the initial state with no predecessor state.

θ: state transfer probability, the probability of transferring the previous state S_i to the next state

S_j: calculated by の.

の: state transfer function, which is Q × S, θ = の (S_j, S_i, (S_j, S_i) ∈ Q).

Y: the output result, containing the response to the appearance of the state machine.

In order to precisely and concisely portray the operation of M and detect attacks in real time, we converted the raw packets into events that could be recognized by a finite-state machine, such that the raw packet p at a certain moment is written as (p_i, t_i, Σ) and materialized in the form {(p₀, t₀), (p₁, t₁), (p₂, t₂),…,(p_n, t_n)}. Based on expert empirical knowledge, p_i is converted into s_i to obtain (s_i, t_i, Σ) as the input of M. Σ is the set of s_i and t_i. Only if p_i can be converted into s_i is s_i used as the input of M. In other words if p_i → s_i cannot be completed, Y outputs fault injection attack alarms. If p_i → s_i, M is input for state detection, when s_i is able to enter M, the following definition for detecting semantic attacks.

Definition 2.

In the finite-state machine M = (Q, S, θ, の, Y), when S_i is transferred to S_j, θ_ij is computed after の (S_i, S_j). θ_ij represents the transfer probability of S_i transfer to S_j, if θ_ij < 1, then Y outputs a semantic attack alarm. Otherwise, S_i transfers to S_j, belonging to the legitimate transfer state and then continues to detect the next S_j+1. The finite-state machine M is always in operation and processes S in real time.

5.2. Model Training

5.2.1. Definition

We use the finite-state machine model to model the normal traffic behavior of the industrial bus network. The state event set Q = {SD1, SD2, SD3, SD4, SC} of the finite-state machine determines the specific state according to the message text segment. Taking SD3 of the PROFIBUS-DP protocol as an example, the information in the message text segment was analyzed to obtain the message information <sd, da, sa, fc, du, fcs, ed>, and the content of this message information was used to determine the current state. Within this set of information, (1) st indicates the start of the frame definer; (2) da indicates the device destination address; (3) sa indicates the device source address; (4) fc indicates the protocol function code; (5) du indicates the protocol parameter data, including I/O, GSD file configuration, and rate; (6) checksum indicates the frame check data; and (7) ed indicates the end of the frame definer.

Regarding the message information, the state events mapped by the message sequence of PROFIBUS-DP are unique and ensure that the finite-state machine model M can work effectively. If the message sequence mapping state event fails, the message is a malicious fault injection frame. In addition, each state event is used to determine whether the state violates the state machine by calculating the state transfer probability through the state transfer function, thus detecting semantic attacks.

5.2.2. Building the FSM Model

We used historical messages from industrial bus networks to automate the construction of the finite-state machine model and generate a state transfer diagram. The following 11 packets are presented as an example to illustrate the construction process of the finite-state machine M. For more information on the related constant values, refer to [32,33].

(1)

sd = “DC”, da = “02”, sa = “02”

(2)

sd = “10”, da = “08”, sa = “02”, fc = “49”, fcs = “53”, ed = “16”

(3)

sd = “10”, da = “02”, sa = “08”, fc = “00”, fcs = “0A”, ed = “16”

(4)

sd = “68”, ler = “05”, lers = “05”, sd = “68”, da = “88”, sa = “82”, fc = “6D”, du = “3C 3E”, fcs = “F1”, ed = “16”

(5)

sd = “A2”, da = “82”, sa = “88”, fc = “08”, du = “3E 3C 00 04 00 FF 00 00”, fcs = “8F”, ed = “16”

(6)

sd = “68”, ler = “10”, lers = “10”, sd = “68”, da = “88”, sa = “82”, fc = “5D”, du = “3D 3E B8 1E 01 00 42 24 01 40 01 00 42”, fcs = “A3”, ed = “16”

(7)

sd = “E5”

(8)

sd = “68”, ler = “09”, lers = “09”, sd = “68”, da = “88”, sa = “82”, fc = “7D”, du = “3E 3E 00 20 20 10”, fcs = “53”, ed = “16”

(9)

sd = “E5”

(10)

sd = “68”, ler = “05”, lers = “05”, sd = “68”, da = “02”, sa = “08”, fc = “08”, du = “01 00”, fcs = “AA”, ed = “16”

(11)

sd = “68”, ler = “05”, lers = “05”, sd = “68”, da = “08”, sa = “02”, fc = “5D”, du = “00 01”, fcs = “79”, ed = “16”

These 11 packets represent a complete process, in which packets (1)–(9) are the initialization phase and packets (10) and (11) are the data exchange phase. These constant values are necessary for attackers to launch semantic attacks. The semantic attack packets listed in Table 1 contain these constant values, such as the master address, slave address, function codes, and data values. The finite-state machine model used the above historical messages to construct two phases, i.e., the initialization and data exchange phases. In the initialization phase, packet (1) indicates that the current token frame belongs to the master with address 2, which controls the slaves on the bus network. Packet (2) indicates that the master with address 2 diagnoses whether the slave with address 8 is alive or not, and if the slave is alive, it replies with packet (3), informing the master that the current slave is active. Next, packet (4) queries the master regarding the slave’s relevant parameter configuration, such as its rate. Packet (5) indicates that the slave replies to the master with the configuration parameters. After the master confirms the configuration parameters, the communication interface configuration is entered and packet (6) indicates that the input and output formats specified by the master and slave are the same. If the slave confirms the same input and output as the master, it replies to master packet (7). When the previous state is completed, the master sends out a data packet (8) to perform a final check with the slave. The slave station replies to master packet (9) after confirming that there is no error, thus completing the initialization phase and generating a state transfer diagram, as shown in Figure 3.

The data exchange phase can only be entered when the initialization phase is completed. Hence, if an attacker launches a semantic attack, he must complete the initialization phase, including state transfer (e.g., diagnosis, configuration parameter determination, and consistency checking). Additionally, there is no transfer to the initialization phase when the data exchange phase is in progress. For example, packet (10) activates the I/0 point position of the slave station and collects data from that point. The slave receives the packet (10) and communicates to the master that the current I/O position is in the active state packet (11). Next, the master and slave communicate in a cyclic manner, i.e., packets (10) and packets (11) appear periodically in the industrial bus network. The state transfer diagram of the data exchange phase is shown in Figure 4.

5.3. Detection Algorithm and Its Description

Using the finite-state machine M model constructed by automation, fault injection and semantic attacks can be detected in two stages, and the occurrence of illegal state events, i.e., fault injection attacks, can be detected based on the statical analysis of the message segment information. Specifically, considering the captured industrial bus network messages, if s ∉ Q, then a fault injection attack is detected. To detect semantic attacks, one calculates the probability of the transfer of state s_i to state s_j at a certain moment through the state transfer function の using historical data, obtaining a stable value θ with a certain amount of data training. If the industrial bus network message s ∈ Q, there is no fault injection attack in the message. Therefore, the state transfer probability is next determined to detect whether there is a semantic attack in this message. The detection algorithm for the finite-state machine M is shown below (Algorithm 1).

Algorithm 1 Finite-state Machine (M) Detection Attack Algorithm

Input: real-time industrial bus network message serial (p0, p1, …, pn)

Input: set of state events Q, state transfer function の, state transfer probability θ

Output: fault injection and semantic attack alarm serial (p0, p1, …, pn)

While$\mathrm{i}<\mathrm{n}$:

${\mathrm{S}}_{\mathrm{i}}\leftarrow \mathrm{extract} \left({\mathrm{p}}_{\mathrm{i}}\right)$

if  ${\mathrm{S}}_{\mathrm{i}}\notin \mathrm{Q}$ then:

Alert (“Fault Injection Attack”)

else:

RecoderState $\left({\mathrm{S}}_{\mathrm{i}}\right)$

θ (${\mathrm{S}}_{\mathrm{i}}+1$) = の (${\mathrm{S}}_{\mathrm{i}}$, ${\mathrm{S}}_{\mathrm{i}}+1$)

if    $\mathsf{\theta }<1$ then:

Alert (“Semantic Attack”)

else:

Next (serial)

endif

endif

END

6. Evaluation

6.1. Experimental Environment

In order to experimentally verify the effectiveness of DpGuard detection, we used a real system. In this experiment, a miniature industrial bus system was built, in which the master device was an S7-300, the slave device was an ET200, the relays were connected to the slave output points, and the communication protocol was PROFIBUS-DP. The main configuration parameters of the ET200 DP slave are shown in

Table 2. Details of ET200 slave configuration parameters.

Parameter Name	Parameter Value
Communication Rate	19,200
GSD	si03806a.gsd
Address	8
Power Modules	6ES7 138-4CA01-0AA0 PM-E DC24V
DO Module	6ES7 132-4BB30-0AA0 2DO DC24V
DI Module	6ES7 131-4BD01-0AA0 4DI DC24V

. Under normal operation, the relay was closed and the DpGuard ran on the window host, which was connected to the control system via an RS485. The bus system architecture diagram is shown in Figure 5.

DpGuard is also effective in other network structures. If there are multiple master or slave devices in the bus network, DpGuard can build corresponding FSM models from messages as long as it is deployed in a suitable location where it can monitor the traffics of the bus network. Then, DpGuard can detect fault injection and semantic attacks according to the proposed detection method. Additionally, our solution can be adapted to detect attacks in other fieldbus protocol (e.g., MODBUS RTU and CAN) networks by incorporating additional protocol analysis processes. We will address this in our future work.

6.2. Experimental Data Generation

In this experiment, to ensure that the finite-state machine model could learn the state of the whole system, a total of 20,000 packets were collected, containing the function codes and states described by the PROFIBUS standard document. After the finite-state machine model is constructed, we capture 1h of packets in the micro-industrial bus system, totaling 6288 packets. These comprised 3588 normal packets, 2150 fault injection packets, and 1500 semantic attack packets. Compared with [2], we generated a more robust and advanced attack dataset. Detailed packet descriptions are shown in

Table 3. Details of the generated experimental data.

Type of Attack	Attack Description	Quantity
Normal	Normal packets	3588
Fault attack	Replay of non-normal-format messages	300
	Replay of function code message	450
	Replay of SC frames	200
	Replay of SD1 frame	200
	New function code	200
	New destination address	200
	New length of main site SD2 news	300
	New length of slave SD2 message	300
Semantic attack	Register tampering	250
	Diagnosis message tampering	250
	Input data tampering	250
	Slave address tampering	250
	Output data tampering	250
	Token frame tampering	250

.

6.3. Experimental Results and Analysis

6.3.1. Detection Capabilities

To assess the effectiveness of DpGuard, common evaluation metrics for dichotomous test were used, i.e., accuracy, true positive, false positives, true negatives, and false negatives, as shown in

Table 4. Evaluation indicators.

	Predicted Positive	Prediction Negative
Reference Positive	TP (true positive)	FN (false negative)
Reference Negative	FP (false positive)	TN (true negative)

.

According to the confusion matrix shown in Table 3, the accuracy rate represents the proportion of the total sample that the model predicted correctly, calculated as:

Accuracy = (TP + TN)/(TP + TN + FP + FN)

(1)

where TP is a correctly classified malicious packet sample, FP is a misclassified normal packet sample, TN is a correctly classified normal packet sample, and FN is a misclassified malicious packet sample.

Recall, also known as TPR, indicates the percentage of positive samples that the model predicted correctly as a percentage of all samples that were actually positive. The precision rate represents the percentage of positive cases correctly predicted by the model as a percentage of all samples predicted to be positive. The precision and recall rates were calculated as follows:

Precision = TP/(TP + FP)

(2)

Recall = TP/(TP + FN)

(3)

The F1 score combines precision and recall as the summed average of the two. The calculation formula is:

F1 = (2 × Precision × Recall)/(Precision + Recall) = 2TP/(2TP + FP + FN)

(4)

As shown in

Table 5. Test results and comparison.

Method	Fault Attack				Semantic Attack
	Accuracy	Precision	Recall	F1	Accuracy	Precision	Recall	F1
Ref. [35]	0.6745	0.7201	0.6700	0.6884	0.6011	0.5945	0.6043	0.6100
Ref. [34]	0.8780	0.8921	0.8442	0.8792	0.7109	0.7034	0.7034	0.7264
Ref. [36]	0.8827	0.8930	0.8672	0.8703	0.7133	0.7254	0.7197	0.7206
Ref. [2]	0.9990	0.9964	0.9892	0.9928	0.7300	0.7292	0.7255	0.7273
DpGuard	1.0000	1.0000	1.0000	1.0000	0.9980	0.9922	0.9980	0.9912

, DpGuard evaluated the results to detect fault injection with 100% accuracy and semantic attacks with 99.80% accuracy. Our algorithm can sensitively identify and detect power failure/instabilities and token resets or the re-initialization of the token frame. We supplemented the experiments by constructing tests of power failure/instability, which would cause the PROFIBUS network to be reinitialized. The experimental results showed that DpGuard detected these exceptions and raised alerts. DpGuard detected the wrong protocol state transition and considered it as semantic attacks. Although this result was a “false positive” (this was not a real cyber attack), this alarm was also meaningful for the protection of the PROFIBUS network. Drawing a comparison with other algorithms, DpGuard outperformed SVM [34], logistic [35], DT [36], and the algorithm presented in [2]. Logistic had the lowest average detection accuracy of 0.6378, because detecting anomalies is a dichotomous problem and logistic is suitable for regression problems. In addition, because PROFIBUS-DP messages require expert-knowledge preprocessing and feature selection before they can be used as an input for machine learning/deep learning, and because they comprise raw data with no obvious features that are the same as normal communication data, machine learning/deep learning is ineffective, especially for detecting semantic attacks. The algorithm presented in [2] can effectively detect fault injection attacks but is poor at detecting semantic attacks. In addition, DpGuard did not achieve 100% accuracy in detecting semantic attacks because there was an “unknown_type” message in the packet, which was not specified in the PROFIBUS standard; we suspect that this was a private-protocol-specified packet. Therefore, the DpGuard caused a false alarm.

6.3.2. Selection of θ

Since fault injection attacks were detected by DpGuard in the first stage, there was no need to evaluate the impact of θ selection on the detection performance. Therefore, the selection of θ values only needed to be evaluated for the detection of semantic attacks by DpGuard. In our experiments, we set θ = 0.5, 0.6, 0.7, 0.8, 0.9, 1.0. The experimental results are shown in Figure 6, showing that as θ increases, the accuracy of semantic attacks by DpGuard also increased. When θ = 1, the accuracy of DpGuard reached a maximum of 99.80%. We trained the finite-state machine model when θ = 1 to ensure that an effective state machine model was constructed. Therefore, we set θ as 1 to ensure the effectiveness of DpGuard when detecting semantic attacks.

6.3.3. Impact Performance

The normal communication of the industrial bus network may be affected by the access to the attack detection system in the bus, resulting in the interruption or even complete stoppage of the communication between the master and slave PLC devices. Therefore, we needed to pay attention to whether normal communication between the master and slave PLCs was affected after DpGuard was deployed on the PROFIBUS-DP system. To demonstrate that our detection scheme has little to no impact on the normal operation of the PROFIBUS-DP system, we used an oscilloscope to capture waveforms in order to evaluate the performance impact. We captured waveforms during the normal operation of the PROFIBUS-DP system without any external dry winding for a total of 72 h. Afterward, we captured waveforms with an oscilloscope for a total of 48 h, after deploying DpGuard in the system. As shown in Figure 7, the waveforms for the normal operation of the PROFIBUS-DP system and the waveforms after DpGuard was deployed were the same, which proved that our testing solution had almost no effect on the normal operation of the PROFIBUS-DP system. The experimental result showed that DpGuard had little impact on the operation of the bus system, causing only a microsecond delay. At the same time, the results showed that our detection solution has wide application prospects. We also monitored the bus fault lights for the master and slave stations, which reported no errors during the DpGuard deployment, indicating that our detection method did not interfere with the bus communication of the PROFIBUS-DP system.

7. Conclusions

Applying attack detection technology to PROFIBUS-DP fieldbus systems is a meaningful way to improve their security. DpGuard is capable of detecting not only fault injection attacks but also semantic attacks that cannot be detected by existing tools. By correlating the initialization phase and data exchange phase through the characteristics of the ICS traffic, DpGuard provides ICS operators with contextual alerts in response to attacks on ICS devices through real time attack detection and the detection of attackers entering abnormal initialization phases during atypical data exchange phases. We evaluated DpGuard using a PROFIBUS-DP protocol communication system built with a Siemens S7-300 PLC and a Siemens ET200 PC. The experimental results showed that the solution is able to detect fault injection and semantic attacks accurately in real time without affecting the normal operation of industrial bus network systems. Compared with four other representative detection methods, our scheme presented a superior detection performance. In particular, the detection accuracy for semantic attacks reached 99.80%.