Time-Sensitive Network Simulation for In-Vehicle Ethernet Using SARSA Algorithm

Abstract

In order to more accurately analyze the problem of time delay simulation and calculation in the time-sensitive network (TSN) of vehicular Ethernet, a TSN reservation class data delay analysis model improved based on the State–Action–Reward–State–Action (SARSA) reinforcement learning algorithm is proposed. Firstly, the TSN data queue forwarding delay model and reservation class data delay analysis intelligent body model are established, then the TSN traffic scheduling mechanism is improved by the SARSA reinforcement learning algorithm, and the improved TSN network reservation class data analysis model is established for the uncertainty of traffic scheduling in the network; finally, the fitting performance of the proposed method is verified by simulation and experimental validation. The results show that the deviation between the two is less than 5% under different BE loads, i.e., the established reservation class data delay analysis model is able to correctly fit the scheduling mechanism of the vehicle-mounted TSN network, which proves the reasonableness of the model simulation.

1. Introduction

In recent years, there has been a rapid development of intelligent and connected vehicles, leading to a new trend in the automotive engineering field. An increasing number of electronic control systems, such as antilock braking systems (ABSs), advanced driver assistance systems (ADASs), and in-vehicle infotainment systems (IVIs) [1], have been implemented in automobiles. Although these electronic control systems enhance the handling, safety, and comfort of vehicles, they also result in the installation of a greater number of electronic control units (ECUs) in automobiles [2]. Therefore, smart connected vehicles require high bandwidth from the in-vehicle network, while also necessitating a network that is reliable enough to maintain low latency and jitter [3]. The traditional in-vehicle network has gradually become inadequate in meeting the diverse needs of products, thus emphasizing the need to expand and standardize the protocols for in-vehicle networks based on Ethernet. With these requirements in mind, it is necessary to conduct research and apply advances in in-vehicle Ethernet.

For in-vehicle Ethernet, the IEEE802.1 Working Group has been working on the development of Ethernet Audio Video Bridging (AVB) technology [4,5,6,7,8,9,10,11]. AVB technology supports various features such as clock synchronization, bandwidth reservation, low-latency traffic specification, and more. The AVB protocol cluster has been extended with the introduction of TSN technology, which has attracted the interest of many researchers. TSN builds upon the foundation of AVB technology and incorporates new mechanisms and enhancements in IEEE 802.1 bridges and end stations. This ensures a low-latency, high-reliability, bandwidth-isolated, and zero-blocking-loss network, with deterministic transmission time for Ethernet Local Area Networks (LANs). TSNs offer an enhanced functional implementation of traditional Ethernet in specific application scenarios such as automotive. Additionally, TSNs enable a single physical network for applications with different communication requirements, thus providing multiple network solutions [12]. Therefore, with its distinct benefits, TSN is increasingly utilized in various fields such as autonomous driving, intelligent cockpit, vehicle-to-circuit coordination, and more [13,14,15,16].

Regarding TSN technology, there have been numerous studies conducted both domestically and internationally. Scholars have focused on researching and discussing TSN’s practical application scenarios and trends [17]. Regarding delay analysis, Jahanzaib Imtiaz examined the delay of AVB in industrial real-time communication using the Priority Queuing (PQ) and Class Based Queuing (CBQ) methods [18]. Dorin Maxim studied issues related to upper bounds on the delay in AVB data streams and the transmission delay of different types of CDT time slots [19]. Sivakumar Thangamuthu conducted an analysis and comparison of the worst-case end-to-end delay and the impact on AVB traffic in TSN networks for three shapers: Burst Limiting Shaper (BLS), Time Aware Shaper (TAS), and Peristaltic Shaper (PS) [20]. Daniel Thiele thoroughly examined the worst-case delay in TSN networks using the TAS and PS mechanisms. This analysis encompasses the impact of various service types as well as the impact within the same services [21]. Luxi Zhao and Paul Pop proposed a network calculus-based approach to calculate the worst delay value for AVB data streams in TSN networks. They conducted an analysis and comparison of the upper bounds of delay in two different cases: with and without the frame preemption mechanism [22]. Ehsan Mohammadpour analyzed the worst end-to-end delay for reservation class traffic in TSN networks, focusing on the CBS and asynchronous traffic shaping mechanisms [23]. Yuefei Wang proposed a hierarchical scheduling mechanism and performed an analysis of the worst end-to-end delay for reservation class traffic. This analysis takes into account the influence of FIFO (First In First Out), CBS (Credit-Based Shaper), and TAS mechanisms [24]. Hao-Liang Xu put forward a frame preemption delay characterization model utilizing a hybrid probability model. Additionally, the impact of delay influencing factors on the component weights of the hybrid probability model is analyzed through simulation [25].

TSN guarantees real-time control of data traffic based on AVB, but it can impact the transmission delay of reservation class data within the network. Currently, there is ample literature on the time delay of reservation class traffic in TSNs, but much of it focuses on extreme cases, considering all possible factors that may disrupt reservation class traffic. In vehicular TSN networks, on the other hand, due to their unpredictability, it is difficult to accurately model or analyze the delay of reserved class data under extreme conditions. Therefore, it is essential to account for traffic scheduling randomness, develop a suitable set of delay analysis models, and conduct further research on the transmission delay of reservation class data.

This paper aims to analyze the data flow scheduling mechanism and algorithms within TSN networks. Due to non-deterministic scheduling challenges, this paper proposes an intelligent SARSA reinforcement learning algorithm for delay analysis of reservation class data in vehicular TSN networks, which will provide a more accurate simulation and analysis of TSN network.

2. TSN Data Queue Forwarding Latency Modeling

2.1. Analysis of Mathematical Models

The TAS scheduling mechanism guarantees the real-time transmission of Control Data Traffic (CDT) while non-CDT traffic is subject to the First-In-First-Out (FIFO), Credit-Based Shaper (CBS), and Time Aware Shaper (TAS) mechanisms, which can contribute to significant latency. Consequently, the primary factors influencing queue delays in non-CDT traffic include FIFO delay, CBS delay, TAS delay, and data transmission delay [26].

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}R=R_{FIFO}+R_{CBS}+R_{TAS}+R_T\end{array}\end{array}\end{array}\end{array}$$

(1)

where $R$ is the queue forwarding delay for data frames, $R_{FIFO}$ is the $FIFO$ delay, $R_{CBS}$ is the $CBS$ delay, $R_{TAS}$ is the $TAS$ delay, and $R_T$ is the data frame forwarding delay.

2.1.1. Mathematical Modeling of FIFO Delay

The FIFO delay formula [27] in equation is shown below:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{FIFO}=\frac{M_j}{C}\end{array}\end{array}\end{array}\end{array}$$

(2)

where $C$ is the transmission rate and $M_j$ is the frame length of the same type of data interference frame.

Taking into account the impact of the CBS algorithm [28] on the Stream Reservation (SR) class data flow, the aforementioned equation requires correction. As illustrated in Figure 1, when multiple SR class frames with the same priority enter the forwarding queue, the second SR class frame must wait for the credit value of the first SR class frame to reach a non-negative value from a negative value after completion of transmission, before it can initiate transmission. The recovery time is determined by the ratio of the absolute value of the minimum credit value to the credit value growth rate. Considering the CBS algorithm, the modified FIFO delay is as follows:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{FIFO}=\frac{M_j}{C}+\left|\frac{loCredit}{idleSlope}\right|=\frac{M_j}{C}\left(1+\frac{sendSlope}{idleSlope}\right)\end{array}\end{array}\end{array}\end{array}$$

(3)

where $loCredit$ is the rate of credit accumulation, $idleSlope$ is the rate of credit accumulation, and $sendSlope$ is the rate of credit consumption.

2.1.2. Mathematical Model of CBS Delay

As depicted in Figure 2, the CBS algorithm classifies data streams into three categories: L (low) priority, M (medium) priority, and H (high) priority, which correspond to BE, SR_B, and SR_A, respectively. During the transmission of a BE frame, both a Class B frame (B₁) and a Class A frame (A₁) are simultaneously added to the queue [29]. Therefore, the CBS delay can be calculated as follows:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{CBS}=t_{LPB}+t_{HPB}\end{array}\end{array}\end{array}\end{array}$$

(4)

where $t_{LPB}$ is the L-level data stream delay, and $t_{HPB}$ is the H-level data stream delay.

2.1.3. Mathematical Modeling of TAS Delay

As illustrated in Figure 3, the operation cannot be executed when the target data frame arrives as the interfering frame is being transmitted. Once the transmission is finished, the interfering frame is inserted into the protection window, during which non-CDT data frames cannot be transmitted. Therefore, the time delay caused by the TAS mechanism is calculated as follows:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{TAS}=L_P=L_{CDT}+L_{GB}\end{array}\end{array}\end{array}\end{array}$$

(5)

where $L_P$ is the frame length of the protection window, $L_{CDT}$ is the frame length of the $CDT$ time slot, and $L_{GB}$ is the frame length of the protection band.

2.2. Delay Modeling

2.2.1. Class A Data Delay Modeling

Following the aforementioned analysis, the delay of Class A data comprises four types of time delays, namely FIFO, CBS, TAS, and forwarding delay [30], i.e.,

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_A=R_{FIF{O}_A}+R_{CB{S}_A}+R_{TA{S}_A}+R_{T_A}\end{array}\end{array}\end{array}\end{array}$$

(6)

In accordance with the FIFO algorithm, Class A data frames cannot be transmitted when they enter the queue due to the presence of interfering frames. These interfering frames are transmitted with unknown conditions and varying delays. As a result, the unfinished transmission portion can be regarded as an equivalent frame. The FIFO delay of a Class A data frame is represented by the following equation:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{{FIFO}_A}=\frac{{M^{\prime }}_{\tau }}{C}\left(1+\frac{{sendSlope}_A}{{idleSlope}_A}\right)\end{array}\end{array}\end{array}\end{array}$$

(7)

where: ${sendSlope}_A$ is the rate of credit decline for Class A, ${idleSlope}_A$ is the rate of credit growth for Class A, and ${M^{\prime }}_{\tau }$ is the equivalent frame length.

Within the context of the CBS algorithm, time delays can be classified into two categories, namely low and high priority traffic blocking delays. As Class A data frames are assigned the highest priority, only low-priority delays may occur [31]. If the transmission of a low-priority data frame disrupts the transmission of a Class A data frame, then the completion of the latter is uncertain, and the length of the unsent segment can be regarded as the equivalent frame length. Therefore, the CBS delay for a Class A data frame is expressed as follows:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{{CBS}_A}={t^A}_{LPB}=\frac{{M^{\prime }}_L}{C},{M^{\prime }}_L={M^{\prime }}_B or {M^{\prime }}_{BE}\end{array}\end{array}\end{array}\end{array}$$

(8)

where ${t^A}_{LPB}$ is the L-class blocking delay for Class A frames under the CBS algorithm, and ${M^{\prime }}_L$ is the equivalent frame length for L-class interference frame blocking under the CBS algorithm.

Within the TAS mechanism, the portion of the protection window that cannot be transmitted due to the arrival of Class A data can be considered as the equivalent length. Consequently, the TAS delay for a Class A data frame is expressed as follows:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{{TAS}_A}={L^{\prime }}_P\end{array}\end{array}\end{array}\end{array}$$

(9)

where ${L^{\prime }}_P$ is the equivalent length.

In summary, the Class A data delay is equal to the following:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_A=R_{FIF{O}_A}+R_{CB{S}_A}+R_{TA{S}_A}+R_{T_A}\\ =\frac{{M^{\prime }}_{\tau }}{C}\left(1+\frac{{sendSlope}_A}{{idleSlope}_A}\right)+\frac{{M^{\prime }}_L}{C}+{L^{\prime }}_P+\frac{M_A}{C}\\ =\frac{M_A+{M^{\prime }}_L}{C}+\frac{{M^{\prime }}_{\tau }}{C}\left(1+\frac{{sendSlope}_A}{{idleSlope}_A}\right)+{L^{\prime }}_P\end{array}\end{array}\end{array}\end{array}$$

(10)

where $M_A$ is the frame length of the Class A target data frame.

2.2.2. Class B Data Delay Modeling

Similarly, the queue forwarding delay expression for Class B data frames is

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_B=R_{FIF{O}_B}+R_{CB{S}_B}+R_{TA{S}_B}+R_{T_B}\end{array}\end{array}\end{array}\end{array}$$

(11)

The expression for the queue forwarding delay under the FIFO algorithm is not repeated as it follows the same principle as that of Class A data frames.

In the CBS algorithm, Class B data frames have a lower priority compared to Class A and a higher priority compared to Class BE. This leads to the coexistence of traffic blocking delays with smaller priorities and higher traffic blocking delays. The completion of transmission for Class BE interfering frames is uncertain, and the remaining portion can be designated as the equivalent length. Hence, the CBS delay for Class B interference frames can be expressed as follows:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{t^B}_{LPB}=\frac{{M^{\prime }}_{BE}}{C}\end{array}\end{array}\end{array}\end{array}$$

(12)

where ${t^B}_{LPB}$ is the L-level blocking delay of Class B data frames under the CBS algorithm, and ${M^{\prime }}_{BE}$ is the equivalent frame length of the blocking of Class BE interference frames under the CBS algorithm.

Similarly, the high-priority traffic blocking for Class B data frames can be obtained as shown in the following equation:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{t^B}_{HPB}=R_A-{t^A}_{LPB}-R_{{TAS}_A}=R_{FIF{O}_A}+R_{T_A}\end{array}\end{array}\end{array}\end{array}$$

(13)

where ${t^B}_{HPB}$ is the H-level blocking delay for Class B data frames under the CBS algorithm.

So, the CBS delay for Class B data frames is

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{{CBS}_B}={t^B}_{LPB}+{t^B}_{HPB}=\frac{{M^{\prime }}_{BE}}{C}+R_{{FIFO}_A}+R_{T_A}\end{array}\end{array}\end{array}\end{array}$$

(14)

The Class B delay under the TAS algorithm can be coterminous with Class A, and therefore will not be repeated, i.e.,

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{{TAS}_B}={L^{\prime }}_P\end{array}\end{array}\end{array}\end{array}$$

(15)

In summary, the Class B data delay is equal to the following:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_B=R_{{FIFO}_B}+R_{{CBS}_B}+R_{{TAS}_B}+R_{T_B}\\ =\frac{{M^{\prime }}_{\tau }}{C}\left(1+\frac{{sendSlope}_B}{{idleSlope}_B}\right)+\frac{{M^{\prime }}_{BE}}{C}+{L^{\prime }}_P+\frac{M_B}{C}+R_{{FIFO}_A}+R_{T_A}\\ =\frac{M_B+{M^{\prime }}_{BE}}{C}+\frac{{M^{\prime }}_{\tau }}{C}\left(1+\frac{{sendSlope}_B}{{idleSlope}_B}\right)+R_{{FIFO}_A}+R_{T_A}+{L^{\prime }}_P\end{array}\end{array}\end{array}\end{array}$$

(16)

where $M_B$ is the frame length of the Class B target data frame.

2.2.3. Modeling of Latency for BE Class Data

Similarly for Class A and B, the queue forwarding delay for Class BE data frames can be expressed as

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{BE}=R_{FIF{O}_{BE}}+R_{CB{S}_{BE}}+R_{TA{S}_{BE}}+R_{T_{BE}}\end{array}\end{array}\end{array}\end{array}$$

(17)

Under the FIFO algorithm, the Class BE is congruent to the Class A and B. Since there is no credit value for the Class BE data stream, the FIFO delay of the Class BE data frame is shown in the following equation:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{FIF{O}_{BE}}=\frac{{M^{\prime }}_{\tau }}{C}\end{array}\end{array}\end{array}\end{array}$$

(18)

Under the CBS algorithm, only a high-priority traffic blocking delay exists for Class BE frames. If a Class BE data frame is blocked by a high-priority interfering frame, it is uncertain how much of the frame has been transmitted, and the remaining portion can be designated as the equivalent frame length. The CBS delay of a Class BE data frame can be expressed using the following equation:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{{CBS}_{BE}}={t^{BE}}_{LPB}=\frac{{M^{\prime }}_H}{C},{M^{\prime }}_H={M^{\prime }}_A or {M^{\prime }}_B\end{array}\end{array}\end{array}\end{array}$$

(19)

where ${t^{BE}}_{LPB}$ is the H-level blocking delay of the BE class data frame under the CBS algorithm, and ${M^{\prime }}_H$ is the blocking equivalent length of the class H interfering frame under the CBS algorithm.

Under the TAS algorithm, the treatment of Class BE frames is similar to that of Classes A and B. Thus, the TAS delay for Class BE data frames can be expressed using the following equation:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{{TAS}_{BE}}={L^{\prime }}_P\end{array}\end{array}\end{array}\end{array}$$

(20)

In summary, the queue forwarding delay of a BE class data frame can be expressed as

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{R}_{BE}=R_{FIF{O}_{BE}}+R_{CB{S}_{BE}}+R_{TA{S}_{BE}}+R_{T_{BE}}\\ =\frac{{M^{\prime }}_{\tau }}{C}+\frac{{M^{\prime }}_H}{C}+{L^{\prime }}_P+\frac{M_{BE}}{C}\\ =\frac{M_{BE}+{M^{\prime }}_H+{M^{\prime }}_{\tau }}{C}+{L^{\prime }}_P\end{array}\end{array}\end{array}\end{array}$$

(21)

3. Forwarding Strategy Model Optimization

3.1. SARSA Reinforcement Learning Theory

SARSA reinforcement learning is a network synchronous learning method based on the temporal difference algorithm [32,33,34,35]. This approach updates state–action values using feedback information from the environment, without requiring knowledge of the state transfer, environmental model, or behavioral strategy. Consequently, the convergence speed of the algorithm can be significantly accelerated. The algorithmic procedure for SARSA reinforcement learning is as follows:

(1)

Initialize the Q-value table.

(2)

Repeat the following steps: In states, use the greedy algorithm to select action a based on the Q-value table. Obtain the reward r and transition to the next states. Update the Q-value table using the iterative equation.

(3)

Output the optimal policy.

The selection of action a and the iterative formula for updating the Q-value table are crucial components of the entire process. The ε-greedy algorithm is the strategy employed by the SARSA algorithm for selecting actions. This strategy allows the intelligent agent to randomly choose an action with a probability of ε, and choose the current optimal action with a probability of 1 − ε, aiming to achieve the global optimum. The specific algorithm is as follows:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{\pi }^{\epsilon }\left(s,a\right)=\left\{\begin{array}{c}\pi \left(s,a\right) ,1-\epsilon \\ random from A ,\epsilon \end{array}\right.\end{array}\end{array}\end{array}\end{array}$$

(22)

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}\pi \left(s,a\right)=argmaxQ\left(s,a\right)\end{array}\end{array}\end{array}\end{array}$$

(23)

As the SARSA algorithm updates the Q-value table based on the next actual action taken, the iterative equation for the Q-value table is as follows:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}Q\left(s,a\right)=Q\left(s,a\right)+\alpha \left[r+\gamma Q\left(s^{\prime },a^{\prime }\right)-Q\left(s,a\right)\right]\end{array}\end{array}\end{array}\end{array}$$

(24)

where $\alpha$ is the learning rate and $\gamma$ is the reward discount.

3.2. Q-Table Determination

The action space A of the model can be quantized as $A=[\mathrm{1,2},\mathrm{3,4},\mathrm{5,6}]$, where 1–6 are idle, entering protection window, transmitting Class A frames, transmitting Class B frames, transmitting Class BE frames, and maintaining the current transmission action [36], respectively.

The states of each type of data flow are taken as state quantities in the state space S. The state space S is as follows:

$$\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}S=\left[\begin{array}{ccc}{Credit}_P& {Amount}_P& {Transmit}_P\\ {Credit}_A& {Amount}_A& {Transmit}_A\\ {Credit}_B& {Amount}_B& {Transmit}_B\\ {Credit}_{BE}& {Amount}_{BE}& {Transmit}_{BE}\end{array}\right]\end{array}\end{array}\end{array}\end{array}$$

(25)

The credit value of Class A and B data streams is an indefinite value, while Class BE data streams and protection windows do not have credit values. However, in order to maintain consistency, the credit values of Class BE data streams and protection windows are consistently set to 0, thus establishing the state space, i.e., ${Credit}_{BE}\equiv 0$, and ${Credit}_p\equiv 0$. The state space S is partitioned according to the number of states in the state space, and the results are shown in

Table 1. Status classification table.

s	AP	AA	CA	AB	CB	ABE	a
1	≠0	random	random	random	random	random	2
2	=0	≠0	≥0	≠0	≥0	≠0	3
3	=0	≠0	≥0	random	<0	≠0	3
6	=0	≠0	≥0	≠0	≥0	=0	3
7	=0	≠0	≥0	=0	≥0	=0	3
8	=0	≠0	<0	≠0	≥0	≠0	4
25	=0	=0	<0	≠0	≥0	=0	4
26	=0	=0	<0	random	<0	=0	1
27	=0	=0	<0	=0	≥0	=0	1
	TP		TA	TB		TBE
28	0		0	0		1	6
29	0		0	1		0	6
30	0		1	0		0	6
31	1		0	0		0	6

.

In order to facilitate the analysis, the state S is divided according to the relationship of each variable with 0, and then the similar results are combined, and the results obtained are shown in Table 1.

Each state is assigned a reward value when it performs an action, with the highest reward value given to the best action. The maximum reward is set to 10, while the remaining actions are assigned a reward value of 0.

After performing each action, the state of the environment is updated thus iterating the Q-table, thus allowing the intelligent body to keep performing actions to accomplish learning. In this paper, $\alpha =0.1$ and $\gamma =0.9$ are set to obtain the final Q-table.

3.3. Calculation of Time Delays

After initializing the Q-value table and states, the greedy algorithm filters and executes the action a, followed by updating the state space parameters and calculating the time delay value to update the states. This process is cyclically executed until $t<\mathrm{10,000,000}$. Afterward, the average time delay value is calculated for each type of data. The computational flowchart of SARSA reinforcement learning is illustrated in Figure 4.

4. Simulation and Result Analysis

The data flow at the gateway of an in-vehicle TSN network usually belongs to the periodic traffic that helps to calculate the delay value. The delay values of different types of data frames are calculated so as to verify the reasonableness of the model proposed in this paper.

Referring to previous studies [37], the total link bandwidth is set to 100 Mbit/s, each gating period is set to 500 μs, and the non-CDT transmission bandwidth is set to 90 Mbit/s. where the CDT time slot length and the non-CDT time slot length are set to 150 μs and 350 μs, respectively. the lengths of the data frames of Class A, B, and BE are all set to 400 bytes. The Class A data frame period and Class B data frame period are set to 125 μs and 250 μs, respectively, and the reserved bandwidth of the data stream is set according to the actual situation. The reserved bandwidths for the Class A and Class B data streams are 400 Mbit/s and 20 Mbit/s, respectively.

The corresponding program is run in MATLAB-R2020b and the results are shown in Figure 5, Figure 6 and Figure 7. Firstly, we compare and analyze the mean values of the delay of the data frame group of the SR class data frames under AVB and TSN networks, respectively, so that the parameters of the SR class data frames remain unchanged. Figure 5 shows the delay averages of Class A data frames in both TSN and AVB network environments when the BE load is 5 Mbit/s. It can be seen that although the protection window in TSN guarantees the transmission delay of CDT data frames, it has a great impact on the queue forwarding of SR class data frames, which is in line with the theoretical situation.

Secondly, the data delay of the two SR class data frames is compared and analyzed in the TSN network. Figure 6 shows the comparison of the average delay of traffic for Class A data frames and Class B data frames when the BE class load is 10 Mbit/s. The results are 110.93 μs and 181.35 μs, respectively, and also the discrete situation of Class A data frames and Class B data frames can be seen from the figure, i.e., the Class A mean square deviation is smaller than that of Class B. The results show that the transmission real time of Class A data frames is prioritized higher in TSN networks, i.e., Class A traffic is prioritized higher than Class B, which is in line with the previous data type analysis.

Next, the BE data load is analyzed to see if there is any significant effect on the delay of SR class data. As shown in Figure 7, Class A data frames are taken as the object of study, and the delay values of Class A data frames under different BE data frame loads are simulated and analyzed. It can be seen that the transmission delay of Class A data frames increases significantly with increasing load.

In summary, the TSN data forwarding delay analysis model developed in this paper matches the theoretical situation, and the fitting performance for the above scenarios is also better, thus verifying the reasonableness of the model.

5. Test Validation

The simulation testbed consists of RAD_Galaxy, gateway, surround view system and PC as shown in Figure 8. The video data are captured by the four cameras of the surround view system, and the data are transmitted to the gateway at the same time. RAD_Galaxy acts as an intermediary device connecting the gateway to the PC and the monitor, and also acts as a sending node for the audio data and control data. This allows the data messages at the gateway to be monitored at the PC, thus realizing a semi-physical simulation; the test bench is shown in Figure 9.

The experiment concludes by comparing the theoretical delay, obtained through simulation in MATLAB, with the actual measured delay values on the experimental bench. This comparison aims to verify the reasonableness of the TSN delay analysis model developed in this paper.

Due to the large variance of the individual time delay data of the measured samples, the average of the time delays was used as a comparison reference, with 40 data frames per subgroup. The results of the time delay comparison are shown in Figure 10 and Figure 11. The results show that the delay values obtained from both model predictions and experimental measurements increase with the increase in Class BE data load, and the deviation values of both have a small increase. The theoretical and actual measured delay values for Class A and Class B data frames are shown in

Table 2. Theoretical and measured delay results for Class A data.

	Class A (Mbps)	Class B (Mbps)	Class BE (Mbps)	Theoretical Delay (μs)	Measured Delay (μs)	Deviation (%)
1	50	25	10	110.93	109.35	−1.44
2	50	25	15	114.47	115.91	1.24
3	50	25	20	118.60	122.76	3.39
4	50	25	25	122.07	128.43	4.95

and

Table 3. Theoretical and measured delay results for Class B data.

	Class A (Mbps)	Class B (Mbps)	Class BE (Mbps)	Theoretical Delay (μs)	Measured Delay (μs)	Deviation (%)
1	50	25	10	181.35	183.15	0.98
2	50	25	15	185.81	187.97	1.15
3	50	25	20	189.94	193.61	1.90
4	50	25	25	194.03	198.38	2.19

. The deviation of both the theoretical and measured values of the reserved class data transmission delay is less than 5% for different BE data loads, with a maximum deviation of 4.64%.

Furthermore, in order to validate the feasibility of the method,

Table 4. Delay results based on Q reinforcement learning.

	Class A (Mbps)	Class B (Mbps)	Class BE (Mbps)	Theoretical Delay (μs)	Measured Delay (μs)	Deviation (%)
1	50	25	10	177.84	181.36	1.94
2	50	25	15	179.48	183.95	2.43
3	50	25	20	183.40	189.34	3.14
4	50	25	25	187.64	196.26	4.39

is included, which presents a comparison of delay results using the Q-reinforcement learning approach. Through this comparative test, it becomes apparent that the delay analysis model put forth in this paper successfully captures the delay characteristics of reservation classes in TSN networks.

6. Conclusions

In this paper, we propose a TSN data forwarding delay model using the SARSA reinforcement learning algorithm for simulation studies. Building upon the FIFO, CBS, and TAS delay models, we establish delay models for both Class A and Class B data. To optimize the queue forwarding delay model, we utilize the SARSA reinforcement learning algorithm. We then verify the validity of our simulation model through simulations and physical object experiments, ensuring its accuracy and reliability.