Bounding the Upper Delays of the Tactile Internet Using Deterministic Network Calculus

Abstract

With the increasing popularity of time-sensitive network applications and the gradual integration of the Tactile Internet into people’s lives, how to ensure ultra-low latency has become a demand and challenge for network performance. Therefore, it is extremely important to analyze the performance of the Tactile Internet. In this paper, we propose an analytical model based on deterministic network calculus (DNC) to quantitatively derive the end-to-end performance bounds of the Tactile Internet, develop a tandem model describing the communication of the Tactile Internet network, and analyze delay-related traffic parameters, such as arrival rate and burst size. We investigate the variation of the accuracy of the DNC analytical model and the measurement model under different parameters, and verify the accuracy of the proposed DNC analytical model by theoretical derivation and analysis and comparison with the measurement model under the NS3 platform. We discuss the impact of relevant parameters on the delay boundaries to determine which network configuration enables the end-to-end delay to meet the established requirements. This will provide valuable guidance for the design of Tactile Internet architectures.

1. Introduction

Nowadays, the Internet has pervaded all aspects of people’s lives, and traditional network architectures have struggled to deliver satisfactory network performance. However, in some emerging areas such as self-driving cars, virtual reality, augmented reality, health-care 4.0 and industry 4.0, there are more stringent requirements for quality of service (QoS) such as low latency and high reliability, which pose further challenges for network performance.

For the challenges facing the network in the new era, the concept of Tactile Internet is proposed [1]. The IEEE P1918.1 Working Group defines the Tactile Internet as a network, or a network of networks, for remotely accessing, sensing, manipulating, or controlling real and virtual objects or processes that are sensed in real time [2]. To provide a medium for delivering control, touch and sensing and driving information in real time, the Tactile Internet requires highly reliable, responsive and intelligent connectivity. The envisioned Tactile Internet requires a suitable architecture to meet its design requirements, and it needs to provide extremely low round-trip latency of 1 ms or less [3]. If the delay in tactile information exceeds 1 ms, it can lead to unwanted effects such as cyber sickness, whose symptoms are comparable to motion sickness [1]. For example, in virtual reality, if the time difference between the virtual picture and human motion is more than 1 ms, “halo screen” may occur, which leads to user dizziness [3]. Therefore, end-to-end delay will be a key factor in the performance of the Tactile Internet architecture, and worst-case delay performance should be considered during the design phase of the Tactile Internet.

Providing such stringent QoS requires network modeling and performance analysis as an integral part, in addition to key technologies such as 5G that can guarantee ultra-reliable and low-latency communication for the Tactile Internet [4]. For the low-latency scenarios required by the Tactile Internet, we can estimate the performance metrics of the Tactile Internet in advance through network modeling, and then turn them back to guide the resources that need to be provided, which is very helpful for the design of the Tactile Internet architecture.

Network modeling can be divided into measurement models, analytical models and machine learning models [5]. The measurement model is a simulation model for studying network systems built by various network simulators, such as NS3 and OMNeT++. We usually run this model on a computer and then analyze the output results. Measurement models require accurate simulation of network systems, which requires higher costs on large network architectures. Analytical models describe the research object and system according to certain qualifications and reasonable assumptions, abstract the mathematical analytical model of the research object, and use the mathematical analytical model to solve the problem. However, when a system is complex, it becomes difficult to describe the system accurately with some restrictive assumptions. Machine learning models can be helpful when analytical models are challenged by system complexity [5,6]. Machine learning is able to model the relationship between output variables (e.g., latency) and one or more independent input variables (e.g., software and hardware parameters) [7], and this approach is not limited by system complexity and often achieves good results, but it also lacks a reasonable explanation for network models, and for the interaction aspects of system parameters.

As far as we know, it is rare to see measurement models and machine learning models for performance analysis of Tactile Internet. The queuing models [4,8,9] are commonly used to analyze the network; however, the queuing models on the one hand analyze the average delay, which cannot meet the strict requirements of the Tactile Internet for the worst-case delay, on the other hand queuing models use a random traffic model, which may not match the realistic traffic characteristics [10]. Most importantly, the above models are difficult to make accurate predictions under the performance requirement of 1 ms, especially in the case of Tactile Internet, which is extremely sensitive to delay. While deterministic network calculus (DNC) shows great potential in dealing with these aspects, it is a system theory for studying communication networks [11], specifically calculating worst-case bounds for delay requirements in networks. It focuses on worst-case performance guarantees and is considered as a key technology for future Internet performance analysis. Worst-case delay is the most important issue for Tactile Internet service scenarios, and these services are critical for applications such as remote surgery and smart driving. Considering that the analytical model can simplify the network environment and relies on qualifying conditions and assumptions, we combine the analytical model with the measurement model and verify the results obtained from the analytical model by the measurement model to prove the rationality and accuracy of the proposed analytical model.

Curve-based system models such as network calculus, real-time calculus (RTC) and compositional performance analysis (CPA) are the basis of modern analytical frameworks [12]. Network calculus is compositional in the sense that the system under analysis is decomposed into a set of components connected by streams. Each component is analyzed individually by computing for given curves on the input streams the respective curves bounding the output streams (local analysis). RTC is an adaptation of NC to real-time systems focusing, for instance, on domain-specific scheduling problems. Despite notational differences, the underlying models are considered as equivalent [13]. CPA is a compositional analysis framework for distributed real-time systems which has been developed independently from NC and RTC [14].

Compared with traditional real-time applications, emerging time-sensitive network (TSN) applications have more stringent requirements for latency, and there is a more urgent need for network calculus. Many studies (such as [15,16,17,18,19]) have used network calculus to analyze the performance of TSN, especially those with strict requirements on latency. Therefore, in this paper, we use DNC to propose a method for modeling end-to-end delay upper bounds for the Tactile Internet.

The main contributions of this paper are as follows:

• We establish a network domain analytical model of Tactile Internet based on DNC and derive the delay bound of single-flow with single-switch, multi-flow with single-switch and multi-flow with multi-switch.
• We carry out extensive simulation experiments on measurement models for different network configurations, combine the theoretical model with the measurement model, verify the accuracy of the proposed analytical model, and compare it with existing CPA model.
• We perform numerical analysis of the model according to different parameters to evaluate the effect of the number of switches and traffic competition on Tactile Internet latency.

The organization of this paper is summarized as follows. Related works are presented in Section 2. Section 3 introduces the Tactile Internet end-to-end architecture and the basic theory of DNC. Section 4 gives a network model of Tactile Internet based on DNC. Section 5 performs the derivation of delay bounds for the model. Section 6 obtains simulation results by computing the model experimentally, compares them with the results derived in Section 5, and performs a comprehensive performance analysis based on the proposed model. Finally, Section 7 concludes this paper.

2. Related Works

In 2014, the Telecommunication Standardization Sector of the International Telecommunication Union published a report on the Tactile Internet [1], which outlined the potential of the Tactile Internet, explored its prospects in various application areas, described the needs of the Tactile Internet for future digital infrastructure and its expected impact on society, and included the Tactile Internet in the Technology Watch area. Since the release of the report, the Tactile Internet has grown rapidly in various fields. However, ultra-low latency, the most important feature of the Tactile Internet, meeting 1 ms end-to-end delay has become a major challenge for the Tactile Internet.

There are many analytical studies of Tactile Internet performance. She et al. [4] analyzed the queuing delay for vehicular communication using a queuing model, and since the queuing dynamics in medium access control was affected by the randomness of the time-varying channel and the random geometry of the physical layer, it was very difficult to make the instantaneous queuing delay bounded by a deterministic value, so the focus was on the average delay constraint, which had limited application in future low-latency services. Maier et al. [20] developed an analytical framework to calculate the average end-to-end delay and its distribution for both local and non-local remote operations in the presence of human-to-machine traffic in a telemedicine setting, discussing different arrival models including deterministic, gamma, generalize Pareto, and Poisson distributions, but did not combine these arrival models with a service model as a way to obtain worst-case delay constraints. Kim et al. [21] proposed a new performance evaluation method to evaluate strategies to assess the delay constraints and reliability constraints of base stations and users in the Gangnam station area in Seoul, Korea, which combined ray tracing tools with system-level simulation, but the method had certain requirements for specific environments and was difficult to apply to other specific scenarios. Aijaz et al. [22] used a 5G design based on orthogonal frequency division multiplexing technique to ensure that the air interface transmission budget was limited to 100 μs in a factory automation use case with a delay requirement of 1 ms, focusing on radio resource allocation, but they only considered queuing delays without transmission delays and network delays.

For various application scenarios of Tactile Internet, it is important to reduce delay and it is also important to classify traffic based on traffic characteristics and network device conditions. However, there is a lack of classification consideration for the complex network environment of Tactile Internet in the above work, and the methods in the above work either focus on average delay or have limited capability for latency constraints, while the DNC analyzes the worst-case delay.

DNC has been used in the past to model different communication networks. Georges et al. [23] demonstrated the application of DNC in switched real-time networks. Yang et al. [24] obtained strict delay bounds for avionics full duplex switched networks with realistic traffic patterns and network workloads based on DNC. Zhang et al. [25] constructed a framework for analyzing the stochastic end-to-end delay of long term evolution (LTE) networks by modeling the arrival and service models of LTE networks using stochastic network calculus. De et al. [26] build a model for calculating worst-case delays in a critical embedded system environment for audio/video bridging based on DNC. Geyer et al. [27] demonstrated the use of DNC to model worst-case delays in communication networks used in the automotive and aerospace industries. Duan [28] used network calculus for service function chain (SFC) modeling and performance analysis in software defined network (SDN)/network function virtualization (NFV) to determine the end-to-end delay boundaries that SFC can guarantee to their end-users based on their service capacity and load characteristics, applicable to dynamic elastic service provisioning for various SFC in SDN/NFV. Hu et al. [29] computed delay bounds for asynchronous traffic shaping hops with inconsistent parameters in time-sensitive networks based on DNC. Ren et al. [30] derived an end-to-end delay model and a backlog model for fog server nodes and fog nodes using DNC theory.

DNC analysis uses network information such as network topology, network forwarding capability and flow path to determine the worst-case delay. With the increasing delay requirements of time-sensitive network applications, DNC has been widely used in the above network scenarios, and has achieved good results. In this paper, we use DNC to model and analyze the performance of Tactile Internet and obtain the end-to-end delay bound. The experimental results show that our model can be applied to various application scenarios in Tactile Internet.

3. Background

In this section, we first give the end-to-end architecture of the Tactile Internet, using tele-surgery as an example, and describe the management and implementation of the Tactile Internet. Then, we review the basic idea of DNC.

3.1. Description of the Tactile Internet Architecture

While the traditional Internet transmits text, audio and video, giving people an auditory and visual experience, the Tactile Internet will transmit tactile and driving information on top of the traditional Internet, providing haptic feedback and giving people a real tactile experience.

The Tactile Internet relies on haptic and non-haptic control to communicate between end-to-end operations [3]. The difference between haptic and non-haptic control is that in addition to visual and auditory feedback for haptic control, the system actually has a haptic feedback (kinesthetic or vibrotactile), resulting in a global control loop. The feedback of non-tactile control can only be auditory or visual, so there is no concept of control loop [31].

As shown in Figure 1, the end-to-end architecture of the Tactile Internet is represented by three main parts: the master domain, the network domain and the slave domain. In order to properly evaluate the performance of Tactile Internet, most related studies rely mainly on simulation-based evaluation [32]. It is necessary to abstract the architecture of the Tactile Internet model.

3.1.1. Master Domain

The master domain usually consists of human operator (HO) and human-system interface (HSI). The HSI is a tactile device that serves to convert human control into a tactile input through coding technology, and through command signals, the operator can achieve operation of the slave domain. In some applications, a master domain may consist of multiple operators operating a single slave domain in concert. Such HSI haptic devices are available from vendors such as iFeelPixel Association, Immersion Corporation, Neurodigital Technologies, Sensable and Geomagic [33], they are usually designed in the form of a linkage, which includes a mechanical arm attached to the stylus that moves with the stylus and applies a certain force at its tip. The main entities that connect the master to the network domain are the Network Controller (NC) and the Gateway Node (GN). We can configure them at an edge, collectively called “GNC” [34]. To realize the vision of the Tactile Internet, further improvements in tactile devices are needed, especially in terms of increasing degrees of freedom (DoF) to meet the demand and incorporating network interfaces to ensure communication with the core network.

3.1.2. Network Domain

The network domain provides bi-directional communication between the master and slave domains, enabling the combination of HO and remote environments. Tactile command signals sent from the master domain will pass through routers, switches, gateways, base stations, access points, and servers and reach the tactile support engine in the slave domain, thus enabling control of remote machines. Conversely, the haptic feedback signal is sent from the domain, goes through the above series of transmissions, and finally reaches the master domain, giving the human operator the corresponding haptic feedback. Thus, in the Tactile Internet, the human operator and the remote machine maintain this tactile communication through command signals and feedback signals in the network domain.

3.1.3. Slave Domain

The slave domain consists of a remote operator console (robot) and operates accordingly by receiving command signals sent from the master domain, after which the tactile data is fed back to the human operator console, thus achieving real-time interaction and forming a global control loop.

3.2. Overview of DNC

DNC is a system theory in communication networks [35]. In contrast to traditional queuing theory models, DNC is a theoretical framework that uses minimum plus $(min,+)$ algebra and maximum plus $(max,+)$ algebra to analyze computer network performance and obtain worst-case bounds on delay and buffer queue length in the network. The DNC solves network performance boundary problems using two basic tools: arrival curves, which represent the upper bound on data arriving at the system, and service curves, which represent the minimum service the system is guaranteed to provide. A system S can be a buffer, a simple network node, or a complex network.

3.2.1. Arrive Curve

The cumulative function $R\left(t\right)$ represents the sum of the data volume in the time interval $[0,t]$. When $t=0$, $R\left(0\right)=0$ and $R\left(t\right)$ is a generalized incremental function. Then, define the arrival curve of the data flow F as:

$$R\left(t\right)-R\left(s\right)\le \alpha (t-s)\forall t\ge s\ge 0.$$

(1)

It can also be combined with the $(min,+)$ convolution formula expressed as:

$$R\le R\otimes \alpha .$$

(2)

If the arrival curves of the two flows ($F_1$ and $F_2$) are ${\alpha }_1$ and ${\alpha }_2$, respectively, the arrival curve of their convergent flow is $({\alpha }_1+{\alpha }_2)$.

The leaky bucket model is the most commonly used arrival curve model, and its expression is ${\alpha }_{r,b}=rt+b$, where r denotes the average rate and b denotes the burst size.

3.2.2. Service Curve

Suppose a system S is given an input function of $R\left(t\right)$ and an output function of $R^{\ast }\left(t\right)$, and we can define the service curve $\beta$ provided by system S as:

$$R^{\ast }\left(t\right)\ge \underset{s\le t}{inf}\{R\left(s\right)+\beta (t-s)\}\forall t\ge 0.$$

(3)

It can also be expressed as:

$$R^{\ast }\ge R\otimes \beta .$$

(4)

Obviously, we stipulate that $R\left(t\right)\le R^{\ast }\left(t\right)$.

The latency-rate model is the most commonly used service curve model, and its expression is

$${\beta }_{R,T}\left(t\right)=R{[t-T]}^{+}=\left\{\begin{array}{cc}R(t-T),\hfill & \mathrm{if}t>T,\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(5)

where R is the service rate, T is the delay, and $R\ge 0$, $T\ge 0$.

3.2.3. Tandem Theorem

Suppose a flow passes through systems $S_1$ and $S_2$ in turn, and the service curves provided by these two systems are ${\beta }_1$ and ${\beta }_2$, respectively, then the equivalent service curves of these two systems will be $\tilde{\beta }={\beta }_1\otimes {\beta }_2$. Without loss of generality, assume that a flow passes through n systems in sequence, the equivalent service curve for these n systems is ${\tilde{\beta }}_{sys}={\beta }_1\otimes {\beta }_2\otimes \cdots \otimes {\beta }_n$.

From [11], the equivalent service curves of two systems $S_1$ and $S_2$ with latency-rate service curves ${\beta }_{R_1,T_1}$ and ${\beta }_{R_2,T_2}$ are ${\beta }_{min(R_1,R_2),T_1+T_2}$. Therefore, the equivalent service curve extended to n systems with latency-rate service curves ${\beta }_{R_i,T_i}$ is ${\beta }_{min(R_1,\cdots ,R_n),T_1+\cdots ,T_n}$.

3.2.4. Delay Bound

From the definitions of arrival and service curves, it can be obtained that the virtual delay $d\left(t\right)$ experienced by a flow with arrival curve $\alpha$ as it passes through the node providing service curve $\beta$ is

$$d\left(t\right)\le \underset{t\ge 0}{sup}\{\underset{d\ge 0}{inf}\{d:\alpha \left(t\right)\le \beta (t+d)\}\}.$$

(6)

We can use $h(\alpha ,\beta )$ to denote the maximum horizontal deviation between the arrival curve $\alpha$ and the service curve $\beta$. So the above equation can be expressed as: $d\left(t\right)\le h(\alpha ,\beta )$.

3.3. Problem Description

Tactile Internet architecture shown in Figure 1 is very sensitive to latency. Tactile Internet vision would enable to provide remote real-time control, whether that is human-to-machine or machine-to-machine [36]. It is necessary to ensure that the latency between the master and slave domains is within a certain value. Our work is based on DNC for network modeling to obtain the latency upper bound. This in turn guides the deployment of the network infrastructure based on the given latency requirements in real-world application scenarios. The effect of the relevant parameters on the delay boundaries is discussed through DNC theory to determine which network configuration enables the end-to-end delay to meet the established requirements. This allows for a more rational configuration of network devices and efficient network planning and optimization.

There have been studies [20,37] applying artificial intelligence (AI) to mobile edge computing (MEC) servers for the Tactile Internet. The HSI on the HO side is equipped with an AI edge sample forecast (ESF) module. If a sample is lost or excessively delayed, this module is responsible for providing the HO with a predicted sample (based on previous samples) instead of waiting for a lost or delayed sample, thus allowing the operator to proceed to the next operation smoothly. In addition, this waiting time threshold needs to be set taking into account the worst-case end-to-end delays in the Tactile Internet. Thus, the DNC can be used to analyze the end-to-end delay of the Tactile Internet to obtain a theoretical upper bound on the delay, which can be set as a time threshold for starting the prediction module. This will be shown in our next study.

In the architecture of the Tactile Internet, the network domain is a key part of the implementation of tactile communication. Therefore, we only analyze the network domain transmission latency to obtain a network domain design solution for a given latency requirement. However, in the future Internet, end-to-end latency will also include latency of composite service systems combining network and cloud computing [38], which we will investigate in our future work.

4. Modeling Tactile Internet Using DNC

To quantitatively study the performance of the Tactile Internet network, we abstract the complex network domain in Figure 1 into a mathematical model, as shown in Figure 2. In the abstract Tactile Internet model, $R\left(t\right)$ denotes the input data and $R^{\ast }\left(t\right)$ denotes the output data, and the arrival traffic for each system can be either a single through traffic ${\alpha }_{th}\left(t\right)$ or an aggregated cross traffic ${\alpha }_{cr}\left(t\right)$ synthesized from multiple cross traffic ${\alpha }_{cr}^j\left(t\right)$. Since multiple flows may share servers, the cross-traffic can be used to represent the impact of resource competition.

There are n systems in the abstract model. Suppose the service curve of the $ith$ system is ${\beta }_i\left(t\right)$. We can connect the service curves of the n systems into an equivalent system with the equivalent service curve of $\tilde{\beta }\left(t\right)$ by convolution operation through the series theorem. By combining the traffic characteristics of the Tactile Internet and the network structure into the above model, we can obtain an end-to-end delay upper bound for the Tactile Internet.

5. Determining Delay Bounds

In this section, we derive the end-to-end delay calculation method for the Tactile Internet model abstracted in Figure 2 based on DNC theory. For different characteristics of traffic arrivals and network topology, we have derived models for the single-flow with single-switch, multi-flow with single-switch, and multi-flow with multi-switch cases.

5.1. Single Flow with Single Switch Performance Model

In this section, we will determine the delay bound when a single flow enters a network device. Suppose target flow $f_A$ enters a switch $S_1$ and exits through a port.

Let the average rate of $f_A$ be $r_f$ and the maximum burst be $b_f$. Then, the arrival curve of $f_A$ is obtained from the leaky bucket model as

$${\alpha }_f\left(t\right)=r_{f}t+b_f.$$

(7)

Let the processing rate of the switch be R and the fixed delay be T. Then, the service curve provided by $S_1$ is

$${\beta }_{R_1,T_1}=R{[t-T]}^{+}.$$

(8)

From the classical DNC results [11], the delay bound of $f_A$ can be derived as

$$d=T+\frac{b_f}{R}.$$

(9)

5.2. Multi-Flow with Single Switch Performance Model

In this section, we will determine the delay bound when multiple flows enter into a network device. When multiple flows are encountered for aggregation, we define the flow that requires a delay upper bound as $f_A$, and the others as interfering flows $f_B$. Assume that a single target flow $f_A$ and a single interfering flow $f_B$ enter a switch $S_1$ at the same time and are output through a single port. Let the average rate of the target flow $f_A$ be $r_{f_A}$ and the maximum burst be $b_{f_A}$, and the average rate of the interfering flow $f_B$ be $r_{f_B}$ and the maximum burst be $b_{f_B}$. Then, their arrival curves are ${\alpha }_{f_A}\left(t\right)$ and ${\alpha }_{f_B}\left(t\right)$, respectively.

$$\left\{\begin{array}{c}{\alpha }_{f_A}\left(t\right)=r_{f_A}t+b_{f_A},\hfill \\ {\alpha }_{f_B}\left(t\right)=r_{f_B}t+b_{f_B}.\hfill \end{array}\right.$$

(10)

Let the processing rate of switch S be R and the fixed delay be T. Based on the concept of residual service curve [39], the first-in-first-out (FIFO) residual service curve of the target flow $f_A$ at switch $S_1$ can be obtained as:

$${\beta }_{R_A,T_A}^{res}=R_A^{res}{[t-T_A^{res}]}^{+},$$

(11)

where $R_A^{res}=R-r_{f_B}$, $T_A^{res}=T+\frac{b_{f_B}}{R}$. $R_A^{res}$ is the service rate provided by the switch to the target flow, which is deducted from the total service rate used to service the interfering flow. The increase in fixed delay $T_A^{res}$ is the maximum amount of burst used to service the interfering flows. Thus, the delay bound can be obtained as

$$d=T+\frac{b_{f_B}}{R}+\frac{b_{f_A}}{R-r_{f_B}}.$$

(12)

When L target flows $f_{A_1},\cdots ,f_{A_L}$ and M interfering flows $f_{B_1},\cdots ,f_{B_M}$ enter a network device (with processing rate R and fixed delay T), we assuming that the average rate of the $ith$ target flow is $r_{f_{A_i}}$, the maximum burst is $b_{f_{A_i}}$, and the average rate of the $jth$ interfering flow is $r_{f_{B_j}}$ and the maximum burst is $b_{f_{B_j}}$, their aggregated flow arrival curves can be expressed as

$$\left\{\begin{array}{c}{\alpha }_{A_{agg}}\left(t\right)=\sum _{i=1}^L{r}_{f_{A_i}}t+\sum _{i=1}^L{b}_{f_{A_i}},\hfill \\ {\alpha }_{B_{agg}}\left(t\right)=\sum _{j=1}^M{r}_{f_{B_j}}t+\sum _{j=1}^M{b}_{f_{B_j}}.\hfill \end{array}\right.$$

(13)

Based on the above analysis of the residual service curve, the residual service curve of the aggregated target flow can be introduced as

$${\beta }_{R_{A_{agg}},T_{A_{agg}}}^{res}=R_{A_{agg}}^{res}{[t-T_{A_{agg}}^{res}]}^{+},$$

(14)

where $R_{A_{agg}}^{res}=R-{\sum }_{}^{}{r}_{f_{B_j}}$, $T_{A_{agg}}^{res}=T+\frac{{\sum }_{}^{}{b}_{f_{B_j}}}{R}$. Then, we can derive the delay bound as

$$d_{agg}=T+\frac{{\sum }_{}^{}{b}_{f_{B_j}}}{R}+\frac{{\sum }_{}^{}{b}_{f_{A_i}}}{R-{\sum }_{}^{}{r}_{f_{B_j}}}.$$

(15)

5.3. Multi-Flow with Multi-Switch Performance Model

This section determines the delay bounds for multiple flows passing through multiple network devices connected in series. Let the processing rate of each of the N switches $S_1,\cdots ,S_N$ be $R_1,\cdots ,R_N$ and the fixed delay $T_1,\cdots ,T_N$. Then, by the tandem theorem, the service curve ${\beta }_{sys}$ provided by the equivalent system obtained from these N switches can be obtained as

$${\beta }_{sys}=min(R_1,\cdots ,R_N)·{[t-\sum _{i=1}^N{T}_i]}^{+}.$$

(16)

Thus, the upper bound $d_{single}$ for the delay of a single flow (with arrival curve ${\alpha }_f\left(t\right)=r_{f}t+b_f$) into the system can be obtained as

$$d_{single}=\sum _{i=1}^n{T}_i+\frac{b_f}{min(R_1,\cdots ,R_N)}.$$

(17)

The upper bound on the delay $d_{mixed}$ for a single mixed flow (single target flow arrival curve for ${\alpha }_{f_A}\left(t\right)=r_{f_A}t+b_{f_A}$ and single interference flow arrival curve for ${\alpha }_{f_B}\left(t\right)=r_{f_B}t+b_{f_B}$), entering the system is

$$\begin{array}{cc}\hfill d_{mixed}=& \sum _{i=1}^n{T}_i+\frac{b_{f_B}}{min(R_1,\cdots ,R_N)}\hfill \\ \hfill & +\frac{b_{f_A}}{min(R_1,\cdots ,R_N)-r_{f_B}}.\hfill \end{array}$$

(18)

The upper bound on the delay $d_{ag{g}_{mixed}}$ for multiple mixed flows (aggregated target flow arrival curve for ${\alpha }_{A_{agg}}\left(t\right)={\sum }_{i=1}^L{r}_{f_{A_i}}t+{\sum }_{i=1}^L{b}_{f_{A_i}}$ and aggregated interference flow arrival curve for ${\alpha }_{B_{agg}}\left(t\right)={\sum }_{j=1}^M{r}_{f_{B_j}}t+{\sum }_{j=1}^M{b}_{f_{B_j}}$) entering the system is

$$\begin{array}{cc}\hfill d_{agg\_mixed}=& \sum _{i=1}^n{T}_i+\frac{{\sum }_{}^{}{b}_{f_{B_j}}}{min(R_1,\cdots ,R_N)}\hfill \\ \hfill & +\frac{{\sum }_{}^{}{b}_{f_{A_i}}}{min(R_1,\cdots ,R_N)-{\sum }_{}^{}{r}_{f_{B_j}}}.\hfill \end{array}$$

(19)

5.4. Tactile Internet Performance Model

Some use cases and traffic requirements for the Tactile Internet are given by the IEEE 1918.1 “Tactile Internet” standard working group [2]. We will use the traffic characteristics of the tele-operation use case to validate our model.

We first consider the case when there is only one switch. When the master domain sends control commands to the slave domain in tele-operation, there is only one traffic flow “Haptic”, which corresponds to the single flow single switch case of Section 5.1, so there is

$$d_{ctrl}=T+\frac{b_{Haptic}}{R}.$$

(20)

When the slave domain sends feedback signals to the master domain, there are three types of traffic: “Video”, “Audio”, and “Haptic feedback”, which correspond to the case of Section 5.2, a multi-flow single switch, so there is

$$d_{fdbk}=T+\frac{b_{Video}+b_{Audio}}{R}+\frac{b_{Hapticfdbk}}{R-(r_{Video}+r_{Audio})}.$$

(21)

Thus, the round-trip delay of tele-operation is

$$d_{round\_trip}=d_{ctrl}+d_{fdbk}.$$

(22)

Following the above idea, we extend it to the case of N switches, and the delay of the control command and the delay of the feedback signal can be derived from Section 5.3, respectively:

$$d_{ctrl}=\sum _{i=1}^n{T}_i+\frac{b_{Haptic}}{min(R_1,\cdots ,R_N)},$$

(23)

and

$$\begin{array}{cc}\hfill d_{fdbk}& =\sum _{i=1}^n{T}_i+\frac{b_{Video}+b_{Audio}}{min(R_1,\cdots ,R_N)}\hfill \\ \hfill & +\frac{b_{Hapticfdbk}}{min(R_1,\cdots ,R_N)-(r_{Video}+r_{Audio})}.\hfill \end{array}$$

(24)

Since we use switches with the same attributes, there are ${\sum }_{i=1}^n{T}_i=nT$ and $min(R_1,\cdots ,R_N)=R$ in Equations (23) and (24). We denote each of the above four cases in this section by Cases 1–4, and their descriptions and corresponding variable settings are summarized in

Table 1. Actual situations and parameter Settings in four cases.

	Traffic Direction	Number of Flow(s) and Switch(es)	Variable Setting
Case 1	Master to Slave	1 Flow and 1 Switch	Haptic arrival rate: 1–4 pkts/s
			Haptic burst size: 12–48 B
Case 2	Slave to Master	3 Flows and 1 Switch	Video arrival rate: 1–100 Mbps
			Audio arrive rate: 5–512 Kbps
Case 3	Master to Slave	1 Flow and N Switches	Haptic burst size: 12–48 B
			Number of switches: 2–6
Case 4	Slave to Master	3 Flows and N Switches	Video arrival rate: 1–100 Mbps
			Audio arrive rate: 5–512 Kbps
			Number of switches: 2–6

.

6. Evaluation

In this section, we first validate and evaluate the upper bounds on delay obtained from the theoretical model described above ($accuracy\xi =\frac{actualdelay}{theoreticaldelay}$). Then, we study the impact of traffic arrivals at different rates, interference from mixed flows, and the number of switches on the end-to-end performance of the Tactile Internet.

6.1. Experimental Settings

Referring to the parameters of the real switch used for time-varying analysis provided in [10], we use the switch parameters of 10 G and set the service rate R to 1250 Mbps and the fixed delay T to 8 μs. The packet size is set to 1024 bits. We perform simulation experiments in NS3 network simulator, which generates burst flows arriving according to a Poisson Pareto Burst Process module, with each burst generating a flow with a constant rate. Two terminal nodes and multiple switch nodes are generated through NodeContainer, the nodes are all connected in tandem, where one terminal node connects to the traffic generator as the master domain node and the other terminal node is used as the Receive, as the slave domain node.

6.2. Performance Validation

In order to verify the correctness and validity of the system end-to-end delay upper bound model, the experimental data obtained from the NS3 platform is compared with its calculated delay values to verify the accuracy of the proposed performance model. Also, we added another analytical model of end-to-end delay, CPA [40], to compare with our experiments.

6.2.1. Single Flow with Single Switch

First, we simulate the case of single flow single switch, divide the traffic case of tele-operation, select the burst size of 6 DoF and divide it equally, and simulate the above case in turn and get its theoretical delay upper limit according to Equation (20), and explore the accuracy of the theoretical delay compared with the simulated delay under different burst sizes, the comparison results are shown in Figure 3a.

As can be seen from Figure 3a, when arrival rate is 1000 pkts/s, the upper bound of theoretical and simulation delay increase with the increase of burst size, showing a consistent trend. As can be seen from Figure 3b, when the burst size is 12 B, the theoretical and simulation delays remain basically unchanged with the increase of arrival rate, which is consistent with Equation (20). From these two results, it can be seen that the upper delay bound obtained by the DNC model is closer to the simulation delay than the CPA model. The DNC accuracy is above 99.99% and the CPA accuracy is above 99.60%, both maintain a high accuracy rate. Moreover, when the network is a simple structure in, the end-to-end delay obtained by the DNC theory is approximately equal to that of the delay obtained by the simulation.

6.2.2. Multi-Flow with Single Switch

The arrival rate of the tactile feedback flow in the feedback flow is the same as that in the control flow. Combined with the result of Case 1, we set arrival rate to 1000 pkts/s, treat video and audio flow as interference flow and divide their data rates into five combinations. Because of the additivity of the arrival curve, the two kinds of interference flow can be treated equally. The CPA, DNC and simulation results for the above five cases are shown in Figure 3c. It can be seen that the trends are consistent and the upper delay bound increases with the increase of the interference flow rate. The DNC accuracy is above 95.78% and the CPA accuracy is above 95.30%. The residual service curve analysis gives the maximum residual service rate that can be obtained by the target flow by separating the service required by the interfering flow from the total service capacity. Since the worst-case end-to-end delay of the target flow is considered, the interfering flow will use the switch’s services first and the target flow will use its residual services. Therefore, when the service required by the interfering flow increases, the residual service of the target flow will decrease and the upper bound of delay will increase. This method takes into account the case where the interfering flow does not use up its quota bandwidth and the target flow happens to occupy this part of the free bandwidth. Therefore, from the results, the network calculus is also applicable for the case of multi-flow mixing.

6.2.3. Single Flow with Multi-Switch

In this case, we use the same attribute of switch, we set up 2–6 switches, the service rate and fixed delay of the switch is still 1250 Mbps and 8 μs, and they are connected in series to discuss the comparison of DNC delay and simulation delay of various switches respectively under different burst size.

Figure 4a shows the relationship between the end-to-end delay and the number of switches for a burst size of 12 B, and it is easy to find that the CPA, DNC and simulation results show the same upward trend with the increase of switches and are relatively close to each other, which has a good fit. However, the DNC is still closer to the simulation result than the CPA, which is similar to the result of Case 1. Since we assume that the switches have the same properties and each switch has a fixed delay T, the upper bound on the delay is a simple linear summation under the (min,+) convolution operation. Figure 4b shows the variation of the accuracy with the burst size and the number of switches, we can see that the accuracy increases gradually with the increase of the burst size. This trend is consistent with that of Case 1. Moreover, the accuracy remains high as the number of switches increases, indicating that the Tactile Internet network model based on network calculus is also applicable to complex networks with multiple nodes.

6.2.4. Multi-Flow with Multi-Switch

Combined with Case 2 and Case 3, this case will discuss the relationship between the upper bound of the DNC delay and the simulation delay in the case of multiple flows and multiple switches.

As can be seen from Figure 5a, when the properties of the interference flow are fixed and the number of switches is increased, the CPA, DNC and the simulation delay are in the same upward trend with higher accuracy and DNC is closer to the simulation result, the DNC accuracy is above 95.26% and the CPA accuracy is above 92.84%. As can be seen from Figure 5b, when the arrival rate of the interfering flow increases, the end-to-end delay of the simulated target flow presents an upward trend. This is due to the competition between the interference flow and the target flow. The service rate of the target flow is the maximum remaining service rate that can be obtained by separating the services required by the interference flow from the total service capacity. This is the same analysis as in Case 2. Moreover, when the rate of interference increases, the simulation delay is gradually close to the theoretical delay(from 95.43% to 95.45%). This shows that when the delay of target flow increases, the delay obtained by DNC theory can better reflect the real upper bound of network delay.

Figure 5c gives the variation of the accuracy for different interfering flow attributes and the number of switches. As with the analysis of Figure 4b and Figure 5a, the target flow delay accuracy increases with the increase of the interfering flow rate, while the accuracy decreases when the number of switches increases, but still remains stable at over 95%. Therefore, the network calculus also ensures accurate latency analysis in the case of multiple flows and multiple switches. Reduced to the real scenario of haptic Internet, the end-to-end delay of the feedback signal with haptic information can also be predicted by network calculus theory.

From the above four cases, the DNC model can obtain higher tightness overall than the CPA model. We evaluate the overall situation of the above four cases using four regression indicators: mean squared error (MSE), root mean squared error (RMSE), mean absolute error (MAE) and R square ($R^2$) (as in

Table 2. Overall evaluation of Case 1–Case 4.

	Case 1	Case 2	Case 3	Case 4
RMSE	5.64 $\times {10}^{-7}$	8.34 $\times {10}^{-4}$	2.32 $\times {10}^{-6}$	2.11 $\times {10}^{-3}$
MSE	3.18 $\times {10}^{-13}$	6.95 $\times {10}^{-7}$	5.40 $\times {10}^{-12}$	4.46 $\times {10}^{-6}$
MAE	5.00 $\times {10}^{-7}$	8.34 $\times {10}^{-4}$	2.00 $\times {10}^{-6}$	2.03 $\times {10}^{-3}$
${\mathit{R}}^{\mathbf{2}}$	99.99%	99.73%	99.99%	96.51%

). The results show that the delay bounds obtained using DNC could perfectly fit the worst-case delay bounds of various cases, reflecting the feasibility and accuracy of this analytical model.

6.3. Performance Analysis

In this subsection, we will use the analytical model developed in Section 5 to investigate the performance of the Tactile Internet under different service configurations by varying the properties of the target flows, the number of nodes, and the cross-traffic arrival rate.

6.3.1. Effect of the Number of Tactile Internet Nodes on the Upper Bound of Delay

The number of Tactile Internet nodes is an important aspect of Tactile Internet architecture deployment, and this subsection aims to investigate the effect of the number of Tactile Internet nodes on delay bounds with different target flow burst sizes. The network setup is described as follows: the arrival traffic is modeled as a Poisson process and the arrival rate is set to 1000 pkt/s. In this section, we study the case of tandem connection of similar nodes, i.e., each node’s service process has a fixed service rate and delay, as can be obtained from Figure 6a. In that case, the upper limit of delay is linearly related to the number of Tactile Internet nodes and tends to increase as the number of nodes increases. This is because the tandem formula obtained from the $(min,+)$ convolution formula, with the same node properties, only increases the fixed delay of that node for each additional node in the equivalent service curve, so the overall upper limit of delay is linearly increasing with the number of nodes. In addition, by comparing the delay bounds under the burst size of the four target flows, it is found that the delay bounds increase with the burst size because when the burst size increases, packets generate more congestion and wait in the cache before the service node, which increases the delay of the target packet that has to wait for the previous packet transmission to be processed, which also corresponds to the positive correlation between the DNC delay formula and the burst size.

6.3.2. Effect of Interference Flow Properties on the Upper Bound of Delay

While the Tactile Internet gives people haptic feedback in real time, it cannot be separated from picture and sound feedback. In many Tactile Internet applications, the types of traffic from the domain to the master domain include video, audio, and haptic feedback, and competing traffic other than haptic feedback will be queued with haptic traffic for shared services, which puts a premium on end-to-end delay of haptic information. This subsection investigates the effect of the properties of the interference flow on the upper bound of the tactile flow delay. As shown in Figure 6b, the interference traffic has a significant effect on the upper bound of the target flow delay in the Tactile Internet. A large amount of interfering traffic results in a higher delay upper bound. This is because under the FIFO scheduling algorithm, the interference traffic consumes a large amount of service resources, which leads to a reduction of available resources for the target traffic and negatively affects the service provisioning of the target traffic.

As analyzed above, the developed analytical model can be used as a practical tool for the deployment and management of Tactile Internet services. In particular, the developed analytical model can help service providers improve the performance of the placement of network nodes such as Tactile Internet servers or switches during the design phase of Tactile Internet architecture. In the performance analytical modeling process, we use a parametric approach to develop the resolution model. As a result, service providers and network operators can customize parameter values and perform performance evaluation based on their usage scenarios.

7. Conclusions

In this paper, we analyze the architecture of the Tactile Internet and model it abstractly as a tandem system based on its architectural features. The system can be described as a two-way communication between the master and slave domains, and one of the most important features of the communication is the ultra-low latency. We therefore apply DNC theory to the traffic characteristics of the Tactile Internet and perform a performance analysis. Using DNC theory, we investigate the impact of different traffic characteristics of the Tactile Internet and the number of switches on the end-to-end delay and the accuracy of DNC theory. The results show that DNC provide tighter upper bounds on delay in all cases. From the simulation experiments, it can be concluded that the main factor affecting delay is the number of switches. If reverted to a realistic scenario, it could be the number of servers deployed. This also implies that edge computing will be the trend in the deployment of Tactile Internet applications. In future research, we will use DNC to analyze the performance of more complex Tactile Internet network architectures.