Which Simulator to Choose for Next Generation Wireless Network Simulations? NS-3 or OMNeT++ ^†

Abstract

A common medium access control (MAC) protocol used in Wi-Fi networks is the distributed coordination function (DCF). In the past few decades, a number of network simulators have been developed to validate their analytical models for DCF and other MAC protocols, among which OMNET has gained popularity. NS3 and OMNET provide fundamental modeling framework for the IEEE 802.11 MAC and PHY layers. In this paper, we conducted a comparative study of a unified IEEE 802.11 DCF analytical model with NS3 and OMNET for performance evaluation. Various access modes and different topologies were examined for comparing the theoretical study with the simulators. This study provided a basis that the results obtained from NS3 and OMNET Wi-Fi modules are also credible just like in the theory.

1. Introduction

Network simulations are commonly used to study the behavior of computer networks and communication systems with flexibility, repeatability and scalability in experimentation. Many emerging and promising discrete event simulators are being used today by students, developers and researchers. The comparison among performance evaluation tools is shown in Figure 1. Simulation tools such as NS3 and OMNET provide the highest reproducibility, least complexity and a low-cost environment as compared to the emulation tool such as MiniNet and testbed tool such as PlaneLab. The drawback for these simulation tools is that these are less realistic as compared to PlanetLab for testbed. In recent years, NS-2 has been lacking in support. Among the emerging commonly used simulators are NS3 and OMNET. NS3 has a syntax structure same as the C++ but the scripting interface is designed in Python, allowing for software integration and providing a simulation environment in real time. On the other hand, OMNET is a C++ library is component-based, modular and extendable. It also supports GUI, making it easy to incorporate the simulation kernel into user applications.

The future internet will have a major usage of the Wi-Fi networks due to its simple and low-cost installation. Research has been conducted in order to understand and optimize the Wi-Fi networks and protocols through mathematical modeling, simulations and emulations. The DCF of the Wi-Fi networks is simplified and expressed mathematically by the Dai model [1]. The behavior of each wireless packet is studied through a discrete time renewal process which includes successful transmissions and backoff collisions. The Dai model is different when compared to the classical Bianchi model [2], as it includes all system parameters that effect the wireless transmission. The Dai model was validated earlier through an NS2 simulator, reflecting its accuracy and precision.

NS3 and OMNET have built-in IEEE 802.11 models that operate according to MAC layer following the CSMA/CA and DCF rules and regulations. Because of its complicated structure, a few researchers have thought about validating MAC layer of the NS3 and OMNET. There are a number of research papers highlighting the validation of the physical and the MAC layer in NS3 and OMNET [3,4,5,6,7,8,9,10]. A testbed is made to validate the MAC model in NS3 [11]. Similarly, the MAC model in the OMNET is validated through a testbed [12]. The medium access control (MAC) in NS3 is validated by increasing the number of wireless devices/nodes. In the proposed method, a comparison is made between the NS3 MAC model, OMNET MAC model and the mathematical model proposed by Dai. The proposed work is the first of its kind to present the detailed comparison of MAC model through a mathematical approach and simulator results.

The paper is categorized as follows. Section 2 presents the results obtained from the analytical unified framework. Section 3 describes the setup about simulators, how to produce the simulation results and how those results compare to theoretical outcomes. The conclusion of the paper is reported in Section 4.

2. Method

DCF in Wi-Fi networks is reported in this section along with an overview of how throughput is interpreted for saturated and unsaturated networks. The basic access mode and RTS/CTS mode are taken into consideration. The aim was to compare the simulators result with the mathematical model results produced by Dai’s model.

2.1. DCF in Wi-Fi Networks

The IEEE 802.11 DCF network may broadcast each wireless packet an unlimited number of times with an infinite number of nodes n and with infinite buffer capacity. The backoff parameters are associated with each node. The parameters include cutoff phase K and starting window size W. Each node also has a λ traffic arrival rate.

It is crucial to find the holding times in the basic access mechanism for successful transmission and collision rates as the MAC model parameters in both the simulators, the node waits a specific time intervals (ACK time-out and CTS timeout period) after a collision. The holding times for successful transmission and collision rates are

$${\mathsf{\tau }}_{\mathrm{T}}^{\mathrm{ba}}=\frac{\left(\frac{8\mathrm{PL}}{{\mathrm{R}}_{\mathrm{D}}}+\frac{8\mathrm{MH}}{{\mathrm{R}}_{\mathrm{D}}}+2\mathrm{PH}+\frac{8\mathrm{ACK}}{{\mathrm{R}}_{\mathrm{B}}}+\mathrm{SIFS}+\mathrm{DIFS}\right)}{\mathsf{\sigma }}$$

(1)

and

$${\mathsf{\tau }}_{\mathrm{F}}^{\mathrm{ba}}=\frac{\frac{8\mathrm{PL}}{{\mathrm{R}}_{\mathrm{D}}}+\frac{8\mathrm{MH}}{{\mathrm{R}}_{\mathrm{D}}}+\mathrm{PH}+\mathrm{ACKTimeout}+\mathrm{DIFS}}{\mathsf{\sigma }}$$

(2)

respectively. The unit of bytes is used to measure MACheader (MH) and ACK frames. PHY header (PH) DCF interframe space (DIFS) and Short interframe space (SIFS) are measured in the units of μs. R_B is the basic data rate and measured in Mbps.

In the RTS/CTS mode, the holding times for successful transmissions and collision rates are determined using the following equation:

$${\mathsf{\tau }}_{\mathrm{T}}^{\mathrm{rts}}=\frac{\frac{8\mathrm{PL}}{{\mathrm{R}}_{\mathrm{D}}}+\frac{8\mathrm{MH}}{{\mathrm{R}}_{\mathrm{D}}}+4\mathrm{PH}+\frac{8\left(\mathrm{RTS}+\mathrm{CTS}+\mathrm{ACK}\right)}{{\mathrm{R}}_{\mathrm{B}}}+3\mathrm{SIFS}+\mathrm{DIFS}}{\mathsf{\sigma }}$$

(3)

and

$${\mathsf{\tau }}_{\mathrm{F}}^{rts}=\frac{\frac{8\mathrm{RTS}}{{\mathrm{R}}_{\mathrm{B}}}+\mathrm{PH}+\mathrm{CTSTTimeout}+\mathrm{DIFS}}{\mathsf{\sigma }}$$

(4)

respectively. Byte is the standard measuring unit for RTS and CTS.

2.2. Wi-Fi Module of NS3

The Wi-Fi module of the NS3 is depicted in Figure 2. The connection layer is the top layer, which acts as a bridge between the module and the management layer. The WiFiNet-Device acts as an interface and sends the packet to the MAC high module at the management layer, when the transmission is initiated by an application. Several functions are performed by the MAC high module such as beacon, probing, connection and rate control. In OMNET, the Ieee80211MgmtSta module has the same working principles as the MAC high module in the NS3, but with some extra algorithms such as rate control. The MAC high model is categorized into three layers such as:

• The ns3: regularWi-FiMac is a parent MAC model that ensures quality of service (QoS);
• The ns3: APWi-FiMac supports the networking in infrastructure mode;
• The ns3: StaWi-FiMac supports the association states and the active probing states;
• The ns3: AdhocWi-FiMac supports the networking in the ad hoc mode.

Similarly, the low module in the MAC layer has the following functions:

• MacLow: keep records of the transactions involving ACK/DATA/RTS/CTS;
• DcfManager and ns3: DcfState: keep track of the distributed coordination functions;
• DcaTxop and ns3: EdcaTxopN: controls the packet transmission, packet fragmentation and packet queues.

The WiFiPhy module is responsible to handle the reception/transmission of frames along with energy management. The error rate model, which is not present in the PHY layer of OMNET simulator, provides an additional feature of approximating the probability of successful frames revived.

2.3. Wi-Fi Module of OMNET

INET framework is used in OMNET for the TCP/IP. The simulation model suite supports all the link layer models, Internet protocol version 4 (IPv4), Internet protocol version 6 (IPv6), transmission control protocols (TCP), user datagram protocols (UDP) including Ethernet, multi-protocol label switching (MPLS) modeling and point-to-point protocol (PPP). OMNET uses the network interface card (NIC) which is configured using the following interfaces:

• Ieee80211Nic: is the network interface card with generic functions;
• Ieee80211NicAdhoc: supports the ad hoc network configuration;
• Ieee80211NicAP: supports the access point functionalities;
• Ieee80211NicSTA: supports the networking in the infrastructure mode.

There are four submodules in the Wi-Fi network interface card used in OMNET. The submodules are categorized as management module, MAC module, connection module and physical layer module. The connection layer is responsible to instruct the management layer to perform beaconing, channel scanning and association/authentication. The nodes dynamic behavior is modified during the handover mechanism when the Ieee80211AgentSta module is replaced. To facilitate the exchange of management frames between entities such as nodes and access points (APs), the management layer utilizes the Ieee80211MgmtSta module. The layer is also responsible for periodically switching channels during scanning and collecting information from received probes and beacons. The Ieee80211Mac submodule of the MAC layer is in charge of transmitting frames using the CSMA/CA protocol. In OMNET, the MAC incorporates several pre-defined policies including ACK, RTS/CTS, TXOP, fragmentation, DCF and HCF policies, as well as a statistics policy. The physical layer deals with the modeling of reception and transmission of frames. Radio module at PHY layer is responsible for the error free reception of frames in case of interference or low signal powers. This layer also includes a model for energy use. The block diagram for OMNET IEEE 802.11 NIC is depicted in Figure 3.

3. Performance Evaluation

In this section, the comparative performance evaluation of the mathematical model and both the simulators is reported. In order to analyze and simulate Dai’s theoretical model in NS3 and OMNET, the network is first loaded from unsaturated to saturated condition by changing the traffic arrival rate. The results are obtained by varying the number of nodes n, window size W and the cut-off phase K.

3.1. RTS/CTS Access Mode vs. Basic Mode

There are two mechanisms in the DCF: basic access mechanism and the RTS/CTS mechanism. Basic access mechanism is a two-way handshake in which the node after waiting a DIFS period transfers the packet on the condition if the channel is idle. If the channel is not idle, then the backoff process is initiated which is decided on the basis of backoff window size. On the successful transmission of the packet, the node receives the Acknowledgment frame. If the ACK time-out period is over and no ACK frame is received, the backoff procedure is again initiated by the node.

The RTS/CTS mechanism is a four-way handshake in which the channel is reserved by the node first. The channel reservation is carried out by transmitting a RTS frame. On receiving the RTS frame, a CTS frame is sent to all the nodes. This exchange of RTS and CTS frames suspend the transmissions of all the nodes until the existing transmission is complete. The backoff procedure is initiated by the node when the CTS time-out period is over unless the reception of the frame is verified by the ACK frame.

The performance evaluation of the packet loads PL against the throughput $\hat{\mathrm{D}}$ is depicted in the in Figure 4 for the infrastructure mode and ad hoc mode. The data rate is set to 24 Mbps and 54 Mbps. By observing Figure 4, we found that the simulation results of the NS3 simulator exhibited a closer trend to that of the mathematical model results. It was observed that the MAC layer in both NS3 and OMNET can be relied upon while performing simulation tests related to IEEE 802.11 DCF networks.

3.2. Ad Hoc vs. Infrastructure

When the devices communicate with each other directly, an ad hoc network is formed reflecting a decentralized architecture. As each node both receives and emits packets to other nodes, it functions as a transceiver. The network architecture becomes centralized in the infrastructure mode when nodes connect to an AP before communicating with one another. The infrastructure mode is different to the ad hoc mode as it involves the additional channel activity related to association and de-association. Active scanning and beacon transmission are the common actions involved in associations and de-associations.

Figure 5 clearly illustrates how the overall throughput performance was impacted by association in infrastructure mode. In the comparison between the throughput and number of nodes while in the ad hoc and infrastructure mode, we can observe that the total throughout achieved in the ad hoc mode was slightly higher than the throughput achieved in the infrastructure mode. The reasons for the lower throughout gain in the infrastructure mode is due to the fact that association and de-association activities occur frequently. The trends of the OMNET simulator were almost the same as the NS3 simulator, where NS3 exhibited closer results to the Dai model.

4. Conclusions

In this paper, we performed extensive experiments to compare the IEEE 802.11 DCF analytical model against the IEEE 802.11 simulation model in NS3 and OMNET. The comparison was validated by varying system parameters such as the number of nodes, window sizes, cutoff phases, tuning packet payloads in the RTS/CTS mode and the basic access mode, switching the network from unsaturated to saturated conditions. The proposed study has shown that the MAC model of both simulators exhibit close behaviors to Dai’s model proposed for homogeneous IEEE 802.11 DCF networks. Our results show that both the simulators showed close trends with the mathematical model and both the simulators can be used for the simulations for the Wi-Fi networks. In the future, we plan to design a software/hardware platform to validate the MAC models of NS3 and OMNET in a dense Wi-Fi network.