Improving the Performance of ALOHA with Internet of Things Using Reinforcement Learning

Abstract

Intelligent medium access control (MAC) protocols have been a vital solution in enhancing the performance of a variety of wireless networks. ALOHA, as the first MAC approach, inspired the development of several MAC schemes in the network domain, with the primary advantage of simplicity. In this article, we present design, implementation, and performance evaluations of the ALOHA approach, through significant improvements in attaining high channel utilization as the most important performance metric. A critical emphasis is currently focused on removing the burden of packet collisions, while satisfying requirements of energy and delay criteria. We first implement the ALOHA protocol to practically explore its performance behaviors in comparison to analytical models. We then introduce the concept of dynamic payload instead of fixed-length packets, whereby a dynamic selection of the length of each transmitted packet is employed. Another specific contribution of this paper is the integration of the transmission policy of ALOHA with the potential of Internet of Things (IoT) opportunities. The proposed policy utilizes a state-less Q-learning strategy to achieve the maximum performance efficiency. Performance outputs prove that the proposed idea ensures a maximum throughput of approximately 58%, while ALOHA is limited to nearly 18% over a single-hop scenario.

1. Introduction

A huge focus of recent studies on medium access control (MAC) for wireless networks has been placed for some time on the development of cost-effective solutions. This is because the components of such a wireless network often share and contend with the wireless communication medium, resulting in inefficient utilization of network capacity and sources. This requires intelligent coordination among the components to increase channel utilization. The key concept of MAC approaches relies on the efficient usage of a shared wireless channel through coordination of the channel access strategy of the network components. Existing MAC approaches can be categorized as contention-based and schedule-based with respect to their channel access strategies [1]. In addition to this, there has been a further prominent number of research efforts extending this categorization towards a hybrid type that takes the advantages of MAC strategies while alleviating their drawbacks [2]. Contention-based protocols allow users to initiate their transmissions without pre-coordination, resulting in excessive collisions and unacceptable overall network performance. To alleviate these weaknesses, schedule-based protocols introduce an intelligent transmission strategy through a dynamic schedule, but at the expense of complexity and overheads. The feasibility of a typical MAC scheme is, therefore, important in many practical deployments that require a diminution in the burden of a computationally-complex design structure [3]. The main attributes when designing an efficient MAC protocol have been broadly researched, including energy-efficiency, throughput, scalability, latency, adaptability, and fairness [4].

The development of MAC protocols has been recently studied in the major areas of network science. A series of recent studies investigated several MAC protocols for Internet of Things (IoT) environments with a key goal of addressing MAC behaviors [5,6,7]. A performance evaluation of a modified existing MAC approach for a Cognitive Radio Based Wireless Network (CRWN), as an emerging trend in the current network era, was undertaken using game theory leading, to significant enhancements of several of the aforementioned attributes [8]. Due to the widespread application areas of wireless sensor networks (WSNs), a great number of research activities have focused on developing energy-efficient MAC schemes to prolong the overall lifetime of a typical WSN application, while satisfying other attributes at a desirable level [9,10]. Efficient MAC solutions are required to maintain an efficient utilization of limited resources for Flying Ad-hoc Networks (FANETs), as the mobility of Unmanned Aerial Vehicles (UAVs) is quite high, subject to frequent topology changes. To provide a robust quality of service (QoS) in terms of efficient and intelligent communication among UAVs, novel MAC protocols have been proposed for multi-hop FANETs [11,12]. In underwater networking, the major bottleneck associated with sophisticated underwater channel characteristics is the long propagation latency, which has encouraged researchers to explore application-specific MAC ideas [13,14]. MAC studies in mobile ad hoc networks (MANETs) have drawn a distinct interest, with the aim of overcoming the limited energy issue with battery-powered mobile devices [15].

The ALOHA protocol, with its nearly 50 year history, has been gaining much research interest owing to its simplicity in implementation [16]. An ultimate bottleneck of ALOHA is the low level of the maximum achievable channel throughput due to the superior number of collisions caused by its random-access nature. Therefore, the practicality of the ALOHA scheme is more suitable for sporadic data transmissions when the network is lightly loaded. The simplest version of the ALOHA protocol, the pure ALOHA, starts transmission immediately when there is a packet ready to be transmitted, with no control mechanism for the current status of the transmission medium. A successful reception of a packet is then acknowledged by an immediate response, called the acknowledgement (ACK) packet. This will essentially allow concurrent transmissions from multiple transmitters, resulting in packet collisions, either partly or fully overlapping. When modelling this scheme, the collided packets are considered to be corrupted and lost as a critical assumption. Pure ALOHA operates a retransmission policy to retransmit the lost packets that performs successive retransmissions of each collided packet up to seven times. Eventually, the maximum efficiency of pure ALOHA in terms of channel throughput is achieved by nearly 0.18.

The main objective of this paper is to revisit the traditional ALOHA scheme in order to enhance its performance. The first part of the proposed study concentrates on the practical implementation of ALOHA to characterize the protocol performance, taking the issues associated with the practical aspects into consideration. A crucial aspect is known as the capture effect phenomenon, which permits a successful reception of the first captured packet in collisions [17,18]. Bearing the fact of capture effect in mind, this work demonstrates the throughput and delay performances of ALOHA with a finite number of users. An analytical form of the throughput with the capture effect, using the unique parameters in these implementations, is derived to compare the theoretical throughput with the practical throughput. The performance measurements clarify an important performance improvement in terms of throughput as an outcome of capture effect. Later, to further improve the throughput performance in practice, we propose a novel selection strategy of variable payload length as another novelty of this work. By performing this strategy, the length of a packet prior to its transmission is dynamically selected, ranging from 1 byte to its maximum length, in place of keeping an equal and fixed-length payload. An explicit target of this solid concept is to reduce the complete length of the transmitted packets, thereby preventing as many collisions as possible, and otherwise reducing the degree of overlapping. Therefore, the likelihood of decoding a packet involved in a collision is increased. The performance results of the proposed dynamic process exhibit interesting improvements.

In recent times, the wireless networking domain has witnessed a rapid technological development with the invention of the Internet of Things (IoT) [19,20,21]. It is well established that IoT creates a network environment for physical objects to directly connect with each other with no human control. This will no doubt make wireless networking a primary component of IoT for pervasive data transfer with ubiquitous Internet connection via available wireless technologies. The progress in developing efficient and intelligent MAC approaches will ultimately benefit from IoT opportunities with a significant influence on the basic design attributes. A significant amount of research effort will rely on adaptation of traditional MAC protocols [22]. A distinct purpose of this article is to take advantage of the IoT theme, accelerating the packet transmission. An adapted version of ALOHA protocol with IoT opportunities is proposed, referred to as IoT-ALOHA, with the feature of transmitting packets via the Internet. The main duty of IoT-ALOHA is to make a smart decision on when to transmit a packet over the Internet, along with the problem of exhausting more energy capacity. The principal basis of the operating decision is built on the recent individual transmission history. In particular, IoT-ALOHA pushes the packets to be transmitted over the Internet, in case of having to face a notable and superior number of collisions. One of the most effective solutions based on previous experience to represent the desirability level of an action is Q-learning [23], which can numerically assign a weight value to be updated based on the results of packet transmissions in this framework. A simple and stateless version of Q-learning is implemented to numerically indicate the desirability level of such a transmission choice.

The other parts of the paper are organized as follows. Section 2 summarizes recent related works specifically designed for ALOHA, underlying their fundamental working principles. The details of the proposed enhancements are presented in Section 3. Section 4 describes the performance evaluation results to prove the superiority of the proposed strategies. We finally sum up the complete work in Section 5 together with possible future research directions.

2. Related Work

There has been tremendous number of previous studies of MAC schemes specifically designed for a plethora of wireless network areas. It is worth noting that the scope of this article covers the presentations of the latest MAC designs with the main emphasis placed on ALOHA. Therefore, this part intends to overview the recent ALOHA-based MAC studies. In [24], the impact of capture effect on the throughput of ALOHA was thoroughly studied by modeling different capture scenarios through extensive experimentation. This work explicitly demonstrates the occurrence of capturing the first-arriving packet in a collision when all collided packets are transmitted at the same transmission power. In addition to analytical derivations in incorporating its impact on the throughput, experimental evaluations are performed using a IEEE 802.15.4-compliant device in a finite-user single-hop case.

An alternative way of mitigating collisions is the capability of the multi-packet reception (MPR) at the expense of advanced signal processing and new coding techniques [25,26,27]. The classical analysis of ALOHA throughput was extended to reassess the effective throughput under the umbrella of MPR. In a MPR-capable network, more than one packet in concurrent transmissions can be successfully decoded, contrary to the traditional collision assumption model accepted in ALOHA. However, the maximum number of packets to be successfully decoded is limited, which indicates the MPR degree. Here, if the number of concurrent transmissions exceeds this degree, none of the resolvable transmissions will be received, leading to an “all-or-nothing” model. The maximum achievable throughput is strongly dependent on a particular MPR degree. In this regard, current studies have attempted to derive the likelihood of a successful transmission and modified throughput in the presence of an MPR property.

ALOHA-based systems have gained much research interest in low-power wide-area networks (LPWANs), with a key focus on IoT acting as random-access strategy [28]. The throughput and delay performances of the ALOHA protocol are analyzed with extensive approximations in different settings. The impact of setting an optimal packet length and number of users on the throughput performance have been investigated through a set of simulations. A major output of this study is performance improvement by reducing the packet length. The effective throughput of ALOHA is close to nearly 0.28 with a low number of users. Increasing the number of users resulted in approaching the theoretical throughput value, as expected.

The main mechanism of ALOHA is improved to raise the maximum throughput capacity in LoRaWAN environments for electronic shelf labels (ESLs) [29]. LoRaWAN employs many intermediate nodes to deliver the same packet to a particular destination with a similar ALOHA mechanism, facing a high level of collisions. This may impede the implementations of LoRaWAN in many industrial areas, rendering it efficient in practical applications with a low data rate. The core part of the proposed improvement is the modification of the retransmission strategy to overcome repeated collisions. Instead of retransmitting the lost packets in sequence, the transmitting node will only send an initial part of the packet to check the availability of the receiving node. Therefore, a synchronization between the overlapping nodes is established that eliminates the regulation of random delay for retransmissions in a row.

A new version of ALOHA, called KALOHA “Knowledge in ALOHA”, was introduced to advance the efficiency of ALOHA by exploiting the local knowledge obtained about the packet transmission, with no extra physical layer assistance [30]. The idea makes use of local times for both received packets and acknowledgements. When the transmission medium experiences a high traffic load, the proposed idea encourages the nodes to decrease their transmission rate. This is achieved by letting the nodes have an explicit coordination via the sharing of perceived channel utilization. The performance results prove that KALOHA doubles the throughput of pure ALOHA with its simplistic nature.

Recently, reinforcement learning (RL) has been incorporated into the development of a novel MAC solution by transforming and adapting ALOHA in underwater acoustic networks [31]. With this reliable schedule, many collisions can be handled via an effective learning process operated by each node independently. The main mechanism lies in an asynchronous operation that optimizes the required transmission slots along with a fresh back-off strategy. The performance evaluations demonstrated significant advantages under varying network topologies. SOMAC (Self-Organizing MAC) is a recent RL-based MAC approach that alters the MAC dynamism with respect to current network requirements [32]. SOMAC has its basics in an intelligent selection of the best MAC scheme through the RL approach employed. Consequently, it is capable of coping with environment changes and dynamic application requirements. LEASCH is a MAC solution of the radio resource scheduling issue in 5G networks based on a deep RL model [33]. The main difference of LEASCH is its mechanism in learning the scheduling goal without knowledge of radio resource scheduling. CoRL is a collaborative RL-based MAC scheme for IoT Networks that prevents collision and mitigates complexity [34]. It benefits from the nature of a framed-slotted contention-based protocol and Q-learning. QL-MAC is another Q-Learning based powerful MAC approach with the aim of optimizing active durations of nodes in a network [35]. The optimization is achieved in association with the traffic conditions and the current transmission state of the neighboring nodes. As a result of examining the existing studies, we infer that ALOHA-based MAC solutions will play a continually key role in many aspects of networks. The principal merit of this paper is to revisit ALOHA protocol for a potential performance improvement with the combination of IoT and RL.

3. Proposed Enhancements and Background

3.1. Background of Experimental Setup and Q-Learning

Simply, we develop a low-cost and energy-efficient object hardware composed of a processor, wireless transceiver, IoT module and energy-harvesting unit. For the processing component, a breadboard-friendly NodeMCU platform is used featuring a small dimension and an ESP8266 microcontroller [36]. NodeMCU is the core part of the object, as the ESP8266 provides a low-cost Wi-Fi connection. An affordable and reliable transceiver module, namely nRF24L01, is the responsible unit for two-way communication with the properties of a transmission coverage of 1000 m with line-of-sight and an adjustable data rate, with a maximum limit of 2 Mb/s [37]. The length of the packet provided by nRF24L01 is configurable with a variable length payload property ranging from 1 to 32 bytes. To make the programming simple, we benefit from the rationale behind the Arduino development environment as it provides an easy programming style along with plenty of open-source software and available libraries. A solar panel supplies the required energy requirement, whereby a rechargeable energy storage unit (lithium polymer (LiPo) battery) stores the harvested energy. A view of the object with all components is depicted in Figure 1.

We consider a fully-connected single-hop type of network topology with six transmitters and one receiver, each of which is equidistant from the receiver, in a circle. This small-scale scenario is not fully realistic in practical implementations, but it can be a representative one-hop sub-network in complex network deployments. It is highly believed that the small scale of experiment can enable us to study the fundamental performance of the proposed strategies for an indoor topology. The packet generation duration by all nodes follows an exponential inter-arrival time by setting each transmitter to the same average inter-arrival time. A view of the topology in an unobstructed area is depicted in Figure 2.

Reinforcement learning, as a reliable learning solution, has certainly been an extremely wide field of research in complex applications to deal with decision-making duties [38]. A particular focus of the RL method is aimed at learning an ideal solution policy through a trial-and-error methodology with no prior experience. The environment of such a complex system is fitted with a set of actions. The system actors or users, so-called agents, interact with the actions to learn a nearly-optimal goal. The outputs of actions taken are interpreted into a cumulative reward mechanism to be fed back into the agent. A key purpose of RL strategy for an agent is to efficiently and successfully select consecutive actions, maximizing the acquired reward. The experience obtained through the reward is utilized for the decisions regarding forthcoming actions. The overall numerical value of reward for a particular action is stored with the aid of a Q-value that represents the desirability level of the action to be chosen.

Q-learning is one of the most widely implemented and powerful algorithms under the horizon of RL [39]. In this study, a simple stateless version of this algorithm is considered since the proposed learning issue requires no state representation, as formulated below:

$$\mathrm{Q}\left(a\right)\leftarrow \left(1-\alpha \right)\cdot \mathrm{Q}\left(a\right)+\alpha \mathrm{r},$$

(1)

where Q(a) is the Q-value of action a, the reward is represented by r and α is the learning rate factor that controls the weight of most recent experience.

3.2. ALOHA with Capture Effect

Recent practical implementations have revealed a successful reception among simultaneous transmissions depending on the packet overlapping length, which is termed the capture effect. Therefore, with the utility of this capture effect, the first arriving packet from a concurrent transmission may be successfully received, subject to the degree of packet overlap. Much of the effort towards capture effect investigates a common capture scenario, known as two-packet-capture scenario, as depicted in Figure 3. In this scenario, the radio transceiver of the receiving unit synchronizes with the first arriving packet, while the second arriving packet, after a brief moment creates a collision. This may lead to the corruption of the successful reception of the first arriving packet, resulting in both packets being lost.

This phenomenon may have an effective impact on the performance metrics, particularly by transmitting packets with equal power, as in the ALOHA network. In our earlier work [24], we thoroughly studied the capture effect experimentally for pure ALOHA by modelling its occurrence probability. A capture coefficient is then derived, acting as a packet reception rate in collisions. A multiplication of the capture probability and coefficient was analytically exploited to incorporate the impact of capture into the throughput of pure ALOHA. The final derivation of the throughput of pure ALOHA, with the assumption of a finite number of users, through binomial distribution was presented as:

$$\begin{array}{ll}np{\left(1-p\right)}^{2\left(n-1\right)}+n(n& - 1)p^2\left(\frac{{\left(1-p\right)}^{2\left(n-2\right)}}{2}+{\left(1-p\right)}^{2\left(n-3\right)}\right)C_{n_2}\\ \stackrel{\begin{array}{l}ALOHAwithout\\ captureefect\end{array}}{\leftrightarrow }& \stackrel{2-packetcapture}{\leftrightarrow }\end{array}$$

(2)

Here, n indicates the total number of users in the network, p is the probability of a user transmitting a packet which is equivalent for each user and ($C_{n_2}$) shows the capture coefficient for two-packet collision. In this study, to account for the capture effect, we rely on this throughput expression by adjusting the required parameters of our experimental setup. For this purpose, the number of users (n) is set to six and the capture coefficient ($C_{n_2}$) is calculated in connection with the maximum packet size provided by the radio transceiver (NRF24L01). We will hereafter denote this analytical form of the throughput as ALOHA-C (ALOHA with Capture) in the rest of the paper.

3.3. ALOHA with Dynamic Payloads

One of the common assumptions in the analytical calculation of the throughput of ALOHA is the generation of fixed-length packets to be transmitted by all users. Keeping an equal and fixed-size payload for pure ALOHA makes such a throughput analysis very convenient with a particular average rate of packet generation. It is well known that the maximum efficiency of pure ALOHA is restricted, mostly due to the packet collisions, regardless of the amount of overlap. Generating packets with the same length to be processed by the process of a blind transmission strategy of ALOHA may cause either partly or fully packet overlapping. The total number of overlapping bytes up to the packet length among two colliding packets can be treated as uniformly distributed. To narrow down the problem of collision, we propose to dynamically adjust the payload length at random, with the objective of reducing the degree of overlapping. In this case, the overall length of the transmitted packets is reduced at the onset of transmission. An example scenario of dynamic selection of payload length is depicted in Figure 4.

It is noted from Figure 4a that all packets with fixed payload size are involved in collision, thus creating a chance of the destruction of all packets. Unlike this unsuitable event, through the proposed dynamic selection of payload size shown in Figure 4b, the first arriving packet with no overlap may be easily decoded. The second packet is strongly expected to survive in the presence of overlapping with the third packet. This type of scenario of interest motivates us to call for an alternation in the payload size in offering better channel utilization efficiency. To operate such a mechanism, as a result, the challenges associated with collisions are partly handled through the proposed approach, whereby each user is allowed to pick a random payload length for each transmitted packet ranging from even 1-byte to the maximum length.

3.4. ALOHA with IoT and Q-Learning

This section deals with the basic motivation of improving practical ALOHA (P-ALOHA) in advancing the throughput owing to the development of IoT-enabled objects. With this solid design, the packets can be transmitted via the Internet, but at the expense of a high energy consumption budget. It is noteworthy that the current IoT modules with powerful components consume more energy than traditional wireless transceiver modules. This makes continuous usage of IoT components inconvenient in energy-constrained networks. An intelligent solution is required to let the objects occasionally transmit their packets through the Internet. This necessitates a decision technique at the initiation of packet transmissions to decide whether a packet will be transmitted through the Internet. For this decision, we benefit from the recent transmission history of each object, as the decision should be performed by objects based on their individual transmission records. The principal emphasis of the decision process is placed on the successful transmission capabilities of the objects. We intend basically to encourage the objects to send packets via the Internet when they face a considerably high level of collisions.

To support an efficient decision process, a simple stateless version of the Q-learning algorithm is employed to numerically denote the desirability level of the transmission choices, either for Internet or Radio module. Each object maintains a Q-value to be updated recursively after transmissions, which is defined below:

Q = Q + α(R − Q).

(3)

Here, R is the reward function, taking two values depending on the transmission result. If the transmission becomes successful, R typically takes a positive value of +1 and −1 for a failed transmission. The parameter α is defined as learning speed by weighting the impact of recently acquired experience, which is usually assigned to a small constant value, 0.1 in the experiments in this paper. In the literature, there has been no agreed consensus for the choice of numerical values of R and α, which are often selected in relation to the specific application dynamics. The value of Q for each object is initialized to 1 which may approach 0 after a series of unsuccessful transmissions. Basically, in this strategy, the objects decide to send packets via the Internet with probability 1 − Q. The probability of sending a packet via the Internet is therefore 0 on start-up. When the network starts to run, collisions will cause the decrement of Q value, allocating probability 1 − Q for packets being transmitted via the Internet. Hence, the proportion of packets transmitted via the Internet is dependent on the rate of unsuccessful transmissions of each object independently. ALOHA with this IoT capability will be denoted as IoT-ALOHA in the remainder of the paper. The complete flowchart for IoT-ALOHA is presented in Figure 5.

4. Performance Evaluations

This section presents the outputs of the performance evaluations of the proposed approaches with an experiment duration of 10⁶ slots. We run each experiment 100 times and each performance output is the average of these experiments. The performances of the protocols are measured in terms of channel throughput and delay. The channel throughput can be described as the fraction of total channel occupancy, where a full channel usage will be demonstrated by one Erlang unit. The delay of a packet is defined as the total time taken from the packet generation to the successful packet transmission. In all performance evaluations related to the delay, we present an average value of the delay of all generated packets, so-called end-to-end delay performance. All parameters used in the experiments are summarized in

Table 1. Summary of experiment parameters.

Parameters	Values
Experiment Period	1,000,000 slots
Topology	Single-Hop
Number of Nodes	6
Data Rate	1 Mbps
Power Levels	0 dBm
Packet Length	32 Byte
Packet Generation Model	Poisson
Learning rate (α)	0.1

. Another significant issue in practical implementation is the impact of propagation delay with respect to the distance among nodes. A successful packet transmission may not be received at the receiver if the distance is not properly arranged. Prior to running the experiments, we manually place the nodes and check the transmission link between each transmitter and receiver.

The first experiments target the demonstration of effective channel throughput performances for the ALOHA, ALOHA-C and P-ALOHA, as a function of varying generated traffic levels, in terms of Erlang unit. One of the main concerns in beginning practical implementation, as an underlying contribution of this paper, is to observe the throughput performance of P-ALOHA without the ACK mechanism. In this scenario, all generated packets are transmitted once without ACK and retransmission policy. It is worth mentioning that re-transmission policy causes much higher traffic loads, as a packet can be re-transmitted up to seven times. This kind of heavy traffic load would potentially result in many packet collisions, consuming much of the energy budget unnecessarily. By reducing the number of packet collisions, the performance efficiency is strongly expected to improve as a consequence of the proposed idea. The most significant drawback of this strategy is that the nodes have no feedback for the transmitted packets. Consequently, the main purpose of this idea is to illustrate the throughput characteristic of ALOHA-type approaches by eliminating the rule pertaining to the ACK. Accordingly, the throughput performances of the protocols observed are depicted in Figure 6. In this figure, the throughput curves for ALOHA and ALOHA-C are derived from their analytical expressions reported in the previous section. The results comfortably prove that P-ALOHA outperforms ALOHA, offering a throughput performance of nearly 0.38 Erlang when the generated traffic is 1 Erlang. It might be noted that the P-ALOHA experiences a sense of increase, with increasing generated traffic level approaching its maximum throughput at generated traffic of 1 Erlang. On the other hand, the practical results of P-ALOHA closely match the analytical results of ALOHA-C with a very small negligible discrepancy. To sum up, these results clarify the advance obtained by ignoring a common assumption in ALOHA, leaving the uncertainty of outcome of the packet transmissions as a considerable design parameter.

We then observe the throughput performance of P-ALOHA with a varying range of dynamic payload length as a trend of increasing generated traffic value. Overall observations from the outputs presented in

Table 2. Throughput performances of P-ALOHA with varying range of dynamic payload length.

Range of Payload	Generated Traffic (Load)
	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1.0
1–32	0.0893	0.1635	0.2192	0.2660	0.3055	0.3343	0.3448	0.3704	0.4163	0.4175
5–32	0.0950	0.1721	0.2301	0.2827	0.3071	0.3486	0.3696	0.3973	0.4210	0.4383
10–32	0.0963	0.1775	0.2433	0.2876	0.3185	0.3708	0.3887	0.4237	0.4386	0.4645
15–32	0.0979	0.1808	0.2525	0.2995	0.3330	0.3698	0.3860	0.4365	0.4367	0.4728
20–32	0.0962	0.1789	0.2482	0.2966	0.3306	0.3689	0.3847	0.4315	0.4318	0.4696
P-ALOHA	0.0866	0.1610	0.2158	0.2702	0.3110	0.3407	0.3549	0.3710	0.3739	0.3806

indicate that the goal of dynamic payload yields better throughput efficiency in all circumstances. We systematically change the range of the payload length to precisely determine the impact of the payload range on the throughput performance. The minimum payload length is selected as 1-byte long, which is boosted to be a multiple of 5 bytes. The maximum payload size is always equal to the maximum packet size provided by the radio transceiver unit. We see from the corresponding results that the throughput efficiency faces lower performance with smaller values of minimum payload size. This is because many packets with low payload length would probably collide with the packets that have higher packet length. These small packets are likely to be lost in collisions because of the potentially full overlapping. To mitigate this issue, an effective solution is required to avoid very small packets generated in the network. Hence, P-ALOHA experiences higher throughput performance by increasing the minimum payload length. However, this increase results eventually in P-ALOHA approaching its standard performance. It is observed that the throughput performance starts to reduce beyond the minimum packet length of 15 bytes. A superior throughput of approximately 0.47 Erlang is obtained with 15–32 bytes of dynamic payload length at the traffic load of 1 Erlang. It is therefore concluded that the policy of dynamic payload length can be considered as a potential means of improving the overall performance of ALOHA-type protocols.

One of the main practical characteristics in the ALOHA protocol is to provide an immediate response regarding the successful reception of transmitted packets, whereby a small-length packet, i.e., acknowledgement (ACK), is sent back to the corresponding transmitter. Such an ACK mechanism supplies ideal flexibility in controlling network environments, with an extra overhead due to exchanging ACK packets. A critical part of the performance evaluations is therefore dedicated to the throughput and delay performances of P-ALOHA under the presence of an ACK mechanism. Figure 7 points out the throughput increases of P-ALOHA over pure ALOHA, revealing a similar performance trend for both P-ALOHA and P-ALOHA with a dynamic payload. P-ALOHA with a dynamic payload typically reaches slightly higher throughput, thanks to its basic mechanism outlined above. The maximum achievable throughput performances for both schemes, around 0.38 and 0.47 Erlang, respectively, without ACK mechanism, are lower, accomplishing around 0.29 and 0.31 Erlangs for the generated traffic of 1 Erlang. This performance degradation is due to the broadcasting of so many ACK packets, resulting in retransmissions of the packets that are not acknowledged. This certainly increases the total number of packet transmissions in the network, causing an undesirable proportion of collisions. Figure 8 presents the delay performances of both schemes in terms of average end-to-end delay with varying network traffic load. For the generated traffic level of 0.1 Erlang, both schemes acquire a similar delay experience as most packets do not wait for long in the queue. However, by increasing the traffic load, both schemes begin to postpone the transmissions, which also allows some generated packets to be destroyed when there is no available place in the queue. P-ALOHA with dynamic payload offers the lowest delay performance, because its throughput performance is higher corresponding to a fast packet delivery property.

We finally report the throughput and delay performances of IoT-ALOHA with respect to varying traffic loads. It is clearly seen from Figure 9 that the effective throughput exhibits an increasing trend along with traffic load, meaning a high rate of successful packet reception. In particular, IoT-ALOHA significantly outperforms P-ALOHA, reaching a maximum achievable throughput of nearly 0.6 Erlang. For dense traffic loads, IoT-ALOHA is able to ensure almost 50% superior throughput performance than P-ALOHA, thanks to the IoT mechanism of transmission via the Internet. Another important superiority of the IoT mechanism can be seen simply in the delay performance, which is illustrated in Figure 10 below. At low traffic loads, IoT-ALOHA and P-ALOHA operate with a similar performance because the majority of the packets require no need for Internet access. When the traffic load is increased, IoT-ALOHA starts to obtain better delay performance because the packets being sent via the Internet spend less time in the queue. More specifically, at the 1 Erlang traffic load, the average delay of IoT-ALOHA is around 40 ms. while the ideal delay of P-ALOHA is achieved at over 70 ms and 60 ms with dynamic payload property.

5. Conclusions

This paper emphasizes the practical aspects of an enhanced design of an ALOHA protocol for a wireless network domain, combined with reinforcement learning and internet of things (IoT) phenomena. A literature review of existing medium access control solutions designed for the theme of ALOHA was conducted, pointing out the working fundamentals and properties. A significant focus is placed on minimizing the problem of packet collisions, maintaining the main performance metrics of throughput and delay at a high level. The main contribution of this article is to introduce lower extra overhead and complexity compared with the state-of-the-art medium access control (MAC) approaches. The first part of the paper presents a practical implementation of ALOHA to observe the performance fluctuations in the light of capture effect. To further improve the performance, we then propose to use dynamic payload length in place of fixed-length packets, an approach which is believed to be a solid contribution. To eliminate deficiencies in a blind transmission policy of ALOHA, IoT and Q-learning are employed to replace the random access nature in an intelligent transmission strategy. The performance evaluations for a representative single-hop topology indicate that the maximum achievable throughput is significantly improved with a better delay effort. To extend the current work, future work will focus on expanding the network topology to multi-hop scenarios, which will require more intelligent solutions against unexpected issues. Another significant shortcoming of the paper is the lack of energy performance measurement. We understand that real test-bed results can be of value, but they are, of course, quite specific to the specific hardware platform used. Such results are rarely found in the literature. An important part of future studies will be dedicated to practical energy measurements by adding the required hardware components to the current node architecture.