Constraints and Recent Solutions of Optical Camera Communication for Practical Applications

Abstract

Visible light communication (VLC) has emerged as a promising technology for wireless communication due to its advantages of the vast optical spectrum, high energy efficiency, and no electromagnetic interference radiation. With the widespread adoption of LED infrastructure and camera-equipped smart devices, optical camera communication (OCC) has gained momentum as a pragmatic version of VLC based on commercial off-the-shelf (COTS) devices. Compared with VLC systems based on photodiodes (PD), the information-carrying capability of OCC enables it to provide a wide range of services in the areas of intelligent transportation, indoor positioning, underwater communication, and the Internet of Things (IoT). This paper presents a brief overview of the OCC system, focuses on the constraints affecting OCC performance, and offers feasible solutions for dependable data transmission in complex and diverse scenarios. Finally, this paper summarizes the potential extended applications of OCC, hoping to push this advanced form of optical wireless communication toward practical deployments in our daily lives.

1. Introduction

With the unprecedented development and breakthrough of wireless device technology, the functions of wireless communication have been greatly expanded. However, delivering high-quality multimedia services to users takes a sizable quantity of bandwidth and extensive network coverage, which means that the current radio spectrum capacity is in a “spectrum crunch” due to its constrained availability [1]. To alleviate the radio frequency (RF) spectrum congestion, researchers are exploring the use of higher frequency bands such as millimeter wave [2,3], terahertz [4], and petahertz bands. As an attractive complementary solution, optical wireless communication (OWC) has been considered for easing the bandwidth usage pressure on RF wireless systems, as it covers the entire optical spectrum ranging from 350 to 1550 nm. In contrast to the bandwidth of 300 GHz in RF technology, OWC technologies provide an enormous theoretical bandwidth of up to 20 THz for infrared (IR), 320 THz for visible light (VL), and 30 PHz for ultraviolet (UV).

The current research on OWC can be categorized into four types based on the spectrum utilization [5]: visible light communications (VLC) [6,7], light-fidelity (Li-Fi) [8], optical camera communications (OCC) [9,10], and free space optical (FSO) communications [11], as shown in Figure 1. As a subset of OWC, OCC is thought to be a practical way to implement VLC on commercial off-the-shelf (COTS) devices and has attracted extensive attention from academic and industrial communities. In particular, the recent development of image sensors assembled in pervasive consumer electronics has created significant opportunities for the practical application of OCC [12]. To be specific, the OCC system includes independent transmitter and receiver modules. For the transmitter module, an OCC system reuses the pervasively deployed light-emitting diode (LED) lighting infrastructure to generate modulated light signals. For the receiver module, an OCC system reuses smart devices (such as smartphones and tablets) equipped with cameras to capture incident light signals in a sequence of image frames [13]. In contrast to VLC systems based on photodiodes (PDs), OCC systems can provide ubiquitous coverage both indoors and outdoors leveraging the existing infrastructure. In addition, OCC systems also have unique features such as spatial and wavelength separation capability, as well as low complexity and high cost-effectiveness. Due to these advantages, OCC is emerging as an essential wireless technology in the foreseeable future.

Upon analysis of previous literature [14,15,16,17,18,19,20,21], it has been observed that the primary emphasis of research on OCC technology has been on increasing its data rate. As the sampling rate of OCC is limited by the camera hardware, several studies have investigated various opportunities for using advanced modulation formats to achieve higher data rates in OCC systems [14,15,16,17,18]. H. Lee et al. designed and implemented a line-of-sight (LOS) OCC system, named RollingLight, which enables for communication via light with various rolling shutter cameras [14]. Specifically, RollingLight employs a high-order frequency shift keying (FSK) modulation technique to transmit data, enabling a throughput of 11.32 bytes per second. Between 2014 and 2018, P. Luo et al. [15] introduced a series of undersampled modulation schemes, including undersampled phase shift ON–OFF keying modulation (UPSOOKM [16]) and undersampled pulse amplitude modulation (UPAM [17]), to realize a flicker-free OCC system with improved spectral efficiency. In addition, Y. Yang et al. presented the novel idea of generating amplitude-shift keying (ASK) through physical light composition and an efficient demodulation algorithm, achieving a throughput of 80 kbps at a distance of 2 m [18]. Another idea for improving the data rate is to add the capacity of multiplexing or MIMO (multi-input and multi-output) to OCC [19,20]. N. Jiang et al. designed and implemented a wavelength division multiplexing (WDM)-MIMO NLOS OCC system with multi-level pulse width modulation and difference-based pulse width recognition schemes to realize a data rate of 10.8 kbit/s over an NLOS link of more than 2 m [19]. Y. Yang et al. presented a practical OCC system named ReflexCode, which adopts reflected light as its communication media and codes information by superposing light emissions from multiple transmitters [20]. Experimental results demonstrate that ReflexCode can achieve a throughput of up to 3.2 kbit/s at a distance of 3 m.

Over the past decade, researchers have conducted detailed surveys/reviews on different aspects of OCC systems, in addition to striving to improve data rates [9,10,22,23,24,25]. N. Saha et al. [22] presented a study covering the challenges and opportunities of OCC systems. N. Le et al. [23] published a survey regarding the design and implementation of OCC, while offering insights into potential future directions. N. Saeed et al. [24] presented a comprehensive survey on optical camera-based communications, localization, navigation, and motion capture. In addition, W. Liu et al. conducted a survey that delved into practical constraints and solutions for OCC [9]. In another survey, S. Mohsan et al. examined principles and standardization activities related to OCC, exploring topics such as practical constraints and modulation schemes for OCC systems [10]. In a recent survey published in 2022, A. Liu et al. presented a comprehensive system model to characterize OCC systems [25]. Despite this literature providing a generalized overview of the OCC technologies, there appears to be a deficiency in discussing the constraints associated with implementing OCC systems for realistic applications. The performance of OCC systems during practical implementation is primarily constrained by the following factors:

• Time Consuming in Region of Interest (RoI) Extraction;
• Vulnerable to Complex Image Background Interference;
• Non-uniform Grayscale Distribution;
• Low Signal-to-Noise Ratio (SNR).

In light of these constraints, there is a need for an in-depth study on OCC systems and for seeking out potential and feasible solutions to push OCC into real-life deployments. To this end, this paper first attempts to provide a brief investigation of OCC which mainly focuses on the practical constraints and state-of-the-art solutions, then it takes into account potential applications and future research directions, thereby contributing to the development of OCC. The rest of the paper is structured as follows. Section 2 presents an overview and related studies on OCC. Section 3 examines the key factors that impact the performance of OCC and provides corresponding solutions. Section 4 discusses several potential applications of OCC. Section 5 articulates future research directions related to OCC. Finally, Section 6 concludes the paper.

2. Brief Overview of Led-Based OCC

There are three main types of OCC based on their transmitter types: (i) liquid crystal display-based OCC, such as screen–camera communication [26,27], defined as LCD-OCC; (ii) LED panel-based display–camera communication, defined as LED-DCC [28]; and (iii) LED-based OCC, such as ColorBar, CASK [29,30,31], defined as LED-OCC. Specifically, LED-OCC adopts high-efficiency LED lighting infrastructure as the transmitter and camera-equipped smart devices as receivers in order to realize ubiquitous data communication. LED-DCC uses LED panels to display information that needs to be transmitted and a camera to receive the information. The key difference between LED-DCC and LED-OCC is that the former uses LED displays that cannot be used for illumination, while the latter uses LEDs that can also serve as illumination sources. As a result, LED-DCC technology is more suitable for specific applications, such as advertising and interactive displays. In contrast, LED-OCC can be used in a wider range of applications where illumination is also required, such as indoor/outdoor lighting. In addition, LCD-OCC employs the liquid crystal display as the transmitter, but the expensive screen, complicated decoding, and limited range hinder LCD-OCC from having an enormous market like LED-OCC. Therefore, this section focuses on a brief overview of LED-based OCC technology.

2.1. OCC Standardization

IEEE specifications and/or standards act as an important bridge to push academic research into industrial products and stipulate application scenarios and fundamental technical requirements for emerging technologies. In order to facilitate the implementation and commercialization of VLC, the IEEE Standards Association published the VLC technology standard, IEEE 802.15.7-2011 [32], which includes the link layer (DLL) and physical (PHY) layer design specifications. Due to the growing interest in OCC technology, the IEEE Standards Association granted a new project authorization request from IEEE 802 for VLC to amend the IEEE 802.15.7-2011 standard for better and more application-enabling standards, IEEE.802.15.7m [33]. The IEEE 802.15.7m OWC Task Group (TG7m) is dedicated to revising the previously established IEEE VLC standards, proposing different PHY layer modes for LED-ID and OCC with revised media access control (MAC) methodology [34], which greatly influenced the advancement of the image sensor communication technology, a.k.a OCC. Interested readers are directed to Reference [34] for the standardization of OCC and achievements of TG7m.

2.2. OCC Principles

The standardization of IEEE 802.15.7m has significantly facilitated the implementation and commercialization of OCC systems. Figure 2 illustrates the basic principle underlying the OCC system. It can be seen that the OCC system is composed of three primary components, namely a transmitter, channel, and receiver. In the transmitter module, data can be modulated using different modulation techniques, such as on–off keying (OOK) [16], color shift keying (CSK) [35], multilevel intensity modulation (m-IM) [36], etc. Afterward, the LED driver is employed to activate the LED to emit modulated light. In the receiver module, different optical beams carrying the data stream arrive at the receiver through the wireless channel and are captured and distinguished simultaneously by the image sensor [12]. The image sensor initially converts the incident light signal into an electrical signal, which is subsequently quantized into an image containing stripes and ultimately compressed into a specific image format. Subsequently, the communication stripes present in the image are converted into grayscale values by means of RoI partitioning and pixel sampling [37]. The demodulator is finally required to convert the grayscale values into bits.

2.3. OCC Transceivers

2.3.1. OCC Transmitter

As a subset of OWC, OCC typically uses high-frequency strobe light signals from daily lighting fixtures to transmit data that cannot be perceived by the naked eye. Currently, the most commonly used lighting fixtures have LED lamps, fluorescent lamps, and incandescent bulbs. Across various lighting fixtures, LED-based lighting infrastructure is widely deployed due to its high energy efficiency, small size, and long lifespan. The large-scale promotion and popularity of LEDs provide a practical application foundation for the development of OCC systems. In addition, its fast rate of ON/OFF switching characteristics also makes LEDs the preferred choice for transmitters in OCC systems. In this vein, OCC systems can transmit signals directly utilizing commercial LED light sources, significantly reducing the cost of implementation and installation.

Considering that white light is more suitable for people’s daily lighting needs, the OCC system usually adopts white LEDs as its transmitter. In fact, white light is a kind of mixed light which is formed by the mixture and superposition of several colors of light. There are currently three varieties of white LEDs based on their hybrid light-emitting principle: (i) Blue LED + yellow phosphor (PC-LED). The blue LED chip is used to excite the yellow phosphor to produce yellow light, and then the blue light and yellow light are mixed to form white light. (ii) RGB-LEDs. Package monochromatic LED chips of three primary colors of light (red, green, blue) together so that they intersect together to synthesize white light. (iii) Ultraviolet LED + RGB phosphors (RGB+UV-LED). The ultraviolet LED is coated with a tricolor phosphor, and white light is produced under ultraviolet irradiation. Different types of white LEDs have different properties that may impact the energy efficiency of the system—for example, PC-LED, with its low cost and straightforward design, but slightly lacking luminous efficiency. RGB-LEDs can avoid the energy loss of phosphors during the light conversion process, thereby obtaining higher luminous efficiency. However, the use of multiple LED chips in an RGB-LED package can lead to increased complexity and higher power consumption compared to PC-LED. Therefore, careful selection and optimization of the LED light source are critical for achieving optimal performance in an OCC system.

2.3.2. OCC Receiver

Unlike traditional PD-based VLC systems, OCC systems employ an image sensor-based commercial camera as a detector to receive data, where a typical camera structure consists of an imaging lens, image sensor, and readout circuit [38]. Among them, the imaging lens is a sophisticated optics device and is used to focus the image of an object onto an image sensor. The image sensor is a two-dimensional array of photodetectors used to receive spatial information and then form images on the display. Therefore, the image sensor is able to classify multiple spatially separated light sources with high resolution. In addition, it has a color filter that independently illuminates multiple colors, so it can act as a natural multi-color receiver. In summary, image sensor-based receiving devices are suitable for imaging optical MIMO [39] and can support the parallel transmission of large data volumes. Charge-coupled devices (CCDs) and complementary metal oxide semiconductors (CMOSs) are two types of image sensor technologies used in digital cameras and other imaging devices. The appearance of CCDs and CMOS is shown in Figure 3. Generally, CCDs require more power than CMOS sensors, as they use an external power source to shift the charge between pixels. CMOS sensors, on the other hand, use internal transistors to amplify the charge, which reduces power consumption. Besides, CMOS sensors are generally cheaper to produce than CCDs, as they require less-complex manufacturing processes. This makes them more affordable for consumer-grade cameras and other imaging devices. The image sensors have two modes of operation: global shutter operation and rolling shutter operation, as shown in Figure 3. Global shutter is normally used in CCD image sensors [40], while rolling shutter is more suitable for low-cost CMOS image sensors. In the global shutter mode, each light sensor unit synchronously collects light and is simultaneously exposed. After the exposure is completed, the information for each pixel is read out sequentially, this mechanism is beneficial for capturing moving objects, thus global shutter is a great way to obtain high-quality images as far as multimedia services are concerned. However, the global shutter mode only records the information sent at a certain moment by the transmitter. In the rolling shutter mode, instead of collecting all the light simultaneously, the photodiode sequentially scans (column-by-column or row-by-row) the pixels using the CMOS image sensor to generate the image. The rolling shutter offers advantages for OCC, as multiple ON/OFF states of the light source are also captured in the image to represent the binary bits “0” or “1”.

2.4. OCC Channel Transmission Types

In the OCC systems, the transmitters (illumination sources) are generally deployed on ceilings or walls of indoor spaces. The light signal can be transmitted either by direct line-of-sight (LOS) or indirect non-line-of-sight (NLOS) [41], as shown in Figure 4. In a LOS-OCC system, the camera faces directly toward the LED transmitter and captures the light emitted by the light source. The receiver can receive data and observe the position of the transmitter at the same time. Normally, its maximum data rate is determined by the size of the LED luminaire, as only the RoI contains valid data bits in the received frames. Therefore, the transmission rate is significantly impacted by the size of the spot and the distance between the transmitter and receiver. In addition, direct light cannot pass through obstacles and is prone to signal interruption. In an NLOS-OCC system, the camera is positioned towards an observation plane and detects the light reflected from the observation plane (e.g., walls or floors). Therefore, the maximum data rate of an NLOS-OCC system depends on the size of the illuminated area rather than the size of the LED luminaire. However, the majority of reflection occurrences in nature entail diffuse reflections, resulting in the dispersion of parallel light following reflection. Thus, the signal power that ultimately reaches the receiver is diminished. In real-world scenarios, it may be beneficial to choose a particular channel transmission type that suits the specific application context.

3. Analysis of Key Factors Affecting OCC Performance

Despite the significant advantages and potential demonstrated by OCC, there are still challenges that restrict its performance in practical implementation. During the implementation phase, a variety of issues are encountered in the OCC system, such as time consumption involved in RoI extraction, interference from complex image backgrounds, non-uniform grayscale distribution, blooming effect, extinction ratio, and low SNR. Therefore, this section aims to present a condensed overview of key technologies that can effectively address above constraints.

3.1. Time Consumption Involved in RoI Extraction

The extraction of RoI is a critical step in the decoding process of OCC since only the strip-shaped region (Region of Interest) carries the desired optical communication information in the received image. Despite the ease with which the naked eye can identify RoIs in photographs, the automatic extraction of RoIs by smartphones is far from trivial due to the spatial distribution of the transmitter. Fortunately, with the help of advanced computer vision (CV) technology, precise optical signals can be effectively identified from received frames [37]. The conventional process for detecting RoIs using CV methods is depicted in Figure 5. After capturing the received image, it is first converted to grayscale, blurred, and then passed through a binary OTSU filter [42]. Subsequently, the edge detection algorithm is utilized to detect and outline the boundaries of each transmitter and ascertain the minimal enclosing circle for each contour. Finally, each subregion of the image is examined independently in order to decode data from each light. It is evident that RoI extraction involves a tedious process that must be executed with the utmost attention to detail. Meanwhile, the process of Gaussian blur involves constructing a matrix of weights for filtering, which can lead to significant time consumption.

Recent breakthroughs and rapid adoption of deep learning (DL) methods, such as R-CNN, YOLO, etc., have been instrumental in enabling accurate object detection. In the field of OCC, these methods have been widely employed for detecting the RoI from the captured image [44,45]. For example, T. Pham et al. applied the full and tiny YOLOv2 model for RoI detection and high tracking performance in vehicular OCC systems [44]. In addition, D. Choi et al. presented a DL-based RoI detection method that rapidly and accurately extracts OCC data from successive image frames while simultaneously detecting the RoI using a trained neural network model [45]. However, a common problem of DL-based methods is the high computational complexity and long time required for the model to figure out the RoI. A viable solution to extenuate the computational burden and processing time is to implement a lightweight RoI detection method. Recently, X. Hu et al. presented a fast RoI extraction method based on image segmentation in order to alleviate the delay of RoI extraction in the OCC system [43]. Currently, the research on lightweight RoI extraction is limited, necessitating further investigation and the development of novel methodologies by researchers to propel this study forward.

3.2. Complex Image Backgrounds Interference

Upon completion of the RoI extraction process, it is crucial to execute stripes demodulation within the RoI for the purpose of recovering data. However, the close interactions between LOS-OCC/NLOS-OCC systems and the indoor spaces inevitably expose the communication stripes to interference from complex image backgrounds, making it difficult to achieve low complexity and accurate demodulation. Therefore, the resolution of this crucial matter is imperative in the realm of OCC research.

3.2.1. LOS-OCC Scenarios

In a LOS-OCC scenario, direct line-of-sight is required between the transmitter and receiver devices, which minimizes susceptibility to interference from complex image backgrounds. However, when utilizing LED panels embedded with advertisements as the transmitter in LOS-OCC systems, the communication stripes may suffer from interference arising from the complex image backgrounds in the LED billboards. This interference can lead to communication errors, reduced data rates, and decreased reliability, as illustrated in Figure 6a. In order to mitigate the effect of complex image backgrounds in LED billboards, various techniques have been proposed [46,47,48,49]. These techniques include background subtraction [46], which involves subtracting the background image from the received image in order to extract the signal, and adaptive thresholding, which involves adjusting the threshold for image detection based on the intensity of the received signal. Additionally, some researchers have addressed similar issues by using machine learning (ML) approaches. In Reference [48], Y. Chuang et al. suggested using logistic regression classification to mitigate the high noise ratio of advertising panels. In Reference [49], K. Hsu et al. proposed and demonstrated an OCC system based on LED display panels that uses grayscale value distribution together with an ML algorithm to enhance the demodulation. Arguably, the implementation of these innovative schemes has augmented the practicability of OCC systems in real-world scenarios.

3.2.2. NLOS-OCC Scenarios

In the LOS-OCC system, the camera requires constant alignment with the LED luminaire (transmitter), which leads to an unusual use case since the lighting fixtures are typically installed on the ceiling of indoor space. The NLOS-OCC system presents itself as a highly practical option for users as it can effortlessly capture OCC signals reflected from any surface. Recently, NLOS-OCC systems have gained widespread attention in both academic and industrial communities [20,50,51,52,53,54]. In 2017, Y. Yang et al. designed a grayscale superposition higher-order modulation and a robust demodulation algorithm based on collaborative transmission, which effectively suppresses inter-illumination interference and enhances the communication performance of the NLOS-OCC systems [20]. During the same year, W. Wang et al. proposed and demonstrated an approach for detecting long-distance light signals in an NLOS system by utilizing the rolling shutter pattern of a commercial mobile phone camera [50]. Despite the studies conducted, the NLOS-OCC systems described above assume that the reflection surfaces are pure white, which may overlook possible interference from patterns or colors present in complex image backgrounds.

In the NLOS-OCC scenario, light can be reflected from the surface of any object located inside a room. However, the communication stripes suffer from negative impact due to intense reflection or absorption when light encounters glossy or dull materials. As a result, the SNR of the optical signal will have a significant decline. As shown in Figure 6b, the black sections of the ink painting pattern (reflective surface) are highly light-absorbing and severely affect the display of communication stripes. Considering these issues, numerous research teams have conducted systematic and in-depth studies within this particular field [51,52,53,54]. In 2018, F. Yang et al. designed and implemented an NLOS-OCC system that separated the captured optical signals from the complex image background, thereby improving the robustness of the NLOS-OCC systems [51]. In 2019, J. Lain. et al. proposed a modified background subtraction method to restore the damaged pixels caused by the heterogeneous reflective background [52]. In 2021, L. Liu et al. presented a real-time flicker-free OCC system that allowed for the retrieval of descriptions from the complex optical signals reflected by an exhibition poster or artwork using a hand-held smartphone [53]. In 2022, P. Zhang et al. introduced an innovative algorithm named Gsnake, which aimed to restore damaged pixels disturbed by the color or texture of the reflective surface in realistic scenarios [54]. These methods have proven to be effective in mitigating the adverse effects of complex image backgrounds on optical communication stripes, thereby enhancing the reliability and robustness of NLOS-OCC systems in practical applications.

3.3. Non-Uniform Grayscale Distribution

In an NLOS-OCC system, the receiver device is placed facing an observation plane (e.g., a wall) and captures the light reflected from the observation plane. Thus, the maximum data rate of the NLOS-OCC system is typically determined by the size of the observation plane being illuminated. However, the irradiance distribution on the observation plane is non-uniform, resulting in lower SNR in the received frames and limiting the data transmission rate of the OCC system. As shown in Figure 7, the non-uniform illumination pattern on the observation plane generates higher grayscale in the center and gradually decreases towards the edges. Therefore, the SNR of the frame in the edge area is much lower than that in the central area, which complicates the demodulation process and seriously degrades the communication performance of the camera-based OCC system.

In prior studies, several approaches have been proposed to mitigate the effects of non-uniform SNR and improve demodulation accuracy [55,56,57,58,59,60]. For example, C. Chow et al. proposed and demonstrated the use of histogram equalization and a Sobel filter to reduce the impact of the background noise and transform the grayscale levels with high uniformity [56]. In the subsequent year, the aforementioned research team suggested two entropy thresholding algorithms aimed at addressing the issue of non-uniform SNR in OCC systems [57]. In 2018, L. Liu et al. proposed a novel frame averaging-based signal tracing algorithm for improving the OCC performance [58]. That same year, Z. Zhang et al. proposed a thresholding scheme based on boundary pixels of stripes to deal with non-uniform SNR [59]. In order to further reduce the consumption of computing resources and make OCC more suitable for mobile devices with limited energy and computation. In 2020, Z. Chen et al. took advantage of the uniform light emission of an LED luminaire with the lens to increase SNR [60]. In 2021, the same group proposed a free-form lens-based LED transmitter design method that uses a free-form lens to process the modulated light to form a uniform illumination spot on the observation surface [55]. The high uniformity of the illumination pattern makes it simple and accurate when demodulating the received signal, which has a uniform and high SNR.

3.4. Signal Demodulation and Decoding

Signal demodulation plays a crucial role in enhancing the performance of OCC systems, which has garnered significant attention in the OCC research. Selecting the appropriate column matrix and threshold scheme is the key step in optimal signal demodulation for OCC, while the process of signal demodulation is usually affected by problems such as blooming effect [61], extinction ratio (ER) fluctuation [62], or low SNR [63]. First, the blooming effect is caused by the overflow of charge from the saturated pixels into the neighboring pixels. Thus, the width of these stripes is distorted in the blooming area. Second, the high data rates will cause high ER fluctuation, thereby impeding the accurate restoration of logic data through the use of a threshold. Finally, the occurrence of low SNR can be attributed to signal fading and significant noise interference. The accuracy of converting grayscale values to logical values can be adversely affected by the aforementioned phenomena, ultimately resulting in the deteriorated performance of the OCC system. In order to enhance the decoding effectiveness of OCC in real-world scenarios, a multitude of research investigations related to column matrix selection and threshold schemes have emerged in the past few years.

Numerous methodologies for column matrix selection have been presented in prior research [61,62,63,64,65,66]. For example, C. Chow et al. suggested a method for mitigating the blooming effect by grouping every 60 rows of pixels in the column matrix selection for grayscale values. This approach enabled the OCC system to realize a data rate of 1.68 kbit/s [61]. In 2017, Z. Zhang et al. suggested a method for sorting the elements of each row matrix of grayscale in descending order to reduce random noise interference of adjacent pixels and jumps in grayscale values [62]. In 2018, J. He et al. presented a dynamic column matrix selection algorithm supporting user mobility, which allowed for a data rate of 4.08 kbps [64]. In 2019, the same research team proposed a new column matrix selection method that utilized the energy diffusion of LEDs to mitigate the blooming effect and enhance the demodulation performance of OCC systems [65]. C. Chow et al. introduced a novel column matrix selection approach that can achieve a data rate of 1.51 kbit/s when the receiver undergoes translational or rotational motions at a distance of 100 cm [66]. In 2021, P. Zhang et al. proposed a signal enhancement scheme that superimposed the grayscale matrix by columns in order to obtain a new grayscale vector for demodulation [63]. The above scheme is a compelling illustration of the efficacy associated with the selection of a suitable column matrix when it comes to demodulating signals.

In addition to the column matrix selection scheme, many researchers have proposed various innovative methods related to threshold schemes for signal demodulation [67,68,69]. In 2017, C. Chow et al. proposed and demonstrated a novel threshold scheme that achieved a net data rate of 7.68 kbit/s [68]. In 2018, the same research team designed a modified adaptive threshold scheme for demodulation in order to reduce the effect of the high signal fluctuation caused by the uneven light exposure [69]. Furthermore, several researchers have suggested the utilization of other cutting-edge strategies for decoding [49,70,71,72,73]. For example, L. Liu et al. proposed a two-dimensional convolution neural network (CNN) architecture for symbol decoding, enabling a high data rate of 47 kbit/s [70]. During the year 2020, Y. Meng et al. suggested a precise image decoding algorithm, replacing function fitting with critical grayscale to decode images [71]. From 2020 to 2023, J. He et al. designed a CNN-based decoding scheme [72] to alleviate the stripe distortion in the mobile environment and a sub-column pixel neural network-based scheme [73] to effectively mitigate the effects of inter-system interference (ISI) and noise on the signal decision. It is evident that implementing an efficient signal demodulation scheme can mitigate the negative effects experienced during the signal demodulation process, thereby improving the performance of the OCC system.

4. Potential Applications

OCC has been considered as a practical wireless communication technology due to the reuse of existing camera-based receivers that are widely readily in smart devices. This flexibility of the camera-based receivers has led to OCC being applied in diverse fields, such as intelligent transportation systems, indoor localization and navigation, underwater communication, and IoT connectivity. Thus, this section summarizes several potential applications for OCC.

4.1. OCC-Based Intelligent Transportation Systems

The current outdoor environment features a significant amount of LED infrastructure, and most vehicles are equipped with integrated cameras, making OCC a highly attractive communication technology. LEDs have the characteristics of low power consumption and shorter response time. The spatial separation capability of the image sensor incorporated in the camera offers significant immunity from interference. The aforementioned features are crucial for vehicular communication, rendering OCC substantially more advantageous than other OWC technologies. As shown in Figure 8, the infrastructure located alongside roads, such as traffic signals, nearby vehicle headlights, brake lights, and streetlights, sends data that is received by the image sensors installed on the vehicles [74,75]. Thus, OCC-based intelligent transportation systems (ITSs) enable vehicle-to-infrastructure (V2I), infrastructure-to-vehicle (I2V), and vehicle-to-vehicle (V2V) communication.

In OCC-based ITSs, the data transmitted through LEDs encompasses parameters such as the vehicle’s longitude, latitude, and safety-related information. Moreover, driving assistance data can be communicated from one vehicle to another in order to enhance safety and the driving experience. In recent years, vehicle OCC systems have shown tremendous promise for enhancing the effectiveness and security of surface traffic. Therefore, OCC-based ITS has sparked the interest of the research community and the industry [76,77,78]. For example, in 2014, P. Ji et al. investigated the possibility of using existing LED lights in a vehicle or a traffic light and the camera of a smartphone to carry out Vehicular VLC (V2LC) [76]. In the identical year, Z. Cui et al. took the first look at the channel fading of a V2LC link caused by vehicle mobility [77]. In 2023, K. Xu et al. unveiled its latest project, the NeuromorphicVLC. This prototype is equipped with neuromorphic cameras that aim at vehicular communication and networking [78]. These above studies have contributed to the advancement of its implementation in practical applications.

4.2. OCC-Based Indoor Positioning Systems

With the popularity of LED lighting technology and the rapid development of high-resolution CMOS sensor, OCC-based indoor positioning systems (IPSs) have ushered in vigorous development and broad prospects [79]. In OCC-based IPSs, multiple LEDs transmit their identifiers or coordinate information. At the receiver, the camera is used to decode that information and determine its position and orientation relative to the LEDs [22]. Compared to other indoor positioning systems that utilize RF, such as RF identification (RFID), wireless local area network (WiFi), and Bluetooth positioning technologies, indoor positioning systems based on visible light present several advantages [80]. These include a high level of accuracy in positioning, immunity to electromagnetic interference, low-cost front ends, and the capability to perform positioning and illumination tasks simultaneously. In particular, the positioning accuracy of OCC-based IPSs is within the centimeter range [81,82], which is superior to RF-based positioning technology.

In recent years, the integration of OCC into indoor positioning technologies has been extensively researched [81,82,83,84,85,86]. For example, Y. Li et al. presented a real-time OCC-IPSs that employs a novel non-iterative perspective-n-point algorithm to estimate the camera position [81]. B. Lin et al. experimentally demonstrated a VLC-IPS based on a commercial camera, which can provide error-free data transmission and reasonable positioning errors for a height of 180 cm [83]. X. Liu et al. designed and implemented an indoor visible light localization system under dimmable LEDs, which can handle the blurring effects and use only two LEDs for positioning [84]. A study conducted by H. Song et al. introduced a decoding scheme that is both versatile and efficient, as it can be utilized not only in OCC-based VLP but also in other communication domains to enhance decoding performance [85]. B. Hussain designed an IPS based on pedestrian dead reckoning (PDR) that uses VLP for pedestrian step length estimation and heading angle calibration, while addressing the device heterogeneity and user diversity challenges of PDR [86].

4.3. OCC-Based Underwater Communication

Underwater wireless optical communications (UWOC) have gradually attracted considerable attention as an alternative technology to conventional acoustic communication [87,88]. This can be attributed to the unique and remarkable advantages of UWOC technology, including its available bandwidth and the feasibility of utilizing COTS devices like LED lighting lamps. The Underwater Optical Camera Communication (UWOCC) system, as a subsystem of UWOC, has sparked the research interest of researchers by using LEDs as transmitters and the embedded CMOS camera of smart devices as receivers for underwater communication [89,90]. During the past years, researchers have explored UWOCC through theoretical and experimental studies [89,90,91,92]. For example, M. Akram et al. proposed a MIMO OCC-based underwater wireless link and provided a comprehensive framework of its design and implementation techniques [89]. Z. Zhou et al. introduced a de-bubble algorithm and binary fringe correction to enhance the robustness of the UOCC system in the presence of air bubbles. This approach enables the UOCC system to achieve a data rate of 7.2 kbit/s [91]. A. Shigenawa et al. suggested a predictive equalization technique for UWOCC to eliminate the effect of the attenuation of light intensity in underwater environments [92]. We believe that the development of related technologies will help drive the future deployment of UWOCC.

4.4. OCC-Based IoT Connectivity

The Internet of Things, commonly referred to as IoT, represents the network of physical devices and sensors within smart environments. This interconnectivity enables objects to communicate and exchange data between themselves. Nowadays, IoT has demonstrated its tremendous potential by providing the ability for billions of connections of devices around the world. The IoT paradigm opens the door to innovations, enabling the realization of smart cities, infrastructures, and services to enhance the quality of life and improve the utilization of resources [93,94]. As a particular type of OWC, OCC with a huge frequency spectrum integrated with IoT can offer extensive possibilities for indoor and outdoor applications in future smart environments like smart cities, smart homes, smart farming, and smart factories, as shown in Figure 9.

Within an IoT network, the use of cameras in OCC systems enables the efficient acquisition and monitoring of data emanating from diverse devices and sensors. After receiving data from multiple devices, the OCC system will process the information and transmit it to a gateway using either a wired or wireless network. The data will be forwarded to backhaul networks, such as 5G or satellite networks, which will then be stored in a server or the cloud to enable IoT-based applications. In 2017, P. Chavez-Burbano et al. proposed a long-distance OCC system for relatively slow data rate applications in smart cities [95]. In 2019, M. Chowdhury et al. published a review that clearly described how OWC technologies will be an effective solution for the successful deployment of IoT systems [5]. In 2020, N. Van et al. presented an analysis of the prospective progress of OCC-based IoT networks [96]. We believe that the implementation of OCC technology can enable a wide range of IoT network applications.

Figure 9. OCC–IoT-based smart farming [97].

Figure 9. OCC–IoT-based smart farming [97].

5. Future Research Directions

The integration of sensing and communication (ISAC) emerged as a new enabling technology due to recent advances in wireless communication and the huge demand for sensing capabilities. This technology offers two primary advantages over dedicated sensing and communication functions [98]: (i) the integration gains from the efficient use of congested wireless/hardware resources and (ii) the coordination gains to balance dual-functional performance or the execution of mutual assistance. The issue of enabling both communication and sensing at the same time is a significant challenge for visible light-based applications since it unlocks the full potential of widely adopted LED lighting infrastructures and embedded cameras. With regards to the LED lighting infrastructure, the utilization of light as a sensing medium has the capability to detect and recognize gestures and postures [99,100]. With regards to the embedded camera, leveraging the immense capabilities of image sensors can perform other roles while maintaining communication [101,102]. A potential course of action would be to explore the possibilities of conducting additional research on the application of ISAC technology in the OCC domain.

In general, surveillance cameras transmit data by connecting to existing Internet or internal networks. Therefore, OCC can quickly connect to existing networks without the need for additional backbone networks. This not only simplifies system setup and deployment, but also improves data transmission security and reliability, avoiding potential signal interference and eavesdropping issues that may exist in traditional wireless communication technologies. With the continuous expansion of OCC application scenarios, achieving large-scale OCC networking is an important research direction. Researchers are exploring new topologies and routing algorithms to achieve large-scale OCC networking. In addition, drones typically carry one or more cameras and LED indicators that can form OCC communication links. Therefore, OCC applications within drones have the potential to enable communication between drones. Drones equipped with OCC can perform the following functions: (i) transmit control information to other drones for situational awareness, surveillance, and other applications; (ii) exchange sensor data between drones, such as location, altitude, speed, and other telemetry data; etc. However, drones may move rapidly and change their orientation, which can cause occlusion and shadowing of the light signals. This can lead to interruptions in communication or even complete loss of signal. Thus, the reliable optical communication of drones in motion is a potential research direction in the future. Meanwhile, drone-to-drone (D2D) communication may require high data rates for video streaming and sensor data exchange. Therefore, the design of the OCC system needs to be optimized for the specific application requirements and constraints. This includes the modulation scheme, encoding, and decoding techniques, as well as the hardware design of the camera and LED light sources.

6. Conclusions

In this paper, we briefly introduce OCC technology that combines lighting, communication, and imaging on a single platform. Specifically, this paper describes several key aspects of OCC technology, including IEEE standardization activities, principles, transceivers, channel transmission types, practical constraints, and feasible solutions. Additionally, this paper highlights the potential applications of OCC technology and reports on future research directions in this field. It is noteworthy that this article discusses the key factors affecting the performance of OCC and seeks potential and feasible solutions to push OCC into real-life deployments. In a nutshell, the present article provides valuable insights to novice readers on the current state of OCC development from multiple perspectives. Based on current technological advancements and ongoing research, we believe that OCC will unlock substantial potential in the next era of wireless technologies.