Indoor Localization Based on Integration of Wi-Fi with Geomagnetic and Light Sensors on an Android Device Using a DFF Network

Abstract

Sensor-related indoor localization has attracted considerable attention in recent years. The accuracy of conventional fingerprint solutions based on a single sensor, such as a Wi-Fi sensor, is affected by multipath interferences from other electronic devices that are produced as a result of complex indoor environments. Light sensors and magnetic (i.e., geomagnetic) field sensors can be used to enhance the accuracy of a system since they are less vulnerable to disturbances. In this paper, we propose a deep feedforward (DFF)-neural-network-based method, termed DFF-WGL, which integrates the data from the embedded Wi-Fi sensor, geomagnetic field sensor, and light sensor (WGL) in a smart device to localize the device in an indoor environment. DFF-WGL does not require complex and expensive auxiliary equipment, except for basic fluorescent lamps and low-density Wi-Fi signal coverage, conditions that are easily satisfied in modern offices or educational buildings. The proposed system was implemented on a commercial off-the-shelf android device, and performance was evaluated through an experimental analysis conducted in two different indoor testbeds, one measuring 60.5 m² and the other measuring 38 m², with 242 and 60 reference points, respectively. The results indicate that the model prediction with an input consisting of the combination of light, a magnetic field sensor, and two Wi-Fi RSS signals achieved mean localization errors of 0.01 m and 0.04 m in the two testbeds, respectively, compared with any subset of combination of sensors, verifying the effectiveness of the proposed DFF-WGL method.

1. Introduction

The indoor localization of smartphones is typically achieved by using embedded sensors to gather information about the surrounding environment [1,2,3,4,5,6,7]. These sensors can detect a variety of signals, including electronic signals from Bluetooth or Wi-Fi sensors; light signals from light or camera sensors; motion or inertial signals from accelerometers, gyroscopes, and magnetometers given in inertial measurement units (IMU); etc. These signals are then processed to estimate the smartphone’s location within an indoor environment. In addition to the schemes based on using the signals acquired from smartphone platforms for indoor localization, wireless sensor networks (WSNs) [8,9] and Frequency-Modulated Continuous Wave radar (FMCW) [10,11,12]-based schemes have garnered research attention in recent years.

On a smartphone platform, Wi-Fi and Bluetooth signals are the two main electronic signals used for indoor localization. Even though channel state information (CSI)-based Wi-Fi [13] can be used to track the location of smartphones with higher accuracy than received signal strength (RSS)-based Wi-Fi localization, this method requires specific hardware and software that are not commonly found in most commercial devices. However, RSS-fingerprint-based radio map methods are commonly researched with regard to both Wi-Fi [14] and Bluetooth localization [15]. These methods require a minimum infrastructure consisting of Wi-Fi access points (AP) and Bluetooth Low-Energy (BLE) beacons.

On the contrary, the geomagnetic field is a naturally occurring, ubiquitous, and relatively stable phenomenon. Furthermore, the use of geomagnetic field fingerprinting as a localization technique presents an appealing alternative to Wi-Fi- and Bluetooth-fingerprinting methods, as it does not rely on any infrastructure. Some authors have performed geomagnetic fingerprinting for indoor localization [6,16,17,18]. On the one hand, electronic signals are less stable and pervasive than the geomagnetic field. On the other hand, the geomagnetic field has a consistent presence, and its force fingerprinting can exhibit distinctiveness within a confined indoor space, making it a viable option for fingerprint-based indoor localization.

In comparison to electronic signals and geomagnetic field fingerprinting, light signals are more stable and less prone to interference, making them a potentially reliable option for indoor localization. Moreover, the unique fingerprinting of light signals can offer precise localization information within a confined indoor space. Some studies have explored the use of visible light communication signals and smartphones for indoor localization [19,20]. However, light sensor localization schemes suffer from the drawback of being limited by dark or non-line-of-sight (NLOS) environmental conditions.

In conventional RSS-based Wi-Fi localization schemes that do not use fusion, numerous Wi-Fi RSS samples are required during the site surveillance process to maintain high localization accuracy. Due to hardware limitations with regard to scanning Wi-Fi signals, the Wi-Fi scan speed is usually not very fast, making the site surveillance process laborious and tedious. Approaches involving crowdsourcing [21,22,23] have been commonly employed to tackle this issue; however, they are susceptible to a decrease in accuracy due to variations in device accuracy. In the event of the unfeasibility of crowdsourcing, we aim to achieve precise localization and reduce site-surveying time by incorporating data from light and magnetic sensors with a higher sampling rate than that of Wi-Fi scanning whilst minimizing the number of Wi-Fi sample data.

Fingerprint augmentation (FA) is another method used in Wi-Fi-fingerprint-based localization schemes in which the fingerprint of each reference point (RP) in a fingerprint database is used to generate a virtual RSS fingerprint via interpolation [24,25,26]. This technique leverages the known RSS fingerprints at RPs to estimate the signal strengths at locations in between, effectively expanding the coverage of the database and improving localization accuracy. Linear and k-nearest neighbor (KNN) interpolation [27] are commonly employed for this purpose, allowing a system to make accurate predictions even in areas with limited or no direct fingerprint measurements.

To construct the training dataset for the deep learning model by combining data from multiple sensors, we employed Sequential Feature Selection [28]. In this technique, there is an initially empty set of features, which are incrementally added through a search across the feature space. Then, the features that contribute to enhancing the overall accuracy of the model are selected. This systematic approach ensures that the most informative sensor data are included in the training dataset, facilitating the development of a robust and accurate model. Furthermore, it allows for the efficient utilization of sensor data, optimizing a model’s performance for various applications in sensor-based tasks.

The proposed method is designed for localizing robots by equipping a tablet-like device on the top part of the robot rather than for human targets equipped with smartphones in their hands or pockets. This is because the direction of and gesturing applied to smartphones are more complex, and in such scenarios, an accelerometer and a gyroscope yielding measurements in IMU should be used, while the collected light sensor data may be less related to the actual target’s position. In summary, the main contributions of this study are as follows:

• We proposed a fusion method to integrate the fingerprint information from the embedded light sensor, magnetometer, and Wi-Fi sensor in a commercial off-the-shelf Android device for indoor localization. The method does not require heavy equipment or expensive infrastructure, so it can be quickly deployed for a new region of interest.
• We utilized a DFF neural network model as a tool for analyzing and detecting specific signal features within the fingerprint signals collected from the embedded sensors in commercially available mobile devices. By leveraging the DFF model, we achieved significant improvements compared to conventional fingerprint localization schemes. The proposed method demonstrates a potential for practical application in real-world scenarios.
• We assessed the varying impacts of fingerprint data, such as light-illumination-level signals, magnetic field strengths, and Wi-Fi signals, on localization performance. We conducted a series of comparative experiments to analyze the contribution of each sensor type. Through the results obtained, one can establish criteria to determine sensor combinations adaptively for diverse application scenarios.

The subsequent sections of this paper are structured as follows. Section 2 provides an overview of the relevant literature and distinguishes the proposed approach from the current state-of-the-art systems. Section 3 outlines the localization model that utilizes the DFF network. Section 4 delineates the experimental design, data analysis, and system performance evaluation. Finally, Section 5 summarizes the conclusions drawn from this study and outlines possible future research directions.

2. Related Works

In recent years, various researchers have proposed a multitude of valuable approaches, employing a fusion of diverse sensors embedded within smart devices to achieve reliable indoor localization for location-based services (LBS) in indoor environments. For instance, in [29], a fusion method combining BLE and inertial navigation based on a particle filter was proposed. In [30], the fusion process involves CSI and magnetic field strength (MFS) for smartphones. Additionally, the authors of [31] presented a fusion technique that combines crowdsourced Wi-Fi fingerprinting with micro-electro-mechanical system sensors using an enhanced complementary filter. In [32], the fusion method incorporates accelerometers, gyroscopes, and magnetometers in smartphones along with Wi-Fi fingerprinting for pedestrian dead reckoning (PDR). Migicol [33] introduced a fusion method that combines Wi-Fi and magnetic signals based on a two-pass bidirectional particle-filtering approach to enhance accuracy. In [34], a fusion algorithm based on PDR and Wi-Fi RSS fingerprinting was proposed for achieving high accuracy in indoor positioning. This approach utilizes PDR data for step counting and distance estimation and RSS fingerprinting for correcting position errors. VMag, proposed in [35], is an infrastructure-free indoor-positioning method fusing geomagnetic and visual images captured using smartphone cameras; it also utilizes particle-filtering and neural network techniques. Moreover, a pedestrian-positioning method [36] fusing IMU data of smartphones and surveillance video was proposed, although these data may not always be readily available in certain environments.

Additionally, the combination of magnetic sensor data and deep learning has also been researched. In [37], the researchers proposed a deep learning method for creating a permanent magnet localization model for tongue tracking by training a feedforward neural network. In [38], a deep-learning-based neural network is introduced within the context of DC magnetic cleanliness for space missions entailing the modelling of magnetic dipoles. In [39], a real-time magnetic localization method was introduced, combining a hybrid feedforward neural network and the Levenberg–Marquardt (LM) algorithm.

In recent times, machine learning and deep learning have yielded successful results in the realm of indoor localization. In [40], the authors proposed a deep neural network (DNNs)-based indoor localization method for a smartphone integrating Wi-Fi fine-timing measurement (FTM) and RSS. Convolutional neural network (CNN)-based schemes have been proposed in recent years [41,42]. In [42], a method fusing Wi-Fi and MFS signals to fingerprint images was proposed in order to implement an accurate and orientation-free positioning system. Approaches based on recurrent neural networks (RNNs) and their variants, long-short-term-memory (LSTM)-based methods, were proposed in [16,43]. In [43], the authors formulate the indoor localization problem as a recursive function approximation problem, which was solved by using an LSTM network to fuse Wi-Fi and PDR signals obtained from smartphones. The neural networks used in these methods are complex and require careful parameter tuning and long training times. The authors of [44] proposed an LSTM-network-based method fusing Wi-Fi RSS measurements and PDR techniques to estimate the location of a smartphone. The accuracy of this system is influenced by Wi-Fi signal quality and requires periodic calibration for accurate PDR estimates. The author of [45] utilized a deep LSTM neural network to integrate data from magnetic and light sensors in smartphones for indoor localization. This method leverages the high sampling rate of magnetic and light sensors to improve the accuracy of indoor localization. The drawback of this approach is that it requires a large quantity of labeled training data for the deep LSTM model, which may be time consuming and costly to obtain.

In summary, most of the aforementioned methods rely on extensive offline fingerprint databases, complex infrastructure, or intricate neural networks that incur long training times or high hardware costs. In this paper, we propose an infrastructure-free method making use of a relatively lightweight DFF neural network, along with the integration of data from light, magnetometer, and Wi-Fi sensors in a smart device, designed to achieve stable and accurate indoor localization. The advantage of sensor fusion lies in the reduction in site-surveying time for building fingerprint radio maps in comparison to relying on a single type of sensor data, such as Wi-Fi RSS signals. Moreover, the use of multiple sensor sources can enhance robustness against environmental interference.

3. Proposed DFF-WGL Framework

We present DFF-WGL, an Android-device-based multiple sensor fusion scheme for localizing a mobile device (MD) in an indoor environment. DFF-WGL employs a deep learning model—the DFF neural network—to extract fingerprint features from a radio map, which is created using signals from the embedded light sensor, magnetometer, and Wi-Fi sensor in an Android device. DFF-WGL provides a real-time indoor localization framework consisting of two phases, namely, offline site surveillance and online localization, as illustrated in Figure 1.

In the offline phase, an MD is used to survey the region of interest (RoI), collecting fingerprint data at predefined reference points. The fingerprints of light-illumination-level signals, MFS signals, and Wi-Fi signals from two preselected access points (APs) are collected as prior information. Data preprocessing, including filtering, augmentation, and normalization, is performed after data collection. The index of each RP in the RoI is stored and used as a label to train the DFF network. After training the model, it is deployed on the Android platform to enable location estimation prediction during the online phase. During the online phase, when an MD is located in an unknown position, it collects new sensor data from the three embedded sensors and inputs them into the trained model. After the preprocessing process, the raw data are then fed to the trained network model, and the location estimate of the MD is calculated using the output probability from the DFF network’s output layer weighted according to the RP’s coordinates.

3.1. System Description

We developed an Android application for collecting fingerprint data. To expedite site surveillance, we implemented a multi-threaded approach for reading data from the light sensor, magnetometer, and Wi-Fi scanner, as depicted in Figure 2. Each sensor-reading process is handled via a separate thread. The light sensor and magnetometer require approximately 200 ms to read each sample of data, whereas the Wi-Fi scanner extracts the RSS data for specific Wi-Fi signals at one sample per second. As all three threads begin executing simultaneously upon a device’s readiness at the RP, the threads for the light sensor and magnetometer complete their respective reading processes before the Wi-Fi-scanning thread. Consequently, the total time spent at each RP depends on the Wi-Fi-scanning time required for 100 samples. Detailed information about the data collection process is given in Section 4.1.

Light-illumination-level data, MFS data, and Wi-Fi RSS data from two APs at k RPs are collected from the embedded sensors in the MD to construct the fingerprinting database, which can be described as

$$\mathit{F}=\left[\begin{array}{ccccccc}{\mathit{L}}_1& {\mathit{M}}_1^x& {\mathit{M}}_1^y& {\mathit{M}}_1^z& {\mathit{W}}_1^1& {\mathit{W}}_1^2& S_1\\ {\mathit{L}}_2& {\mathit{M}}_2^x& {\mathit{M}}_2^y& {\mathit{M}}_2^z& {\mathit{W}}_2^1& {\mathit{W}}_2^2& S_2\\ ⋮& ⋮ & ⋮ & ⋮ & ⋮ & ⋮ & ⋮\\ {\mathit{L}}_k& {\mathit{M}}_k^x& {\mathit{M}}_k^y& {\mathit{M}}_k^z& {\mathit{W}}_k^1& {\mathit{W}}_k^2& S_k\end{array}\right]$$

(1)

where ${\mathit{L}}_k$, ${\mathit{M}}_{\mathit{k}}^x$, ${\mathit{M}}_{\mathit{k}}^y$, ${\mathit{M}}_k^z$, and ${\mathit{W}}_k^1$, ${\mathit{W}}_k^2$ denote the light-illumination-level signal vector; the MFS signal vector in the x, y, and z axes; and the Wi-Fi RSS signal vector from two different Aps collected at the kth RP, with the same vector length. In our experiment, we collected 100 sample data at each RP; hence, the vector length is 100. $S_k$ denotes the label of the kth RP, for example, ‘RP0’.

3.2. Data Preprocessing

3.2.1. Filtering for Light Illumination Level Data and Wi-Fi RSS Data

To eliminate the initial instability of the data collected from a tablet’s embedded light sensor and Wi-Fi sensor, several low-pass filtering methods can be employed. Notable options include the exponential moving average (EMA) filter [46], the simple moving average (SMA) filter [47], and the moving median (MM) filter [48]. These filters are adept at smoothing erratic or unstable data, resulting in a more consistent and representative signal. As shown in Figure 3, the SMA, however, has a drawback: it initially produces a filtered value of zero. This characteristic may not align with the typical behavior of light sensor data, where real-world light illumination levels do not always start at zero. The EMA, on the other hand, tends to generate non-integer values, which may not be suitable for training neural network models that work optimally with integer data. The Moving Median (MM) filter, however, maintains the integer data type in its output. This feature makes it particularly well suited for light sensor data, where the original readings are in integer form. When training neural network models for tasks involving light illumination levels, using integer values as inputs is advantageous. Therefore, the MM filter is a fitting choice for filtering such data. Because MFS is less susceptible to interference, we did not apply any filtering to the MFS data in this scheme.

3.2.2. Data Augmentation

Linear interpolation [49] was used as a data augmentation method for enhancing our sensor data for three critical sensor types: light illumination level, MFS, and Wi-Fi RSS data. This method involves filling in data gaps between reference points by estimating intermediate values based on known data points. For example, in the case of light illumination data, we used linear interpolation to predict illumination levels at unmeasured points within the data range. This process expanded our dataset, increasing its granularity and coverage, while preserving the original data’s characteristics, as shown in Figure 4.

3.2.3. Normalization for MFS signal

To enhance the practicality of the proposed technique, we utilized pre-installed signal sources, such as ceiling lamps and Wi-Fi access points. As both light illumination and MFS signals can be obtained from any location, we selected the Wi-Fi signals that exhibit stable signal strength and can cover the entire RoI to generate a useful signal fingerprint. Specifically, as the MFS data stem from three different axes, it is necessary to normalize the data from each axis to mitigate potential errors. The values $M=\left[m_x,m_y,m_z\right]$ obtained from the magnetometer can be normalized using $\mathcal{l}2 norm$:

$$M_{norm}=[\frac{m_x}{ {\left|\left|M\right|\right|}_2},\frac{m_y}{ {\left|\left|M\right|\right|}_2},\frac{m_z}{ {\left|\left|M\right|\right|}_2}]$$

(2)

where ${\left|\right|M\left|\right|}_2=\sqrt{m_x^2+m_y^2+m_z^2}$, $m_x$, $m_y$, and $m_z$ represent the magnetic signal from the x-axis, y-axis, and z-axis, respectively.

3.2.4. Feature-Level Fusion for Multiple Sensor data

To enhance the localization accuracy of the deep learning model, feature-level fusion [50] was employed in the construction of the training dataset. This method entails concatenating feature vectors from three distinct embedded sensors, as illustrated in Figure 4. The dataset matrix includes raw filtered light-illumination-level data, normalized MFS data, and two Wi-Fi signal RSS data, each placed in a separate row. These rows correspond to different RPs and serve as inputs to the deep learning model, directly connecting to the input layer of the DFF model. This approach optimizes the model’s ability to leverage information from multiple sensor sources, ultimately improving its accuracy in localization tasks.

3.3. Structure of DFF Networks

A deep feedforward network [51], also known as a deep feedforward (DFF) neural network, is a type of artificial neural network that is widely used in machine learning and deep learning applications. In a DFF network, the neurons are organized into layers, with each layer having a specific role in the computation of the network’s output. The input layer receives the input data, which are processed by hidden layers so that complex representations and model hierarchical features can be learned. The output layer produces the network’s multi-class classification output. Our proposed scheme, shown in Figure 4, adopts a multi-class classification approach. The network takes the light-illumination-level signal; MFS signals from the x, y, and z-axes; and two Wi-Fi RSS signals as inputs.

For a multi-class classification problem with classes denoted as K, the DFF network will have K output neurons, each corresponding to the prediction probability value for each possible class. Let X be the input data and Y be the output data. We define a DFF with three layers as follows.

The first layer takes the input $\mathit{X}$ and applies an affine transformation with weights ${\mathit{W}}_1$ and biases ${\mathit{b}}_1$, followed by a non-linear activation function (Relu) $f_1$:

$${\mathit{Z}}_1=f_1\left({\mathit{W}}_1·\mathit{X}+{\mathit{b}}_1\right)$$

(3)

For each subsequent layer $l=2\cdots L$, we apply another affine transformation with weights ${\mathit{W}}_l$ and biases ${\mathit{b}}_l$, followed by a non-linear activation function (Relu) $f_l$:

$${\mathit{Z}}_l=f_l\left({\mathit{W}}_l·{\mathit{Z}}_{l-1}+{\mathit{b}}_l\right)$$

(4)

The output layer applies another affine transformation with weights ${\mathit{W}}_l$ and biases ${\mathit{b}}_l$, followed by a softmax activation function to obtain the predicted class probabilities:

$$\widehat{\mathit{Y}}=softmax\left({\mathit{W}}_L·{\mathit{Z}}_{L-1}+{\mathit{b}}_L\right)$$

(5)

where $softmax\left(z_i\right)=e^{z_i}/{\sum }_{j=1}^K{e}^{z_j}$, $z_i$ denotes the scores inferred by the network for each class in $K$. The loss function used during training is the cross-entropy loss function:

$$loss\left(\mathit{Y},\widehat{\mathit{Y}}\right)=-{\sum }_{j=1}^K{Y}_i·\log \left({\widehat{Y}}_i\right)$$

(6)

We train the network using backpropagation via the RMSprop optimization algorithm, where the weights are updated using

$$v_{dw}=\beta · v_{dw}+\left(1-\beta \right)·{dw}^2$$

(7)

$$v_{db}=\beta · v_{db}+\left(1-\beta \right)·{db}^2$$

(8)

$$W=W-\frac{\eta }{\sqrt{v_{dw}+\epsilon }}dw$$

(9)

$$b=b-\frac{\eta }{\sqrt{v_{db}+\epsilon }}db$$

(10)

where $v_{dw}$ is the moving average of the squared gradients of the weight parameter $w$, calculated over time using the previous value $v_{dw}$ and the current squared gradient ${dw}^2$. $v_{db}$ is the moving average of the squared gradients of the bias parameter $b$, calculated over time using the previous value $v_{db}$ and the current squared gradient ${db}^2$. $dw$ and $db$ are the gradients of the loss function with respect to the weight parameters $w$ and $b$, respectively. $\beta$ is the hyperparameter that controls the decay rate of the moving average, with a typical value of 0.9. $\eta$ is the learning rate that controls the step size for updating the parameters. $\epsilon$ is a small constant added to the denominator of the update rule to avoid division by zero and improve numerical stability. Finally, $W$ and $b$ are the weight and bias that are being updated.

The detailed DFF network structure we adopted in our scheme includes the following layers: the first layer, which is the input layer with six input nodes; the second Dense layer, consisting of 256 nodes with a ReLU activation function; and the third Dense layer, which also includes 256 nodes with a ReLU activation function. The output layer contains either 242 or 60 nodes (depending on the experiment in testbed 1 or testbed 2) with a Softmax activation function.

3.4. Generation of Location Estimate

In the online phase, when the MD is located in an unknown position, a set of newly collected sensor sample data is denoted as $F_t=[l_t,m_t^x,m_t^y,m_t^z,w_t^1,w_t^2]$, where $l_t$, $m_t^x$, $m_t^y$, $m_t^z$, $w_t^1$, and $w_t^2$ are the mean values received over time t. The output layer of the DFF network, acting as a multi-class classifier in Figure 4, consists of a group of probability values that correspond to the number of RPs in the RoI. $y_i$ represents the probability value output from the last layer of the DFF model when the MD is located at the $i$th RP with the current sample data $F_t$ as an input. Since the summation of all the probability values in the output layer equals one, the final location estimate can be calculated as follows:

$$\widehat{x}={\sum }_{i=1}^{\mathrm{K}}{y}_i \times {\mathit{l}}_{{rp}_i}$$

(11)

where K denotes the quantity of RPs in RoI, and ${\mathit{l}}_{{rp}_i}$ represents the centroid coordinate of the $i$th RP.

4. Experiments and Analysis

4.1. System Description

This study was conducted in two distinct indoor testbeds located on the sixth floor of the Hwado Building at Kwangwoon University in Seoul, South Korea. The first testbed was a wide rectangular corridor area measuring $60.5 {\mathrm{m}}^2(11 \mathrm{m}\times 5.5 \mathrm{m})$, while the second testbed was a narrow rectangular corridor area measuring $38 {\mathrm{m}}^2(19 \mathrm{m}\times 2 \mathrm{m})$. A total of 242 and 60 RPs were designated for the two areas, respectively, and the distances between adjacent points were 0.5 m and 1 m as determined according to Euclidean distance. The experimental setup, including a floor plan and a photographic depiction of the testbeds, can be found in Figure 5 and Figure 6, respectively.

During the site surveillance stage, an Android tablet serving as the MD was placed on a box atop a swivel chair with the monitor oriented vertically towards the ceiling to enable accurate measurements of environmental light illumination from lamps. The swivel chair was only moved to marked RPs, and at each RP, a period of approximately 1 min 40 s is required for the MD to collect 100 samples each of light sensor data; MFS data from the x-axis, y-axis, and z-axis of the magnetic sensor; and RSS data from two selected Wi-Fi signals. The label assigned to each RP was used to annotate the corresponding environment data in the dataset, which was temporarily stored in an Excel file format on the tablet. Once the site survey stage was completed, the dataset was uploaded to a computer using UART.

In a single round of data collection at an RP, a total of 100 measurements of the light illumination level; 100 measurements of the MFS along the x-axis, y-axis, and z-axis; and 100 measurements of RSS for two selected Wi-Fi RSS signals were acquired. These measurements were utilized to construct four independent fingerprint datasets. In testbed1, datasets 1 and 2 each have a length of 145,200 data points (242 × * 100 × 6), representing 100 measurements for 242 RPs and six sensor signals. Similarly, in testbed 2, datasets 3 and 4 each have a length of 36,000 data points (60 × 100 × 6), representing 100 measurements for 60 RPs and six sensor signals. The detailed information about the four datasets is summarized in

Table 1. Detailed information about datasets.

Dataset	Location	Samples/Signal	Signal Kinds	RP	Total Data	Date
1	Testbed1	100	6	242	145,200	12 June 2022
2	Testbed1	100	6	242	145,200	22 June 2022
3	Testbed2	100	6	60	36,000	13 April 2023
4	Testbed2	100	6	60	36,000	20 April 2023

. A 3D visualization of the four datasets is shown in Figure 7.

The DFF model was trained on Google Colab using a dataset partitioned into training, validation, and test sets in a ratio of 6:2:2. Keras, a deep learning API within the TensorFlow framework, was utilized to develop the model. Specifically, the TensorFlow API version was 2.12.0, and the Keras version was 2.12.0.

During the online localization process, the trained neural network runs on the Android platform, while the MD collects new environment data and then inputs them to the trained model. Finally, the estimated position is calculated and subsequently displayed on the tablet’s monitor.

4.2. Fingerprint Correlation Analysis

To establish the practicability of the proposed scheme, which involves the integration of data from a light sensor, a magnetometer, and a Wi-Fi sensor to achieve localization, Pearson correlation coefficient computation was performed using the four authentic fingerprint datasets. The Pearson correlation coefficient $\mathit{r}$ is calculated as follows:

$$\mathit{r}=\frac{\sum (x_i-\overline{x})(y_i-\overline{y})}{\sqrt{\sum {(x_i-\overline{x})}^2\sum {(y_i-\overline{y})}^2}}$$

(12)

where $x_i$ and $y_i$ are the values in the first and second datasets, and $\overline{x}$ and $\overline{y}$ are the mean values for the first and second datasets, respectively.

Figure 8a,b present the correlation matrices for testbeds 1 and 2, respectively, based on four distinct fingerprint datasets. The figure reveals that the light sensor data demonstrate the highest correlation coefficient, which can be attributed to their relatively stable illumination intensity, which persists unless there is a change in the light source position or damage to the source itself. Conversely, the MFS data exhibit a lower correlation coefficient compared to the light-illumination-level data, indicating that the MFS signal varies over time. Lastly, the Wi-Fi RSS signal exhibits the lowest correlation coefficient across both testbeds, further accentuating the inherent instability of Wi-Fi RSS signals in indoor environments and their proneness to electronic interference.

4.3. Impact of Different Sensor Combinations

To assess the impact of various sensor combinations on the localization error distribution, we devised multiple groups of sensor signal combinations. We conducted a comparative analysis to elucidate the distinct contributions of these sensor signals using a test dataset sourced from the offline Dataset1. For ease of exposition, the MFS signals from the x, y, and z-axes were treated as a single signal type, while the two Wi-Fi RSS signals, despite their intrinsic similarity, were categorized as two distinct signal types. Consequently, cases C1 to C4 encompass a single type of sensor signal, cases C5 to C8 correspond to two types of sensor signals, cases C9 to C11 involve three distinct signal types, and case C12 integrates four diverse signal types. The cumulative distribution function (CDF) of the localization error under the proposed model is presented in Figure 9.

Firstly, we present the experimental results for single-sensor signal-based localization (C1~C4). Beginning with case C1, it is evident that prediction performance remained subpar, and localization accuracy was consistently low across various learning rates. Although the light-illumination-level signal exhibits remarkable stability, its coverage area is limited. As illustrated in Figure 7a,g, the light illumination levels recorded on different dates are nearly identical. However, valid illumination is detected only within specific regions beneath the lamps, distinct from the case for other RPs. In most other areas, the illumination levels are close to zero. Consequently, the network model struggled to acquire sufficient location-related information solely from the light-illumination-level signal. This deficiency is reflected in the notably low prediction accuracy within the test datasets.

Similarly, the performance of a single Wi-Fi signal (C1~C4) also proved unsatisfactory. The highest accuracy within 2 m did not exceed 20%, and more than 50% of localization errors exceeded 3 m, as evidenced in Figure 9C3,C4. These outcomes underscore the unreliability of single-Wi-Fi-signal-based fingerprinting methods in indoor environments. The presence of severe multipath effects and interference results in frequent fluctuations in indoor Wi-Fi signals, thereby exacerbating challenges within the localization system. Contrastingly, the results obtained from the MFS signal (C2) outshine those of the other single-signal experiments. Notably, the embedded magnetometer captured three-dimensional data from the x, y, and z axes, providing C2 with more information than other single-signal approaches and consequently yielding superior prediction performance.

In the experiments conducted from C5 to C8, we investigated the performance when two sensor signals were used. A comparison between C8, C3, and C4 highlights the advantage of employing two Wi-Fi signals over a single Wi-Fi signal, resulting in improved prediction accuracy. However, the test accuracy of 21.7% achieved in C8 remains insufficient for accurate indoor localization. The second lowest test accuracy, 54.2%, was observed in C6, where one light-illumination-level signal and Wi-Fi signal 1 were used. Comparing C6 (54.2%) with C8 (21.7%) and C3 (6.8%), we can observe that the light-illumination-level signal increases test accuracy by 47.4%, while Wi-Fi signal 2 only boosts it by 14.9% when compared to C3 (6.8%). This suggests that the light-illumination-level signal, due to its stability and strong location-related characteristics, may be a preferable choice over Wi-Fi in specific applications.

The highest test accuracy was achieved in C5, amounting to 92.1%, followed closely by that of 89.1% in C7. The discrepancy in accuracy between C5 and C7, attributed to the use of a light-illumination-level signal versus Wi-Fi signal 1, aligns with the earlier analysis.

Moving on to C9~C11, designed to assess performance under three sensor signals, we found that the inclusion of the light-illumination-level signal in C9 (96.05%) contributed a 6.95% accuracy increase when compared to C7 (89.1%). Furthermore, in C9 (96.05%) and C11 (93.5%), the illumination-level signal provided a 2.55% accuracy boost over Wi-Fi signal 2. Comparing C8 (21.7%), C10 (78.05%), and C11 (93.58%), we can observe that the MFS signals outperform the light-illumination-level signal, with the former exhibiting a substantial 71.88% accuracy increase, which can be compared to the 56.35% accuracy boost provided by the latter. These results align closely with the comparisons between C3 (6.8%), C6 (54.2%), and C7 (89.1%), demonstrating accuracy improvements of approximately 82.3% and 47.4%, respectively.

The final experiment, denoted as C12, employed all available sensor signals, achieving a remarkable 96.85% test accuracy with a learning rate of 0.01 and an even higher accuracy of 97.32% with a learning rate of 0.001. Notably, C12 demonstrated the highest test accuracy among all twelve experiments and exhibited the smallest localization error within the test dataset. C12 (97.32%) and C9 (95.85%) displayed similar performance, with the slight variance observed being attributable to the influence of Wi-Fi signal 2. Despite its relatively modest contribution, Wi-Fi signal 2 still contributed to a 1.47% accuracy improvement.

4.4. Impact of Learning Rate

To assess the influence of different learning rates ($lr$) on the neural network model’s performance, we selected four representative values, namely, 0.1, 0.01, 0.001, and 0.0001, for a comprehensive evaluation across all comparative experiments. Underfitting occurred when $lr=0.1$, as this learning rate proved excessively large in relation to the dataset, hindering the model’s ability to capture intricate dataset features. This resulted in non-convergence, elevated test loss, and a test accuracy lower than the training accuracy during the training process. Conversely, overfitting occurs at an $lr=0.0001$, at which point the model learns an excessive number of intricate details, including noise and inaccuracies, leading to the highest training and test losses among the four learning rates.

In Figure 9, encompassing all twelve experiments, $lr=0.0001$ yields the lowest average localization error, with $lr=0.1$ ranking as the second lowest, except for the specific cases of Figure 9C1,C3,C4, where the models fail to converge and attain exceedingly low accuracy. Comparatively, the model employing $lr=0.01$ exhibited slightly improved localization accuracy. This improvement is attributed to the larger step length in the gradient descent algorithm associated with $lr=0.001$, which enhances the learning process’s efficiency based on the current dataset.

4.5. Performance in Real-Time Localization

Based on previous experimental findings and parameter analyses, we conducted real-time online experiments to assess the practical performance of the proposed DFF-WGL scheme within two distinct testbeds. Separate experimental paths were designed for testbeds 1 and 2. In Figure 10a,b, we present 2D plots depicting twelve sets of localization errors obtained under varying sensor signal combinations while the MD collected real-time environmental data in both testbeds. To facilitate the comparative evaluation of system performance across different signal combinations, we employed box plots in Figure 10, revealing the distribution, spread, and central tendency (median) of the localization errors. Subsequently, in Figure 11, we compare the mean, median, and variance values of the localization errors for each result group.

In Figure 10, showing the box plot, and Figure 11, presenting the histogram, we can observe that in testbed1, for the results for C1, C3, C4, C6, and C8, the combination of light-illumination-level data and two Wi-Fi RSS signals failed to provide sufficient and stable location-related information. Consequently, the DFF network struggled to converge and make accurate localization predictions in testbed1. Even with the addition of light and Wi-Fi signals in C10, the localization error only marginally decreased, resulting in a mean error of approximately 0.87 m. In contrast, the MFS signal from the magnetometer, specifically in C2, achieved a localization error as low as 0.15 m. However, this signal exhibits instability, as indicated by the 2D figure and variance values in Figure 11. Combining the MFS signal with the light sensor in C5 yields superior results compared to the combination with Wi-Fi in C7, with the latter combination producing lower mean localization errors and variances.

Comparing the results from C9 to C12, we discerned that C10 performs the poorest, relying solely on the light sensor signal and two Wi-Fi RSS signals. Next in performance is C11, followed by C9, as the light-illumination-level signal offers more stable information than a Wi-Fi signal. The most favorable localization error is achieved in the C12 combination, which provides sufficient and stable fingerprint information to enable the DFF network to make accurate localization predictions.

For the results obtained in testbed2, the majority of outcomes align consistently with those observed in testbed1. It is important to highlight that in the case of the C1 combination, there are three ceiling lamps in testbed2, resulting in a higher volume of light-illumination-level data collected by the MD compared to that in the testbed1 experiment. Consequently, the mean localization error for C1 in testbed2 is approximately 1.1 m, an improvement over the 2.1 m error observed in testbed1. Due to the contribution of the light illumination signal, both C6 and C10 failed to achieve satisfactory localization performance in testbed1 but exhibited nearly zero median values in testbed2.

In conclusion, the sensors’ ability to furnish adequate and stable fingerprint information corresponds to the following sequence: magnetometer > light sensor > Wi-Fi sensor. These real-time experimental results align with the analyses conducted using offline datasets (Figure 9).

4.6. Performance Comparison

To evaluate the performance of the proposed scheme, we compared the localization error with that of the existing state-of-the-art schemes, including EZ [52], BPNN [53], and Magicol [33], as shown in Figure 12. It can be observed that the proposed DFF-WGL scheme outperforms Magicol by more than 80 percentile error within 2 m. The Magicol fusing Wi-Fi signals, MFS signals, and dead reckoning with a particle filter achieved high accuracy but also presented high computational complexity. EZ and BPNN utilize only Wi-Fi signals to estimate the target’s position. Conversely, the proposed scheme achieved the best localization performance by integrating signals from multiple sensors with a deep feed-forward network.

5. Conclusions

In this study, we proposed a novel approach for indoor active localization using a DFF neural network for classification by fusing fingerprint data obtained from multiple sensors, including the light sensor, magnetometer, and Wi-Fi sensor embedded in a commercial Android device. We adopted a neural network as a multi-class classifier to learn the fine features from the signal fingerprints collected from each RP within the region of interest. Different sensor combinations were evaluated, and localization performance was compared in two real indoor environments. Since the magnetometer is sensitive to the direction and position of the device, in this scheme, we kept the mobile device in a stable state and facing the same direction and oriented the device’s monitor vertically towards the ceiling. This ensured that the embedded light sensor near the front camera collected stable light-illumination-level data throughout the experimental process.

Since magnetic measurements can be obstructed by magnetic materials or 50 Hz signals and harmonics in the power network in indoor environments, we will set about solving this problem by implementing more stringent measurement procedures, as described in [54,55], to adapt to more complex application scenarios in the future. We also aim to increase the practicality of our proposed scheme in larger regions of interest under more complex indoor environments and test the performance of the proposed scheme at different times of day. We also aim to solve the direction-aware problem. Additionally, the proposed scheme will be further developed to adaptively sense variations in light intensity and generate appropriate localization strategies to enable the constant usage of this scheme.