Carrier Phase Residual Modeling and Fault Monitoring Using Short-Baseline Double Difference and Machine Learning

Abstract

Global Navigation Satellite Systems (GNSS) are used to provide accurate position, navigation, and time (PNT) information to users in various sectors of our society including transportation. Augmentation systems such as differential GNSS (DGNSS), real-time kinematics (RTK), and Precise Point Positioning (PPP) improve the GNSS performance, and providing reliable measurements from its reference station is very crucial. To ensure safe and accurate PNT solutions, code and carrier measurements must be monitored for potential faults or a performance degrade. Although there exist numerous methods to model and monitor the measurements, research on the carrier phase measurements is not as extensive as the code measurements. This paper introduces a split of residuals into receiver noise and multipath components to customize their estimation according to their respective statistical properties. This study also proposes a method to use machine learning-based non-linear regression to effectively model and monitor potential faults in the GNSS measurements including the carrier phase. A training dataset is used to model the nominal quantities of GNSS measurement residuals, and inflation factors are applied to over-bound the fault-free residuals. These inflated residuals are coupled with uncertainty factors to compute thresholds for monitoring carrier phase residuals, and the effectiveness of the thresholds is validated with a test dataset by achieving the false alarm rate of $6.61\times {10}^{-6}$, slightly lower than the desired level of ${10}^{-5}$.

1. Introduction

Global Navigation Satellite Systems (GNSS) provide position, navigation, and time (PNT) information to users all around the world. The concept of GNSS is based on processing known GNSS satellite positions and times with the distances between the satellites and the user to compute estimated user position, velocity, and time (PVT). There are two major types of GNSS positioning: Single Point Positioning (SPP) and Differential Positioning (DP). The standalone method uses only the GNSS measurement observations available to the receiver, while the Differential GNSS (DGNSS) method uses additional information from nearby monitoring stations. Types of information provided by the monitoring stations include GNSS measurements, and the corresponding integrity information. In case of GPS, using the uncorrected measurements results in an expected horizontal accuracy of an over 5 m root mean square error (RMSE), while the DGNSS method provides an expected accuracy of a $1$ m RMSE [1,2,3,4]. Although the DGNSS has the potential to provide improved accuracy and integrity PVT, the system accuracy and integrity are dependent on how well the monitoring stations can assess the quality of the measurements and detect anomalies such as cycle slips, a non-pre-surveyed multipath, or a signal blockage prior to reporting them to the user [5]. Therefore, all the GNSS measurements must be monitored for faults in real time to ensure safe and reliable navigation for various applications including autonomous ground vehicles and unmanned aerial vehicles (UAV) [6,7].

To monitor the validity of the carrier phase measurement, modeling the residual under a normal condition should be preceded so that thresholds for the nominal expected residual can be defined to monitor outliers. One form of residual modeling is using the theoretically expected carrier phase noise from the third-order phase-lock loop (PLL) used in the carrier phase tracking [8]. Although it is a theoretical and straightforward way for the modeling, it has the limitation of not accounting for other normal errors, such as a tolerable multipath under typical signal reception environments, i.e., open-sky, sub-urban, and urban [9]. Another usual way to estimate the expected normal residuals is using the elevation angle of the satellites. The elevation angle is generally inversely proportional to the carrier phase noise due to an increased path length between the satellites and the receiver [10,11]. Even a multipath error under a typical condition can also be roughly modeled with the elevation angles [12,13]. However, the error of each satellite cannot be properly modeled according to the azimuth because it is based on the assumption that a noise and multipath error have the same magnitude at the same elevation angle. It cannot describe a non-uniform multipath error for the same elevation angle due to the site visibility-related effect.

There also exist several methodologies for modeling and defining the thresholds based on linear or differential combination of GNSS measurements. First, there are code carrier divergence tests that monitor the variations in the code measurement noise and ionosphere [14]. Although this method is effective for monitoring code measurement quality and the health status, carrier measurement noise should be assumed to be negligible without any outlier residuals, so carrier monitoring is difficult based on the code carrier divergence test. Carrier residual computation methodologies including the zero-baseline double difference (ZBDD), carrier detrending, and short-baseline double difference (SBDD) can be utilized for monitoring [15]. All of these differential methods are used to detrend the geometry terms between a satellite and a user in the carrier phase measurements. ZBDD detrends the geometry term but also removes common noise terms and even a multipath present between the satellite and the antenna splitter and accordingly underestimates the noise present in the signals. The geometry distance detrending can be also carried out using filters such as the sixth order Butterworth Filter or polynomial fitting. Although this method is effective in monitoring large carrier phase anomalies such as ionospheric scintillations [16], the detrending performance is dependent on how the filter parameters are accurately estimated. Initial measurements should be used for this parameter estimation process, and the magnitude of the variations in the detrended carrier is generally larger than the expected noise of the measurements, making the measurement monitoring less effective. The SBDD, on the other hand, can be an option to mitigate the aforementioned issues, as all the noise and multipath errors are not cancelled out due to separate antennas as much as the ZBDD, and the detrended carrier is not as filter parameter-dependent.

In order to carry out effective carrier phase noise and multipath modeling and monitoring, other GNSS common error terms should be effectively differentiated. To extract the site-dependent error terms, we suggest utilizing the SBDD-based residuals. The precise and accurate relative vector between two GNSS antennas of a reference station can be used to calculate the residuals instead of using their absolute positions. Since the obstacle geometry and its multipath effect cannot be exactly identified, the residual error regression process is necessary to estimate and model the nominal residuals. The azimuth angle as well as elevation should be considered as the input parameters for the model, and C/No from both satellites used in the residual computation can be added. In order to be employed to a safety-critical system by ensuring correct false alarm probability, inflation of the standard deviation for residual errors with respect to the tail portion needs to be considered. Finally, the inflation factors should be computed so that the normal distribution over-bounds the actual residual distribution to meet the false alarm requirement.

In Section 2, we discuss the GNSS measurements that are used for PVT computation and assess the various methods that can be used for GNSS residual computation and modeling. In Section 3, our proposed methodology is introduced and how it is implemented to model and monitor the residuals is described. In Section 4, we describe how the training and test data were collected. In Section 5, we validate our methodology using a validation set of data. Finally, we end with a conclusion about our study and how it can be applied for accurate and reliable GNSS PNT.

2. GNSS Measurement Residuals

There are three measurement types used to quantify the line-of-sight (LOS) distance and velocity between the satellite and the user: code phase, carrier phase, and doppler measurements. Code measurements provide information about the absolute distance between a satellite and a user as described in (1). The carrier phase measurements, which are shown in (2), provide higher resolution distance information, but it is difficult to resolve the integer ambiguities that are inherent in the measurements, so they are often used as a relative metric unless the ambiguities are resolved [17]. Doppler measurements provide the LOS doppler frequency between the satellite and the user. They are similar to the rate of carrier phase measurements, but doppler is estimated by looking at the frequency of the signal with respect to the base frequency instead of computing the difference in the number of cycles between measurement epochs.

$${\rho }_A^P=R_A^P+c\left(\delta t_A-\delta t^P\right)+T+I+M+{ϵ}_{{\rho }_A}^P$$

(1)

$${\varphi }_A^P=R_A^P+c\left(\delta t_A-\delta t^P\right)+T-I+M+N_A^P\lambda +{ϵ}_{{\varphi }_A}^P$$

(2)

where subscript $A$ denotes a receiver, and superscription $P$ stands for a visible satellite. The pseudorange, carrier phase, geometric range, integer ambiguity in cycles, wavelength of the carrier, and speed of light are $\rho$, $\varphi , R$, $N,$ $\lambda$, and $c$, respectively. The common errors to be removed by the SBDD such as clock error and tropospheric and ionospheric errors are denoted as $\delta t$, $T$, and $I$, respectively. The multipath and noise, which are residual errors to be monitored, are represented as $M$ and $ϵ$, respectively.

Although code measurements have been mainly used for PVT estimates and navigation of vehicles, carrier phase measurements are also necessary to achieve reliable sub-meter level accuracy navigation [18]. In order to estimate the expected accuracy of the PVT solutions, an accurate uncertainty model of the measurements is required [19]. Additionally, to ensure the integrity of the solutions, anomaly monitoring of the measurements is necessary. Although there exist multiple code measurement modeling and monitoring studies, work on the carrier phase monitoring methodologies is not as abundant [20]. In this paper, we review the modeling and monitoring methodologies for carrier phase measurements, which are accompanied by integer ambiguity resolution that had not been considered when applying the doppler or time-differenced carrier phase [21].

2.1. Residual Computation

The carrier phase measurements represent the line-of-sight distance between the satellites and the user, so the satellite motion needs to be removed prior to modeling and monitoring. There are several methods for removing the geometry variation, which are detrending or differencing the carrier phase measurements.

One way to remove the satellite dynamics is carrier detrending that uses polynomials to estimate satellite motion. Similarly, high-pass filters can be used to separate noise from satellite motion and a multipath. For example, a sixth order Butterworth Filter can be used to remove the slowly varying satellite motion [16]. These detrending methods are often used to obtain residuals for the detection of GNSS anomalies such as ionospheric scintillation. Although the detrending method is effective for a single antenna, the accuracy and precision of the residuals are environment- and receiver-dependent, and the presence of atmospheric and clock variations makes the assessment of receiver noise difficult. It is also available that the satellite motion can be predicted from the GNSS ephemeris files, but the exact location of the receiver should be exactly surveyed to assess their effects on the carrier phase, and the ephemeris degradation over time might cause a buildup of residual inaccuracies. Furthermore, the biggest problem with detrending the carrier phase measurements is that whenever the monitor is reset due to initialization or fault detection, a number of samples are required to bring the filtered residuals back to nominal levels [21].

Code and carrier phase measurements both have an identical geometric range according to Equations (1) and (2). Therefore, if we take the difference between the measurements, Code Minus Carrier (CMC), the satellite motion with common errors is removed. However, the problem with using this metric is that the noise and multipath in code measurement noise are dominant compared to those in the carrier. Kee observed that the noise of the code is approximately 1000 times greater than that of the carrier [22]. Furthermore, while the effect of the troposphere is removed, the ionosphere effect as a delay on the code and as an advance on the carrier results in the ionosphere divergence.

When we difference ($\Delta$) the carrier phase measurements between receivers $A$ and $B$ for a common satellite $P$, the Single Difference between Receivers, as shown in (3), the effects of the ionosphere and troposphere will be cancelled out as long as the antennas are close to each other. If the receivers share a common antenna, the difference in the geometric range and the multipath cancels out. Otherwise, the difference in the geometric range should be estimated to be removed, which will be outlined further in this paper. The integer ambiguities should be especially fixed and removed as demonstrated by Tiberius [8] when we difference carrier phases unlike code measurements. The difference in the receiver clock stability will still be a part of the residuals.

$$\Delta {\varphi }_{AB}^P=\left(R_A^P-R_B^P\right)+c\left(\delta t_A-\delta t_B\right)+\left(N_A^P-N_B^P\right)\lambda +\left(M_A^P-M_B^P\right)+{ϵ}_{AB}^P$$

(3)

The double difference is a between-satellite single difference of a between-receiver single difference as shown in (4). The between-satellite difference is useful because the combination eliminates clock errors, both the satellite and receiver clock errors. The removal of the clock errors in the double difference makes it possible to reduce all the non-integer biases and determine the integer ambiguity.

$$\nabla \Delta {\varphi }_{AB}^{PQ}={\varphi }_A^P-{\varphi }_A^Q-{\varphi }_B^P+{\varphi }_B^Q$$

(4)

ZBDD is a popular method used by many academic and industrial entities to estimate GNSS receiver noise [15]. It is computed by taking the double difference of measurements between two satellites and two receivers that share a single antenna. First, we take the difference of the carrier phase for satellite $P$ between receivers $A$ and $B$ as shown in (5). As the receivers share the same antenna, the effects of the atmosphere including ionospheric and tropospheric delays are cancelled out along with the multipath.

$$\Delta {\varphi }_{AB}^P=c\left(\delta t_A-\delta t_B\right)+\left(N_A^P-N_B^P\right)\lambda +{ϵ}_{AB}^P$$

(5)

If we take the difference of the single differences between two satellites, $P$ and $Q$, we obtain the final ZBDD ($\nabla \Delta$) residual shown in (6).

$$\nabla \Delta {\varphi }_{AB}^{PQ}=\nabla \Delta N_{AB}^{PQ}\lambda +{ϵ}_{AB}^{PQ}$$

(6)

Although this method is effective in easily removing most of the non-noise terms, it has the disadvantage of underestimating the noise as well. All of the multipath and atmospheric noises between the satellites and the antenna splitter are common, so they are cancelled out [15]. Therefore, ZBDD will only be able to model the independent pure thermal noise terms present in the devices.

SBDD is essentially the same as ZBDD, but the configuration does not share a single antenna. Therefore, the issues that stem from the use of a single antenna, which include underestimated noise and no information about the multipath and atmospheric noise, have been mitigated. However, the SBDD geometric range term and integer ambiguity resolution must be resolved as shown in (7).

$$\nabla \Delta {\varphi }_{AB}^{PQ}=\nabla \Delta R_{AB}^{PQ}+\nabla \Delta N_{AB}^{PQ}\lambda +\nabla \Delta M_{AB}^{PQ}+{ϵ}_{AB}^{PQ}$$

(7)

The geometric range term can be estimated if we know the baseline vector between the antennas corresponding to antennas $A$ and $B$. Assuming the baseline $\Delta {\overrightarrow{x}}_{AB}$ is short enough to let the line-of-sight vectors ${\overrightarrow{G}}_A^P$ and ${\overrightarrow{G}}_B^P$ be the same as ${\overrightarrow{G}}^P$ shown in (8), the relative geometry can be computed as (9). The geometry double difference ($\nabla \Delta R_{AB}^{PQ}$) can be calculated with the satellite-difference line-of-sight term ($\nabla {\overrightarrow{G}}^{PQ}$) and the baseline vector ($\Delta {\overrightarrow{x}}_{AB})$ as described in (10).

$$\begin{array}{c}{\overrightarrow{G}}^P=\raisebox{1ex}{$\left({\overrightarrow{x}}^P-{\overrightarrow{x}}_A\right)$}\!\left/ \!\raisebox{-1ex}{$\left|{\overrightarrow{x}}^P-{\overrightarrow{x}}_A\right|$}\right.\\ {\overrightarrow{G}}^Q=\raisebox{1ex}{$({\overrightarrow{x}}^Q-{\overrightarrow{x}}_A)$}\!\left/ \!\raisebox{-1ex}{$\left|{\overrightarrow{x}}^Q-{\overrightarrow{x}}_A\right|$}\right.\end{array}\}$$

(8)

where $\overrightarrow{x}$ is the position vector of the receiver or the satellite.

$$\Delta R_{AB}^P={\overrightarrow{G}}^P·\left({\overrightarrow{x}}_B-{\overrightarrow{x}}_A\right)={\overrightarrow{G}}^P·\Delta {\overrightarrow{x}}_{AB}$$

(9)

$$\nabla \Delta R_{AB}^{PQ}=\Delta R_{AB}^P-\Delta R_{AB}^Q=\left({\overrightarrow{G}}^P-{\overrightarrow{G}}^Q\right)·\Delta {\overrightarrow{x}}_{AB}=\nabla {\overrightarrow{G}}^{PQ}·\Delta {\overrightarrow{x}}_{AB}$$

(10)

For the integer ambiguities, there exist several methods to estimate and resolve the unknowns. These might include fixing the ambiguities, grid searching candidates, and variance reduction using the sequential quasi-Monte Carlo method [18,22,23]. For accurate noise modeling, it is important to exclude the outliers in carrier phase measurements, such as a cycle slip or clock jump, as they can introduce significant errors in the model. Fortunately, the detection of these outliers can be achieved with relative ease and accuracy through post-processing. The integers are fixed throughout the pass-over by dividing all the double-difference terms in the outlier-free session by the wavelength ($\lambda )$, and they are rounded to the nearest integer as shown in (11).

$$\nabla \Delta N_{AB}^{PQ}=round\left(\overline{\raisebox{1ex}{$\left(\nabla \Delta {\varphi }_{AB}^{PQ}-\nabla \Delta R_{AB}^{PQ}\right)$}\!\left/ \!\raisebox{-1ex}{$\lambda $}\right.}\right)$$

(11)

Consequently, we can estimate the residual terms that include GNSS receiver noise and the influence of a multipath on the GNSS measurements. In particular, since safety-of-life facilities generally employ multi-antenna and multi-receiver systems for redundancy, SBDD is a very suitable algorithm to the facility and applications.

2.2. Residual Modeling

There are several different methods to model the GNSS residuals; for example, exponential representation of carrier phase noise, elevation angle-dependent noise estimation, and theoretical computation of the carrier phase noise from the third-order PLL. Furthermore, another novel method recently proposed and explored is the machine learning approach that models the residuals with respect to the input parameters using non-linear regression.

Measurement noise is well known to be inversely proportional to the elevation angle of the satellites. Bischoff proved that the carrier phase variance is strongly dependent on the elevation of the satellites [24]. Additionally, there have been numerous elevation-dependent models for the residual estimation. Equation (12) has been proposed by Vermeer and Rothacher [10,11]. Equation (13) was proposed by Ha, and Equation (14) by Wang et al. [25,26]. These are used by various software packages such as the Bernese GNSS software [27].

$${\sigma }^2=\frac{1}{sin{\left(El\right)}^2}$$

(12)

$${\sigma }^2=\frac{1}{sin\left(El\right)}$$

(13)

$${\sigma }^2=\frac{1}{E{l}^2 }$$

(14)

where $El$ is the satellite elevation angle in degrees, and $\sigma$ is the noise error standard deviation.

Another GNSS residual modeling method uses a well-known relationship between the Carrier-to-Noise Ratio (C/No) and expected carrier phase noise for the third-order PLL used by GNSS receivers [28]. This relationship is often used to theoretically assess the expected carrier phase noise, but it does not consider the GNSS receiver hardware clock and voltage-controlled oscillator (VCO), which also contribute to the total noise of the receiver [29].

$${\sigma }_{\varphi }^2=\frac{B_n}{C/No}\left(1+\frac{1}{2T_{c}C/No}\right)$$

(15)

where $B_n$ is the PLL loop bandwidth, and $T_C$ is the integration time of the tracking loop in milliseconds.

In this study, we propose applying a machine learning algorithm to consider various GNSS signal reception environment factors at the receiver site, as opposed to using deterministic methods. Decision trees and support vector machines (SVM) are popular machine learning tools used in data classification and regression. For the residual modeling, we use both methodologies due to their inherent characteristics. The decision tree method makes no assumptions on the distribution of the data nor the structure of the model, so it is effective for modeling data with discrete behavior such as multipath behavior [30]. On the other hand, SVM are effective for continuous data such as multi-variable-dependent noise, as they are able to effectively model the noise while mitigating portions of data with anomalies or unavailability.

In order to construct the regression tree, we followed the method outlined by Hastie et al. [31] by defining two regions $R_1$ and $R_2$ based on a splitting variable $j$ and a split point $s$.

$$\{\begin{array}{c}{R}_1\left(j,s\right)=\left\{X|X_j\le s\right\}\\ R_2\left(j,s\right)=\left\{X|X_j>s\right\}\end{array}$$

(16)

where $X$ is the data. We find the pair $\left(j,s\right)$ such that we minimize the cost function $J$.

$$J=\underset{j,s}{\min }\left[\underset{c1}{\min }{{\displaystyle \sum }}_{x_i\in R_1\left(j,s\right)}{\left(y_i-c_1\right)}^2+\underset{c2}{\min }{{\displaystyle \sum }}_{x_i\in R_2\left(j,s\right)}{\left(y_i-c_2\right)}^2\right]$$

$$\mathrm{where} \{\begin{array}{c}{c}_1=\overline{(y_i|x_i\in R_1\left(j,s\right))}\\ c_2=\overline{(y_i|x_i\in R_2\left(j,s\right))}\end{array}$$

(17)

In Equation (17), $x$ is the input variable, and $y$ is the predictor. After splitting the data into two regions, they are split further again until the minimum node size is reached.

For the SVM, we find non-linear boundaries for the data by constructing a hyperplane. The input data are transformed using kernel functions. There are various options for the kernel including polynomial and Gaussian. In our case, the third-order polynomial kernel is selected. Afterwards, a hyperplane is selected such that the distance between the plane and the data points is maximized.

3. Machine Learning-Based GNSS Residual Error Modeling Methodology

The flowchart for the monitoring scheme determination is outlined in Figure 1. Using measurements from the receivers, we compute the GNSS measurement residuals observed with the receivers. We used the SBDD residuals as the ZBDD underestimates the noise, and the detrending of the carrier phase overestimates noise. As the SBDD residuals have effects of both the multipath and noise, the effect of the components should be separately examined. The resulting residuals are each modeled using machine learning regression. Finally, the estimated residuals from the models are used to compute the thresholds for carrier phase residual monitoring.

3.1. Residual Modeling Using Machine Learning Regression

The residuals obtained from SBDD consist of mainly noise and multipath components. Therefore, these terms must be separated prior to modeling as shown in Figure 2. While the noise components are rapidly fluctuating close to white noise, the multipath (MP) is changing slowly. Therefore, the results of a high-pass filter (HPF) are representative of receiver noise, while the outputs of a low-pass filter (LPF) are appropriate for modeling the effects of the multipath on the measurements. The cutoff frequency for the HPF and LPF was selected as 0.3 Hz because this was the optimal value used by Forte [32] to detrend the carrier phase measurements for scintillation detection purposes. Finally, both are modeled separately using machine learning regression.

The input parameters for the machine learning are the normalized East, North, and Up (ENU) of both the satellites with respect to the receivers, and the satellite $\mathrm{C}/\mathrm{No}$. Existing methods presented models with only elevation and/or $\mathrm{C}/\mathrm{No}$ input variables, but the suggested method accounts for the azimuth as well, and the model is constructed empirically using GNSS data collected by the receivers. We used ENU instead of elevation and azimuth of the satellites, as the regression model will not recognize that $0°$ azimuth is equivalent to $360°$ azimuth. To calculate the residuals, we employed the root-mean-square (RMS) of residuals accumulated over a minimum of 5 continuous seconds. If the residuals exceeded the predetermined bin size, the accumulation was reset, and the RMS value was treated as a single residual corresponding to the respective bin for model training. The bin sizes can be adjusted based on the available sample data and the short-term variability of the parameters. Choosing wider bin widths would yield more conservative residual estimates, whereas opting for finer bin widths would necessitate a significantly larger number of samples to accurately model the residuals.

In our case, the bin widths were defined as 0.01 m for East and North, 0.05 m for the Up direction, and 2 dB-Hz for $\mathrm{C}/\mathrm{No}$. This is because we wanted finer resolution for azimuth-dependent multipath modeling, and more conservative estimates for elevation dependent residual estimation. Additionally, the 2 dB-Hz frequency was selected, because for our receiver and given ENU bin sizes, the $\mathrm{C}/\mathrm{No}$ variability for each continuous residual used for RMS computation was within this range. However, for other receivers with different tracking loop bandwidth and antenna characteristics, different bin sizes should be used.

For the machine learning models, we considered SVM for HPF and a tree for LPF. This is because thermal noise has a strong Gaussian property, and the SVM would model the residuals as being continuous to the input features described above, which will assist in modeling the residuals for bins with a limited number of samples. Multiple SVM models including linear, second-order, third-order, and Gaussian have been considered, but the Gaussian SVM were finally chosen as their regression modeling errors with respect to the training data were the smallest compared to other models. The smaller the kernel scale means higher variations in the response function, so a kernel scale of $\frac{\sqrt{P}}{4}$, where $P$ is the number of predictors, is selected. In contrast to the HPF, a tree is used as the model type for the LPF, as a multipath is discrete and environmentally dependent. The greater the number of nodes means higher flexibility in the response function, so a minimum node size of four was chosen. The regression modeling errors corresponding to each machine learning model considered are outlined in

Table 1. Regression modeling errors of the machine learning models with respect to the training data for the HPF and LPF components.

HPF		LPF
Model	Regression Modeling Errors (RMSE)	Model	Regression Modeling Errors (RMSE)
Linear SVM	0.51 mm	Tree (Min. Node = 16)	0.32 mm
Second-order SVM	0.49 mm
Third-order SVM	0.47 mm	Tree (Min. Node = 8)	0.31 mm
Gaussian SVM (Kernel = $4\sqrt{P}$)	0.39 mm
Gaussian SVM (Kernel = $\sqrt{P}$)	0.39 mm	Tree (Min. Node = 4)	0.30 mm
Gaussian SVM (Kernel = $\frac{\sqrt{P}}{4}$)	0.38 mm

. The error results confirm that the proposed models are the most optimal selections.

3.2. Inflation Factor and Threshold Computation

An integrity monitoring system is responsible for detecting potential threats by comparing the residuals to specified thresholds. Typically, the residuals reflect the characteristics of the target threat and can be used to evaluate the system’s integrity performance [33]. The thresholds are determined based on the statistic distribution of the residuals and the false alarm rate required to support specific applications [34]. These thresholds are determined using the required probabilities of missed detection and a false alarm as specified by the system. Both of these probabilities must be considered to satisfy the system integrity risk and the continuity of service requirements [35].

A false alarm poses a problem for the continuity risk, which is the probability that the system will be interrupted despite there being no issues present [36,37]. Particularly, for typical safety-of-life applications, an allowable maximum false alarm rate is allocated, as unscheduled system interruptions can lead to fatal accidents [38,39]. Therefore, it is critical to determine the proper detection threshold so that the probability of a false alarm does not exceed the desired requirements. The probability of missed detection refers to the possibility that the threshold is not exceeded even though anomalies, such as carrier phase cycle-slips, may be present [40].

However, in this study, the focus is on the probability of a false alarm rather than the missed detection. This is because the residual modeling methodology of this study is designed for nominal conditions without any measurement anomalies. This approach aligns with the typical practice of initially determining the fault-detection thresholds using only the false alarm probability allocated from the continuity requirement of the system [41].

To determine the threshold, it is assumed that the residuals under nominal conditions can be bound by a normal distribution with a zero mean and a specified standard deviation [37]. If the normal distribution was correctly modeled and the estimated residuals effectively bound the actual residuals, the estimated metrics should over-bound the actual metrics at the tails of the normal distribution. However, the two tails of the actual residual distribution may not be Gaussian or may be larger than expected, due to effects of a multipath or the influence of measurement noise [42,43]. Therefore, if the tail statistics associated with the threshold are not sufficient for over-bounding of the residuals, the probability of a false alarm may be larger than the system requirement. Hence, inflation factors for the estimated residuals from the machine learning regression need to be calculated in order to over-bound the tails of the actual residual distribution.

To compute the inflation factor, the residuals need to be converted prior to being compared to the standard normal distribution. The conversion is carried out by normalizing the actual residuals using the estimated residuals [33], as expressed in Equation (18).

$$re{s}_{norm}=\frac{res}{ {\sigma }_{estimated}}$$

(18)

In Equation (18), $res$, $re{s}_{norm}$, and ${\sigma }_{estimated}$ represent the actual residuals, normalized residuals, and standard deviation of the residuals estimated using machine learning, respectively. As different machine learning models have been applied to the HPF and LPF data, the inflation factors should also be computed for each dataset, respectively. For the HPF, as the residuals represent measurement noise with a zero mean, the resulting ratio distribution is close to a Cauchy distribution. For the LPF, the corresponding ratio distribution is bimodal with peaks at $\pm 1$ due to the presence of non-zero mean effects of a multipath with both positive and negative signs. However, if the zero mean modeling error is sufficiently large, the ratio distribution for LPF will also be a Cauchy distribution. Although the shapes of the center areas for the normalized HPF and LPF may not perfectly match that of the standard normal distribution, our goal is to over-bound the tails of these distributions in order to meet the false alarm statistics required for monitoring purposes, so only the total area under the probability density function (pdf) bounded by the tails is of interest. If the proportion of normalized data exceeding the threshold is greater than the desired false alarm rate at the tail of the normal distribution pdf, an inflation factor (IF) needs to be applied. To determine the inflation factor, we use Equation (19) to search for the two points at which the actual residual accumulation from the end satisfies half of the maximum allowable false alarm rate.

$${{\displaystyle \int }}_{-\infty }^{M_L}{p}_{true}\left(x\right) dx={{\displaystyle \int }}_{M_R}^{\infty }{p}_{true}\left(x\right) dx={{\displaystyle \int }}_K^{\infty }{p}_{N\left(0,1\right)}\left(x\right) dx=\frac{1}{2}\ast P_{FA}$$

(19)

where $p_{true}\left(x\right)$ and $p_{N\left(0,1\right)}\left(x\right)$ represent the pdfs of actual residuals and standard normal distribution, respectively. $P_{FA}$ is the allowed probability of a false alarm and $K$ is the coverage factor of the standard normal distribution, which is calculated using Equation (19) [44,45,46,47,48,49,50]. $M_L$ and $M_R$ refer to the points at both ends of the actual data distribution that satisfy half of the allowed probability of a false alarm ($P_{FA}$). If the $M_L$ or $M_R$ is larger than $K$, it indicates that the estimated standard deviation needs to be inflated to meet the false alarm requirements. Therefore, the IF can be calculated using Equation (20).

$$IF=\frac{max\left(M_L, M_R\right)}{K}$$

(20)

After applying the inflation factors to the standard deviation, $\sigma$, of the HPF and LPF, the variance of SBDD residuals should be calculated by summing the squares of the inflated HPF and LPF standard deviations. Then, we multiply the expected residuals by the coverage factor to derive the thresholds. The final equation for the threshold is shown in Equation (21).

$$Threshold=K\sqrt{{\left(I{F}_{HPF}·{\sigma }_{HPF}\right)}^2+{\left(I{F}_{LPF}·{\sigma }_{LPF}\right)}^2}$$

(21)

4. Analysis of Field Test Results

Data collection for both model training and testing was carried out at Sejong University, Seoul, South Korea. Residual modeling and inflation factor computations were performed using the training dataset, and the created monitoring architecture was validated using the test data. The analysis for the model implementation was conducted using four parameters for each satellite: normalized satellite ENU direction components and the satellite $\mathrm{C}/\mathrm{No}$. As this model training is based on SBDD, a total of eight input variable parameters from a pair of satellites were employed for modeling the residuals. In order to simplify the visualization of the trained and modeled data, all the input parameters from the two satellites were set as identical.

4.1. Field Test Configuration

The GNSS receiver antennas were set up as shown in Figure 3. Two identical NovAtel OEMV6 receivers were used to collect the data, with each receiver connected to a NovAtel pinwheel antenna. The training data were collected on 21 July 2020 for $24$ h, and the test data were collected on 22 February 2021 for 24 h using the identical setup. The data were logged in the RINEX 3.1 format using NovAtel commercial software. To determine the true locations of each antenna, $24$ h of GPS data were post-processed using Trimble Business Center (TBC). The relative vector between the two receiver antennas was used for the residual computation, not absolute coordinates. The length of the short baseline used for the DD process was 0.409 m. The MATLAB regression package was utilized for machine learning data processing and modeling.

4.2. Residual Computations

SBDD residuals were computed for each satellite pair combination using Equation (7). The SBDD geometric range can be calculated using the relative vector, without the need for absolute position information for each antenna, as discussed in Section 2.1 For example, Figure 4 illustrates the SBDD residuals obtained for the pair of GPS PRN 1 and 3.

Next, a high-pass filter was applied to the SBDD residuals to separate the noise and multipath components from the residuals of the training dataset. The filtered output was then subtracted from the residuals to obtain the LPF. Figure 5 shows the resulting plot, where the HPF values are a zero mean and have smaller magnitudes compared to the LPF, which has a non-zero mean.

To incorporate all the training parameters into machine learning, the ENU components and $\mathrm{C}/\mathrm{No}$ of each satellite were recorded along with the corresponding residuals. The observed values of azimuth and elevation angles, which were converted from the ENU components, as well as $\mathrm{C}/\mathrm{No}$ for both satellites of the PRN 1 and 3 dataset are shown in Figure 6.

4.3. Machine Learning Residual Estimation

The machine learning models for the HPF and LPF have multivariable inputs from both satellites. However, to visualize the models in this section, the parameters for both satellites were set to be identical, and the $\mathrm{C}/\mathrm{No}$ was fixed at $45$ dB-Hz. The results for both the HPF and LPF models are presented in Figure 7. As expected, the modeled residuals are higher at lower elevations, and the residuals in the presence of a multipath increase as well. There are empty spaces in the northern part of the skyplot because no satellites are visible from the ground near $0°$ azimuth in the northern hemisphere. Additionally, there are no signals for elevations below $15°$ and azimuth between $135°$ and $180°$ due to obstruction from a nearby concrete structure, which is shown in the environment overlay provided in Figure 8. We can see that the visibility limitations coincide with the presence of the building, which blocks up to $14.5°$ elevation when surveyed using a geodetic GNSS receiver on top of the structure. Between $180°$ and $210°$ azimuths, the floor of the building rooftop consists of metal, leading to the effects of a multipath at low elevations. From the skyplot and the two-dimensional (2D) projections in Figure 8, we can see that the combined model is successful in estimating the residuals while accounting for the elevation-dependent receiver noise and effects from an environmental multipath.

We assessed the effectiveness of various methodologies in estimating the observed GNSS residuals for anomaly monitoring by comparing the statistical resemblance of each normalized distribution to the standard normal distribution. The models considered were the proposed algorithm that incorporates SBDD and machine learning, ZBDD, elevation-based models (EL), and $\mathrm{C}/\mathrm{No}$-based models (CN0). The normalized residuals ($Z$) for the proposed algorithm can be calculated using the following equation:

$$Z=\frac{re{s}_{HPF}+re{s}_{LPF}}{\sqrt{\left({\sigma }_{HPF}^2+{\sigma }_{LPF}^2\right)}}$$

(22)

where $re{s}_{HPF}$ and $re{s}_{LPF}$ represent the results obtained from a high-pass filter and a low-pass filter, respectively. These filter results are equivalent to receiver noise and a multipath effect. The values ${\sigma }_{HPF}$ and ${\sigma }_{LPF}$ are sigma values of the noise and multipath for each satellite parameter, which are modeled using the training dataset as described in Section 3.1.

The validation was conducted by assessing how well each method modeled the residuals observed for the 1-day training data on 21 July 2020. To evaluate the modeling performance statistically, we normalized the residuals observed from the training data with the residuals modeled by each method. Then, to find the best match, we compared the percentage of normalized samples within the $1\sigma$ and $2\sigma$ bounds. This approach is chosen instead of comparing the shapes or other portions of the distribution because most GNSS fault monitoring methodologies define requirements and performances in multiples of $\sigma$ [51].

Figure 9 displays the normalized residuals of the training data using various modeling approaches. For the training set, all methods, except for our proposed algorithm, generated a Gaussian-shaped distribution, indicating the lack of consideration for multipath effects by the other models. In contrast, our machine learning-based SBDD model resulted in a bimodal shape because it accounted for a multipath effect, $re{s}_{LPF}$ in Equation (22), as discussed in Section 3.2. While the normalized HPF data, $\frac{re{s}_{HPF}}{\sqrt{\left({\sigma }_{HPF}^2+{\sigma }_{LPF}^2\right)}}$, follows a Cauchy distribution, the LPF magnitude, $\frac{re{s}_{LPF}}{\sqrt{\left({\sigma }_{HPF}^2+{\sigma }_{LPF}^2\right)}}$, is the dominant component of the mixed distribution, leading to a distribution that deviates from the standard normal distribution. As the effects of the multipath are non-zero, the normalized distribution has peaks near $\pm 1$ when the multipath has been modeled correctly for the training dataset. However, if the modeling error is sufficiently large, the distribution will return to being of a Gaussian shape.

On the contrary, the statistics within the $1\sigma$ and $2\sigma$ bounds reveal different results than those inferred from the discrepancies in the distribution shapes near the center. For fault monitoring purposes, our primary focus lies on the statistics within the $1\sigma$ and $2\sigma$ bounds. The model that exhibits the closest match to these bounded statistics is the one we need to utilize. Although the distribution shape is close to Gaussian, the ZBDD underestimated the residuals because it removes all the multipath and atmospheric noise, as previously explained. As a result, probabilities bound to $\pm 1$ and $\pm 2$ by ZBDD were only 0.073 and 0.153, respectively, instead of being 0.686 and 0.954 of a unit normal distribution. As summarized in

Table 2. Probability bounding for $\pm 1$ and $\pm 2$ of each normalized model for the training data.

Model	$\pm 1$	$\pm 2$
ZBDD	0.073	0.153
Elevation	0.455	0.761
C/No	0.974	0.999
SBDD Machine Learning	0.683	0.980
Unit Normal Distribution	0.686	0.954

, the elevation-based function also underestimated the residuals, while the $\mathrm{C}/\mathrm{No}$-based modeling method overestimated them. Despite the difference in the shape of the probability density function, the statistical properties in the central part of the SBDD are the closest to those of the unit normal distribution, with probabilities of 0.683 and 0.980 for the $\pm 1$ and $\pm 2$ bounding, respectively.

Figure 10 and

Table 3. Probability bounding for $\pm 1$ and $\pm 2$ of each normalized model for the test data.

Model	$\pm 1$	$\pm 2$
ZBDD	0.087	0.184
Elevation	0.547	1.836
C/No	0.990	1.000
SBDD Machine Learning	0.731	0.953
Unit Normal Distribution	0.686	0.954

show the results of residual modeling for the test data, using the models obtained from the training data. In contrast to Figure 9, where the SBDD distribution was concentrated at $\pm 1$, the SBDD distribution exhibits a unimodal shape due to the increase in the modeling error. Using the same training dataset for both model generation and residual normalization resulted in a significant effect of the multipath, leading to a bimodal distribution. However, when the test dataset was used for the normalization, which differs from the sigma modeling, the model’s uncertainty increased. As a result, the influence of $\frac{re{s}_{HPF}}{\sqrt{\left({\sigma }_{HPF}^2+{\sigma }_{LPF}^2\right)}}$ in Equation (22) becomes more pronounced, causing the bimodal distribution to be less noticeable. Similar to the statistics in Table 2, other methods over- or under-estimated the actual residuals, but the proposed method produced a distribution that is statistically most similar to the standard normal distribution. The metrics for bounding are provided in Table 3, and it is evident that the proposed model surpasses its counterparts in performance.

4.3.1. Inflation Factor Calculation for Tail-over-Bounding

Figure 11 shows that the proportions of central area within the $1\sigma$ and $2\sigma$ bounds for both the normalized HPF and LPF residual distributions are similar to those of the standard normal distribution. However, the tails of the normalized residual distributions are heavier than the standard normal distribution as shown in Figure 12, potentially leading to an increased number of false alarms. To address this issue, the estimated residuals must be inflated.

In this study, we demonstrate the computation of the inflation factor and resulting threshold to meet a false alarm requirement of ${10}^{-5}$, which has been used in various previous works [41,52,53]. We computed the terms from the training dataset and then applied them to the test dataset to verify the false alarm probability. The coverage factor, k, needed to meet the requirement of ${10}^{-5}$ is 4.4172 for both sides of the standard normal distribution. If the estimated residuals accurately reflect the tail area with the given probability, then 99.999% of the normalized residuals should fall inside the bounds.

To calculate the inflation factors for the HPF and LPF residuals, we searched for the actual residuals in the normalized training dataset that correspond to half of ${10}^{-5}$ at each tail end, as expressed in (19). We chose the larger of the two values to obtain the conservative inflation factors, which were 8.31 for HPF and 7.82 for LPF. Finally, the inflation factors of HPF and LPF were computed to be $1.88$ and $1.77$, respectively, using (20).

4.3.2. Residual Monitoring Results

After applying the inflation factors, the $K$ factor of $4.4172$ was multiplied to the thresholds, as shown in (21), to calculate the residual monitoring thresholds with a ${10}^{-5}$ uncertainty level. We evaluated the performance of the proposed monitoring methodology using the test data. Figure 13 shows example cases of residual anomaly monitoring for GPS satellites PRN 31 and 32. The blue line illustrates the SBDD residuals, while the magenta dots denote the thresholds obtained using the machine learning-based residual modeling engine, which takes satellites’ $\mathrm{C}/\mathrm{No}$ and elevation and azimuth angles of the satellites as input arguments. When the elevation angle or the $\mathrm{C}/\mathrm{No}$ increases, the expected receiver noise decreases and the threshold becomes tighter. When only the $\mathrm{C}/\mathrm{No}$ drops due to the effects of a multipath, the threshold takes this into account and loosens the threshold despite no significant change in the elevation. Therefore, the proposed methodology not only tightly bounds the residuals but also considers multiple parameters for more robust anomaly monitoring. Figure 14 shows the residual monitoring results for multiple PRN combinations. From the results, it is evident that the thresholds consistently tightly bound the SBDD residuals under nominal, fault-free conditions.

The thresholds are conservative and tend to over-bound the residuals most of the time, but may be too tight for certain epochs. Prior to applying the inflation factor, the false alarm rate, which represents the proportion of the residuals that exceeded the computed threshold, was $1.1\times {10}^{-3}$, which is significantly higher than the assigned false alarm requirement. However, after applying the IF to the threshold according to Equation (21), warnings were issued for 22 samples out of a total of 3,328,150 residuals in the 24 h test dataset, even though no actual anomaly event occurred. This corresponds to a false alarm rate of $6.61\times {10}^{-6}$, which is slightly lower than the desired level of ${10}^{-5}$. It is suspected that this is due to choosing a larger value from the two tails in Section 4.3.1 in order to calculate a more conservative IF, and also computing IF for HPF and LPF separately and combining them together assuming data independence.

5. Discussion of the Results and Conclusions

In this study, we used SBDD to accurately compute the GNSS measurement residuals, and proposed machine learning models, namely tree and Gaussian SVM regression, to effectively model the nominal values of the residuals under fault-free conditions. We evaluated the effectiveness of our suggested residual estimation and modeling method in the light of anomaly monitoring, and compared it to several common residual estimation methods. The assessment validated that the proposed methodology was more accurate than other existing models in estimating the residuals affected by both receiver noise and a multipath environment. Afterwards, the proposed model was used to normalize the training data and compute the IF to over-bound the fault-free measurements. Finally, uncertainty level multipliers were applied to derive the residual thresholds required to satisfy the required system false alarm probability.

The results of our study demonstrate that the residuals obtained from the test data were successfully bound by the thresholds derived from the trained model. Although the threshold was determined conservatively, the observed false alarm rate was less than but similar in magnitude to the theoretically expected rate, suggesting that the threshold was sufficient for the probability of a false alarm but not too loose for the probability of misdetection. This validates the effectiveness of the proposed GNSS carrier phase residual monitor for enhancing GNSS operation integrity. This monitoring architecture can also be used for code and doppler measurements. The implementation for code measurements would be identical, with the exception of the integer ambiguity. For doppler measurements, neither the integer ambiguities nor the geometric range terms need to be considered.

While our study primarily focused on modeling residuals under fault-free conditions and conducting a feasibility test of the proposed algorithm, we have future plans to engage in extensive experimental validation across multiple sites over an extended duration. Moreover, our future work will involve investigating methods for detecting faults using the developed model. Specifically, we aim to study the detection of usual fault events such as cycle slips and a multipath at a reference station. By considering the geometric correlation between the two antennas, we hope to estimate the minimum detectable error that could facilitate effective detection and exclusion of multipath effects, which is one of the most unsolved problems.