Comparison of NeQuick G and Klobuchar Model Performances at Single-Frequency User Level ^†

Abstract

In this study, the NeQuick G and Klobuchar models are evaluated by monitoring performance issues related to ionosphere activity for single-frequency users. The effects of radio frequency (RF) signal propagation through the ionosphere may have a significant impact on satellite communication and navigation systems because of geomagnetic field geometry near the magnetic equator and in the proximity to the high- and low-latitude zones. An ongoing challenge is determining how accurate the ionospheric models employed by existing Global Navigation Satellite Systems (GNSSs) are. This work investigates the patterns of total electron content (TEC) fluctuations over distinct zones from 1 January 2019 to 30 June 2022. Measurements are collected at station networks deployed worldwide. Firstly, monthly and seasonal variations of TECs are analysed. Secondly, the TEC ’availability’ parameter, as the percentage of time when the TEC error is compliant with the specification of the Galileo Single-Frequency Ionosphere Algorithm (’NeQuick G’ model), is introduced. The TEC error defines the difference between (a) the model TEC, obtained by either the NeQuick G or the Klobuchar model over a given station, and (b) the reference TEC, based on observations from networks of GNSS receivers. Finally, the position, velocity, and time (PVT), along with broadcast group delays (BGDs) are analysed and the PVT accuracy is compared between the NeQuick G and Klobuchar models. In 3.5 years, the seasonal behaviour of TEC shows maxima during the March and October equinox and minima during the June and December solstice. Moreover, an increase in the TEC values and the amount of TEC errors are visible as we are approaching the next solar maximum. Preliminary results show a larger associated positioning error using the Klobuchar than the NeQuick G model. However, the difference is zone-dependent, most evident in equatorial regions. This collaborative study of the GRC, NMA, TUW, and SRC was performed under the Framework Partnership Agreements (GSA/GRANT/04/2016).

1. Introduction

The collaborative study presented in this paper was performed under the European Union Agency for Space Programme (EUSPA) Framework Partnership Agreements (GSA/GRANT/04/2016) involving the entities Galileo Reference Centre (GRC) [1], Norwegian Mapping Authority (NMA), Technische Universität Wien (TUW), and Space Research Centre (SRC) of Polish Academy of Sciences, whose main objectives were to investigate the GNSS performance at different levels. The ionosphere, the ionized part of the Earth’s upper atmosphere, is highly dynamic and is influenced by various space weather phenomena. The presence of free electrons can significantly impact radio wave transmission, altering the electromagnetic radio frequency waves [2]. Satellite communication and navigation systems, such as the Galileo and the Global Positioning System (GPS), rely on radio signals transmitted through the ionosphere. However, the ionosphere can cause significant errors in these signals, which can lead to degraded performance and reduced accuracy, especially at the equatorial and high latitudes. To mitigate the effects of the ionosphere on satellite systems for single-frequency (SF) applications, ionospheric models have been developed [3]. Their final scope is to provide accurate estimates of the total electron content (TEC). Two widely used ionospheric models, enabling the ionospheric delay corrections computations for SF users, are the NeQuick G model, developed for Galileo, and the Klobuchar model, developed for GPS. NeQuick G [3] is a three-dimensional and time-dependent ionospheric electron density model based on an empirical climatological representation of the ionosphere, which predicts monthly mean electron density from analytical profiles, depending on solar-activity-related input values: sunspot number or solar flux, month, geographic latitude and longitude, height, and universal time (UT). On the other side, the Klobuchar model [2] is designed to minimise user computational complexity and user computer storage as well as to keep a minimum number of coefficients to transmit on satellite-user links. In this study, the comparison of NeQuick G and Klobuchar models at the SF user level is shown. Three parameters are used to present the results: the TEC accuracy (also called TEC error) defined as the difference between the model TEC obtained by either the NeQuick G or the Klobuchar model over a given station and the reference TEC based on observations from networks of GNSS receivers; the TEC availability characterization defined as the percentage of time when the TEC error is compliant with the specification of the Galileo Single-Frequency Ionosphere Algorithm (NeQuick G model); the Position, Velocity, and Time (PVT) accuracy characterization used to assess the impact of ionospheric disturbances on the accuracy of PVT estimates. The results are presented for a network of stations distributed worldwide. The computation method used for the TEC estimates is different for the networks of stations used by the involved entities GRC, NMA, TUW, and SRC. The results cover the period from 1 January 2019 to 30 June 2022. The comparison of the performance of the two models in the different networks is the key element of this work, providing a better understanding of the model performances. Moreover, this study provides valuable insights into the performance of the NeQuick G and Klobuchar models at the SF user level.

2. Data and Methods

This section explains the data and methods used to calculate the TEC accuracy, TEC availability, and SF position accuracy. The stations used by the different entities are spread worldwide as shown in Figure 1.

These stations are therefore divided into four groups according to the different geomagnetic latitude zones. Stations considered by individual entities are marked with different colours; in the case of TUW, in the mid-latitude region, 5 stations per quarter were considered. The station choice for each quarter was mainly dependent on the site location within the mid-latitude region and RINEX data availability (2019 Q1: AJAC, GRAS, JOZ2, LEOP, NICO; 2019 Q2–2019 Q4: AJAC, GRAS, LEOP, NICO, WROC; 2020 Q1–2021 Q4: EBRE, GRAS, GRAZ, NICO, WROC). Due to urgent maintenance activities at the Data Centre of the Bundesamt für Kartographie und Geodäsie (BKG) of the International Geodesy Service (IGS), and hence RINEX data unavailability, the majority of used validation sites had to be replaced in the 1st and 2nd quarter of 2022 (2022 Q1: DYNG, GRAS, MELI, NICO, WARN; 2022 Q2: DYNG, EBRE, GOP6, GRAS, WSRT).

As described in the introduction, this paper presents two ionospheric models used for SF receivers, the NeQuick G and Klobuchar models. The NeQuick G model [4] is designed to reach a correction capability of at least 70% of the ionospheric code delay in terms of root mean square (RMS), with a lower Slant total electron content (STEC) residual error bound of 20 TEC Unit (TECU) for any location, time of day, season, and solar activity, excluding periods where the ionosphere is largely disturbed due to, for instance, geomagnetic storms. On the other hand, the Klobuchar model [2] procides for GPS SF users an accuracy of about 50% RMS of the ionospheric range delay, depending on the solar activity or the region. Hence, its accuracy is often unsatisfactory even for absolute positioning, especially in the case of heightened solar activity [5].

2.1. TEC Accuracy

To test the accuracy of the broadcast ionospheric models for both NeQuick G and Klobuchar, the observations derived from the selected networks of stations described above, are compared. For the investigation of the accuracy of the modelling we present results in a vertical total electron content (VTEC) format. This is directly comparable to common ionospheric models and avoids having the elevation angle as a dimension in the plots. We note this because for other parts of this paper STEC is used in the underlying calculations, as the NeQuick G requirement is defined using STEC. The VTEC accuracy, performed for the Galileo and the GPS constellations, is obtained following the three steps below:

• The computed VTEC values are acquired using the navigation models. The models are different depending on the constellation, NeQuick G for Galileo and Klobuchar for GPS. The computation follows the algorithm as described in [3] and in [2] for NeQuick G and Klobuchar, respectively. The different institutions involved in the study, used different approaches, in particular, for NMA/TUW, the VTEC for NeQuick G and Klobuchar is calculated every 5 min using the received ionosphere parameters, for the set of satellites observed by the GNSS reference stations (Figure 1), using the actual observation geometry (receiver and satellite positions).
• For the same set of points, the reference VTEC is computed. A description of the methodologies is provided, detailing the computation of the reference VTEC. For NMA, the observed VTEC values are calculated every 5 min based on dual-frequency measurements from the GNSS reference stations (Figure 1). For the high-latitude stations, differential code biases (DCBs) are retrieved from the NMA’s ionosphere monitoring system as the receivers used here are part of the much larger network processed by that system. For the low-latitude stations, DCBs are retrieved from the International GNSS Service (IGS) archives. For TUW, the observed VTEC values are calculated every minute based on dual-frequency measurements from the GNSS reference stations (Figure 1). For the mid-latitude stations, DCBs are obtained by a least squares adjustment based on P1 and P2 code observations, as well as satellite DCBs downloaded from the Astronomical Institute of the University of Bern (AIUB) CODE database. The standard Single Layer Model (SLM) is used to convert slant TEC to vertical TEC (ionosphere altitude 350 km).
• The VTEC error is finally calculated as the difference between the model-derived VTEC values and the reference VTEC values obtained through the observations.

2.2. TEC Availability

To obtain the TEC availability, the STEC parameter is considered, instead of the VTEC one. The reason for this choice is to be in line with the NeQuick G requirements specification, described below. Based on the NeQuick G specification, described in Section 2, this section introduces the TEC availability parameter which assesses the capability to correctly estimate the ionospheric delay affecting the Signal in Space (SiS) measurement. The TEC availability is defined as the number of epochs (for a single station) where the STEC error is under the following Galileo NeQuick G requirements [3]:

• For an STEC ≤ 66.7 [TECU]: Threshold = 20 TECU;
• For an STEC > 66.7 [TECU]: Threshold = 30% of STEC.

From the GRC network of stations, the STEC is obtained for a grid of points using the navigation model and averaging the contribution of the different satellites; while the reference STEC is obtained with the tomographic ionosphere model (TOMION) software V1.6.4. [6] developed at the Universitat Politècnica de Catalunya (UPC). The STEC error is therefore defined as the difference between the STEC as computed from the navigation model and the reference STEC as computed using the TOMION software. To perform a fair and comprehensive performance assessment between the NeQuick G and Klobuchar ionospheric models, the Galileo SF ionospheric algorithm specification is used as a common benchmark for both models.

2.3. SF Position Accuracy

This section describes the position accuracy computation algorithm used by the different institutions. The algorithm consists of three main steps, which are detailed below.

• The Galileo E1 SF solutions are calculated in post-processing mode based on RINEX files, whereby the settings are applied as shown in

  Table 1. Settings applied by each institution to calculate SF position accuracy.

  	GRC	NMA	SRC	TUW
  Software	TOMION [6]	Where [7]	GALAT 1	raPPPid 2 [8]
  Sampling rate	30 s	30 s	30 s	30 s
  Elevation cut-off mask	5 deg	5 deg	5 deg	5 deg
  Troposphere delay	GPT2	GPT2w [9]	GPT [10]/GMF [11]	VMF3 [12]
  Solid Earth tides	applied	applied	applied	applied
  Elev.-dep. weighting	1/sin (elev.)	1/sin (elev.)	1/sin (elev.)	1/sin2 (elev.)
  Estimation	Epochwise least	Epochwise least	Epochwise least	Kalman filter
  	square	square	square
  Estimated parameters	coordinates,	coordinates,	coordinates,	coordinates,
  	receiver clock	receiver clock	receiver clock	receiver clock,
  				ZWD 3
  PDOP threshold	6	6	6	N/A

  ¹ SRC internal software; ² TUW open-source software—https://github.com/TUW-VieVS/raPPPid, https://vievswiki.geo.tuwien.ac.at/raPPPid (accessed on 10 May 2023); ³ Zenith Wet Delays.

  . In the first run, the NeQuick G model is applied for ionospheric corrections, while in the other one, the Klobuchar model is used.
• The reference site positions are based either on the official International Terrestrial Reference Frame 2014 (ITRF2014) SINEX file station position and velocity solution (epoch 1 January 2010) extrapolated to the selected epoch from the analysed quarter of observation (TUW) or yearly calculated ITRF14 coordinates, which refer to an epoch on 1st January of each year (NMA, GRC).
• Coordinates obtained from the two SF solutions are directly compared epoch-wise in a local topo-centric coordinate system (East, North, Up) to the reference values in ITRF 2014, whereby differences are computed for east, north, and upward coordinates. The horizontal positioning error (HPE) and vertical positioning error (VPE) can be derived based on the differences in east, north, and up ($\mathsf{\Delta }E$, $\mathsf{\Delta }N$, $\mathsf{\Delta }U$) for each epoch i, as follows:
  $$\begin{array}{cc}\hfill HP{E}_i& =\sqrt{\mathsf{\Delta }{E}_i^2+\mathsf{\Delta }{N}_i^2}\hfill \end{array}$$

  (1)

  $$\begin{array}{cc}\hfill VP{E}_i& =|\mathsf{\Delta }{U}_i|\hfill \end{array}$$

  (2)

Then, the single-frequency position errors were calculated as a mean of the 95th percentile (HPE 95% and VPE 95%) for the reference sites on a daily basis. Hereby, epochs exceeding the position dilution of precision (PDOP) threshold are excluded. The results in Figure 2 show the monthly average of these daily HPE 95% and VPE 95% site values grouped into high-latitude, middle north, middle south, and low-latitude regions.

3. Results

3.1. TEC Accuracy

Figure 3 shows the monthly mean VTEC error for the high/middle/low latitudes and for the NeQuick G and Klobuchar model for the NMA and TUW network of stations as described above. Annual variations of the errors can be seen for the high and mid-latitudes. These variations are of opposite signs for the two models, i.e., when the Klobuchar-derived error decreases, the NeQuick G-derived error increases, and vice versa.

At high latitudes, the NeQuick G errors vary around zero, approximately between −2 and +2 TECU. The Klobuchar errors have an offset from zero of about 5–6 TECU, and slightly higher peak-to-peak values during a year. There may be a long-term decreasing trend in the Klobuchar error values. In the last quarter of 2021, the largest discrepancy (values comprises between 10 TECU and 15 TECU) in the time series error of the two models over the analysed period is observed. This may be related to the increasing ionization and space weather activity as the solar activity increases in the current solar cycle 25. At mid-latitudes, NeQuick G and Klobuchar have similar error magnitudes but with opposite offsets from zero. On average NeQuick G is slightly closer to zero, but this difference between Klobuchar and NeQuick G errors is smaller than the offsets of their individual time series and the inherent variations in the time series. At low latitudes, there is no obvious annual trend. Instead, both error graphs appear to have a long-term trend. This time series is not long enough to determine if this is related to the 11-year solar cycle, but this would be the most plausible explanation. It is known that the general ionization level varies with the solar cycle, and as the low latitudes have the largest TEC levels, they will also have the largest magnitude of long-term variation.

3.2. TEC Availability

This section describes the monthly mean TEC availability results for the high/middle north/middle south/low latitudes and for the NeQuick G and Klobuchar model for the GRC network of stations as described above. Numerical results are presented in Figure 4. As explained in Section 2.3, the Galileo SF ionospheric algorithm specification serves as a benchmark for both models. Being the specification not designed to fit the Klobuchar model, its worst performances with respect to the NeQuick G one are evident. However, the patterns of the Klobuchar follow as seeing for the VTEC accuracy a seasonal variation, showing the worst results during equinoctial months. On the contrary, NeQuick G performs better in the middle south, middle north, and high-latitude regions. In both models, this behaviour is more visible in low latitudes, in agreement with the disturbed ionospheric conditions at equatorial regions. In addition, during the last period analysed it is observed that the performances for both models are worst in the low-latitude regions. As described in Section 3.1, this time series is not long enough to conclude that it is driven by the 11-year solar cycle.

3.3. SF Position Accuracy

In Figure 2, SF monthly mean horizontal (HPE 95%) and vertical (VPE 95%) position accuracies are displayed, respectively, for the used stations for the high-/middle north/middle south/low-latitude regions. The position errors are estimated twice, once when the NeQuick G ionospheric model was applied to the positioning solution and another time when the Klobuchar model was applied. The monthly HPE 95% show, in general, smaller magnitudes compared to the monthly VPE 95%, which is mainly the case at high and both middle latitudes, especially for the Klobuchar model. Differences between the GRC, SRC, and NMA/TUW can be seen in the SF position accuracy solutions. This could be explained that each institution uses different kinds of station networks and that different software packages, models, and configurations are applied for the HPE 95% and VPE 95% calculation (see Table 1). At high latitudes (Figure 2) a more-or-less linear trend line of HPE 95% can be noted, which is gradually rising over time and reaching its maximum in April 2022 (beginning of Q2 2022), after which it declines. Before April 2022, the position error is on average 1 m for the NeQuick G model (maximum HPE 95% reached among all the results 2.35 m in 2022 Q2), while for the Klobuchar model values are slightly larger (maximum HPE 95% reached 2.52 m in 2022 Q2). Slightly—but not significantly—larger HPE values for the Klobuchar model compared to the NeQuick G model can be observed at mid-latitudes. The low latitudes for both tested models have 2–4 times larger magnitudes compared to the two higher latitude regions. The largest errors are reached in March and April 2022 (end of Q1 2022, beginning of Q2 2022), being ≈5.5 m for the NeQuick G model and 6.93 m for the Klobuchar model. After those maxima are reached, a rapid decline in monthly HPE 95% at low latitudes can be noticed, approaching the high and middle latitude error values.

Significantly higher differences in positioning errors between the NeQuick G and the Klobuchar model are reflected in the monthly VPE 95% (Figure 2). Again, a slight linear trend can be recognized at high latitudes for the NeQuick G model (maximum VPE 95% reached 3.5 m in December 2021). On the other hand, the Klobuchar model does not indicate such a trend. The Klobuchar VPEs 95% are throughout the whole period larger compared to the NeQuick G values, with the largest difference being up to 5.36 m in December 2021. The difference between the NeQuick G and the Klobuchar model is also evident at middle latitudes. Moreover, in the case of SRC, for stations located below the 30${}^{°}$ N parallel, in 2021, there is a significant difference between VPE 95% for Klobuchar and NeQuick G, reaching even 4 m in May 2021. There is also a visible difference between the two groups of mid-latitude stations, while the northern ones behave similarly to the high-latitude group, the southern ones have characteristics closer to equatorial stations. The vertical positioning errors from NeQuick G (maximum VPE 95% reached 3.7 m in February 2022) show, as well, for the whole period smaller magnitudes (up to 1 m smaller compared to Klobuchar). The VPE 95% values at low latitudes demonstrate a similar pattern as the horizontal positioning errors for both tested models. In fact, NeQuick G values are on a comparable level (difference |HPE 95%–VPE 95%| at low latitudes approximately up to 0.5 m) with the exception in April and May of 2022 (Q2 2022), where the difference between horizontal and vertical errors reached almost 1 m. During the analysed period, Klobuchar VPEs 95% were overall larger compared to the NeQuick G VPE 95% (one exception in March 2022).

4. Discussion

4.1. Network-Wise Performance Comparison

This study explores the benefits of using multiple networks of stations around the world to improve ionospheric models for SF users. NeQuick G and Klobuchar models are compared across a wide geographic range and under different ionospheric conditions to evaluate their performance. Data collection from multiple networks provides a more comprehensive evaluation of the models’ ability to mitigate ionospheric effects. SF positioning accuracy is considered for each network of stations, but results may vary due to differences in ionospheric characteristics and data quality. Variations in the number and distribution of ground-based stations, data quality, and local ionospheric conditions could lead to differing results. Overall, this study highlights the benefits of using multiple networks to improve ionospheric models for SF users.

4.2. Model-Wise Performance Comparison

The NeQuick G and Klobuchar models are compared in their ability to mitigate ionospheric effects on GNSS signals for SF users. NeQuick G uses a physical-based approach and is more accurate than Klobuchar, especially at high latitudes and during intense solar activity, while the Klobuchar model uses an empirical approach. NeQuick G can model smaller ionospheric plasma structures and the 3D distribution of plasma density better than Klobuchar. The results, therefore, show better performances of NeQuick G compared to Klobuchar. TEC availability is more stable for NeQuick G at high middle north and middle south latitudes, and the effect on TEC availability is more evident at low-latitude regions with the best performances achieved by NeQuick G. Although, in terms of horizontal positioning, both models provide similar results at all analysed periods and stations, NeQuick G performs better than Klobuchar for vertical positioning.

4.3. Implications of the Results for SF Users

The implications of the results for SF users are significant, as these users typically have more limited access to multiple GNSS frequency signals and are, hence, more susceptible to the effects of ionospheric disturbances. The study founds that in terms of TEC accuracy, TEC availability, and SF position accuracy, the NeQuick G model outperformed the Klobuchar model. It suggests that the NeQuick G model is more effective for mitigating the effects of ionospheric disturbances on SF GNSS signals. For SF users, the improved TEC accuracy and availability offered by the NeQuick G model can result in more accurate position solutions and improved navigation performance. The improved SF position accuracy offered by the NeQuick G model may also lead to increased confidence in the accuracy of positioning solutions in challenging ionospheric conditions. Additionally, the results of this study may be useful in informing future developments of ionospheric models for SF users. Overall, the study highlights the importance of selecting an appropriate ionospheric model for mitigating ionospheric effects on GNSS signals, particularly for SF users who may have limited access to multiple-frequency signals.