Online State-of-Health Estimation for NMC Lithium-Ion Batteries Using an Observer Structure

Abstract

State-of-health (SoH) estimation is one of the key tasks of a battery management system, (BMS) as battery aging results in capacity- and power fade that must be accounted for by the BMS to ensure safe operation over the battery’s lifetime. In this study, an online SoH estimator approach for NMC Li-ion batteries is presented which is suitable for implementation in a BMS. It is based on an observer structure in which the difference between a calculated and expected open-circuit voltage (OCV) is used for online SoH estimation. The estimator is parameterized and evaluated using real measurement data. The data were recorded for more than two years on an electrified bus fleet of 10 buses operated in a mild European climate and used regularly in the urban transport sector. Each bus is equipped with four NMC Li-ion batteries. Every battery has an energy of 30.6 kWh. Additionally, two full-capacity checkup measurements were performed for one of the operated batteries: one directly after production and one after two years of operation. Initial validation results demonstrated a SoH estimation accuracy of ±0.5% compared to the last checkup measurement.

1. Introduction

Steadily increasing pollution is especially harmful to urban areas [1,2,3,4] and the health of the people living in them [5,6,7]. The EU Parliament agreed on a regulation for the transport sector which calls for a 30% reduction in CO₂ emissions by 2030 relative to 1990 [8]. To achieve this goal, electrification of the private and especially the public transport sectors is of great importance. Primarily through the electrification of buses and trains, it has become possible to reduce emissions in urban areas [9]. Local public transportation services are usually well established in regard to their structure. Problems such as the range of the battery electric vehicles and the required charging infrastructure, which are quoted frequently as exclusion criteria for electrification in the private transport sector, are rated differently in the microsystems of public transportation [10].

Due to high loads and long daily periods of use, battery systems used in commercial vehicles face much greater challenges than those in passenger cars. Market success is closely linked to durability [11,12,13], which makes constant monitoring of the battery’s operating conditions indispensable [14,15,16]. Consequently, existing monitoring concepts such as those suggested by Pop et al. [17] need to be enhanced. This is especially the case for battery-aging monitoring systems to achieve a more sustainable battery lifecycle [18,19,20].

In principle, two main effects can be observed in battery aging: battery capacity losses by losing lithium inventory, and active material and increasing internal resistances, where the former leads to a loss in drive range and the latter to a reduction in performance. This work focuses on the estimation of state of health (SoH) in terms of capacity losses over a battery’s lifetime.

We define capacity SoH $So{H}_{\mathrm{C}}$ according to Bauer [21] and Jossen et al. [22] as the ratio of the actual capacity $C_{\mathrm{a}\mathrm{c}\mathrm{t}}$ and the nominal capacity $C_{\mathrm{n}\mathrm{o}\mathrm{m}}$:

$$So{H}_{\mathrm{C}}=\frac{C_{\mathrm{a}\mathrm{c}\mathrm{t}}}{C_{\mathrm{n}\mathrm{o}\mathrm{m}}}.$$

(1)

Closely related to SoH is the state of charge (SoC) of a battery, which we define according to Fleckenstein and Jossen et al. [15,22] as the ratio of the charge $Q$ currently stored in the battery and the actual capacity $C_{\mathrm{a}\mathrm{c}\mathrm{t}}$:

$$SoC=\frac{Q}{C_{\mathrm{a}\mathrm{c}\mathrm{t}}}=\frac{Q}{C_{\mathrm{n}\mathrm{o}\mathrm{m}}·So{H}_{\mathrm{C}}}.$$

(2)

A good SoH estimation is important for two reasons in particular. First, safety risks arise when a battery is operated outside its specified SoH limits; that is, the cells are used longer than agreed with the cell manufacturer. Second, a poor SoH estimate affects the SoC estimation, as can be seen in Equation (2). This can lead, for example, to the battery being discharged more deeply than intended, which in turn can cause permanent damage to the battery.

SoH estimation methods can typically be divided into three groups. Yao et al. [23] distinguishe between model-based-, data-driven- and fusion-technology-methods. The group of model-based methods includes equivalent circuit models as well as electrochemical models. Neural networks and statistical filtering methods belong to the data-driven methods. The last group describes a combination of several methods. Some concepts are listed below.

Soon et al. [24] describe an energy throughput-based method for SoC and SoH calculation. The authors show the dependency of the SoC accuracy on the precision of the residual capacity estimation. If the SoH confidence level is low, SoC deviations of up to 10% can occur. Both variables interact with each other, demonstrating the necessity of an additional function which recalibrates SoH and SoC.

Zou et al. [25] choose an approach for SoC and SoH estimation based on two Extended Kalman Filters (EKF) with different time scales. One of them is running online to calculate SoC and the other one is used offline to estimate SoH. The EKF for SoC estimation is very accurate, with an average relative error of 0.7%. But the EKF for SoH is also precise: within a few hours, the estimation error is less than 3%. This method is highly dynamic. It only needs a few cycles to predict the correct remaining capacity value. However, due to its complexity, it needs to be computed offline instead of being directly computed on the vehicle’s battery management system (BMS). Similar results are provided by Fang et al. [26] and Sun et al. [27].

Li et al. [28] introduce some different analysis methods, like differential analysis (DA), differential voltage analysis (DVA) or differential thermal voltammetry (DTV). With the results of these measurements, they train neural networks (NN) and give an outlook over the advantages and disadvantages.

Wen et al. [29] analyze the correlation between the characteristics of the incremental capacity (IC) curve and SOH at different ambient temperatures. They establish a SoH prediction model at different temperatures by using a least square method.

Other studies focus on SoH estimation based on changes in the characteristic open-circuit voltage (OCV) curve or the final charging and discharging characteristic curve. Both Weng et al. [30] and Wang et al. [31] use the incremental capacity analysis (ICA) derived from the OCV curve to detect and interpret the changes. One disadvantage of the suggested approaches is that a broad SoC window must be charged or discharged at a constant current rate, which is unfeasible in most real operations.

Kong et al. [32] performed a regression for an aging model based on the changing voltage curves by observing a fixed voltage value. The authors assume that at one SoC-OCV point there is a linear correlation between the gradient and the aging. Maher et al. [33] show the behavior of the OCV across the SoC and the amount of charged energy as a function of aging using a Lithium Cobalt Oxide cell. As the differently aged OCV capacity curves are plotted on top of each other, the partial gradient of the curves can be observed to become steeper. This effect is to be exploited in the SoH estimator presented in this paper. Maher et al. themselves do not use the dependence on the capacity but look at the temperature behavior of the OCV and the resulting entropy and enthalpy over SoC to rate the state of the cell. Lavigne et al. [34] also give an example of state estimation via OCV modelling.

More detailed studies on the evaluation and classification of different SoH estimation methods can be found in the reviews of Ungurean et al. [35] and Komsiyska et al. [16].

In contrast to most of the methods mentioned before, the SoH estimation approach presented in the remainder of this paper is designed to run during normal operation of the battery. Its functionality is demonstrated on real field data available for a period of two years. Additionally, the data are complemented by two full-capacity checkup measurements, allowing the evaluation of the SoH estimate at these points in time. Another distinguishing feature is the low computational complexity of the estimator, which favors its use on common BMS hardware.

The remainder of this article is structured as follows: First, the basic idea and the concept behind the presented SoH estimator are explained in Section 2. Next, the experimental data used to evaluate and parameterize the estimator are presented in Section 3. The results are then presented and discussed in Section 4. Finally, the paper is concluded in Section 5.

2. Online State-of-Health Estimator

2.1. OCV Curve

For an accurate state estimation of a lithium-ion battery (LIB) within the battery’s BMS, it is essential to know the nonlinear monotonic correlation between SoC and OCV. The main factors that influence the characteristic OCV curve are aging, temperature and usage history [36]. The “usage history” can also be described as “path dependency” of aging, as Ma et al. [37] or Gering et al. [38] describe it. In the context of SoH estimation, the main challenge is to correctly track and interpret these factors over the battery’s lifetime.

2.1.1. Basics

In general, the OCV $U_{\mathrm{O}\mathrm{C}\mathrm{V}}^{\mathrm{c}\mathrm{e}\mathrm{l}\mathrm{l}}$ of a battery cell is described by the difference between open-circuit potentials of both electrodes

$$U_{\mathrm{O}\mathrm{C}\mathrm{V}}^{\mathrm{c}\mathrm{e}\mathrm{l}\mathrm{l}}=U_{\mathrm{O}\mathrm{C}\mathrm{V}}^{\mathrm{p}\mathrm{o}\mathrm{s}.\mathrm{e}\mathrm{l}.}-U_{\mathrm{O}\mathrm{C}\mathrm{V}}^{\mathrm{n}\mathrm{e}\mathrm{g}.\mathrm{e}\mathrm{l}.},$$

(3)

where $U_{\mathrm{O}\mathrm{C}\mathrm{V}}^{\mathrm{p}\mathrm{o}\mathrm{s}.\mathrm{e}\mathrm{l}.}$ and $U_{\mathrm{O}\mathrm{C}\mathrm{V}}^{\mathrm{n}\mathrm{e}\mathrm{g}.\mathrm{e}\mathrm{l}.}$ are defined as the positive and negative electrode potentials measured against a reference lithium metal electrode, respectively [39].

To determine the OCV, the cell is set to different SoCs and temperatures in sequence. At these setpoints, the cell is allowed to relax for a constant time. After these defined periods, it is assumed that the cell is relaxed, and the voltage is measured for the OCV look-up-table (LUT). The OCV depends strongly on the previous aging path:

• Due to current direction, the OCV potential can be higher or lower; therefore, a hysteresis is observed for most cell chemistries. While the hysteresis for NMC cells is only a few mV, it can be a multiple of 10 mV for lithium iron phosphate (LFP) cells. Baghdadi et al. [40] and Lavigne et al. [41] both show in their studies that the hysteresis increases at low and high SoC. In typical SoC usage windows, the effect is very small. In this investigation, the focus is on NMC. Therefore, hysteresis effects are neglected for the moment.
• Besides hysteresis, temperature has a large influence on OCV. With higher temperatures, the chemical reactivity increases. The warmer it is, the faster the cell reaches its equilibrium state [36,39].
• Often the relaxation time also becomes larger with older cells [36].

These effects show that the characteristic OCV curve gives a lot of information about the cell’s health condition. However, it is not easy to measure and interpret the OCV curve during battery operation.

2.1.2. Aging Characteristics

The characteristic shape of a battery’s OCV curve changes over its lifetime [34,36]. Figure 1 shows the changes of a characteristic OCV curve measured from the beginning of life ($So{H}_{\mathrm{C}}=100\%$) to end of life ($So{H}_{\mathrm{C}}\approx 80\%$).

As can be seen, with advancing age the OCV shape becomes increasingly compressed and steeper. The idea of the SoH estimator presented next is based on detecting this compression and using it to estimate the battery’s SoH.

2.2. Approach

The SoH estimator presented in this paper exploits the changes in the OCV curve that occur due to aging, as described in Section 2.1.2. The basic idea is to estimate the battery’s SoH based on a correction factor

$$m=\frac{m_{\exp }}{m_{\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}},$$

(4)

which is calculated by relating the OCV curve’s expected gradient $m_{\mathrm{e}\mathrm{x}\mathrm{p}}$ with the gradient $m_{\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}$ calculated with the help of a pseudo-OCV value determined with the help of an equivalent circuit battery model. The factor $m$ is then used in a feedback loop $L\left(m\right)$ to gradually estimate $So{H}_{\mathrm{C}}$ so that $m$ becomes asymptotically $m=1$. The difficulty of this approach is that the pseudo-OCV value which is needed to calculate $m_{\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}$ cannot be determined under all operating conditions (see also Section 2.1.1). Therefore, we introduced a weighted rule-based feedback $L\left(m\right(\mathit{w},\mathit{R}\left)\right)$. The rules $\mathit{R}$ ensure that the correction factor $m$ is only calculated when a trustworthy calculation of $m_{\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}$ is to be expected. The individual rules can be weighted with a weighting factor $w\in \mathit{w}$ to control their influence on $m$.

Figure 2a shows the schematic structure of the proposed SoH estimator. The idea for this estimator structure branches from the observer structure known from control theory [42], which is shown for comparison in Figure 2b. An observer is used to estimate a real system’s states $x$ in cases where only certain states can be measured directly. This is carried out by calculating the error $e=y-\widehat{y}$ between the available measurement data $y$ and a model-based prediction of this measurement data $\widehat{y}$ using the same input $u$ for the real system as well as for the model. Then, a feedback $L\left(e\right)$ is to be found that corrects the model’s state estimates $\widehat{x}$ so that $e$ is minimized. For linear systems, it can be shown that this structure is able to correctly estimate the real system’s states under some secondary conditions towards the system’s structure. For nonlinear systems, as is the case with Li-ion batteries, this statement cannot be generalized. In this case, stability should be investigated throughout simulations and field tests. The implementation and results of such tests for the case at hand are described in Section 3. A more detailed introduction to the concept of observers can be found in [43].

2.3. Realization

A programmatic flow chart of the developed SoH estimator is depicted in Figure 3. As can be seen, first the SoH estimator is initialized. Then, a continuous loop is entered during which up to four tasks are executed: (1) weighting, (2) OCV gradient calculation, (3) averaging and (4) a conditional SoH update. If a SoH update is performed, the second and third task are reinitialized. A detailed description of the individual steps of each task is presented in the following subsections.

2.3.1. Weighting

To be able to calculate a trustworthy correction factor $m$ (Equation (4)), a set of rules is introduced, from which a weighting factor $w\left(k\right)$ is obtained. High values of $w\left(k\right)$ indicate a trustworthy calculation of $m$ at time step $k$; that is, it should be used to calculate a new SoH value. In contrast to that, low values of $w\left(k\right)$ indicate a less trustworthy correction factor $m$ at time step $k$; that is, the calculation of a new SoH value should be omitted. The applied set of rules is listed below:

$$\mathrm{Current} \mathrm{Limitation}: w_{\mathrm{i}}\left(k\right)=abs\left(i\left(k\right)\right)\le i_{\mathrm{l}\mathrm{i}\mathrm{m}}$$

(5a)

$$\mathrm{Current} \mathrm{History}: w_{\mathsf{\Delta }\mathrm{i}}\left(k\right)={\displaystyle \sum _{j=k-k_{t_{\mathsf{\Delta }\mathrm{i},\mathrm{l}\mathrm{i}\mathrm{m}}}}^k}abs\left(i\left(j\right)-i\left(j-1\right)\right)\le i_{\mathsf{\Delta }\mathrm{i},\mathrm{l}\mathrm{i}\mathrm{m}}$$

(5b)

$$\mathrm{Time\; since} \mathrm{last\; update}: w_{\mathsf{\Delta }t}\left(k\right)={\displaystyle \sum _{j=k_{t_{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}}}}^k}t\left(j\right)\le t_{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l},\mathrm{l}\mathrm{i}\mathrm{m}}$$

(5c)

$$\mathrm{Delta\; SoC}: w_{\mathsf{\Delta }SoC}\left(k\right)={\mathsf{\Delta }SoC}_{\mathrm{m}\mathrm{i}\mathrm{n},\mathrm{l}\mathrm{i}\mathrm{m}}\le abs\left(SoC\left(k_{t_{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}}}\right)-SoC\left(k\right)\right)\le {\mathsf{\Delta }SoC}_{\mathrm{m}\mathrm{a}\mathrm{x},\mathrm{l}\mathrm{i}\mathrm{m}}$$

(5d)

$$\mathrm{Temperature}: w_{\mathrm{T}}\left(k\right)=T_{\mathrm{m}\mathrm{i}\mathrm{n},\mathrm{l}\mathrm{i}\mathrm{m}}\le T\left(k\right)\le T_{\mathrm{m}\mathrm{a}\mathrm{x},\mathrm{l}\mathrm{i}\mathrm{m}}$$

(5e)

The limits $i_{\mathrm{l}\mathrm{i}\mathrm{m}}$, $i_{\mathsf{\Delta }\mathrm{i},\mathrm{l}\mathrm{i}\mathrm{m}}$, $t_{\mathsf{\Delta }\mathrm{i},\mathrm{l}\mathrm{i}\mathrm{m}}$, $t_{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l},\mathrm{l}\mathrm{i}\mathrm{m}}$, ${\mathsf{\Delta }SoC}_{\mathrm{m}\mathrm{i}\mathrm{n},\mathrm{l}\mathrm{i}\mathrm{m}}$, ${\mathsf{\Delta }SoC}_{\mathrm{m}\mathrm{a}\mathrm{x},\mathrm{l}\mathrm{i}\mathrm{m}}$, $T_{\mathrm{m}\mathrm{i}\mathrm{n},\mathrm{l}\mathrm{i}\mathrm{m}}$, $T_{\mathrm{m}\mathrm{a}\mathrm{x},\mathrm{l}\mathrm{i}\mathrm{m}}$ are design parameters of the algorithm and need to be set according to the battery and use case at hand (see also Section 3.4).

As can be seen from the defined rules, here only binary weighting factors are used. However, it would be straightforward to use more fuzzy weighting factors ranging from 0 to 1.

Finally, all weighting factors are combined in a logical “AND” manner to obtain an overall weighting factor:

$$w=w_{\mathrm{i}}·w_{\mathsf{\Delta }\mathrm{i}}·w_{\mathsf{\Delta }\mathrm{t}}·w_{\mathsf{\Delta }\mathrm{S}\mathrm{o}\mathrm{C}}·w_{\mathrm{T}}.$$

(6)

2.3.2. OCV Gradients

To calculate the gradients $m_{\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}$ and $m_{\mathrm{e}\mathrm{x}\mathrm{p}}$ (see also Equation (4)), a reference point

$$P_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{r}\mathrm{e}\mathrm{f}}\left(So{C}_{\mathrm{r}\mathrm{e}\mathrm{f}},{ u}_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{r}\mathrm{e}\mathrm{f}}, T\right)$$

(7)

on the OCV curve is needed. This point is calculated each time the battery is assumed to be in electro-chemical equilibrium, and thus, an accurate SoC $So{C}_{\mathrm{r}\mathrm{e}\mathrm{f}}$ can be determined by the voltage measurement $u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{r}\mathrm{e}\mathrm{f}}=u\left(k\right)$ and the OCV-SoC relationship via $So{C}_{\mathrm{r}\mathrm{e}\mathrm{f}}=OC{V}^{-1}\left(u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{r}\mathrm{e}\mathrm{f}}\right)$. Simplified, an electro-chemical equilibrium is assumed, if the battery was without current for more than 15 min. The OCV-SoC relationship is, for example, known from the battery’s datasheet.

Next, in each time step $k$, the expected OCV $u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{e}\mathrm{x}\mathrm{p}}\left(k\right)$ and the calculated pseudo-OCV $u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}$(k) are determined with the help of the available voltage measurement $u\left(k\right)$, current measurement $i\left(k\right)$, temperature measurement $T\left(k\right)$ and SoC estimate $SoC\left(k\right)$. The expected OCV voltage $u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{e}\mathrm{x}\mathrm{p}}\left(k\right)$ is taken from the OCV-SoC relationship:

$$u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{e}\mathrm{x}\mathrm{p}}\left(k\right)=OCV\left(Soc\left(k\right)\right).$$

(8)

The pseudo-OCV $u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}$(k) is calculated assuming the equivalent circuit model depicted in Figure 4. It consists of a voltage source representing the pseudo-OCV $u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}$ and an internal series resistance $R_i(SoC,T)$ accounting for the dynamic voltage behavior of the battery when $i\ne 0$. The internal resistance $R_i(SoC,T)$ must be identified beforehand for the battery system used [39]. $u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}\left(k\right)$ is then calculated as follows:

$$u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}\left(k\right)=u\left(k\right)-i\left(k\right)·R_i(SoC,T).$$

(9)

In ideal conditions, the expected voltage $u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{e}\mathrm{x}\mathrm{p}}\left(k\right)$ and the calculated voltage $u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}\left(k\right)$ match. The more ideal the boundary conditions, such as the actual temperature, the current rate, and the time since the last recalibration, the better the calculated and expected value will match.

Finally, the gradients $m_{\mathrm{e}\mathrm{x}\mathrm{p}}\left(k\right)$, $m_{\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}\left(k\right)$ and the correction factor $m\left(k\right)$ at time step $k$ can be calculated according to

$$m_{\mathrm{e}\mathrm{x}\mathrm{p}}\left(k\right)=\frac{u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{e}\mathrm{x}\mathrm{p}}\left(k\right)-u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{r}\mathrm{e}\mathrm{f}}}{Soc\left(k\right)-So{C}_{\mathrm{r}\mathrm{e}\mathrm{f}}},$$

(10)

$$m_{\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}\left(k\right)=\frac{u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}\left(k\right)-u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{r}\mathrm{e}\mathrm{f}}}{Soc\left(k\right)-So{C}_{\mathrm{r}\mathrm{e}\mathrm{f}}},$$

(11)

$$m\left(k\right)=\frac{m_{\mathrm{e}\mathrm{x}\mathrm{p}}\left(k\right)}{m_{\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}\left(k\right)}=\frac{u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{e}\mathrm{x}\mathrm{p}}\left(k\right)-u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{r}\mathrm{e}\mathrm{f}}}{u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}\left(k\right)-u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{r}\mathrm{e}\mathrm{f}}}.$$

(12)

For a better understanding of these formulas, Figure 5 gives a graphical overview of the quantities involved in the calculations.

2.3.3. Mean Correction Factor

A SoH update is not performed in every time step. It is only performed when the conditions to set a new reference point $P_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{r}\mathrm{e}\mathrm{f}}$ (Equation (7)) are met. This allows, together with the weighting factor $w$ from Equation (6), the calculation of a mean weighted correction factor $m_{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}}$ from the time step $k_{\mathrm{r}\mathrm{e}\mathrm{f}}$, in which the last reference point $P_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{r}\mathrm{e}\mathrm{f}}\left(k_{\mathrm{r}\mathrm{e}\mathrm{f}}\right)$ was taken, to the current time step $k$:

$$m_{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}}\left(k\right)=\frac{{\sum }_{j=k_{\mathrm{r}\mathrm{e}\mathrm{f}}}^{k}m\left(j\right)·w\left(j\right)}{{\sum }_{j=k_{\mathrm{r}\mathrm{e}\mathrm{f}}}^{k}w\left(j\right)}.$$

(13)

Due to this weighting and averaging, $m_{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}}$ is considered to be trustworthy enough to be considered in the estimation of the battery’s SoH in the next step.

2.3.4. Conditional SoH Update

Every time the conditions to set a new reference point $P_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{r}\mathrm{e}\mathrm{f}}$ (Equation (7)) are met, an updated SoH value according to

$$So{H}_{\mathrm{C},\mathrm{u}\mathrm{p}\mathrm{d}}=So{H}_{\mathrm{C},\mathrm{o}\mathrm{l}\mathrm{d}}·\max \left(\min \left(m_{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}},{\gamma }_2\right),{\gamma }_1\right)$$

(14)

is calculated, the new $P_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{r}\mathrm{e}\mathrm{f}}$ is set, and the mean correction factor is reset to $m_{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}}=1$. In this update step, two limits ${\gamma }_1$ and ${\gamma }_2$ were introduced to be able to control the minimal and maximal rate of change of $So{H}_{\mathrm{C},\mathrm{u}\mathrm{p}\mathrm{d}}$ in one step. Additionally, the updated SoH value is also smoothed by a discrete low-pass filter of first order [44] according to

$$So{H}_{\mathrm{C},\mathrm{n}\mathrm{e}\mathrm{w}}=So{H}_{\mathrm{C},\mathrm{o}\mathrm{l}\mathrm{d}}+\alpha ·\left(So{H}_{\mathrm{C},\mathrm{u}\mathrm{p}\mathrm{d}}-So{H}_{\mathrm{C},\mathrm{o}\mathrm{l}\mathrm{d}}\right) ,$$

(15)

yielding the new SoH value $So{H}_{\mathrm{C},\mathrm{n}\mathrm{e}\mathrm{w}}$, which is used in the BMS moving forward.

Limiting (Equation (14)) and smoothing (Equation (15)) are performed to be able to control the dynamic behavior of the SoH estimator, as will be described in Section 3.4.

3. Experimental Evaluation

The presented SoH estimator is parameterized and evaluated on pack level with real measurement data which have been collected for more than two years on an operational bus fleet. The recorded average cell voltage, average cell current and average cell temperature are fed into a MATLAB/Simulink model in which the estimator runs. Additionally, for one battery pack the total capacity was measured in June 2016 as well as in February 2019. These checkup measurements serve as ground-truth SoH checkup values to which the estimated SoH can be compared at these two points in time. In the following, the test battery system, the data acquisition strategy, the checkup measurement procedure and the methodology to parameterize the estimator are described.

3.1. Battery System under Test

The examined battery packs consist of 180 lithium-ion high-power pouch cells with a cobalt manganese nickel (NMC) oxide cathode and a graphite anode. The cells are organized in a 90-series 2-parallel (90s2p) topology.

Table 1. Battery pack specifications.

Specification
Cell chemistry	NMC—Graphite
Cell design	pouch
Pack configuration	90s2p
Nominal capacity	92 Ah
Energy	30.6 kWh
Nominal voltage	333 V
Voltage (min)	243 V
Voltage (max)	378 V
Discharging power max (10 s)	266 kW
Charging power max (10 s)	153 kW
Continuous power (RMS)	77 kW
Charge/discharge current max (10 s)	736 A
Continuous current (RMS)	230 A

lists the battery pack performance specifications in detail. In the application considered here, two to four of these battery packs were combined into one battery system, with corresponding capacities ranging from 92 Ah to 184 Ah.

3.2. Real Usage Data

The data used to evaluate the SoH estimator have been collected for more than two years on a bus fleet equipped with battery packs of the type described before. The buses and battery systems are exposed to a mild central European climate in the middle of Germany and are used in the urban transport sector.

Table 2. Overview of the general operation parameters of the vehicle with which the checkup measurement was performed (see also Section 3.3).

Parameter	Value
Time period	1 December 2016 to 22 January 2019
Operational days/Overall days	712/783
Mean daily operation time in hours	13.8
Mean cell temperature in °C	27
Mean SoC in %	83

gives an overview of the general operation parameters of the vehicle with which the checkup measurement was performed (see also Section 3.3).

To be able to record the data, the buses were equipped with data loggers, each of which is connected to the internal CAN bus of the battery system. To save storage space and computing power, the data logger only stores a value for each signal every 6 s without any filtering, even though the internal sampling rate of the BMS systems is much higher. Therefore, the results obtained from the estimator fed with this data in the simulations may vary from the results which would have been gained if the estimator had been run directly on the BMS.

3.3. Checkup Measurements

The capacity of the battery systems considered was measured before delivery to the customer. Additionally, for one battery system, a full-capacity measurement was performed after roughly two years of operation in February 2019. These checkup measurements allow the evaluation of the performance of the SoH estimator at certain points in time. The test bench used to perform the capacity measurements consists of a power supply, two relays and a shunt resistor for current measurements. There is one relay for each battery pole. The relays are controlled by the battery’s BMS. The test bench characteristics are listed in

Table 3. Characteristics of the test bench used to perform the capacity measurements [45].

Test Device	Nominal Power	Measurement	Full Scale	Accuracy
Gustav Klein Type 3865	100 kW	Voltage	Up to 1000 V DC	0.5% fs ± 1 Digit/12 Bit
		Current	Up to 600 A	0.5% fs ± 1 Digit/12 Bit

.

The literature defines numerous ways to determine the correct capacity of a battery cell or a whole battery system [22,46]. The method used here is oriented to DIN EN 62620 and ISO 12405 as well as the cell manufacturer’s procedure. At room temperature, the system is charged with a constant current (CC) of 0.4 C up to a max voltage of 4.2 V (on cell level) and a subsequent constant voltage (CV) phase until the current decreases below a threshold of 1/20 C. Afterwards, the system is fully discharged with a constant current (CC) of 0.4 C to the lower cutoff voltage of 2.7 V (on cell level) and then charged again as described previously. The capacity of the battery is then rated by the discharged capacity.

3.4. Parametrization

A total of eleven parameters must be identified before the estimator can be run, eight to parametrize the weighting rules (Equations (5a–e)) and three to set the estimator’s dynamic behavior (Equations (14) and (15)). Next, it is described how these parameters were chosen.

3.4.1. Weighting-Rules Parameters

The weighting-rules parameters used in Equations (5a–e) determine the operating conditions when a trustworthy OCV calculation which can be used in the further course of the SoH estimation process can be awaited. The error between the expected OCV voltage $u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{e}\mathrm{x}\mathrm{p}}$ (Equation (8)) and the calculated OCV voltage $u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}$ (Equation (9))

$$e_{\mathrm{O}\mathrm{C}\mathrm{V}}=u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{e}\mathrm{x}\mathrm{p}}-u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}$$

(16)

is assumed to be only due to aging effects for the data selected by the rules (Equations (5a–e)).

We formulated the rules in a way which should make it easy for engineers to define the weighting-rules parameters based on their system know-how and best practices. But, if available, simulations or real data should be used to validate and refine the parameters selected. Here, we have a large amount of real data available (see also Section 3.2) which allowed us to choose the parameters based on statistical analysis.

First, we look at the root mean square error (RMSE) of $e_{\mathrm{O}\mathrm{C}\mathrm{V}}$ (Equation (16)) to identify a parameter range which yields possibly a small RMSE. Therefore, we set a fixed window width for the respective parameter, slide this window over the relevant parameter range and calculate the RMSE of $e_{\mathrm{O}\mathrm{C}\mathrm{V}}$ (Equation (16)) for each parameter window’s position. The result is a bar plot where each bar represents one parameter window and its related RMSE value. Figure 6 shows such a plot with a window size of 1% for the $\mathsf{\Delta }SoC$ range since the last recalibration (Equation (5d)), which is defined by the parameters ${\mathsf{\Delta }SoC}_{\mathrm{m}\mathrm{i}\mathrm{n},\mathrm{l}\mathrm{i}\mathrm{m}}$ and ${\mathsf{\Delta }SoC}_{\mathrm{m}\mathrm{a}\mathrm{x},\mathrm{l}\mathrm{i}\mathrm{m}}$.

It can be seen that the RMSE is fairly constant over a wide range from $\mathsf{\Delta }SoC=1\%$ to $\mathsf{\Delta }SoC=80\%$. Only for $\mathsf{\Delta }SoC>80\%$, the RMSE increases significantly. It can be assumed that this is due to inaccuracies in the current measurement which accumulate over time in the $SoC$ calculation. Looking at this result, it would be plausible to choose ${\mathsf{\Delta }SoC}_{\mathrm{m}\mathrm{i}\mathrm{n},\mathrm{l}\mathrm{i}\mathrm{m}}$ = 1% and ${\mathsf{\Delta }SoC}_{\mathrm{m}\mathrm{a}\mathrm{x},\mathrm{l}\mathrm{i}\mathrm{m}}=80\%$. But we also evaluated a second measure to refine this choice.

Second, we looked at the distribution of the data selected by the respective parameter using the same window size as before. Figure 7 shows a sample of the distribution of the $\mathsf{\Delta }SoC$ ranges since the last recalibration (Equation (5d)) with a window size of 1% on the available data.

It can be seen that the highest $\mathsf{\Delta }SoC$ occurred around $\mathsf{\Delta }SoC=9\%$ in this application, neglecting the peak at $\mathsf{\Delta }SoC=1\%$ which stems from small $SoC$ changes while the battery’s SoC is recalibrated. $\mathsf{\Delta }SoC>50\%$ are almost non-existent in the data set.

Combing both measurements, we chose ${\mathsf{\Delta }SoC}_{\mathrm{m}\mathrm{i}\mathrm{n},\mathrm{l}\mathrm{i}\mathrm{m}}=6\%$ and ${\mathsf{\Delta }SoC}_{\mathrm{m}\mathrm{a}\mathrm{x},\mathrm{l}\mathrm{i}\mathrm{m}}$ = 31%. By that, we assume we have enough data passing the delta SoC rule (Equation (5d)) and at the same time having an acceptable RMSE of $e_{\mathrm{O}\mathrm{C}\mathrm{V}}$ (Equation (16)). In this way, we proceeded with all the other parameters, coming up with the final choices listed in

Table 4. Final choice of the weighting-rules parameters (see also Section 2.3.1).

Parameter	Value
$i_{\mathrm{l}\mathrm{i}\mathrm{m}}$	12 A
$i_{\mathsf{\Delta }\mathrm{i},\mathrm{l}\mathrm{i}\mathrm{m}}/t_{\mathsf{\Delta }\mathrm{i},\mathrm{l}\mathrm{i}\mathrm{m}}$	15 A/30 s
$T_{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l},\mathrm{l}\mathrm{i}\mathrm{m}}$	3.5 h
${\mathsf{\Delta }SoC}_{\mathrm{m}\mathrm{i}\mathrm{n},\mathrm{l}\mathrm{i}\mathrm{m}}/{\mathsf{\Delta }SoC}_{\mathrm{m}\mathrm{a}\mathrm{x},\mathrm{l}\mathrm{i}\mathrm{m}}$	6%/31%
$T_{\mathrm{m}\mathrm{i}\mathrm{n},\mathrm{l}\mathrm{i}\mathrm{m}}/T_{\mathrm{m}\mathrm{a}\mathrm{x},\mathrm{l}\mathrm{i}\mathrm{m}}$	23 °C/27 °C

.

The overall parameter choice can be validated using the available field data.

Table 5. Proportion of selected data points per selection rule (see also Equations (5a–e)).

Rule	Proportion in %	RMSE(${\mathit{e}}_{\mathbf{O}\mathbf{C}\mathbf{V}}$)
Current Limitation	38.69	0.0211
Current History	35.27	0.0283
Time since last update	44.08	0.0227
Delta SoC	66.26	0.0276
Temperature	30.15	0.0251
All	0.78%	0.0155

lists the percentage of the data selected by each rule and the respective RMSE of $e_{\mathrm{O}\mathrm{C}\mathrm{V}}$ (Equation (16)) together with the overall percentage and RMSE when applying all rules together according to Equation (6).

It can be seen that only 0.78% of the data are finally processed to calculate the mean correction factor $m_{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}}$ (Equation (13)). However, battery degradation is a slow process; therefore, there should still be sufficient data to gradually estimate the SoH value.

Finally, the selected parameters can be evaluated visually. Figure 8 illustrates the point cloud of all the available data before and after applying the rules (Equations (5a–e)). It can be seen that the finally selected data points (marked in red) are in good agreement with the OCV curve. Thus, it can be considered that the applied rules work in the way they were intended. Ideally, the remaining deviations from the OCV curve would be due to aging effects.

3.4.2. Dynamic Parameters

A crucial part of the estimator is the feedback of the updated SoH $So{H}_{\mathrm{C},\mathrm{n}\mathrm{e}\mathrm{w}}$ (Equation (15)) to gradually adjust $u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{e}\mathrm{x}\mathrm{p}}$ to match $u_{\mathrm{O}\mathrm{C}\mathrm{V},\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}$, as depicted earlier in Figure 2a. In general, this feedback can cause the estimator to oscillate or even become unstable. Therefore, we introduced three parameters (Equations (14) and (15)) to be able to control the dynamic behavior of the estimator. The parameters ${\gamma }_1$ and ${\gamma }_2$ (Equation (14)) limit the rate of change of $So{H}_{\mathrm{C},\mathrm{u}\mathrm{p}\mathrm{d}}$ (Equation (14)) and the parameter $\alpha$ (Equation (15)) controls the damping of the feedback. The parameters can be identified best via simulations in which the dynamic behavior of the estimator is systematically excited. Here, we used the available real data (see also Section 3.2) to run multiple simulations with different parameter settings. To especially excite the dynamic behavior of the estimator during these simulations, we intentionally set an initial SoH offset and could find suitable values for ${\gamma }_1$, ${\gamma }_2$ and $\alpha$.

First, we varied the rate limits $\gamma$ while keeping the gain $\alpha$ constant. The results can be seen in Figure 9 for a negative initial deviation (Figure 9a) and a positive initial deviation (Figure 9b). For negative initial deviations, it can be seen that wider limits result in a steeper initial slope. Besides this, only a small variation can be seen after the initialization phase in which the estimator compensates for the initial SoH offset. In contrast, for positive initial deviations, only the latter can be observed. The initial slope only shows small variations for different values of $\gamma$. Additionally, in comparison to the slopes observed for negative initial deviations, the slope for positive initial deviations seems to be generally lower. This possibly stems from asymmetrical sensitivities of the correction factor $m$ (Equation (12)) to changes in $m_{\mathrm{e}\mathrm{x}\mathrm{p}}$ (Equation (10)) and $m_{\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{c}}$ (Equation (11)), which lead the correction factor to slightly favor higher values, in turn leading to a possible overestimation of the true SoH value. This could be compensated by using asymmetrical limits in $\gamma$ favoring smaller correction factors.

Second, we varied the gain $\alpha$ while leaving $\gamma$ constant. The results can be seen in Figure 10 for a negative initial deviation. An increasingly oscillating behavior of the estimator can be observed with increasing gain $\alpha$.

In general, we favor the SoH estimator to have a high damping to avoid overshooting or oscillating behavior as much as possible. In terms of safety, it is also better to overestimate the aging of the battery, meaning estimating a lower SoH value than the actual one. Therefore, we chose the smallest parameter values in each case, as can be seen in

Table 6. Final choice of the dynamic parameters (see also Section 2.3.4).

Parameter	Value
${\gamma }_1/{\gamma }_2$	0.90/1.05
$\alpha$	0.01

with asymmetrical rate limits $\gamma$, for the reason described before.

4. Result and Discussion

After all parameters have been chosen, the resulting SoH estimation performance on pack level is evaluated and discussed in the following two subsections.

4.1. Result

Figure 11 shows the simulatively determined SoH course of the presented estimator (red solid line) based on the available field data (see also Section 3). For comparison, a lookup table (LUT)-based approach similar to the one from Huynh [47] is shown (blue dashed line). This approach is chosen for comparison, as it is another SoH estimation method which can also be implemented directly in BMS because of its low computational complexity. Additionally, both available checkup measurement results are shown: $SoH\approx 101.1$%, measured at the production end of line test in June 2016, and $SoH\approx 95.9$%, measured in February 2019.

Both algorithms started with an initial SoH of 99.3%, which is the value determined by the LUT-based approach at the beginning of the operation. It results from the calendric aging that occurred within the six months that elapsed between production and operation of the pack. Cyclic aging of the pack is not present during this period.

The SoH of the LUT-based approach continuously decreases at an almost constant rate. This is not surprising, since an increase in SoH is not possible with this approach. In contrast to that, the value estimated by the observer increases within the first four months to a value of 101.3% before it then decreases at an approximately constant rate. We consider this initial behavior with an increasing SoH to be plausible, since the first checkup measurement at the end of production showed a SoH of 101.1%. The value is over 100% because the battery pack’s capacity at the start of life is determined from the cell’s datasheet. However, more charge could be drawn from the battery pack during the first checkup measurement. This explains the SoH value of over 100%. The SoH observer thus goes through a transient phase within the first four months from an initial value that is lower than the true SoH, a value of 101.3% that is most likely closer to the real value, as it is in the range of the first checkup measurement.

From April 2017 on, both the curve of the estimator and the curve of the LUT-based approach decrease almost in parallel, where the estimator’s values are about 1.9% higher than the LUT-based values. Additionally, the estimated SoH almost always remains in the envelope defined by the two checkup measurements depicted by the dashed horizontal lines. Therefore, the transient behavior and the subsequent constant progression of the estimated SoH seem to be plausible, since the battery was delivered with a SoH of more than 100% and is operated with a recurring daily duty cycle from which continuous aging can be expected. In contrast, the LUT-based approach cannot compensate for the initial deviation, as an increase in SoH is not modeled and therefore yields a too-low SoH value.

However, the estimated SoH is not strictly monotonically decreasing after April 2017. A “ditch” in the curve can be observed shortly before the second checkup measurement in February 2019. It cannot be said whether this behavior reflects the true SoH development or whether it is a dynamic phenomenon of the estimator. But, at the time of the second checkup measurement, the estimator meets exactly the measured value of 95.9%, whereas the LUT-based approach reports a lower value of 94.2%. This clearly shows that the new estimator approach outperforms the LUT-based approach, which is not able to compensate for initial deviations or individual aging mechanisms occurring depending on the respective application.

4.2. Discussion

4.2.1. Dynamic Behavior

• The estimator is capable of compensating for initial SoH deviations. With the chosen parameters, the initial deviation of approximately 2% is compensated for in about four months. This might seem slow, but compared to the expected lifetime of the battery used it is rather a short period. The dynamic parameters were intentionally chosen to be conservative to avoid overshooting and to obtain a smooth curve. In practice, unknown initial deviations significantly larger than 2% are not to be expected.
• The recorded data used for the evaluation have a sample time of 6 s. Typically, BMS tasks have a cycle time in the range of several milliseconds. Therefore, we expect to see a different dynamic behavior when the observer runs directly on a BMS, as the observer will be acting on more dynamic data because of the higher sample rate. This will primarily affect the data being selected by the weighting rules (Equations (5a) and (5b)), which in turn results in a different calculation of the mean correction factor m_mean (Equation (13)). Therefore, it is expected that it is necessary to adapt the dynamic parameters of the algorithm when the observer is run directly on a BMS.
• The accuracy and dynamic behavior of the estimator needs to be better verified. The presented results are initial validation results that give an impression of the performance and behavior of the estimator. The two available checkup measurements are not enough to evaluate the accuracy of the estimator in general. It was not possible to carry out more checkup measurements because the bus with the battery pack under consideration was in operational use most of the time and the workshop routine for carrying out the checkup measurement (see Section 3.3) is time-consuming. Therefore, the fleet operator did not agree to any further checkup measurements. But a new field study has already been started to validate the estimator, integrated in a BMS, with more frequent checkup measurements. The results will be published in future work.
• It must be noted that the observed period of about two years is short in comparison to the expected lifetime of the battery examined. Therefore, the dynamic behavior in the future cannot fully be inferred from the assessed data. It is planned to further monitor the system and to perform additional checkup measurements to obtain more confidence in the dynamic interpretation.

4.2.2. Parametrization

• The parameterization demonstrated in Section 3.4 is based on statistical analysis of the available real operating data. In cases where this is not possible, it might be an option to use synthetic data from simulations to find a suitable parameter set for the respective system. Ultimately, the option remains to set the parameters based on know-how about the system used, as the rule-based design of the estimator gives an intuitive way to understand the influence of each parameter.

4.2.3. Alternative Methods and Possible Extensions

• A common approach to SoH estimation is to use a Kalman filter (KF) [48]. The difference to the approach chosen in this work lies in the calculation of the feedback term L (see also Figure 2). In the Kalman filter, the feedback is computed recursively by a computational rule which results from minimizing the mean square estimation error while considering uncertainties in the system model as well as in the measurements. It can be proved that the KF gives the optimal minimum mean square error estimate under some specific conditions. While in practice these conditions are often violated, the KF still results in acceptable performance in most cases. Thus, the KF is widely used in practice. The “magic” to acquire an acceptable dynamic behavior lies mainly in the parameterization of the system noise covariance matrix. However, there is no universal approach to this, leaving engineers with a lengthy trial-and-error process. In contrast, our motivation was to develop feedback that could be parametrized in an intuitive way, since the meaning of each parameter is directly interpretable.
• Currently, the internal resistance R_i in Equation (9) is considered to be constant over the battery’s lifetime, which is not true, as the resistance increases significantly as the battery ages. Therefore, the estimate of SoH_c unintentionally also compensates for aging effects originating from the resistance increase. The solution would be to implement a separate estimator for the aging effects in R_i and to use the estimate in Equation (9). For example, the structure shown here could also be applied to implement such an R_i estimator.
• Another possible extension to the estimator would be to add an additional model which accounts for calendric aging during long standstill phases where the estimator is not updated. However, in the commercial vehicle applications the standstill phases are significantly lower in comparison to passenger car applications, so that the estimator would be most probably able to self-correct for these deviations during the runtime after standstill phases.

5. Conclusions

In this paper, an on-board applicable estimator for capacity SoH estimation for NMC lithium-ion batteries which exploits changes in the OCV curve that occur as the battery ages is presented. Its structure is motivated by the observer concept from control theory. However, deviating from the standard structure, a rule-based feedback is introduced. This allows for an intuitive way to parameterize the estimator. Additionally, the utilized observer structure is computationally beneficial, which favors its use on BMS hardware.

The estimator is evaluated in MATLAB/Simulink with the help of real data collected over more than two years on an electrified bus fleet. Additionally, two checkup measurements were performed for one of the used battery packs: one directly after the production of the battery pack and one after roughly two years of operation. By that, at the time of the checkup measurements, the battery’s real capacity could be determined, which serves as ground-truth reference points helping to evaluate the estimator’s behavior.

By running the estimator in MATLAB/Simulink using the recorded data, its basic functionality could be demonstrated. The estimated SoH at the time of the second checkup measurement deviates less than ±0.5% from the measured SoH. Furthermore, it is demonstrated that the estimator is capable of correcting for initialization offsets. The next step we want to address in the future is to evaluate the estimator’s behavior when it is running directly on a BMS.