Next Article in Journal
Field Test Method and Equivalence Analysis of Delay Characteristics of DC Electronic Current Transformer
Previous Article in Journal
Direct Tilt Controller Design with Disturbance Compensation and Implementation for a Narrow Tilting Electric Vehicle
Previous Article in Special Issue
An Overview of Methods and Technologies for Estimating Battery State of Charge in Electric Vehicles
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Predicting Li-Ion Battery Remaining Useful Life: An XDFM-Driven Approach with Explainable AI

1
Department of Mechanical Engineering, School of Technology, PDEU Gandhinagar, Gandhinagar 382426, India
2
Mechanical Engineering Department, Medi-Caps University, Indore 453331, India
3
Department of Operations Management and Quantitative Techniques, Indian Institute of Management, Bodh Gaya 824234, India
*
Author to whom correspondence should be addressed.
Energies 2023, 16(15), 5725; https://doi.org/10.3390/en16155725
Submission received: 12 June 2023 / Revised: 21 July 2023 / Accepted: 24 July 2023 / Published: 31 July 2023
(This article belongs to the Special Issue Machine Learning Applied in Energy Storage Systems)

Abstract

:
The accurate prediction of the remaining useful life (RUL) of Li-ion batteries holds significant importance in the field of predictive maintenance, as it ensures the reliability and long-term viability of these batteries. In this study, we undertake a comprehensive analysis and comparison of three distinct machine learning models—XDFM, A-LSTM, and GBM—with the objective of assessing their predictive capabilities for RUL estimation. The performance evaluation of these models involves the utilization of root-mean-square error and mean absolute error metrics, which are derived after the training and testing stages of the models. Additionally, we employ the Shapley-based Explainable AI technique to identify and select the most relevant features for the prediction task. Among the evaluated models, XDFM consistently demonstrates superior performance, consistently achieving the lowest RMSE and MAE values across different operational cycles and feature selections. However, it is worth noting that both the A-LSTM and GBM models exhibit competitive results, showcasing their potential for accurate RUL prediction of Li-ion batteries. The findings of this study offer valuable insights into the efficacy of these machine learning models, highlighting their capacity to make precise RUL predictions across diverse operational cycles for batteries.

1. Introduction

Lithium-ion (Li-ion) batteries have received extensive recognition in recent times and have emerged as a crucial energy source for a diverse range of applications, such as portable electronic devices, electric vehicles (EVs), hybrid electric vehicles (HEVs), and renewable energy storage systems [1,2]. The exceptional energy density, prolonged cycle life, and swift charging capabilities of the batteries quickly made them the favored choice, leading to a significant transformation in energy storage and utilization practices across various industries. The adoption of Li-ion batteries is known to promote ecological sustainability by mitigating the release of greenhouse gases, air pollutants, and other environmental impacts that are typically associated with traditional energy sources [3]. Various chemistries of Li-ion batteries, including LiCoO2, LiFePO4, and LiNiMnCoO2, are currently accessible [4,5,6]. Out of the available alternatives, LiFePO4 is distinguished as the most environmentally friendly option owing to its non-hazardous structure and the plentiful occurrence of its component elements. In addition, the process of recycling batteries plays a pivotal role in mitigating waste generation and recovering valuable resources, thereby supplementing the overall ecological sustainability of the technology. The assurance of the dependability, security, and optimal functionality of the batteries has gained considerable importance due to their widespread integration in diverse domains. Several methodologies have been devised to monitor Li-ion batteries, with a particular emphasis on evaluating the State of Health (SOH) and State of Charge (SOC) [7,8]. By analyzing variables, like capacity fading, impedance changes, and aging effects, SOH monitoring approaches aim to assess the overall health and deterioration of Li-ion batteries over time and to predict the remaining useful life (RUL) [9,10]. Capacity estimate and impedance spectroscopy are standard approaches for SOH monitoring, providing useful insights into battery degradation patterns and the need for maintenance or replacement. Conversely, techniques for monitoring SOC concentrate on the estimation of the present energy level or residual charge within a battery. The estimation of SOC is accurately achieved through the utilization of voltage-based methods, current integration, coulomb counting, and advanced algorithms, as indicated in sources [11,12]. Accurately evaluating the RUL of Li-ion batteries is a crucial component of battery monitoring, as well as the number of charge and discharge cycles left before the battery fails, as assessed by SOH [13]. The estimation of the RUL of batteries is a crucial aspect that offers significant benefits, such as facilitating preventive maintenance, optimizing resource utilization, and mitigating the likelihood of catastrophic failures. The continuous monitoring of the RUL of batteries enables individuals to make well-informed decisions about battery replacement or maintenance. This, in turn, enhances operational efficiency and reduces downtime. It is noteworthy that numerous monitoring techniques currently in use utilize simplified models and assumptions that may not comprehensively encompass the intricacies of battery behavior. The aforementioned methodologies may exhibit the susceptibility to external factors, computational complexities, and difficulties in accurately forecasting the RUL or SOC. Hence, it is imperative to devise innovative methodologies that can efficiently overcome these constraints and facilitate accurate and robust surveillance of Li-ion batteries. While classical methods, such as Tremblay [14] and Volterra’s integral equations [15], have been successfully applied in various fields, including power systems [16], they may have limitations in capturing complex nonlinear relationships and dealing with high-dimensional data. In particular, they may require a large number of parameters to accurately describe the behavior of a system, which can lead to overfitting and poor generalization performance. Additionally, these equations may not be able to capture the full complexity of a system, especially if the system exhibits highly nonlinear or chaotic behavior [17]. In comparison, machine learning (ML) techniques can be more flexible and adaptable to different types of data and can potentially provide more accurate predictions or classifications in certain cases. Both methods need data to conduct training and prediction, but models based on physics, like those of Tremblay and Volterra, frequently need larger datasets that include precise electrochemical readings and operating characteristics. Further, external elements that may affect the battery’s RUL but are not explicitly included in the integral equations, such as temperature, cycling profiles, and usage patterns, can be more simply incorporated into ML models. The aging process and total RUL of the battery can be considerably impacted by these outside variables.
The potential of ML techniques in mitigating the limitations of conventional Li-ion battery monitoring methods has been demonstrated recently by various authors [18,19]. ML algorithms possess the capability to analyze intricate and multi-dimensional data obtained after conducting experiments, and subsequently identify significant patterns and correlations. Through the utilization of ML models, intricate relationships between input parameters and the output can be discovered by creating models based on historical data, which enables the capability of accurate predictions and real-time monitoring [20,21]. Nuhic et al. [22] gathered multiple variables across various aging scenarios to assess the state of health (SOH) of batteries. In a related study, the authors presented a feed-forward neural network model in [23] for the purpose of predicting the RUL of Li-ion batteries. The accuracy of the proposed approach was validated through experiments and numerical comparisons. Zhao et al. [24] have proposed a hybrid model that combines the broad learning system (BLS) algorithm with the LSTM neural network. This model has been designed to provide precise predictions regarding the capacity and remaining useful life (RUL) estimation of batteries. However, one key disadvantage of traditional ML models is their lack of interpretability and transparency, which prevents them from being widely adopted in crucial applications. The emergence of Explainable AI (XAI) has been identified as a critical research direction to overcome the aforementioned limitations. The objective of XAI techniques is to augment the interpretability and transparency of ML models, enabling users to understand and trust the decisions made by AI systems. The integration of XAI techniques in Li-ion battery monitoring can facilitate the creation of models that not only yield precise RUL estimations, but also furnish valuable information regarding the underlying factors that positively impact the forecasts and predictions. The ability to interpret data not only enhances the quality of decision-making, but also enables the detection of potential biases, errors, or anomalies in the process of monitoring batteries.
This study introduces a novel methodology for precise and comprehensible prediction of the RUL of Li-ion batteries by utilizing an XAI-assisted XDFM model. The proposed model integrates deep learning and feature interactions, leveraging Shapley techniques to effectively identify the most suitable set of features for accurate RUL estimation at different operational stages. The incorporation of XAI into Li-ion battery monitoring represents a significant advancement towards establishing dependable and reliable battery management systems. The study aims to contribute to the broader objective of achieving sustainable and efficient energy solutions by optimizing the utilization and lifespan of Li-ion batteries. The organization of the proposed methodology is as follows: Section 2 provides a comprehensive overview of the dataset used and machine learning algorithms employed along with the XAI method for feature selection. Section 3 presents the analysis results and key findings, and finally, in Section 4, we summarize the significant outcomes and highlight future extensions and areas for further research. The proposed framework for predicting the RUL of Li-ion batteries is depicted in Figure 1.

2. Materials and Methods

2.1. Machine Learning Models

A branch of AI known as ML allows computers to learn and anticipate data without the need for specific programming. Automated systems can learn patterns and correlations from data via computational techniques [25,26]. After training on huge datasets, ML models improve as they evaluate more data. Because of ML’s adaptability, battery monitoring is a feasible application. Battery performance and health are critical for proper device and system operation. Basic battery monitoring methods or static threshold settings may not accurately capture the complex dynamics and unpredictability of battery performance. Machine learning has the potential to improve battery monitoring by accurately predicting residual capacity, health status, and possible malfunctions. In this study, XDFM, A-LSTM, and Gradient Boosting Machine are utilized for predicting Li-ion battery RUL.

2.1.1. Extreme Deep Factorization Machine

The Extreme Deep Factorization Machine (XDFM) is a robust computational technique that integrates the advantages of deep learning and factorization machines to achieve precise and effective predictions. Factorization machines, which are frequently employed to manage sparse, high-dimensional data, are at the core of the XDFM architecture. XDFM demonstrates that deep learning factorization machines can capture subtle feature interactions and derive useful data representations. DNNs are integrated into the factorization machine framework by XDFM [27,28]. By layering several hidden layers, the model captures higher-order feature interactions and produces complicated representations from input data. Mathematically, the XDFM model can be expressed as:
y ^ = σ w linear T a + w d n n T x d n n k + w c i n T p + + b
where σ represents the sigmoid function; a represents raw features; and w , b represent learnable parameters. During the training phase, the model optimizes the model parameters by minimizing a suitable loss function, which is expressed as:
L = 1 N i = 1 N   y i l o g y ^ i + 1 y ^ i l o g 1 y ^ i
N denotes training data, and y ^ i represents the model in various iterations during training. The objective function is defined as follows:
J = L + λ θ ˙
λ denotes the regularization term, while θ refers to the set of parameters in the CIN and DNN components.

2.1.2. Long Short-Term Memory Network with Attention Mechanism

Recurrent neural networks (RNNs) with Long Short-Term Memory (LSTM) and an attention mechanism can capture complex dependencies in sequential input. An attention mechanism helps the LSTM model focus on relevant input sequence components, improving prediction accuracy [29]. The LSTM model’s attention mechanism allows each time step to focus on different input sequence segments. This helps the model prioritize relevant data. The system can accurately predict language translation, emotional analysis, and voice identification by recognizing distant relationships. Mathematically, the LSTM with attention mechanism can be represented as follows:
h t = LSTM x t , h t 1 , c t 1
e t j = v a T tanh W a h t 1 + U a x j
α t j = exp e t j k = 1 T   exp e t k
c t = j = 1 T   α t j x j
y t = σ W o h t + U o c t
In these equations:
x t represents the input at time step t .
h t denotes the hidden state at time step t .
c t represents the context vector at time step t .
e t j denotes the energy score between the previous hidden state h t 1 and input x j at time step t .
v a , W a , and U a are weight matrices used to compute the energy scores.
α t j represents the attention weight assigned to the input x j at time step t .
y t denotes the output at time step t .
σ is the sigmoid activation function.
W o and U o are weight matrices used to compute the output.
In the LSTM with attention mechanism, the energy scores e t j are calculated to measure the relevance of each input element x j to the previous hidden state h t 1 . These scores are then converted to attention weights α t j by applying the SoftMax function, ensuring that the weights sum up to 1. The context vector c t is obtained by taking a weighted sum of the input sequence x j using the attention weights α t j . This context vector represents the important information from the input sequence that the model focuses on. Finally, the output y t is computed based on the current hidden state h t and the context vector c t . The output can be used for prediction or passed to subsequent layers of the neural network.

2.1.3. Gradient Boosting Machine

GBM is a popular machine learning approach in academia and industry. This ensemble learning method has high predicted accuracy and can handle complex datasets [30,31]. To create a strong prediction model, the GBM algorithm iteratively combines many weak learners, such as decision trees [32]. The repetitive process of training weak learners in a boosting algorithm corrects previous errors, creating an accurate and resilient model. The model is founded on the concept of gradient descent optimization, wherein the algorithm iteratively refines model parameters by minimizing a predefined loss function. GBM loss functions are often calculated by comparing expected outcomes to response variable values.
Mathematically, the GBM algorithm can be summarized as follows. Given a training dataset with input features X and target variable y , the algorithm starts with an initial prediction F 0 ( x ) . In each iteration m , a weak learner, typically a decision tree, is trained to fit the negative gradient of the loss function with respect to the previous predictions, denoted as r i m . This weak learner is referred to as a base learner and is denoted as h m ( x ) . The base learner is then added to the ensemble, and the prediction is updated as follows:
F m x = F m 1 x + ν h m ( x )
where F m 1 x is the prediction from the previous iteration, ν is a learning rate that controls the contribution of each base learner, and h m ( x ) is the prediction from the current base learner. To minimize the loss function, GBM employs a gradient descent step by calculating the negative gradient of the loss function with respect to the ensemble’s predictions. This step determines the direction and magnitude of the updates to the model parameters. The learning rate ν scales the impact of each base learner’s prediction to prevent overfitting and stabilize the learning process. The training process continues for a predefined number of iterations or until a stopping criterion is met. Once training is complete, the ensemble of weak learners forms the final predictive model. During the prediction phase, the input features X are passed through the ensemble, and the final prediction is obtained by summing the predictions of all base learners:
y ^ = F M ( x ) = m = 1 M   ν h m ( x )
where M represents the total number of base learners.

2.2. Explainable AI

XAI is a growing AI subfield that aims to make machine learning models more comprehensible, transparent, and dependable. The technique helps researchers to understand and trust prediction results by revealing these models’ underlying workings. XAI describes these models’ decision-making mechanisms to help individuals understand the findings. The aforementioned techniques encompass rule-based models, feature importance analysis, local explanations, and model-agnostic approaches. Local explanations assist in clarifying prediction for a given input by understanding model behavior at an instance level. Model-agnostic methods, like LIME (Local Interpretable Model-agnostic Explanations) and SHAP (SHapley Additive exPlanations), give significance to different characteristics to provide comprehensive explanations.

SHapley Additive exPlanations

In recent years, SHAP has been receiving significant attention and emerges as an effective method within the field of XAI that provides understandable justifications for the predictions generated by machine learning models. The approach is founded on the theoretical framework of cooperative game theory’s Shapley values and presents a systematic and rigorously formulated methodology for assigning credit to individual features for their contributions to the model’s output. The fundamental objective of SHAP is to ascertain the individual impact of each feature on the prediction by exhaustively evaluating all feasible feature combinations and their corresponding contributions. The fundamental concept underlying SHAP is to allocate a numerical value to each feature that denotes its incremental impact when incorporated into a group of features, while taking into account all feasible feature combinations. The aforementioned values are subsequently combined in order to derive the comprehensive feature significance. Mathematically, the SHAP value for a particular feature can be expressed as follows [33]:
ϕ j ( v a l ) = S { 1 , , p } { j } | S | ! ( p | S | 1 ) ! p ! ( v a l ( S { j } ) v a l ( S ) )
In Equation (11), ϕ j (val) is the Shapley value of a specific feature value. S is a subset of features used in the model, x is the vector of feature values for the instance being explained, and p is the total number of features.

2.3. Experimentation

The NASA Ames Prognostics Centre of Excellence (PCoE) Li-ion battery dataset is examined using a custom-built battery prognostics testbed [34]. The dataset includes charging, discharging, and electrochemical impedance spectroscopy-related information. Data were gathered at various temperatures and current loads. To demonstrate deep discharge aging, discharge limits were set at 2.7 V, lower than OEM recommendations [35]. The batteries were cycled until they reached their end-of-life. During cycling, cells measured the voltage, current, temperature, and impedance. The data recording was carried out at an average data capture rate of about 10 Hz. The main data acquisition system, and a PXI chassis-based system, supported the experiment control and data collection. The obtained data were examined using a MATLAB-based system that incorporated the experiment control, data acquisition, and the evaluation of prognostic algorithms. A total of 18,355 data files provide measurements from a single battery cell over the battery’s life cycle. The cyclic structure of Table 1 provides a comprehensive overview of the steps involved in the charge and discharge phases of the batteries. Each cycle in the table represents a unique charging and discharging process. The table presents information on the voltage, current, temperature, and capacity of the battery for each iteration, as well as additional details, such as the state of charge (SoC) and cut-off voltage. Table 2 provides essential data pertaining to the ambient temperature, cycle count, and discharge cut-off voltage for the batteries under consideration in the estimation of their remaining useful life (RUL). Figure 2 indicates the voltage and current values recorded for batteries B0005, 6, 7, and 18 during a single discharge cycle. The batteries are charged according to step 1 of Table 1, where the CC-CV characteristic can be observed as the battery is initially charged at a constant current, followed by a constant voltage where the amount of current is reduced gradually. The discharge cycle follows, constant at 2 A, until the cut-off voltages corresponding to each battery are reached.
The Python-based SHAP library offers a range of tools, such as Tensor-Flow, XG-Boost, and scikit-learn, to facilitate the computation of SHAP values. The process for calculating SHAP values involves the utilization of the SHAP software for running the model with diverse features, while capturing the appropriate feature values for each simulation. The average SHAP score can be used to prioritize features. The bee swarm plot illustrated in Figure 3a provides supplementary insights into the contribution of individual features to the prediction of RUL. Within this plot, the feature “Cycle” exhibits the widest dispersion along the horizontal axis, thereby indicating its prominence among the considered features. Following closely, the “Current charge” feature demonstrates a slightly smaller range, while the distribution of the “Battery” feature appears comparatively narrower, and so on. In the context of battery capacity prediction, the color scheme of the plot assigns red and blue colors to represent the magnitude of the feature values. Specifically, blue signifies lower feature values, whereas red denotes higher feature values. By analyzing the feature contributions depicted in the plot, it becomes evident that the “Cycle” feature showcases a negative relationship with the predicted RUL. Batteries subjected to a higher number of cycles are associated with reduced capacity, thereby having a lower RUL (as indicated by the presence of red points on the left side). Conversely, instances with lower “Cycle” values tend to exhibit a higher predicted RUL (as represented by the presence of blue points on the right side). Turning attention to the “Current charge” feature, it becomes apparent that higher current measurements generally exert a positive influence on the predicted RUL (as evidenced by the prevalence of red points on the right side). Conversely, lower current measurements are inclined to have a negative impact (as indicated by the abundance of blue points on the left side). With respect to the “Battery” feature, the clustering of blue points on the right side suggests that certain battery IDs consistently contribute to a higher predicted RUL. However, the presence of red points on the left side indicates that a specific battery may yield a lower predicted RUL. Figure 3b presents a bar plot representing the mean SHAP values of all the features. Since the “Cycle”, “Current charge”, “Battery”, and “Current measured” features possess positive mean SHAP values, they are deemed significant explanatory features in terms of explainable artificial intelligence (XAI).

3. Results and Discussion

The accurate estimation of the remaining useful life (RUL) of Li-ion batteries is of vital importance in ensuring their optimal and dependable functioning. The assessment is carried out on four distinct battery types, namely B0005, B0006, B0007, and B0018. The present investigation involves a comparative analysis of the efficacy of three distinct machine learning (ML) models, namely, XDFM, A-LSTM, and GBM, in the context of predicting remaining useful life based on the Li-ion battery dataset procured from the NASA AMES prognostic center. The evaluation of the models’ efficacy is conducted by analyzing the root-mean-square error (RMSE) and mean absolute error (MAE) metrics, which are derived from the training process. This assessment takes into account all the features, as well as those that have been chosen through the utilization of XAI methodologies. The training phase of each battery comprises three cycles, namely, Cycle 0, Cycle 55, and Cycle 110. The results of the comparative analysis are shown in Figure 4a–d, which show RMSE metrics for each model and cycle. For Battery B0005, XDFM outperformed A-LSTM and GBM throughout all cycles. When all features were examined, the XDFM model performed better in terms of RMSE values, with a score of 0.06 for Cycle 0. When only XAI-selected features were used, this score was lowered to 0.03 for Cycle 110. The A-LSTM model’s performance improved as the number of cycles increased, reaching a root-mean-square error (RMSE) of 0.06 for Cycle 55 when all features were evaluated, and then dropping to 0.05 for Cycle 110 when only the XAI-selected features were used. The GBM model produced favorable results, with an RMSE of 0.07 during Cycle 0 and 0.04 during Cycle 55 and 0.03 during Cycle 110. Battery B0006 delivered comparable performance to XDFM and GBM, both of which achieved lower RMSE values than A-LSTM. When all features were examined, the XDFM model produced the best results in terms of RMSE, with a value of 0.04 for Cycle 0. Furthermore, when only the features picked by XAI were considered, the RMSE value was lowered to 0.03 for Cycle 110. The A-LSTM model produced consistent RMSE values ranging from 0.06 to 0.05 throughout all cycles. The GBM algorithm performed consistently in terms of RMSE, with a low RMSE of 0.04 for Cycle 0 and 0.02 for both Cycle 55 and Cycle 110. Battery B0007 displayed patterns similar to the preceding batteries. The XDFM model outperformed the A-LSTM and GBM models by achieving the lowest RMSE values. When all features were used, the XDFM model had an RMSE of 0.05 during Cycle 0. However, using only the features selected using XAI techniques, the model was able to reduce RMSE to 0.02 for both Cycle 55 and Cycle 110. The A-LSTM model demonstrated elevated RMSE values ranging from 0.07 to 0.09. The GBM model, on the other hand, consistently achieved low RMSE values of 0.05 for Cycle 0 and 0.03 for Cycle 55 and 0.02 for Cycle 110. RMSE measurements for all cycles in Battery B0018 showed that XDFM performed the best. When all features were included, the XDFM model achieved a RMSE value of 0.01 in Cycle 0, and this value was maintained at 0.03 in both Cycle 55 and Cycle 110 when only the features selected by XAI were evaluated. The A-LSTM model provided RMSE values ranging from 0.05 to 0.06, but the GBM model produced an RMSE of 0.08 across all feature sets and cycles.
In machine learning, MAE is another important metric used to evaluate the performance of regression models. MAE measures the average absolute difference between the predicted values and the actual values of the target variable. MAE values are obtained when training is performed on all three models. Figure 5a–d compares the performance of various ML models in estimating the remaining useful life of Li-ion batteries. Figure 5a–d show the MAE values obtained for each model and cycle. Reduced MAE values are indicative of enhanced precision in prediction. For Battery B0005, XDFM outperformed A-LSTM and GBM throughout all cycles. When all features were analyzed, the XDFM model produced the best results in terms of MAE, with a value of 0.02 for Cycle 0. Furthermore, the model’s performance was improved in Cycle 110, which used only the features chosen by the XAI approach, resulting in a further drop of the MAE to 0.01. The MAE values for the A-LSTM model ranged from 0.04 to 0.05, but the MAE values for the GBM model were 0.05 for Cycle 0 and 0.04 for Cycle 55 and 0.02 for Cycle 110. In terms of the Battery B0006 performance, the results show that XDFM and GBM outperform A-LSTM. The XDFM model had the lowest MAE values, with a score of 0.02 when all features were evaluated during Cycle 110, and then reducing to 0.01 when just XAI-selected features were used during Cycle 110. The MAE measurements for the A-LSTM model were elevated, ranging between 0.03 and 0.05. The GBM algorithm performed consistently in terms of MAE values. GBM, in particular, produced low MAE values of 0.03 for Cycle 0 and 0.01 for both Cycle 55 and Cycle 110. Similar behavior to that of earlier batteries was displayed by B0007, with XDFM showing substantial improvement over A-LSTM and GBM. The XDFM model had the lowest MAE values, registering 0.02 for Cycle 0 when all features were taken into account and keeping the same value of 0.02 for Cycle 55 and Cycle 110 when just XAI-selected features were used. The MAE values for the A-LSTM model were elevated, ranging between 0.06 and 0.07. The GBM model, on the other hand, consistently achieved low MAE values of 0.04 for Cycle 0 and 0.05 for Cycle 55 and 0.02 for Cycle 110. The results show that XDFM has the most consistent low MAE values across all cycles in Battery B0018. When all features were examined, the XDFM model had a MAE of 0.05 in Cycle 0. However, when only the features picked by XAI were used, the MAE was lowered to 0.02 in both Cycle 55 and Cycle 110. The A-LSTM model produced MAE values ranging from 0.03 to 0.04, but the GBM model consistently produced MAE values of 0.07 across all feature sets and cycles. According to the results of the comparison analysis, XDFM outperformed A-LSTM and GBM in estimating the remaining usable life (RUL) of Li-ion batteries. The use of the XAI feature selection approach has been seen to improve the efficacy of the XDFM and A-LSTM models, resulting in a significant increase in their accuracy.
After analyzing the RMSE values obtained after testing from three distinct machine learning models, namely, XDFM, A-LSTM, and GBM, for four Li-ion batteries (B0005, B0006, B0007, and B0018) at Cycle 0, Cycle 55, and Cycle 110, a number of observations can be inferred, as can be seen in Figure 6a–d. In the case of Battery B0005, the XDFM model demonstrated a superior predictive performance across all cycles. The model obtained the minimum RMSE values. It recorded 0.04 for Cycle 0 when all aspects were considered. The RMSE for Cycle 55 decreased to 0.02 and for Cycle 110 decreased to 0.05 when only the features chosen by XAI were utilized. RMSE values for the A-LSTM and GBM models were greater than anticipated, ranging from 0.05 to 0.07. The predictive efficacy of the XDFM model for Battery B006 was exceptional. Consistently, the lowest RMSE values were observed. Cycle 0’s RMSE was calculated, and the resulting value was 0.09 when all features were considered. However, when only the features chosen by XAI were used, the RMSE was reduced to 0.05 for Cycle 55 and 0.07 for Cycle 110. The RMSE values for the A-LSTM and GBM models were greater, spanning from 0.06 to 0.10. Similarly, it was observed that the XDFM model performed better than the A-LSTM model and the GBM model for Battery B0007. The XDFM method obtained the lowest RMSE values, which ranged from 0.07 to 0.10. The RMSE values for the A-LSTM and GBM models were greater than anticipated, ranging from 0.08 to 0.10. The XDFM model consistently predicted the performance of Battery B0018 with high accuracy. Cycle 55 and Cycle 110 recorded RMSE values of 0.05 and 0.02, respectively, indicating that the features selected by XAI produced the smallest RMSE values. The RMSE values for the A-LSTM and GBM models were greater, spanning from 0.07 to 0.11. The aforementioned findings demonstrate that XDFM is capable of accurately predicting the RUL of Li-ion batteries and is suitable for implementation in the field of battery prognosis.
Distinct trends emerge upon analysis of the MAE values obtained (Figure 7a–d) after the testing phase of three ML models and the Li-ion batteries considered similar in earlier cases. The XDFM model consistently outperformed the other models for B0005 and B0006, as indicated by the lowest MAE values over multiple cycles. In comparison, the A-LSTM and GBM models demonstrated elevated MAE values, indicating a considerably lower level of accuracy in their prediction capabilities. When Battery B0007 was examined, it turned out that all three models performed similarly, as evidenced from the MAE values that were generally around 0.1 across multiple cycles. The results show that there was no statistically significant difference in the performance of the various models when compared to the MAE metric for the specific battery under examination. Battery B0018 consistently achieved the lowest MAE values, demonstrating the XDFM model’s superior performance. When compared to XDFM, the MAE values derived from both the A-LSTM and GBM models were greater, indicating a poorer level of precision in predicting outcomes.
Figure 8, Figure 9, Figure 10 and Figure 11 show the RUL predicted by the ML models, overlaid on the actual RUL during three distinct operational phases. The discharge cycles completed by the batteries are denoted in crimson and separated by a vertical perforated line, indicating the battery’s current state. In Figure 8a, the predicted value of the RUL for Battery B0005 in its fresh, unused state is depicted. This prediction is based on the utilization of the run-to-failure data collected from Batteries B0006, B0007, and B0018. By leveraging this training data, the ML models aim to extrapolate and estimate the RUL of Battery B0005 at this early stage. Subsequently, Figure 8b illustrates the predicted value of the RUL for Battery B0005 after 55 charge/discharge cycles, while Figure 8c represents the corresponding prediction after 110 charge/discharge cycles. These figures offer insights into the ML models’ ability to capture and project the progressive decay in the discharge capacity of Battery B0005 over multiple operational cycles.
Furthermore, Figure 9a–c indicate the RUL predictions obtained from all three models for Battery B0006. Analyzing these predictions, it becomes apparent that the models tend to slightly underestimate the values of discharge capacity utilized to project the RUL during the initial cycles, gradually approaching a more accurate estimation as the battery’s operation progresses. Towards the end cycles of Battery B0006, the ML/DL models tend to marginally overestimate this value, which can be attributed to the composition of the training data, which encompasses Batteries B0005, B0007, and B0018. Notably, these training batteries had initial capacities of 1.8 Ah and were monitored until their discharge capacity dropped to 1.3 Ah. In contrast, Battery B0006 possesses an initial capacity of 2 Ah and operates until its discharge capacity reaches 1.1 Ah. The dissimilarities in the initial and final capacity values between the training and predicted batteries contribute to the observed discrepancies in the RUL predictions. Likewise, Figure 10a–c presents the RUL predictions for Battery B0007 utilizing the ML/DL models trained on the run-to-failure data of Batteries B0005, B0006, and B0018. Similarly, Figure 11a–c depicts the remaining useful life (RUL) prediction for Battery B0018, leveraging the data from Batteries B0005, B0006, and B0007. Notably, XDFM demonstrates a closer approximation of the battery’s RUL by effectively capturing its nonlinear decay. This nature of Li-ion batteries is attributed to the capacity regeneration phenomenon in batteries, where longer rest periods between discharge cycles lead to a sudden spike in capacity.
The following are a few potential reasons for XDFM to perform better in the RUL prediction of Li-ion batteries when in comparison to the other two models: Extreme Gradient Boosting (XGBoost) techniques are used in the XDFM model to facilitate group learning. The model can capture complicated associations and produce precise predictions because of ensemble learning, which integrates several predictors into one powerful predictor. In comparison to the individual A-LSTM and GBM models, the ensemble learning and boosting strategy in XDFM is better able to handle the nonlinearities and complexity found in the Li-ion battery data. Second, even though A-LSTM includes an attention mechanism that enables the model to concentrate on particular input sequence segments that are most important to the prediction task, it might not fully capture the intricate dependencies and linkages found in the Li-ion battery data. The A-LSTM’s prediction powers may be constrained by the attention mechanism’s inability to handle the data’s diversity and nonlinearity. Last, but not least, the data from Li-ion batteries show specific temporal patterns and dependencies that are frequently connected to the form of the degradation curves. In order to record and examine the temporal behavior of Li-ion batteries, the XDFM model uses shape-based approaches. The XDFM model learns more about the underlying degradation mechanisms by concentrating on shape-related parameters, like voltage curves, discharge profiles, or capacity degradation patterns. Due to this particular factor, the XDFM model is better able to capture significant temporal relationships, which enhances the RUL prediction performance. In conclusion, the XDFM model’s superior performance in Li-ion battery testing to the A-LSTM and GBM RUL prediction’s capacity to capture intricate nonlinear correlations, thorough feature utilization, targeted feature selection using XAI, and specialized consideration of shape-based traits are all factors. These elements work together to increase the XDFM model’s accuracy, which makes it especially suitable for estimating the useful life of Li-ion batteries.

4. Conclusions

In the present work, the RUL prediction of Li-ion battery using three machine learning models—XDFM, A-LSTM, and GBM—was explored. Training and testing of all batteries are performed, and RMSE and MAE values are examined to analyze the accurate RUL predictions. The shape-based XAI model is selected to identify the key features, which significantly affects the prediction capabilities. Based on the results obtained, considering all features and the features selected through XAI for Li-ion Batteries B0005, B0006, B0007, and B0018 at Cycle 0, Cycle 55, and Cycle 110, noteworthy outcomes are elaborated as follows:
  • When all features are considered, the average lowest RMSE across all batteries and cycles for RUL prediction is 0.042 when XDFM model were trained, whereas considering XAI-selected features, the average lowest RMSE value for RUL prediction is 0.027 from the XDFM model.
  • Considering the MAE values, the average lowest MAE value across all batteries and cycles for RUL prediction is 0.02 with all features, whereas the average lowest MAE value of 0.01 is obtained from XAI-selected features from the trained XDFM model.
  • When testing of ML model is conducted, the average lowest RMSE value observed is 0.06 with all features and 0.03 with XAI-selected features from the XDFM model.
  • A similar trend is observed from MAE values. The average lowest MAE value observed is 0.03 with all features and 0.02 with XAI-selected features, again from the XDFM model.
Overall, the XDFM model with XAI-selected features outperforms the GBM and A-LSTM models for RUL prediction based on the lowest average numerical RMSE and MAE values across all batteries and cycles. By including additional sensor data, such as humidity and vibration, which can potentially have an impact on the battery’s performance and lifespan, the Li-ion RUL prediction capabilities can be further enhanced. The inclusion of such data can improve model accuracy and offer additional insight on the mechanisms underlying battery deterioration. Other energy storage technologies, including supercapacitors, fuel cells, and flow batteries, can also be predicted using the suggested methods to determine how well they will work and how long they will last. This may aid in the creation of energy storage systems for diverse applications that are more effective and dependable.

Author Contributions

Conceptualization, V.V. and P.N.; methodology, V.V. and H.B.; software, P.N. and M.S.; validation, V.W. and V.V.; formal analysis, H.B. and M.S.; investigation, P.N. and M.S.; resources, V.V. and V.W.; data curation, P.N. and M.S.; writing—original draft preparation H.B. and P.N.; writing—review and editing, V.V. and H.B.; visualization, P.N. and M.S.; supervision, V.V. and V.W. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The authors have used a publicly available Li-ion battery dataset. Link: https://ti.arc.nasa.gov/tech/dash/groups/pcoe/prognosticdata-repository/ (accessed on 15 April 2023).

Acknowledgments

The authors would like to thank B. Saha and K. Goebel, NASA Ames Research Center, for providing access to the dataset for research purpose.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Wen, J.; Zhao, D.; Zhang, C. An overview of electricity powered vehicles: Li-ionbattery energy storage density and energy conversion efficiency. Renew. Energy 2020, 162, 1629–1648. [Google Scholar] [CrossRef]
  2. Hannan, M.A.; Faisal, M.; Jern Ker, P.; Begum, R.A.; Dong, Z.Y.; Zhang, C. Review of optimal methods and algorithms for sizing energy storage systems to achieve decarbonization in microgrid applications. Renew. Sustain. Energy Rev. 2020, 131, 110022. [Google Scholar] [CrossRef]
  3. Nitta, N.; Wu, F.; Lee, J.T.; Yushin, G. Li-ion battery materials: Present and future. Mater. Today 2015, 18, 252–264. [Google Scholar] [CrossRef]
  4. Barker, J. Li-ion Rechargeable Batteries: Novel Materials. In Encyclopedia of Materials: Science and Technology; Elsevier: Amsterdam, The Netherlands, 2010; pp. 1–6. [Google Scholar] [CrossRef]
  5. Li, J.; Ma, Z. Past and Present of LiFePO4: From Fundamental Research to Industrial Applications. Chem 2019, 5, 3–6. [Google Scholar] [CrossRef] [Green Version]
  6. Chen, S.; Zhang, X.; Xia, M.; Wei, K.; Zhang, L.; Zhang, X.; Cui, Y.; Shu, J. Issues and challenges of layered lithium nickel cobalt manganese oxides for Li-ionbatteries. J. Electroanal. Chem. 2021, 895, 115412. [Google Scholar] [CrossRef]
  7. Lee, J.-H.; Lee, I.-S. Lithium Battery SOH Monitoring and an SOC Estimation Algorithm Based on the SOH Result. Energies 2021, 14, 4506. [Google Scholar] [CrossRef]
  8. Rivera-Barrera, J.; Muñoz-Galeano, N.; Sarmiento-Maldonado, H. SoC Estimation for Li-ionBatteries: Review and Future Challenges. Electronics 2017, 6, 102. [Google Scholar] [CrossRef] [Green Version]
  9. Vakharia, V.; Shah, M.; Nair, P.; Borade, H.; Sahlot, P.; Wankhede, V. Estimation of Li-ionBattery Discharge Capacity by Integrating Optimized Explainable-AI and Stacked LSTM Model. Batteries 2023, 9, 125. [Google Scholar] [CrossRef]
  10. Galeotti, M.; Cinà, L.; Giammanco, C.; Cordiner, S.; Di Carlo, A. Performance analysis and SOH (state of health) evaluation of lithium polymer batteries through electrochemical impedance spectroscopy. Energy 2015, 89, 678–686. [Google Scholar] [CrossRef]
  11. Gismero, A.; Schaltz, E.; Stroe, D.-I. Recursive State of Charge and State of Health Estimation Method for Li-ionBatteries Based on Coulomb Counting and Open Circuit Voltage. Energies 2020, 13, 1811. [Google Scholar] [CrossRef] [Green Version]
  12. Movassagh, K.; Raihan, A.; Balasingam, B.; Pattipati, K. A Critical Look at Coulomb Counting Approach for State of Charge Estimation in Batteries. Energies 2021, 14, 4074. [Google Scholar] [CrossRef]
  13. Spotnitz, R. Simulation of capacity fade in Li-ionbatteries. J. Power Sources 2003, 113, 72–80. [Google Scholar] [CrossRef]
  14. Tremblay, O.; Dessaint, L. Experimental Validation of a Battery Dynamic Model for EV Applications. World Electr. Veh. J. 2009, 3, 289–298. [Google Scholar] [CrossRef] [Green Version]
  15. Sidorov, D.; Muftahov, I.; Tomin, N.; Karamov, D.; Panasetsky, D.; Dreglea, A.; Liu, F.; Foley, A. A Dynamic Analysis of Energy Storage with Renewable and Diesel Generation Using Volterra Equations. IEEE Trans. Ind. Inform. 2020, 16, 3451–3459. [Google Scholar] [CrossRef] [Green Version]
  16. Potrykus, S.; Kutt, F.; Nieznański, J.; Fernández Morales, F.J. Advanced Li-ionBattery Model for Power System Performance Analysis. Energies 2020, 13, 2411. [Google Scholar] [CrossRef]
  17. Markova, E.; Sidler, I.; Solodusha, S. Integral Models Based on Volterra Equations with Prehistory and Their Applications in Energy. Mathematics 2021, 9, 1127. [Google Scholar] [CrossRef]
  18. Samanta, A.; Chowdhuri, S.; Williamson, S.S. Machine Learning-Based Data-Driven Fault Detection/Diagnosis of Li-ionBattery: A Critical Review. Electronics 2021, 10, 1309. [Google Scholar] [CrossRef]
  19. Choi, Y.; Ryu, S.; Park, K.; Kim, H. Machine Learning-Based Li-ionBattery Capacity Estimation Exploiting Multi-Channel Charging Profiles. IEEE Access 2019, 7, 75143–75152. [Google Scholar] [CrossRef]
  20. Shah, M.; Borade, H.; Sanghavi, V.; Purohit, A.; Wankhede, V.; Vakharia, V. Enhancing Tool Wear Prediction Accuracy Using Walsh–Hadamard Transform, DCGAN and Dragonfly Algorithm-Based Feature Selection. Sensors 2023, 23, 3833. [Google Scholar] [CrossRef]
  21. Vakharia, V.; Shah, M.; Suthar, V.; Patel, V.; Solanki, A. Hybrid perovskites thin films morphology identification by adapting multiscale-SinGAN architecture, heat transfer search optimized feature selection and machine learning algorithms. Phys. Scr. 2023, 98, 025203. [Google Scholar] [CrossRef]
  22. Nuhic, A.; Terzimehic, T.; SoczkaGuth, T.; Buchholz, M.; Dietmayer, K. Health diagnosis and remaining useful life prognostics of lithiumion batteries using datadriven methods. J. Power Sources 2013, 239, 680–688. [Google Scholar] [CrossRef]
  23. Wu, J.; Zhang, C.; Chen, Z. An online method for Li-ionbattery remaining useful life estimation using importance sampling and neural networks. Appl. Energy 2016, 173, 134–140. [Google Scholar] [CrossRef]
  24. Zhao, S.; Zhang, C.; Wang, Y. Li-ionbattery capacity and remaining useful life prediction using board learning system and long short-term memory neural network. J. Energy Storage 2022, 52, 104901. [Google Scholar] [CrossRef]
  25. Vinay, V.; Kumar, G.V.; Kumar, K.P. Application of Chi Square Feature Ranking Technique and Random Forest Classifier for Fault Classification of Bearing Faults. In Proceedings of the 22nd International Congress on Sound and Vibration, Florence, Italy, 12–16 July 2015; pp. 12–16. [Google Scholar]
  26. Patel, D.R.; Kiran, M.B.; Vakharia, V. Modeling and Prediction of Surface Roughness Using Multiple Regressions: A Noncontact Approach. Eng. Rep. 2020, 2, e12119. [Google Scholar] [CrossRef] [Green Version]
  27. Wang, P.; Sun, M.; Wang, Z.; Zhou, Y. A Novel CTR Prediction Based Model Using xDeepFM Network. In Proceedings of the 2021 IEEE International Conference on Computer Science, Electronic Information Engineering, and Intelligent Control Technology (CEI), Fuzhou, China, 24–26 September 2021. [Google Scholar] [CrossRef]
  28. Luo, T.; Chang, D.; Xu, Z. Research on Apparel Retail Sales Forecasting Based on xDeepFM-LSTM Combined Forecasting Model. Information 2022, 13, 497. [Google Scholar] [CrossRef]
  29. Mamo, T.; Wang, F.-K. Attention-Based Long Short-Term Memory Recurrent Neural Network for Capacity Degradation of Li-ionBatteries. Batteries 2021, 7, 66. [Google Scholar] [CrossRef]
  30. Zhan, X.; Zhang, S.; Szeto, W.Y.; Chen, X. Multi-step-ahead traffic speed forecasting using multi-output gradient boosting regression tree. J. Intell. Transp. Syst. 2019, 24, 125–141. [Google Scholar] [CrossRef]
  31. Thomas, J.; Mayr, A.; Bischl, B.; Schmid, M.; Smith, A.; Hofner, B. Gradient boosting for distributional regression: Faster tuning and improved variable selection via noncyclical updates. Stat. Comput. 2017, 28, 673–687. [Google Scholar] [CrossRef] [Green Version]
  32. Gumaei, A.; Al-Rakhami, M.; Mahmoud Al Rahhal, M.; Raddah, H.; Albogamy, F.; Al Maghayreh, E.; AlSalman, H. Prediction of COVID-19 Confirmed Cases Using Gradient Boosting Regression Method. Comput. Mater. Contin. 2020, 66, 315–329. [Google Scholar] [CrossRef]
  33. Molnar, C. (n.d.) 9.5 Shapley Values | Interpretable Machine Learning. In christo phm.github.io. Available online: https://christophm.github.io/interpretable-ml-book/shapley.html#fnref63 (accessed on 18 May 2023).
  34. Saha, B.; Goebel, K. Battery Data Set. In NASA Ames Prognostics Data Repository; NASA Ames Research Centre: Moffett Field, CA, USA, 2007; [Online]. Available online: https://ti.arc.nasa.gov/tech/dash/groups/pcoe/prognosticdata-repository/ (accessed on 15 April 2023).
  35. Saxena, A.; Celaya, J.; Roychoudhury, I.; Saha, B.; Saha, S.; Goebel, K. Designing Data Driven Battery Prognostic Approaches for Variable Loading Profiles: Some Lessons Learned. In Proceedings of the European Conference of Prognostics and Health Management Society PHM, Denver, CO, USA, 18–22 June 2012; pp. 1–11. [Google Scholar]
Figure 1. Proposed framework for RUL estimation of Li-ion battery using Explainable AI-assisted ExDeepFM Model.
Figure 1. Proposed framework for RUL estimation of Li-ion battery using Explainable AI-assisted ExDeepFM Model.
Energies 16 05725 g001
Figure 2. Voltage and current data were collected during the first cycle of the charge and discharge.
Figure 2. Voltage and current data were collected during the first cycle of the charge and discharge.
Energies 16 05725 g002aEnergies 16 05725 g002b
Figure 3. (a) Bee swarm plot indicating the correlation and contribution of each instance for a feature towards the prediction of battery capacity; (b) Bar plot providing global interpretation of the model’s prediction in terms of mean SHAP scores for feature importance.
Figure 3. (a) Bee swarm plot indicating the correlation and contribution of each instance for a feature towards the prediction of battery capacity; (b) Bar plot providing global interpretation of the model’s prediction in terms of mean SHAP scores for feature importance.
Energies 16 05725 g003
Figure 4. Training RMSE of Li-ion Batteries: (a) B0005, (b) B0006, (c) B0007, (d) B0018.
Figure 4. Training RMSE of Li-ion Batteries: (a) B0005, (b) B0006, (c) B0007, (d) B0018.
Energies 16 05725 g004
Figure 5. Training MAE metrics of Li-ion Batteries: (a) B0005, (b) B0006, (c) B0007, (d) B0018.
Figure 5. Training MAE metrics of Li-ion Batteries: (a) B0005, (b) B0006, (c) B0007, (d) B0018.
Energies 16 05725 g005aEnergies 16 05725 g005b
Figure 6. Testing RMSE of Li-ion Batteries: (a) B0005, (b) B0006, (c) B0007, (d) B0018.
Figure 6. Testing RMSE of Li-ion Batteries: (a) B0005, (b) B0006, (c) B0007, (d) B0018.
Energies 16 05725 g006
Figure 7. Testing MAE of Li-ion Batteries: (a) B0005, (b) B0006, (c) B0007, (d) B0018.
Figure 7. Testing MAE of Li-ion Batteries: (a) B0005, (b) B0006, (c) B0007, (d) B0018.
Energies 16 05725 g007
Figure 8. RUL prediction for Battery B0005 from (a) 0th; (b) 55th; (c) 110th cycle.
Figure 8. RUL prediction for Battery B0005 from (a) 0th; (b) 55th; (c) 110th cycle.
Energies 16 05725 g008
Figure 9. RUL prediction for Battery B0006 from (a) 0th; (b) 55th; (c) 110th cycle.
Figure 9. RUL prediction for Battery B0006 from (a) 0th; (b) 55th; (c) 110th cycle.
Energies 16 05725 g009
Figure 10. RUL prediction for Battery B0007 from (a) 0th; (b) 55th; (c) 110th cycle.
Figure 10. RUL prediction for Battery B0007 from (a) 0th; (b) 55th; (c) 110th cycle.
Energies 16 05725 g010aEnergies 16 05725 g010b
Figure 11. RUL prediction for Battery B0018 from (a) 0th; (b) 55th; (c) 110th cycle.
Figure 11. RUL prediction for Battery B0018 from (a) 0th; (b) 55th; (c) 110th cycle.
Energies 16 05725 g011
Table 1. Description of the experimental cycle.
Table 1. Description of the experimental cycle.
StepActionExit Condition
1CC-CV Charge of batteries4.2 V is reached
22 A Discharge of batteriesCut-off voltages are reached as specified in Table 2
Table 2. Batteries selected, along with the number of cycles, temperature, and cut-off voltages used for analysis.
Table 2. Batteries selected, along with the number of cycles, temperature, and cut-off voltages used for analysis.
Battery ID.Ambient TemperatureNo. of CyclesDischarge Cut-Off Voltage
B000524 °C1682.7 V
B000624 °C1682.5 V
B000724 °C1682.2 V
B001824 °C1322.5 V
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Nair, P.; Vakharia, V.; Borade, H.; Shah, M.; Wankhede, V. Predicting Li-Ion Battery Remaining Useful Life: An XDFM-Driven Approach with Explainable AI. Energies 2023, 16, 5725. https://doi.org/10.3390/en16155725

AMA Style

Nair P, Vakharia V, Borade H, Shah M, Wankhede V. Predicting Li-Ion Battery Remaining Useful Life: An XDFM-Driven Approach with Explainable AI. Energies. 2023; 16(15):5725. https://doi.org/10.3390/en16155725

Chicago/Turabian Style

Nair, Pranav, Vinay Vakharia, Himanshu Borade, Milind Shah, and Vishal Wankhede. 2023. "Predicting Li-Ion Battery Remaining Useful Life: An XDFM-Driven Approach with Explainable AI" Energies 16, no. 15: 5725. https://doi.org/10.3390/en16155725

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop