Next Article in Journal
Artificial Intelligence in Adaptive and Intelligent Educational System: A Review
Next Article in Special Issue
Mathematical and Machine Learning Models for Groundwater Level Changes: A Systematic Review and Bibliographic Analysis
Previous Article in Journal
A Blockchain-Based Framework to Enhance Anonymous Services with Accountability Guarantees
Previous Article in Special Issue
Polarity and Subjectivity Detection with Multitask Learning and BERT Embedding
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Machine Learning for Bankruptcy Prediction in the American Stock Market: Dataset and Benchmarks

Department of Engineering and Architecture, University of Parma, 43124 Parma, Italy
Department of Industrial and Systems Engineering, University of Florida, Gainesville, FL 32611, USA
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Future Internet 2022, 14(8), 244;
Original submission received: 27 July 2022 / Revised: 10 August 2022 / Accepted: 15 August 2022 / Published: 22 August 2022
(This article belongs to the Special Issue Machine Learning Perspective in the Convolutional Neural Network Era)


Predicting corporate bankruptcy is one of the fundamental tasks in credit risk assessment. In particular, since the 2007/2008 financial crisis, it has become a priority for most financial institutions, practitioners, and academics. The recent advancements in machine learning (ML) enabled the development of several models for bankruptcy prediction. The most challenging aspect of this task is dealing with the class imbalance due to the rarity of bankruptcy events in the real economy. Furthermore, a fair comparison in the literature is difficult to make because bankruptcy datasets are not publicly available and because studies often restrict their datasets to specific economic sectors and markets and/or time periods. In this work, we investigated the design and the application of different ML models to two different tasks related to default events: (a) estimating survival probabilities over time; (b) default prediction using time-series accounting data with different lengths. The entire dataset used for the experiments has been made available to the scientific community for further research and benchmarking purposes. The dataset pertains to 8262 different public companies listed on the American stock market between 1999 and 2018. Finally, in light of the results obtained, we critically discuss the most interesting metrics as proposed benchmarks for future studies.

Graphical Abstract

1. Introduction

Since the 2007/2008 financial crisis, most financial institutions, lenders, and academics have become interested in predicting corporate bankruptcy. Usually, corporate bankruptcy costs spread to the whole economy, resulting in cascade effects that impact many companies [1,2].
Despite that different research works have already demonstrated the ability of machine learning (ML) to assess the likelihood of companies’ default, making a fair comparison among all the proposed approaches in the literature remains challenging for several reasons: (a) most of the datasets are not publicly available or are only related to specific economic scenarios like private companies in different countries [3,4]. For private companies, little information is generally available, which makes it difficult to exploit other sources of information that may improve bankruptcy prediction performance (e.g., textual disclosures [5], annual reports [6], stock market data) and that can be used by more complex models; (b) bankruptcy prediction actually involves different tasks: the default prediction in tasks for the next year, using past data, and the survival probability prediction task that aims to predict the probability that a company will face financial distress in k years. Most datasets cannot permit the performing of both tasks, and this is a clear limitation to the development of intelligent models that aim to generalize; (c) bankruptcy prediction models are usually trained on imbalanced data including few examples of the bankruptcy class: there is still no general accepted metric to assess bankruptcy prediction performance with machine learning. Indeed, the prediction accuracy can be misleading since it gives the same cost to false positives and false negatives while making, for example, the wrong prediction on a company that is going to go bankrupt has in general a higher cost than the former.
At the same time, as a means of addressing class imbalance, the common metric reported for bankruptcy prediction is the Area Under the Curve (AUC). This, however, can also be misleading since it refers to the overall performance of a classifier without providing details regarding the performance of each class or the confusion matrix.
In this research work, we investigated the bankruptcy prediction with machine learning on the American stock market, which represents one of the most important drivers of the world economy. The reasons for this choice are summarized in the following: (a) different kinds of data are available for public companies in the American market: stock prices, accounting variables, and annual and quarterly financial reports. It allows the dataset to be improved in the future; (b) a corporate default in the American stock market can have a large impact on the overall economy.
The main contributions of our paper are the following:
  • We collected a dataset for bankruptcy prediction considering 8262 different companies in the stock market in the time interval between 1999 and 2018. The dataset has been made public (, accessed on 14 August 2022) for the scientific community for further investigations as a benchmark and thus it can be easily enriched with data coming from other sources pertaining to the same companies. Our dataset faithfully followed the FAIR Data Principles [7]:
    Findable: our data is indexed in a searchable resource and had a unique and persistent identifier.
    Accessible: our data is understandable to humans and machines and it is deposited in a trusted repository.
    Interoperable: we used a formal, accessible, shared, and broadly applicable language for knowledge representation.
    Reusable: we provided accurate information on provenance and clear usage licenses.
  • We investigated two different bankruptcy prediction tasks: The first task (T1) is the default prediction task which aims to predict the company status in the next year using time series of accounting data or just the last fiscal year available. The second task (T2) is the survival probability prediction task in which the model tries to predict the company status over k years in the future.
  • In light of the results achieved, we critically discuss the most interesting metrics as proposed benchmarks for the future, such as: Area Under the Curve (AUC), precision, recall (sensitivity), type I error, type II error, and macro- and micro-F1 scores for each class.
The paper is organized as follows: In Section 2, we provide an overview the state-of-the-art approaches for bankruptcy prediction. In Section 3, we describe in detail the dataset that has been used for this study. In Section 4, we review and describe all the machine-learning algorithms that we used in our experiments. In Section 5, we introduce the metrics used for the imbalanced scenario encountered in this study. In Section 6, we describe the first task we evaluated in this work, concerning the prediction of a company’s health status based only on data from previous years. In Section 7, we present our second task where we performed a survival probability task on companies within the dataset. In Section 8, we show and describe the experimental results, and finally in Section 9 we summarize the results with a critical discussion about the metrics.

2. Related Works

The recent advancements in machine learning (ML) led to new, innovative, and functional approaches [8,9]. Moreover, they have enabled the development of intelligent models that try to assess the likelihood of companies’ default by looking for relationships among different types of financial data, and the financial status of a firm in the future [10,11,12,13,14,15,16,17]. Different ML algorithms and techniques such as Support Vector Machine (SVM) [18], boosting techniques [19], discriminant analysis [20], and Neural Networks [5] have been used in the literature for this task. Moreover, different architectures have been evaluated to identify effective decision boundaries for this binary classification problem, such as the least absolute shrinkage and selection operator [21], dynamic slacks-based model [22], and two-stage classification [13]. However, although default prediction models have been studied for decades, several issues remain. Interestingly, some new issues have even been introduced with the recently increased exploitation of machine-learning models. Indeed, since the Z-Score model was proposed by Altman in 1968 [23], research mainly focused on accounting-based ratios as markers to detect and understand if a firm is likely to face financial difficulties, such as bankruptcy. Scoring-based models use discriminant analysis to provide ordinal rankings of default risk but are often computed from small datasets using statistical and probabilistic models that focus more on explainability and explicability but miss generalization over time and across different sectors [24]. Other examples are the Kralicek quick test [25] and the Taffler model [26].
In [27], a step towards modern machine learning was made by introducing a binary response model that uses explanatory variables and applies a logistic function for bankruptcy prediction [28]. However, the main goal of these models’ class is not to identify a decision boundary in the feature space but only to select a decision based on an output threshold that was statistically significant in the past for the specific sector. For example, Altman suggested two thresholds, 1.81 and 2.99.
Specifically, an Altman’s Z-score above the 2.99 threshold means that firms are not expected to default in the next two years, below 1.81 that they are expected to default, while the interval between the two thresholds is named the “zone of ignorance” where no clear decision can be taken. However, even though many practitioners use this threshold, in Altman’s view, this is an unfortunate practice since over the past 50 years, credit-worthiness dynamics and trends have changed so dramatically that the original zone cutoffs are no longer relevant [24].
Moreover, we still lack a definite theory for the bankruptcy prediction task [18,29] and in particular, a generally accepted performance metric is missing along with a formal theoretical framework. As a consequence, the most common methodology in bankruptcy prediction tasks is identifying discriminant features using a trial and error approach with various accounting-based ratios [15,16].
Machine-learning models usually need large datasets to be trained and suffer when class imbalance is strong as in bankruptcy, since default events are quite rare. Learning from imbalanced data requires dealing with several challenges, especially when the most important class that should be recognized is exactly the one that is least represented in the dataset. This issue is strongly related to the lack of a general performance metric.
Machine-learning techniques like ensemble methods were firstly explored for default prediction by Nanni et al. [30]. Kim et al. showed a much better performance for the ensembles compared to standalone classifiers, while their results were also confirmed by Kim et al. [31]. Wang et al. further analyzed the performance of ensemble models, finding that bagging outperformed boosting for all credit databases in terms of average accuracy, as well as type I and type II error [32]. In [33], Barboza et al. show that, on average, machine-learning models exhibit 10% more accuracy than scoring-based ones. Specifically, in this study, Support Vector Machines (SVM), Random Forests (RF) as well as bagging and boosting techniques were tested for predicting bankruptcy events and were compared with results from the discriminant analysis, Logistic Regression, and Neural Networks. The authors found that bagging, boosting, and RF outperformed all other models. However, since the dataset has not been released and the models’ hyper-parameters are not reported, it is difficult to replicate such results and understand if the performance improves because of the quality of the models or because the authors take into account some other financial variables as features.
Considering that the comparisons of the models are still inconclusive, new studies exploring different models, contexts, and datasets are relevant. A firm’s failure is likely to be caused by difficulties experienced over time and not just the previous year. In light of this, the dynamic behavior of firms should be considered in a theoretical framework for bankruptcy prediction such as growth measures and changes in some variables [33]. Several research works investigated this aspect, but the results are again inconclusive and often not reproducible. In [28], the authors show that firms exhibit failure tendencies as much as five years prior to the actual event. On the other hand, Mossman et al. [34] pointed out that the models are only capable of predicting bankruptcy two years prior to the event, which improves to three years if used for multiperiod corporate default prediction [35]. In most studies, ratios are analyzed backward in time starting with the bankruptcy event and going back until the model becomes unreliable or inaccurate. Moreover, most of the bankruptcy prediction models in the literature do not take advantage of the sequential nature of the financial data. This lack of multi-period models is also emphasized in a literature review by Kim et al. [36].

3. Dataset

Since most of the bankruptcy models are evaluated on private datasets or small publicly available ones, we provide a novel dataset for bankruptcy prediction related to the public companies in the American stock market (New York Stock Exchange and NASDAQ). We collected accounting data from 8262 different companies in the period between 1999 and 2018. The stock market is dynamic with new companies becoming public every year, changing properties and names, or being removed or suspended from the market as a result of acquisitions or regulatory action. For this reason, we consider the same companies used in [6,37] since this set of firms has been proved to be a fair approximation of the American stock market for each year in that time interval. According to the Security Exchange Commission (SEC), a company in the American market is considered bankrupt in two cases:
  • If the firm’s management files Chapter 11 of the Bankruptcy Code to “reorganize” its business: management continues to run the day-to-day business operations but all significant business decisions must be approved by a bankruptcy court.
  • If the firm’s management files Chapter 7 of the Bankruptcy Code: the company stops all operations and goes completely out of business.
When these events occur we label the fiscal year before the chapter filing as “bankruptcy” (1) for the next year. Otherwise, the company is considered healthy (0). In light of this, our dataset enables learning how to predict bankruptcy at least one year before it happens and, as a consequence, it is possible to deal with the default prediction task using time series and also to deal with the survival probability task looking ahead. Figure 1 shows the percentage of companies’ default for each year in the dataset. This value can be underestimated due to the exclusion of some companies in the past because of their small market capitalization, but it appears to agree with the literature that usually reports that only a percentage below 1% of the available firms in the market is likely to default every year under normal conditions. However, in some periods, bankruptcy rates have been higher than usual, for example during the Dot-com Bubble in the early 2000s and the Great Recession in 2007–2008. Our dataset distribution reflects this condition, see Table 1.
For all the companies and for each year, we selected 18 accounting and financial variables. Features were selected according to the most frequently used ratios, and accounting information to which the literature refers [21,23,38]. The dataset has no missing values or synthetic and imputed added values. Finally, the resulting dataset of 78,682 firm-year observations is divided into three subsets according to the period of time: a training set, a validation set, and a test set. We used data from 1999 until 2011 for training, data from 2012 until 2014 for validation and model comparison, and the remaining years from 2015 to 2018 as a test-set to prove the ability of the models to predict bankruptcy in real never seen cases and macroeconomic conditions. Table 2 shows the full list of the 18 features available in the dataset and their description.
In light of this, the dataset could be used to build and validate different ML models for both of the main two tasks in bankruptcy prediction we show in this research work. Moreover, since the dataset has a temporal dimension, several time series analysis techniques could be exploited as long as there are unsupervised methodologies.

4. Machine-Learning Models

In this section, we briefly review and describe all the machine-learning algorithms we used for the experiments described in the next sections.

4.1. Support Vector Machine

Support Vector Machine is one of the oldest ML algorithms and aims to identify the decision boundaries as the maximum-margin hyperplane separating two classes. The hyperplane equation is given by Equation (1).
f ( x ) = ω T · x + b
where ω is the normal vector and b the bias. The objective function of SVM can be expressed as:
min 1 2 ω 2 s . t . y i ω T x i b ε ω T x i + b y ε
where ϵ is the deviation between f ( x ) and the target y i .

4.2. Random Forest

Random Forest is an ensemble learning algorithm developed by Breiman [39]. Ensemble learning is a way to combine different basic classifiers (“weak” classifiers) to compose a new one (strong learner), more complex, more efficient, and more precise. The weak classifiers should make independent errors in their predictions, and thus a strong classifier can be composed of different algorithms or if the same algorithm is used the models should be trained with different subsets of the training set.
Random Forest is an ensemble bagging tree-based learning algorithm. In particular, the Random Forest Classifier is a set of decision trees that are trained using randomly selected subsets of the training set and randomly selected subsets of features.
A Random Forest Classifier is composed of a collection of classification trees h (estimators):
{ h ( x , T , Θ k ) , k = 1 , 2 , , K }
where ( Θ k ) represents identically and independently distributed random vectors, and each tree casts a unit vote for the most likely class at input x. Each tree in the collection votes (only once) to assign the sample to a class, considering the x feature vector. The final choice is to attribute the example to the class that obtained the majority of votes. A graphic representation of the algorithm is presented in Figure 2.

4.3. Boosting Algorithms

Boosting is a subset of ensemble methods where a collection of models are trained sequentially to permit every model to improve and compensate for the weakness of its predecessor.
Boosting algorithms differ in how they create and aggregate weak learners during the sequential stacking process. In our work, we used various boosting algorithms:
  • AdaBoost [40]: This was the first boosting algorithm developed for classification and regression tasks. It fits a sequence of weak learners on different weighted training data. It gives incorrect predictions more weight in sub-sequence iterations and less weight to correct predictions. In this way, it forces the algorithm to “focus” on observations that are harder to predict. The final prediction comes from weighing the majority vote or sum.
    The algorithm begins by forecasting the original dataset and giving the same weight to each observation. If the prediction is incorrect, using the first “weak” learner, the algorithm will give a higher weight value to that observation. This procedure is iterated until the model reaches a predefined value of accuracy.
    AdaBoost is typically easy to use because it does not need complex parameters during its tuning procedures and it shows low sensitivity to overfitting. Moreover, it is able to learn from a small set of features learning incrementally new information. However, Adaboost is sensitive to noisy data and abnormal values.
  • Gradient Boosting [41]: This algorithm uses a set of weak predictive models, typically decision trees. Gradient Boosting trains many models sequentially that are then composed using the additive modeling property. In each training epoch, a new learner is added to increase the accuracy of the previous one. Each model gradually minimizes the whole system loss function using the Gradient Descent algorithm.
  • XGBoost (Extreme Gradient Boosting) [42]: XGBoost is an optimized distributed Gradient Boosting library designed to be highly efficient, flexible, and portable. XGBoost minimizes a regularized (L1 and L2) objective function that combines a convex loss function based on the difference between the predicted and target outputs and a penalty term for model complexity. The training proceeds by adding new trees that predict the residuals or errors of prior trees that are then combined with previous trees to make the final prediction (4).
    F m ( X ) = F m 1 ( X ) + α m h m X , r m 1 ,
    where α i and r i are the regularization parameters and residuals computed with the i-th tree, respectively, and h i is a function that is trained to predict residuals, r i using X for the i-th tree.
    α i is computed using the residuals, r i solving the following optimization problem:
    arg   min α = i = 1 m L Y i , F i 1 X i + α h i X i , r i 1
    where L(Y, F(X)) is a differentiable loss function.

4.4. Logistic Regression

Binary Logistic Regression models the relationship between a set of independent variables and a binary dependent variable. The goal is to find the best model that describes the relationship between the dependent variable and multiple independent variables.
The Logistic Regression’s dependent variable could be binary or categorical and the independent ones could be a mixture of continuous, categorical, and binary variables.
The general form of Logistic Regression is as follows:
z = a + b 1 x 1 + b 2 x 2 + b 3 x 3 + + b m x m
y = 1 ( 1 + e z )
where x 1 , x 2 , , x m is the feature vector and z is a linear combination function of the features. The parameters b 1 , b 2 , , b m are the regression coefficients to be estimated. The output is between 0 and 1, and, usually, if the output is above the threshold of 0.5 the model predicts class 1 (positive) and otherwise class 0 (negative).

4.5. Artificial Neural Network

An Artificial Neural Network (ANN) is a non-linear function approximator. It consists of an input layer of neurons and an unspecified number of hidden layers and a final output layer. Each neuron performs a weighted sum of its inputs, and finally, it applies an activation function that determines the output of each neuron. When the activation function is a Sigmoid function, the single neuron works as a Logistic Regression without the classification threshold. Figure 3 shows a general structure of an ANN with the input layer, two hidden layers, and the final output layer whose structure strongly depends on the task it should perform. The main common architecture is the feed-forward ANN where each neuron is linked to every neuron in the next layer but without exhibiting any intra-layer connection among neurons belonging to the same layer. Each layer can be seen as a partial approximation of the final function.
Each connection has a weight ω assigned that is randomly initialized at the beginning. The output h i , of a neuron i, in the hidden layer, is:
h i = σ ( j = 1 N ω i j x j + b i )
where σ ( ) is the activation function, N the number of input neurons of the layer, ω i j the weights, x j inputs of the neurons, and b i the bias terms of the hidden neurons.
The goal of the activation function is to bound the value of the neuron so that the Neural Network is not stuck by divergent neurons. Weight estimation for each connection is the main goal of the ANN’s training. This step is usually performed using the back-propagation algorithm [43] that minimizes an objective function that measures the distance between the desired and actual output of the network. Inputs and outputs from a Neural Network can be binary or even symbols when data are appropriately encoded. This feature confers a wide range of applicability to Neural Networks.

5. Metrics for Imbalanced Bankruptcy Prediction Tasks

The two bankruptcy prediction tasks we propose in the following sections have been implemented as binary prediction tasks where the positive class (1) indicates bankruptcy while the negative class (0) means that a company has been classified as healthy. To compare our models and investigate their effectiveness, we used different metrics considering the imbalance condition in the validation and the test sets. These metrics will be critically discussed in Section 8 in light of our results for the two tasks. According to the binary classification task we used to formulate the bankruptcy prediction problem, the following variables represent:
  • True positive (TP): The number of bankrupted companies that have been correctly predicted as such.
  • False negative (FN): The number of bankrupted companies that have been wrongly predicted as healthy firms.
  • True negative (FN): The number of actually healthy companies that have been correctly predicted as such.
  • False positive (FP): The number of healthy companies that have been wrongly predicted as bankrupted by the model.
Since the validation and test sets are both imbalanced for both tasks with a prevalence of healthy companies (∼96–97%), we did not compare the models in terms of accuracy of the model. Indeed, the proportion of correct matches would be insufficient in assessing the model performance. We firstly use the Area Under the Curve (AUC) for all the comparisons as it is commonly used in the literature to compare the performance of models on imbalanced datasets and specifically to evaluate the bankruptcy models in general. AUC measures the ability of a classifier to distinguish between classes and is used as a summary of the Receiver Operating Characteristic (ROC) curve. The ROC curve is created by plotting the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings.
In addition, we investigated other important metrics that can be used to clarify the models’ performance depending on the target stakeholders. These are the precision, recall, and F 1 scores for each class. The precision achieved for a class is the accuracy of that class’ predictions. The recall (sensitivity) is the ratio of the class instances that are correctly detected by the classifier. The F 1 score is the harmonic mean of precision and recall: whereas the regular mean treats all values equally, the harmonic one gives more weight to low values. As a consequence, a high F 1 score for a certain class is achieved only if both its precision and recall are high. Equations (11)–(13) report how these quantities are computed for the positive class. The definitions for the negative class are exactly the same by inverting positives with negatives.
P r e c i s i o n = T P ( T P + F P )
R e c a l l = T P ( T P + F N )
F 1 s c o r e = 2 1 P r e c i s i o n + 1 R e c a l l
Moreover, we computed and reported two other global metrics for the classifier that have been selected because they enable an overall evaluation of the classifier on both classes:
  • The macro- F 1 score: The macro- F 1 score is computed as the arithmetic mean of the F 1 score of all the classes.
  • The micro- F 1 score: It is used to assess the quality of multi-label binary problems. It measures the F 1 score of the aggregated contributions of all classes.
Finally, we used two other metrics that are often evaluated in the bankruptcy prediction models. Because bankruptcy is a rare event, using the classification accuracy to measure a model’s performance is misleading since it assumes that type I error (Equation (12)) and type II error (Equation (13)) are equally costly. Actually, the cost of false negatives is much higher than that of false positives for a financial institution. An error has a different cost depending on the class that has been incorrectly predicted. The cost of predicting a company going into default as healthy is much higher than the cost of predicting a company that will default as healthy. In light of this, we explicitly computed and reported type I and type II errors and compared the classifiers focusing in particular on type II and recall of the positive class.
T y p e I e r r o r = F P T N + F P
T y p e I I e r r o r = F N T P + F N

6. Task T1: Default Prediction with Historical Time Series

The first task we performed using the dataset is training each ML model presented in Section 4 for default prediction with historical time-series accounting data. Our first step was to perform the most classical task of predicting a company’s health status for the next year based solely on data from the previous year. Furthermore, we attempted to answer the open question in the literature regarding the number of years that should be considered in order to maximize the performance of the bankruptcy prediction model. To achieve this, we define a Window Length (WL) variable that refers to the number of fiscal years considered as a temporal window for the prediction. We trained all the models using data between 1999 and 2011, and we made the first comparison using the Validation set (2012–2014) in terms of AUC.
Finally, we report the results of the test set using the best models identified on the validation set. This last step is necessary to verify the ability of the models to generalize. The average number of years available for each company in the dataset is 8 years. However, we evaluated a WL ranging from one to five years for two main reasons:
  • Five years is the general maximum number of years found in the literature to be useful.
  • When increasing the WL considered, the most recent companies are excluded because they do not have available data. In general, considering more years leads to smaller training and test sets. This could introduce a statistical bias, causing the analysis to focus on only the more mature and stable companies that have existed for several years while ignoring the relatively newer, smaller companies that are riskier and have higher default rates. This could introduce a statistical bias forcing the analysis to consider only the more structured and stable companies that have existed for several years while ignoring the relatively new companies with smaller market capitalization and which are thus riskier and have a higher probability of default, particularly during periods of economic decline.
Figure 4 shows the distribution of the two classes of bankruptcy and healthy samples for the training, test, and validation by selecting different Window Lengths (WLs).

6.1. Models Comparison and Selection for Default Prediction

All the models were trained using the same training set (1999–2011) and compared using the same validation set (2012–2014). However, learning from an imbalanced training set led to unsatisfactory results for the bankruptcy class in the validation set. For this reason, we decided to use a random balanced training set that is evaluated for different runs. Indeed, every model is evaluated over 100 independent and different runs: for every run, the training set is balanced with exactly the same number of bankruptcy examples and a random choice of healthy examples from the same period. The number of features changes according to the number of available variables for the temporal window length selected.
A binary classification task is implemented in each model, where the positive class (1) represents a bankruptcy case in the next year, and the negative class (0) represents a healthy case. For RF, AB, GB, and XGB, we used 500 estimators for a fair comparison, while the other specific parameters are the default ones provided in the Scikit-Learn implementations. The ANN is a multi-layer perceptron with three layers. In the first layer, the number of neurons is equal to the number of input features. The hidden layer has half as many neurons as the first layer, and finally, the output layer uses a Sigmoid function to produce a binary prediction. All neurons except for those in the output layer use the Rectified Linear Unit activation (RELU) function. Each Neural Network is trained over 1000 epochs using the early-stopping regularization technique that prevents overfitting when the validation loss tends to increase for a patience number of times.
We compared all the models using the average Area Under the Curve (AUC) over 100 runs. Figure 5 reports the results we achieved trying to predict bankruptcy in the most common setting that exploits only the accounting variables of the last available fiscal year. On average, the ensemble tree-based ML models perform better than the others except for Adaboost. The best average result is achieved with the Random Forest ( A U C = 0.748 ), although the Neural Network achieves the best AUC in a single run ( A U C = 0.856 ). However, the ANN exhibits the highest variability in the results, which led to a lower average AUC ( A U C = 0.653 ). Figure 6 reports the comparison among the models when using more than one year from the prediction (WL from two to five years). As a result of this experiment, we can assert that on average increasing the number of fiscal years in the input does not seem to impact the average AUC significantly:
  • For WL = 2, best average AUC = 0.75 with Random Forest
  • For WL = 3, best average AUC = 0.72 with Random Forest and Gradient Boosting
  • For WL = 4, best average AUC = 0.74 with XGBoost
  • For WL = 5, best average AUC = 0.754 with Random Forest
The highest average value is reached by Random Forest using a window length of 5 years. However, as in the first case, in general, the ensemble models exhibit better performance. However, the best absolute result is reached by the Neural Network again with A U C = 0.85 . It should be highlighted that the Neural Network’s high variability concerning the results can be imputed to the random weights’ initialization for every run. Probably performing several runs over the same training set could reduce this problem. Finally, Table 3 shows the variance for each model considering every window length.

6.2. Results for Default Prediction

Once we compared the algorithms with 100 different runs on the validation set, ranked the average AUC achieved for every temporal window, and selected the best model on the validation set, we performed a final evaluation on the test set (2015–2018). These data have never been used for model training or comparison. It is important to evaluate on a different test set in terms of temporal period because it refers to a different economic cycle. Since we trained on data until 2011 that include the 2007/2008 sub-prime crisis and the European debt crisis in 2010–2011, there could be a bias concerning the knowledge learnt by the models about what is a company that is likely to go into bankruptcy. Evaluating all the models only on the validation set is not enough since we used that dataset to select the hyper-parameters. If a model performs well on both validation (2012–2014) and on the never used test set (2015–2018), we could asses that it is effectively able to generalize concerning bankruptcy companies. The best models on the validation set are in order Random Forest, Gradient Boosting, XGBoost, and Logistic Regression. Figure 7 presents the final results we achieved for this task on the American stock market. We also added the results achieved by the Neural Network with the best model that actually outperforms all the other machine-learning techniques in terms of AUC. It appears that this result contradicts what has been reported in the current literature, which is that bagging and boosting ensemble models perform better. The results achieved on the never seen test set by the ANN demonstrate what is currently known for other machine-learning applications: When a Neural Network is properly designed, trained, and fine-tuned with higher computational costs and when the best model is identified with random and grid searches, then Neural Networks usually outperform all the machine-learning baselines thanks to their ability to deal with non-linear dependencies.

7. Task T2: Survival Probability Task

The second task we performed using the dataset is the years anticipated prediction of the bankruptcy events. This task is implemented by considering the event in the dataset and looking ahead a number of years; we named it the Look-Ahead Window (LAW). In practice, default prediction with WL = 1 is exactly the same as the one with LAW = 1. For this reason we investigated the LAW parameter ranging from two to five years. The main difference between task T1 and this task is that for task T2 the models exploit only a single year of accounting variables in the past depending on the LAW parameter. For example if LAW is selected equal to three years, it means that the model learns how to predict the companies’ status looking over three years. This way of predicting bankruptcy is usually adopted to estimate the survival probability of a company within some years. All the experiments are conducted with the same methodological framework as for task T1. We trained all the models for 100 different runs by randomly undersampling the number of healthy examples in the training set in order to have a balanced set. We compared all the models on the validation set to select the best ones by varying the LAW parameter. Models settings are the same as for task T1. Figure 8 shows the models comparison. We remind readers that the validation set is unbalanced and this is the reason why the performances are measured in terms of the AUC. The results are similar to task T1. Adaboost achieves the worst performance on average along with Support Vector Machine for all the LAW parameters. The other ensemble models seem to outperform the other models and exhibit a small variance by changing the training set for each run. The Neural Network achieves the best absolute result for each window but presents a high performance variance. However, one should highlight that, in general, all the models achieve a better average AUC for the survival probability task rather than on the classical bankruptcy prediction (task T1). Table 3 shows the variance calculated on the AUC values on the validation set for each look-ahead window.

Results for the Survival Probability Prediction

As already carried out for task T1, we selected the best models on the validation set, and we measured the generalization ability of the models on the test set (2015–2018). We compared GB, XGB, LR, and RF. As expected from the previous models’ comparison, the best model is Random Forest also in this case. However, also for this task, the best ANN model found on the validation set definitely outperforms all the other models, showing an additional demonstration of the better ability of this category of models to achieve better performance when properly designed. In general, the results of both validation and test experiments suggest that machine learning can better predict a company’s status over a long period (from 3 to 5 years before) rather than over a short period (1 or 2 years). Results are presented in Figure 9.

8. Results

In this section, we present a deeper analysis of the results for both tasks T1 and T2 by also discussing the other metrics available. In Section 6 and Section 7, we compared all the models in terms of the AUC, and we identified the best model for both tasks using this metric. This metric was chosen for two reasons: first, due to the imbalance test set, but also because it is most commonly used in literature for this task. Now, we would like to analyze the same results by looking at the other metrics in order to find some robust answers about the models for the two tasks. Table 4 and Table 5 show all the results we achieved with all the models on the test set (2015–2018).
We show precision and recall for both the bankruptcy class and the healthy one. Starting from these, we computed the micro- and macro- F 1 scores for the overall classifier and finally the type I and type II errors. All the results were computed for all the temporal window WLs and LAWs of the two tasks.
In the end, we reported the complete confusion matrix of the experiments that we have never seen reported in the other papers because we strongly believe that this may help for correct comparisons and usage as benchmarks for future investigations.

8.1. The Most Suitable Metric

The first question we aim to answer with the experiments is the following: Does a high value of AUC mean that the model is better at predicting bankruptcy or survival probability over time? There is not a unique and simple answer. We compared the overall performance of the models using the AUC and the micro- F 1 score, and the two metrics present similar results both with high values: models that exhibit a high AUC also have a high micro- F 1 score. This direct proportionality is also evident with the macro- F 1 score, which we remind readers is the arithmetic average of the simple F 1 scores on the two classes, but the value is usually definitely lower.
The reason why the macro- F 1 score is much lower is related to the low precision that the models achieve in the bankruptcy class. All the best models selected with the highest value of AUC exhibit a high recall and a low precision for the bankruptcy class and this is not evident looking only at the AUC value that often is reported as something similar to the accuracy. The precision–recall trade-off is well known in the literature: there is an inverse proportionality between the two metrics since precision considers false positives and recall considers false negatives. If the model is good at predicting true positives with a low value of false negatives it will often predict negative samples as positive (High False Positive and so low precision). Moreover, in our dataset and in most of the bankruptcy datasets, the class imbalance is usually very pronounced, with an average of 97% of healthy (negative) samples in the test set versus a small 3% of bankruptcy (positive) samples. For this reason, we can assert that the precision over the bankruptcy class strongly depends on the absolute value of the negative examples in the test set. The reader may observe this dependence by looking at the confusion matrices reported in the two tables.
At the same time, this condition can be considered optimal for some financial stakeholders: If the model has a high recall on the bankruptcy class it will make some wrong predictions on healthy companies but ensure that most of the risky companies are detected and avoided in their investments.
The type II error is the other metric mostly related to the recall of the bankruptcy class, but it considers the ratio of false negatives (default companies that have been wrongly classified as healthy): to minimize this value, it is important to reduce the number of false negatives while for the recall we want to maximize the number of true positives. However, these two metrics should always be highlighted along with the AUC because they provide concrete insight into the ability of the model to correctly identify companies that are going to face financial troubles. Indeed, a higher value of AUC is due to the high precision and recall that the models achieve in the healthy (negative) class since it is over-represented.
The final user, particularly if it is a regulatory body that is responsible for monitoring the status of a company, should not blindly trust the AUC value since all the models selected on this basis may produce several false alarms (low precision for the bankruptcy class, high values of false positives). In this case, they should design and compare the ML models using the macro- F 1 score and then the AUC.

8.2. Best Model and Temporal Windows

In light of our results, we can assert that Neural Networks should be preferred among all the ML models for the bankruptcy prediction tasks since they present the best ability to generalize on new and unseen cases with every metric adopted. However, the training and design of ANNs is usually harder and requests a larger computation time, higher costs, and more experiments with respect to the other models. When computation time and costs are constraints to be kept in account or when the model should be used in high dynamic contexts, the Random Forest algorithm should be preferred since it presents almost similar performance, but it requires fewer parameters, and thus it takes less time to be designed and trained.
The temporal window analysis for tasks T1 and T2 led to the following conclusions:
  • For the default prediction task (T1), the general performance increases when considering more than one year of accounting variables and this is true for both ANNs and Random Forest. Indeed, the 5-year temporal window exhibits the best results in terms of AUC. However, in light of the discussion about the metrics, the temporal window of three years achieves the best trade-off between AUC and macro- F 1 score using fewer variables.
  • For the survival probability prediction task (T2), the best performance is achieved when trying to predict the company status three years in advance (LAW = 3) with an AUC = 0.87 with the ANN. It should be highlighted that ANN reached for LAW = 5 a considerable AUC = 0.86. However, all the models exhibit a really low precision on the bankruptcy class except for SVM and XGBoost. Indeed the best model in terms of macro- F 1 score is XGBoost (LAW = 5)
In general, learning from temporal variables seems to lead to better performance, especially when the model learning capacity can learn complex patterns as happens with Neural Networks.

9. Conclusions

In this research work, we deeply investigated the performance of several machine-learning techniques concerning predicting bankruptcy in the American stock market. We compared the models over two different tasks: (a) default prediction using time series accounting data; (b) survival probability prediction. We performed the tasks using a dataset with 8262 companies in the period between 1999 and 2018. The dataset is also one of the contributions of this paper since it has been publicly released. We used temporal criterion to divide the dataset into training, validation, and test sets. For both tasks, Neural Networks achieve the absolute best results despite exhibiting a great variance in their results, leading to the conclusion that these models can be superior only when opportunely designed and trained with higher computational costs. Finally, we critically discuss the general use of the Area Under the Curve as a common metric to evaluate bankruptcy prediction tasks since in most cases computing precision, recall for the single classes, and the macro- F 1 score would better define the models’ performance. Moreover, we highlighted that using more fiscal years for the prediction can improve the performance for both tasks, as has been proved in the past only for small datasets. In light of this, future works are to be related to the use of Recurrent Neural Networks and attention-models to better exploit the time-series information, considering the possible trade-off of using deep learning models with such short time series and relatively small datasets. Moreover, bankruptcy prediction could also be evaluated with unsupervised models like Isolation Tree and anomaly detection models. The current dataset could also help in that case. Finally, future works are to be related to the possible limitations of this research work. The main issue to be deeply investigated is related to the temporal dimension of the study: we were able to collect reliable data until 2018 and testing on previously unseen example has been promising. However, it would be interesting to evaluate if the current models could also generalize on different economic situations like the ones that come up with the COVID-19 pandemic and the resource crisis. Another limitation of this work is related to the class imbalance. Several techniques of effective sampling methods should be considered in the study in order to evaluate a balanced scenario as long as there is synthetic data generation.

Author Contributions

Conceptualization, G.L.; Methodology, G.A.; Software, M.P.; Supervision, P.M.P. and A.P.; Writing—review & editing, S.C. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Danilov, C.; Konstantin, A. Corporate Bankruptcy: Assessment, Analysis and Prediction of Financial Distress, Insolvency, and Failure. Available online: (accessed on 14 August 2022).
  2. Ding, A.A.; Tian, S.; Yu, Y.; Guo, H. A class of discrete transformation survival models with application to default probability prediction. J. Am. Stat. Assoc. 2012, 107, 990–1003. [Google Scholar] [CrossRef]
  3. Prusak, B. Review of research into enterprise bankruptcy prediction in selected central and eastern European countries. Int. J. Financ. Stud. 2018, 6, 60. [Google Scholar] [CrossRef][Green Version]
  4. Zięba, M.; Tomczak, S.K.; Tomczak, J.M. Ensemble boosted trees with synthetic features generation in application to bankruptcy prediction. Expert Syst. Appl. 2016, 58, 93–101. [Google Scholar] [CrossRef]
  5. Mai, F.; Tian, S.; Lee, C.; Ma, L. Deep learning models for bankruptcy prediction using textual disclosures. Eur. J. Oper. Res. 2019, 274, 743–758. [Google Scholar] [CrossRef]
  6. Adosoglou, G.; Park, S.; Lombardo, G.; Cagnoni, S.; Pardalos, P.M. Lazy Network: A Word Embedding-Based Temporal Financial Network to Avoid Economic Shocks in Asset Pricing Models. Complexity 2022, 2022, 9430919. [Google Scholar] [CrossRef]
  7. Wilkinson, M.D.; Dumontier, M.; Aalbersberg, I.J.; Appleton, G.; Axton, M.; Baak, A.; Blomberg, N.; Boiten, J.W.; da Silva Santos, L.B.; Bourne, P.E.; et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci. Data 2016, 3, 160018. [Google Scholar] [CrossRef][Green Version]
  8. Thakur, N.; Han, C.Y. A study of fall detection in assisted living: Identifying and improving the optimal machine learning method. J. Sens. Actuator Netw. 2021, 10, 39. [Google Scholar] [CrossRef]
  9. Gandomi, A.H.; Chen, F.; Abualigah, L. Machine learning technologies for big data analytics. Electronics 2022, 11, 421. [Google Scholar] [CrossRef]
  10. Schönfeld, J.; Kuděj, M.; Smrčka, L. Financial health of enterprises introducing safeguard procedure based on bankruptcy models. J. Bus. Econ. Manag. 2018, 19, 692–705. [Google Scholar] [CrossRef][Green Version]
  11. Moscatelli, M.; Parlapiano, F.; Narizzano, S.; Viggiano, G. Corporate default forecasting with machine learning. Expert Syst. Appl. 2020, 161, 113567. [Google Scholar] [CrossRef]
  12. Danenas, P.; Garsva, G. Selection of Support Vector Machines based classifiers for credit risk domain. Expert Syst. Appl. 2015, 42, 3194–3204. [Google Scholar] [CrossRef]
  13. du Jardin, P. A two-stage classification technique for bankruptcy prediction. Eur. J. Oper. Res. 2016, 254, 236–252. [Google Scholar] [CrossRef]
  14. Tsai, C.F.; Hsu, Y.F.; Yen, D.C. A comparative study of classifier ensembles for bankruptcy prediction. Appl. Soft Comput. 2014, 24, 977–984. [Google Scholar] [CrossRef]
  15. Wang, G.; Ma, J.; Yang, S. An improved boosting based on feature selection for corporate bankruptcy prediction. Expert Syst. Appl. 2014, 41, 2353–2361. [Google Scholar] [CrossRef]
  16. Zhou, L.; Lai, K.K.; Yen, J. Bankruptcy prediction using SVM models with a new approach to combine features selection and parameter optimisation. Int. J. Syst. Sci. 2014, 45, 241–253. [Google Scholar] [CrossRef]
  17. Bottani, E.; Mordonini, M.; Franchi, B.; Pellegrino, M. Demand Forecasting for an Automotive Company with Neural Network and Ensemble Classifiers Approaches. In IFIP International Conference on Advances in Production Management Systems; Springer: Berlin/Heidelberg, Germany, 2021; pp. 134–142. [Google Scholar]
  18. Geng, R.; Bose, I.; Chen, X. Prediction of financial distress: An empirical study of listed Chinese companies using data mining. Eur. J. Oper. Res. 2015, 241, 236–247. [Google Scholar] [CrossRef]
  19. Alfaro, E.; García, N.; Gámez, M.; Elizondo, D. Bankruptcy forecasting: An empirical comparison of AdaBoost and Neural Networks. Decis. Support Syst. 2008, 45, 110–122. [Google Scholar] [CrossRef]
  20. Bose, I.; Pal, R. Predicting the survival or failure of click-and-mortar corporations: A knowledge discovery approach. Eur. J. Oper. Res. 2006, 174, 959–982. [Google Scholar] [CrossRef]
  21. Tian, S.; Yu, Y.; Guo, H. Variable selection and corporate bankruptcy forecasts. J. Bank. Financ. 2015, 52, 89–100. [Google Scholar] [CrossRef]
  22. Wanke, P.; Barros, C.P.; Faria, J.R. Financial distress drivers in Brazilian banks: A dynamic slacks approach. Eur. J. Oper. Res. 2015, 240, 258–268. [Google Scholar] [CrossRef]
  23. Altman, E.I. Financial ratios, discriminant analysis and the prediction of corporate bankruptcy. J. Financ. 1968, 23, 589–609. [Google Scholar] [CrossRef]
  24. Altman, E.I.; Hotchkiss, E.; Wang, W. Corporate Financial Distress, Restructuring, and Bankruptcy: Analyze Leveraged Finance, Distressed Debt, and Bankruptcy; John Wiley & Sons: Hoboken, NJ, USA, 2019. [Google Scholar]
  25. Kralicek, P. Fundamentals of Finance: Balance Sheets, Profit and Loss Accounts, Cash Flow, Calculation Bases, Financial Planning, Early Warning Systems; Ueberreuter: Berlin, Germany, 1991. [Google Scholar]
  26. Taffler, R.J.; Tisshaw, H. Going, going, gone–four factors which predict. Accountancy 1977, 88, 50–54. [Google Scholar]
  27. Ohlson, J.A. Financial ratios and the probabilistic prediction of bankruptcy. J. Account. Res. 1980, 18, 109–131. [Google Scholar] [CrossRef][Green Version]
  28. Beaver, W.H. Financial ratios as predictors of failure. J. Account. Res. 1966, 4, 71–111. [Google Scholar] [CrossRef]
  29. Wang, G.; Ma, J.; Huang, L.; Xu, K. Two credit scoring models based on dual strategy ensemble trees. Knowl.-Based Syst. 2012, 26, 61–68. [Google Scholar] [CrossRef]
  30. Nanni, L.; Lumini, A. An experimental comparison of ensemble of classifiers for bankruptcy prediction and credit scoring. Expert Syst. Appl. 2009, 36, 3028–3033. [Google Scholar] [CrossRef]
  31. Kim, M.J.; Kang, D.K. Ensemble with Neural Networks for bankruptcy prediction. Expert Syst. Appl. 2010, 37, 3373–3379. [Google Scholar] [CrossRef]
  32. Wang, G.; Hao, J.; Ma, J.; Jiang, H. A comparative assessment of ensemble learning for credit scoring. Expert Syst. Appl. 2011, 38, 223–230. [Google Scholar] [CrossRef]
  33. Barboza, F.; Kimura, H.; Altman, E. Machine-learning models and bankruptcy prediction. Expert Syst. Appl. 2017, 83, 405–417. [Google Scholar] [CrossRef]
  34. Mossman, C.E.; Bell, G.G.; Swartz, L.M.; Turtle, H. An empirical comparison of bankruptcy models. Financ. Rev. 1998, 33, 35–54. [Google Scholar] [CrossRef]
  35. Duan, J.C.; Sun, J.; Wang, T. Multiperiod corporate default prediction—A forward intensity approach. J. Econom. 2012, 170, 191–209. [Google Scholar] [CrossRef]
  36. Kim, H.; Cho, H.; Ryu, D. Corporate default predictions using machine learning: Literature review. Sustainability 2020, 12, 6325. [Google Scholar] [CrossRef]
  37. Adosoglou, G.; Lombardo, G.; Pardalos, P.M. Neural Network embeddings on corporate annual filings for portfolio selection. Expert Syst. Appl. 2021, 164, 114053. [Google Scholar] [CrossRef]
  38. Campbell, J.Y.; Hilscher, J.; Szilagyi, J. In search of distress risk. J. Financ. 2008, 63, 2899–2939. [Google Scholar] [CrossRef][Green Version]
  39. Breiman, L. Random Forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef][Green Version]
  40. Freund, Y.; Schapire, R.E. A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. J. Comput. Syst. Sci. 1997, 55, 119–139. [Google Scholar] [CrossRef][Green Version]
  41. Friedman, J.H. Greedy function approximation: A Gradient Boosting machine. Ann. Stat. 2001, 29, 1189–1232. [Google Scholar] [CrossRef]
  42. Chen, T.; He, T. Xgboost: Extreme Gradient Boosting. Available online: (accessed on 14 August 2022).
  43. Rumelhart, D.E.; Hinton, G.E.; Williams, R.J. Learning representations by back-propagating errors. Nature 1986, 323, 533–536. [Google Scholar] [CrossRef]
Figure 1. Rate of bankruptcy in the dataset (2000–2019) with financial variables in the period (1999–2018). The next subdivision in training, validation, and test is highlighted with different colors.
Figure 1. Rate of bankruptcy in the dataset (2000–2019) with financial variables in the period (1999–2018). The next subdivision in training, validation, and test is highlighted with different colors.
Futureinternet 14 00244 g001
Figure 2. Graphic representation of Random Forest algorithm for classification.
Figure 2. Graphic representation of Random Forest algorithm for classification.
Futureinternet 14 00244 g002
Figure 3. Representation of a generic Neural Network. Nodes are called neurons and the links are called weights.
Figure 3. Representation of a generic Neural Network. Nodes are called neurons and the links are called weights.
Futureinternet 14 00244 g003
Figure 4. Bankruptcy and healthy sample distribution in the training (2000–2011), validation (2012–2014), and test (2015–2018) sets by varying the length of the window (WL) considered.
Figure 4. Bankruptcy and healthy sample distribution in the training (2000–2011), validation (2012–2014), and test (2015–2018) sets by varying the length of the window (WL) considered.
Futureinternet 14 00244 g004
Figure 5. The box plot shows the locality, spread, and skewness groups of AUC values through their quartiles achieved for each model in 100 different runs with a different balance training set. The reported AUC values (horizontal axis) are related to the case of WL = 1 where all models use only variables from the last year. The lines (whiskers) extending from the boxes indicate the variability outside the upper and lower quartiles. The orange line represents the median between the first quartile and the third quartile. The circular points (fliers) are those located past the end of the whiskers.
Figure 5. The box plot shows the locality, spread, and skewness groups of AUC values through their quartiles achieved for each model in 100 different runs with a different balance training set. The reported AUC values (horizontal axis) are related to the case of WL = 1 where all models use only variables from the last year. The lines (whiskers) extending from the boxes indicate the variability outside the upper and lower quartiles. The orange line represents the median between the first quartile and the third quartile. The circular points (fliers) are those located past the end of the whiskers.
Futureinternet 14 00244 g005
Figure 6. The four box-plots show the locality, spread, and skewness groups of AUC values achieved on the validation set for each model in the same conditions as for the first case by varying the WL parameter.
Figure 6. The four box-plots show the locality, spread, and skewness groups of AUC values achieved on the validation set for each model in the same conditions as for the first case by varying the WL parameter.
Futureinternet 14 00244 g006
Figure 7. AUCs achieved by the best models on the test set (2015–2018) for task T1 using the best models selected on the validation set (2012–2014). The ANN is also considered because, although it yields the worst average AUC on the validation set, it achieves the best absolute performance on the test set when using the best models found on the validation set.
Figure 7. AUCs achieved by the best models on the test set (2015–2018) for task T1 using the best models selected on the validation set (2012–2014). The ANN is also considered because, although it yields the worst average AUC on the validation set, it achieves the best absolute performance on the test set when using the best models found on the validation set.
Futureinternet 14 00244 g007
Figure 8. The four box plots show the locality, spread, and skewness groups of AUC values achieved for each model on the validation set for the survival probability task (T2) when the Look-Ahead Window (LAW) is changed between 2 and 5 years.
Figure 8. The four box plots show the locality, spread, and skewness groups of AUC values achieved for each model on the validation set for the survival probability task (T2) when the Look-Ahead Window (LAW) is changed between 2 and 5 years.
Futureinternet 14 00244 g008
Figure 9. AUCs achieved by the best models on the test set (2015–2018) for task T2 (survival probability over time) prediction using the best models selected on the validation set (2012–2014).
Figure 9. AUCs achieved by the best models on the test set (2015–2018) for task T2 (survival probability over time) prediction using the best models selected on the validation set (2012–2014).
Futureinternet 14 00244 g009
Table 1. The table provides the firm distribution by year in the dataset.
Table 1. The table provides the firm distribution by year in the dataset.
YearTotal FirmsBankrupt FirmsYearTotal FirmsBankrupt Firms
Table 2. The 18 numerical bankruptcy features we considered in our tests.
Table 2. The 18 numerical bankruptcy features we considered in our tests.
Variable NameDescription
X1Current assetsAll the assets of a company that are expected to be sold
or used as a result of standard business operations over
the next year
X2Cost of goods soldThe total amount a company paid as a cost directly
related to the sale of products
X3Depreciation and
Depreciation refers to the loss of value of a tangible
fixed asset over time (such as property. machinery,
buildings, and plant). Amortization refers to the
loss of value of intangible assets over time.
X4EBITDAEarnings before interest, taxes, depreciation and
amortization: Measure of a company’s overall
financial performance alternative to the
net income
X5InventoryThe accounting of items and raw materials
that a company either uses in production or sells.
X6Net IncomeThe overall profitability of a company
after all expenses and costs have been deducted
from total revenue.
X7Total ReceivablesThe balance of money due to a firm for goods
or services delivered or used but not yet paid for
by customers.
X8Market valueThe price of an asset in a marketplace. In our
dataset it refers to the market capitalization since
companies are publicly traded in the stock market
X9Net salesThe sum of a company’s gross sales minus its
returns, allowances, and discounts.
X10Total assetsAll the assets, or items of value, a business owns
X11Total Long term debtA company’s loans and other liabilities
that will not become due within one year of the
balance sheet date
X12EBITEarnings before interest and taxes
X13Gross ProfitThe profit a business makes after subtracting
all the costs that are related to manufacturing and
selling its products or services
X14Total Current LiabilitiesIt is the sum of accounts payable, accrued liabilities
and taxes such as Bonds payable at the end of the
year, salaries and commissions remaining
X15Retained EarningsThe amount of profit a company has left over after
paying all its direct costs, indirect costs, income
taxes and its dividends to shareholders
X16Total RevenueThe amount of income that a business has made from
all sales before subtracting expenses. It may include
interest and dividends from investments
X17Total LiabilitiesThe combined debts and obligations that the company
owes to outside parties
X18Total Operating ExpensesThe expense a business incurs through
its normal business operations
Table 3. Variance on the AUC values calculated for each model on the validation set for T2 task (LAW) and for T1 task (WL).
Table 3. Variance on the AUC values calculated for each model on the validation set for T2 task (LAW) and for T1 task (WL).
LAW = 2LAW = 3LAW = 4LAW = 5WL = 1WL = 2WL = 3WL = 4WL = 5
Logistic Regression1.233e-321.232e-326.145e-072.488e-071.233e-322.831e-055.612e-080.000192.427e-06
Random Forest8.301e-058.208e-056.273e-059.655e-058.076e-050.000127.139e-050.000119.349e-05
Gradient Boosting2.887e-051.616e-054.583e-062.462e-053.135e-068.098e-071.801e-061.445e-062.860e-05
Table 4. Overall results on the test set (2015–2018) for the T1 task: default predictions with historical data. Rec indicates the recall and Pr indicates the precision.
Table 4. Overall results on the test set (2015–2018) for the T1 task: default predictions with historical data. Rec indicates the recall and Pr indicates the precision.
WL = 1
TPFNFPTNAUC ScoreMicro-f1Macro-f1I ErrorII ErrorRec BankruptcyPr BankruptcyRec HealthyPr Healthy
Logistic Regression852096922710.7550.7040.48429.90719.0480.810.0810.7010.991
Random Forest8322115820820.7170.6470.45135.74120.9520.790.0670.6430.99
Gradient Boosting852094522950.7590.7120.48829.16719.0480.810.0830.7080.991
WL = 2
TPFNFPTNAUC ScoreMicro-f1Macro-f1I ErrorII ErrorRec BankruptcyPr BankruptcyRec HealthyPr Healthy
Logistic Regression8518102421190.750.6790.47132.5817.4760.8250.0770.6740.992
Random Forest891497921640.7760.6940.48331.14913.5920.8640.0830.6890.994
Gradient Boosting8221107020730.7280.6640.46134.04420.3880.7960.0710.660.99
WL = 3
TPFNFPTNAUC ScoreMicro-f1Macro-f1I ErrorII ErrorRec BankruptcyPr BankruptcyRec HealthyPr Healthy
Logistic Regression8120125917470.6920.5880.42241.88319.8020.8020.060.5810.989
Random Forest851686921370.7760.7150.49528.90915.8420.8420.0890.7110.993
Gradient Boosting782383421720.7470.7240.49527.74522.7720.7720.0860.7230.99
WL = 4
TPFNFPTNAUC ScoreMicro-f1Macro-f1I ErrorII ErrorRec BankruptcyPr BankruptcyRec HealthyPr Healthy
Logistic Regression8511137314510.70.5260.39348.61911.4580.8850.0580.5140.992
Random Forest801680720170.7740.7180.49728.57616.6670.8330.090.7140.992
Gradient Boosting722492518990.7110.6750.46632.755250.750.0720.6720.988
WL = 5
TPFNFPTNAUC ScoreMicro-f1Macro-f1I ErrorII ErrorRec BankruptcyPr BankruptcyRec HealthyPr Healthy
Logistic Regression829117414700.7290.5670.41744.4029.890.9010.0650.5560.994
Random Forest751665119930.7890.7560.5224.62217.5820.8240.1030.7540.992
Gradient Boosting721975918850.7520.7160.49328.70720.8790.7910.0870.7130.99
Table 5. Overall results on the test set (2015–2018) with look-ahead models. Rec means recall and Pr means precision.
Table 5. Overall results on the test set (2015–2018) with look-ahead models. Rec means recall and Pr means precision.
WL = 2
TPFNFPTNAUC ScoreMicro-f1Macro-f1I ErrorII ErrorRec BankruptcyPr BankruptcyRec HealthyPr Healthy
Logistic Regression8518108220610.740.6610.46234.42617.4760.8250.0730.6560.991
Random Forest782585322900.7430.730.49527.1424.2720.7570.0840.7290.989
Gradient Boosting8122100221410.7340.6850.47231.8821.3590.7860.0750.6810.99
WL = 3
TPFNFPTNAUC ScoreMicro-f1Macro-f1I ErrorII ErrorRec BankruptcyPr BankruptcyRec HealthyPr Healthy
Logistic Regression9011113618700.7570.6310.4537.79110.8910.8910.0730.6220.994
Random Forest851678122250.7910.7430.51225.98115.8420.8420.0980.740.993
Gradient Boosting762596220440.7160.6820.46932.00324.7520.7520.0730.680.988
WL = 4
TPFNFPTNAUC ScoreMicro-f1Macro-f1I ErrorII ErrorRec BankruptcyPr BankruptcyRec HealthyPr Healthy
Logistic Regression771974220820.770.7390.50726.27519.7920.8020.0940.7370.991
Random Forest811577020540.7860.7310.50527.26615.6250.8440.0950.7270.993
Gradient Boosting791788019440.7560.6930.48131.16117.7080.8230.0820.6880.991
WL = 5
TPFNFPTNAUC ScoreMicro-f1Macro-f1I ErrorII ErrorRec BankruptcyPr BankruptcyRec HealthyPr Healthy
Logistic Regression781384917950.7680.6850.4832.1114.2860.8570.0840.6790.993
Random Forest791271519290.7990.7340.5127.04213.1870.8680.0990.730.994
Gradient Boosting741786617780.7430.6770.47232.75318.6810.8130.0790.6720.991
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Lombardo, G.; Pellegrino, M.; Adosoglou, G.; Cagnoni, S.; Pardalos, P.M.; Poggi, A. Machine Learning for Bankruptcy Prediction in the American Stock Market: Dataset and Benchmarks. Future Internet 2022, 14, 244.

AMA Style

Lombardo G, Pellegrino M, Adosoglou G, Cagnoni S, Pardalos PM, Poggi A. Machine Learning for Bankruptcy Prediction in the American Stock Market: Dataset and Benchmarks. Future Internet. 2022; 14(8):244.

Chicago/Turabian Style

Lombardo, Gianfranco, Mattia Pellegrino, George Adosoglou, Stefano Cagnoni, Panos M. Pardalos, and Agostino Poggi. 2022. "Machine Learning for Bankruptcy Prediction in the American Stock Market: Dataset and Benchmarks" Future Internet 14, no. 8: 244.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop