Assessing the Performance of Random Forests for Modeling Claim Severity in Collision Car Insurance

Abstract

For calculating non-life insurance premiums, actuaries traditionally rely on separate severity and frequency models using covariates to explain the claims loss exposure. In this paper, we focus on the claim severity. First, we build two reference models, a generalized linear model and a generalized additive model, relying on a log-normal distribution of the severity and including the most significant factors. Thereby, we relate the continuous variables to the response in a nonlinear way. In the second step, we tune two random forest models, one for the claim severity and one for the log-transformed claim severity, where the latter requires a transformation of the predicted results. We compare the prediction performance of the different models using the relative error, the root mean squared error and the goodness-of-lift statistics in combination with goodness-of-fit statistics. In our application, we rely on a dataset of a Swiss collision insurance portfolio covering the loss exposure of the period from 2011 to 2015, and including observations from 81 309 settled claims with a total amount of CHF 184 mio. In the analysis, we use the data from 2011 to 2014 for training and from 2015 for testing. Our results indicate that the use of a log-normal transformation of the severity is not leading to performance gains with random forests. However, random forests with a log-normal transformation are the favorite choice for explaining right-skewed claims. Finally, when considering all indicators, we conclude that the generalized additive model has the best overall performance.

1. Introduction

In the last years, especially in the area of car insurance, many insurers have experienced high fluctuations in their customer portfolio, due to increased competition and the appearance of new technologies (Kamakura et al. 2003). To stay competitive, products must be priced adequately (Bieck et al. 2010; Maas et al. 2008). Traditionally, actuaries rely on linear regression models to calculate the premiums. Such models used explanatory variables, including the characteristics of the policyholder, of the risk insured, and of the contract configuration. Usually, two models are separately calibrated, one for the claim severity and one for the claim frequency (Frees et al. 2016; Ohlsson and Johansson 2010). The pure premium is then obtained by combining both models. In that framework, the severity relates to the average claim amount, while the frequency represents the ratio of the number of claims to the exposure (Bellina 2014; Brisard 2014). Over the last decade, machine learning techniques have won a lot of attention in the area of insurance analytics (Denuit et al. 2019a, 2019b; Quan and Valdez 2018). Machine learning methods are applied in the context of ratemaking (Dalkilic et al. 2009; Huang and Meng 2019; Lowe and Pryor 1996; Pelessoni and Picech 1998; Richman 2018), fraud detection (Li et al. 2018; Wang and Xu 2018), extreme value theory (Velthoen et al. 2021), forecasting (Perla et al. 2020), and in the explanation of the lapse behavior of customers (Guelman et al. 2012; Hu et al. 2020; Staudt and Wagner 2020), among others. While such models are used to select relevant risk factors and automate the creation of categories for continuous variables (Dougherty et al. 1995; Henckaerts et al. 2018), full-pricing applications are scarce, see, e.g., Guelman (2012) and Henckaerts et al. (2020). In claims modeling, most of the current academic research focuses on machine learning methods to develop claim frequency models (Denuit et al. 2020; Ferrario et al. 2018; Noll et al. 2018; Schelldorfer and Wüthrich 2019; Wüthrich and Buser 2018) and much less attention is given to severity modeling (see, e.g., Dewi et al. 2019; Staudt and Wagner 2019).

1.1. Aim and Methodology

In this paper, we focus on insurance claim severity modeling and add insights to the existing body of research (see, e.g., Charpentier 2014, chp. 14, and the literature review below). Our research aims to respond to the following research question: what is the best performing model to model the claim severity in collision car insurance? Driven by our data, we focus on the log-normal distribution assumption which is often used. However, the impact of the distribution assumption on the prediction of damage levels from different models has been given less attention until now. Our objective is to calibrate prediction models for the claim severity and compare the performance of generalized additive models (GAM), generalized linear models (GLM), and random forests (RF). Traditionally, the performance of models is evaluated with the help of goodness-of-fit statistics (GOF). We extend this step by applying goodness-of-lift statistics (GOL), see Denuit et al. (2019). Two reference models are derived, an optimal GAM and a GLM using data-driven binning for continuous covariates following the techniques of Henckaerts et al. (2018). Because the GLM with categorical variables is still the favorite model setup in many insurance companies, it is important for us to measure the performance of the data-driven binning method along GOF and GOL. In the RF setup, we consider two models using the log-transformation of the claims, like in the GLM, and the claim severity without transformation. We compare the performance of these models to the one of the reference models. Our aim is to measure the impact of the log-transformation on the performance of the severity level, i.e., we measure, with the help of GOF and GOL, the performance of the models when back-transforming the log predictions. This back-transformation will be discussed in detail in this paper. We calibrate our models using comprehensive claims data from a Swiss insurer covering 81 309 settled claims. Inn order to measure the prediction performance, we split the data in training and test samples covering the period from 2011 to 2014, respectively, the year 2015.

1.2. Literature on Models and Prediction Performance

GLM introduced by Nelder and Wedderburn (1972) are the “industry standard” to explain the claim severity. In such models, the response variable is distributed according to a distribution of the exponential family (e.g., a Gamma or a log-normal distribution), which is well suited for non-life insurance claims (Ohlsson and Johansson 2010). Traditionally, a log-normal distribution is applied in GLM by log-transforming the dependent variable and then assuming a normal distribution (Frees et al. 2016). While covariates interact as linear predictors in GLM and given that continuous variables rarely interact this way in practice, Hastie and Tibshirani (1990) extend the linear models by relating the response to the continuous variables through a smoothing function in GAM. Because insurers prefer the simplicity of GLM with categorical variables, we use evolutionary trees to derive optimal classes for the continuous variables. The data-driven binning approach closely follows the work of Henckaerts et al. (2018) and Staudt and Wagner (2019). In our procedure, we take particular care in optimizing the number of classes using a penalty function (Grubinger et al. 2014). Following the Bayesian information criteria (BIC), we propose an “optimal” GAM using smoothing functions for the continuous variables and a GLM relying only on categorical variables. While GAM and GLM need a prior variable and interaction selection, this is automated in RF through a specified algorithm. Thus, the interactions that are included in RF models are not limited to a user-specified selection (Hastie et al. 2009; Kuhn and Johnson 2013). Usually, RF models are optimized through the root mean squared error (RMSE) optimization function. However, this optimization function relates to the assumption that the response is normally distributed. Because the claim distribution is right-skewed with only positive values, this assumption is not fulfilled. Hence, we also tune an RF model using the log-transformation used in the regression models. This allows for us to compare the performance among GAM, GLM, and RF models, and to study the impact of the choice of the distribution assumption on the predictions.

While the BIC serves to assess the model performance in GAM and GLM, this function cannot be applied to RF models. Chai and Draxler (2014); Cort and Kenji (2005); Willmott et al. (2009) propose using the RMSE or the mean absolute error (MAE) to assess the performance of machine learning models. Denuit et al. (2019) extend these measures of evaluation and suggest using the area between the concentration and Lorentz curve (ABC) and the integrated concentration curve (ICC). These measures are called GOL statistics and they consider the appropriateness of the predictions, including customers’ claim severity. With the help of the ABC and ICC, we measure the relation between the observed and predicted claim severity (Denuit et al. 2019). On a macro-level, the overall coverage of the total claims by the predictions can be measured with the relative error (RE). In our discussion, we highlight the advantages and disadvantages of the different models. Additionally, we focus on the smoothness of the predictions through contiguous values of the (continuous) risk factors on the individual level. For this purpose, the confidence intervals on the predictions play a major role. However, machine learning and non-parametric models only provide predictions at given points (Khosravi et al. 2011). We build confidence intervals while using the bootstrap method, where new samples are randomly built with replacement and a part of the observations are held out (Carney et al. 2003; Veaux et al. 2014). The so-obtained confidence intervals for the RF model are compared with the ones of the GAM and GLM. We measure the overall prediction performance with the RMSE, MAE, ABC, ICC (individual error), and the RE (overall error).

1.3. Main Results

The main results of our case study are, as follows: in terms of GOL and GOF, GAM is leading to the most appropriate model on the training and test samples. Using a combination of GOL and GOF is bringing enhanced insights in the model interpretation, which GOF alone cannot give. The GLM obtained with the method of Henckaerts et al. (2018) has a good overall performance. However, GAM generates better predictions on the test sample. The log-transformed RF model has a good performance on the right-skewed data, whereas the overall expenses are not covered. The log-transformed RF model shows much lower values for the overall predictions in the violin plot when compared to the other models. The RF model based on a normal distribution assumption covers the total amount of claims well. However, this model is not outperforming other measures. Further, on the selected profiles, RF yields results with low variations along the continuous variables. This helps to explain why RF covers individual claims less well. The log-transformation requires a back-transformation which impacts the performance. Bootstrapping and confidence intervals help to explain the variation of the different factors, so that RF can be an additional asset next to GLM and GAM in practice.

The remainder of the paper is structured, as follows: in Section 2, we describe the available data, analyze the severity distribution, and provide descriptive statistics. In Section 3, we derive the GAM, GLM, and RF models, and then calibrate them on the training data. In Section 4, we describe and discuss the performance of the models on our test sample. We conclude in Section 5.

2. Description of the Dataset

Our study relies on a longitudinal dataset of a Swiss car insurer comprising all closed collision claims that were registered during the period from 2011 to 2015. In Section 2.1, we describe the available data and risk factors. We study the distribution of the claim severity and provide GOF statistics in Section 2.2. We lay out the relative frequencies along the risk factors in Section 2.3.

2.1. Available Data and Variables

We concentrate on the claim severity, as laid out in the Introduction. Because most customers do not declare damage (Denuit and Lang 2004; Klein et al. 2014; Ohlsson and Johansson 2010), no claims are registered for these customers and the number of available observations is typically low, which also explains the limited number of studies (Charpentier 2014, chp. 14). Our data stem from a Swiss collision car insurance portfolio and report the claims that are made by policyholders during the period from 2011 to 2015. A longer history of observations outbalances the disadvantages of the heterogeneity over the years, e.g., varying weather conditions or number of traffic accidents (Denuit and Lang 2004; Denuit et al. 2007). We denote, by S, the claim severity, which is expressed in Swiss francs (CHF). We restrict our study to private policyholders and only consider settled claims in the present work.1 Our full data cover a total of 81 309 claims over five years summing up to CHF 184 mio. In line with the theory of model evaluation, we divide our dataset in a training and a test sample for assessing the prediction performance (Hastie et al. 2009; James et al. 2013; Kuhn and Johnson 2013). Our training data span four calendar years (2011–2014) and they cover 65,950 claims. The test sample consists of the 2015 data and includes 15,359 claims observations. This setup represents, for example, an environment where an insurer uses available data from earlier years (2011–2014) to predict the claim severity for the following period (2015). In terms of financial volume, we consolidate a total claim amount of CHF 149 mio for the training data and CHF 35 mio for the test data.

Table 1. Summary of the variables that are available in the data.

Variable	Description
Continuous variables
$AG$	Age of the policyholder (in years)
$BM$	Bonus-malus level (in %)
$HP$	Horsepower of the vehicle (in hp)
$VC$	Value of the car without accessories (in CHF)
$AP$	Value of the accessories of the car as a percentage of the car value
$AC$	Age of the car (in years)
$WC$	Weight of the car (in kg)
$LO$	Longitude of the policyholder’s main residence ($6.0$–${10.5}^{\circ }$ E)
$LA$	Latitude of the policyholder’s main residence ($45.8$–${47.8}^{\circ }$ N)
Categorical variables
$\mathit{CA}$	26 cantons of Switzerland and Principality of Liechtenstein
$\mathit{LR}$	Language regions along three classes: German, French, Italian
$\mathit{NA}$	Nationality along the following classes: Switzerland (CH), France (FR), Germany and Austria (DE-AT), Spain (ES), Portugal (PT), Italy (IT), Eastern Europe and Turkey (EE-TR), other (OT)
$\mathit{UT}$	Utilization along three classes: private use with/without irregular commuter route (PE), private use with regular commuter route (PR), professional use (PL)
$\mathit{CS}$	Car body style along four classes: hatchback and sedan (CH), limousine (CL), convertible (CC), other (CO)
$\mathit{CB}$	Car brand
${\mathit{CB}}_{\mathit{g}}$	Car brand group
${\mathit{CB}}_{\mathit{c}}$	Car brand country
$\mathit{NS}$	Number of seats along three classes: $4-$, 5 and $6+$ seats
$\mathit{ID}$	Deductible along four classes: CHF 300, 500, 1000 and $2000+$
Binary variables
$GE$	Gender of the policyholder (male/female)
$IT$	Online contract underwriting (yes/no)
$BS$	Bonus-malus level protection (yes/no)
$PM$	Zero-alcohol engagement (yes/no)
$DW$	Driving license withdrawal (yes/no)
$DD$	Driving less than $10,000$ km per year (yes/no)

summarizes the available explanatory variables for the process of ratemaking (Frees 2015), i.e., building homogeneous classes of policyholders (see, e.g., Laas et al. 2016; Staudt and Wagner 2018). Most of the factors are determined at the moment of underwriting of the contract (a priori variables, Denuit and Lang 2004; Denuit et al. 2007; Verbelen and Antonio 2016). The age of the policyholder $AG$, the bonus-malus level $BM$, the horsepower of the vehicle $HP$, the value of the car $VC$, the value of the accessories as a percentage of the car value $AP$, the age $AC$, and weight $WC$ of the car, as well as the longitude $LO$ and latitude $LA$ of the policyholder’s main residence are continuous variables. The bonus-malus level, i.e., the percentage factor applied on the gross premium, measures by experience the riskiness of the policyholder and it is reevaluated every year (a posteriori variable, Antonio and Valdez 2012). If a policyholder has a zero claim history, the value drops below 100%. The variables $LO$ and $LA$, as well as the categorical variables canton $\mathit{CA}$ and language region $\mathit{LR}$, relate to the policyholder’s residence area. The linguistic regions divide Switzerland into three regions, namely, the German-, French-, and Italian-speaking areas (Lüdi and Werlen 2005). A more detailed segmentation is given by the 26 cantons. Further, the values for $LO$ and $LA$ are derived from combining the postal code and the name of the place of residence. In our work, we code $\mathit{LR}$ and $\mathit{CA}$ with numbers for data confidentiality reasons (see, e.g., the descriptive statistics in Section 2.3 and Figure A1 in the Appendix A). The nationality $\mathit{NA}$ is recorded along eight categories, where the following classes are used by the insurer: Switzerland, France, Germany and Austria, Spain, Portugal, Italy, Eastern Europe, and Turkey, as well as a class for all other countries. The car usage $\mathit{UT}$ is divided along private use with/without irregular or regular commuter route and professional use. The car is described along the body style $\mathit{CS}$, the car brand $\mathit{CB}$, and the number of seats $\mathit{NS}$. To aggregate car brands, we also consider the group that the car brand is associated with ${\mathit{CB}}_{\mathit{g}}$ and the country of origin of the brand ${\mathit{CB}}_{\mathit{c}}$. Car styles $\mathit{CS}$ follow four categories, namely, hatchback and sedan, limousine, convertible, and other. The variable $\mathit{NS}$ encompasses three classes: having less than, exactly, or more than five seats. The insurance deductible $\mathit{ID}$ is considered along four categories, as offered by the insurer: CHF 300, 500, 1000, and 2000 (or more). Finally, the binary variables gender ($GE$), buying the contract online ($IT$), bonus-malus level protection ($BS$), zero-alcohol engagement ($PM$), driving license withdrawal ($DW$), and driving more or less than $10000$ km per year ($DD$) are available.

2.2. Claim Distribution

The distribution of the claim severity S is right-skewed (see Figure 1) and it is commonly well described by a Weibull, Gamma, or log-normal distribution (Eling 2014; Ohlsson and Johansson 2010). For choosing the most adequate distribution assumption, we consider three GOF, namely, the Kolmogorov-Smirnov (KS), the Cramer van Mises (CvM), and the BIC measures. Both KS and CvM measures relate to the empirical distribution whereas BIC is linked to the log-likelihood. While the KS measure quantifies the distance between the empirical distribution function of the sample and the cumulative distribution function (Durbin James 1973), CvM extends KS by using the minimum distance estimation (Csorgo and Faraway 1996). The BIC measures the quality of fit (Schwarz 1978). All three measures state that the claim severity S on the training sample is best explained by the log-normal distribution (see

Table 2. GOF statistics for the distribution of the claim severity S in the training dataset.

Distribution	Weibull	Gamma	Log-Normal
KS	0.057	0.068	0.043
CvM	96.328	110.973	34.269
BIC	1,150,200	1,150,469	1,147,293

and Figure 1). In our study, we do not distinguish between the small and large claims and consider all of them together, i.e., we do not split or truncate the dataset, as done in some claim severity models (Albrecher et al. 2017; Denuit and Lang 2004). In collision insurance, and as we only consider settled claims, we do not have as many extremely large claims when compared to other claims data (linked, e.g., to liability insurance). From the graphical illustration, we observe that most of the claims are below CHF 2500 in the training sample. This also holds for the test sample.

2.3. Descriptive Statistics

Figure A1 in the Appendix A illustrates the relative frequency of the claim severity S in the training sample along the risk factors. Given the distribution of S along the ages, we note that the quartile values are at 39, 50, and 62 years. Most policyholders (91%) have the lowest bonus-malus level, which means that they pay 30% of the standard premium. The quartiles along the insured vehicles’ horsepower are at the values of 102, 135, and 163. Twenty-five percent of the cars’ values are below CHF 24,800, the median value is CHF 33,900 and the upper quartile CHF 43,720. Most of the insured cars (78%) have accessories that are worth less than 10% of the car value. Half of the policyholders drive rather new cars that are aged below three years. The quartiles in the car weights are 1200, 1400, and 1600 kg. The distribution of the customers along the cantons (numbered from 1 to 26) and language regions (1, 2, 3) shows that the insurer is particularly active in one canton and one language region. Most customers have Swiss nationality (85%) and they use their car solely for private purposes (81%). They are 69% to drive a limousine with five seats (80%). Moreover, the most popular car brand is VW (23%), which originates from Germany (39%) and it is a member of the group VW. The second-largest car band group is PSA (16%), and it includes the major brands Peugeot, Citroën and Opel. This group contains cars with origins of France and Germany. Most cars are produced in Germany, followed by Japan. The insurance deductible is CHF 300 for 80% of the portfolio. Men are responsible for two-thirds (65%) of the claims. Most contracts are not underwritten online (98%) and they include a bonus-malus level protection (91%). Most of the policyholders do not take the zero-alcohol engagement (98%). Nevertheless, they have no driving license withdrawal (99%) and drive more than 10,000 km per year (82%).

3. Model Parameterization

In this section, we link the claim severity S to the risk factors laid out above. First, we derive optimal GAM and GLM under the BIC (see Section 3.1) following Henckaerts et al. (2018) and Staudt and Wagner (2019). Second, we use RF to model S and $logS$ by considering the $RMSE$ as an optimization function (Section 3.2).

3.1. Optimal Regression Models (GAM and GLM)

From the GOF statistics that are provided in Table 2, we observe that the claim severity S is best approximated by a log-normal distribution. This distribution relates to the exponential family, namely to the normal distribution, by log-transforming S. In the following, we use the identity linear link function in a GAM and consider the risk factors that are summarized in Table 1 in a forward and backward stepwise selection procedure. Thereby, the BIC provides a measure for the model performance. The BIC penalty is more severe then, e.g., the Akaike Information Criterion, and will, therefore, favor less complex models. We rely on the BIC, since we aim to retain well-performing models that are as simple as possible. In the GAM, continuous variables are included through a smoothing function and the spatial longitude and latitude information is integrated through an interaction term (Denuit and Lang 2004; Klein et al. 2014). At this stage, we do not consider further interactions between the other risk factors. Both forward and backward stepwise selection processes lead to the following model:

$$\begin{array}{cc}\hfill \mathbb{E}(log(S\left)\right)=& {\beta }_0+{\beta }_1·BS+{\beta }_{\mathbf{2}}·\mathit{CS}+{\beta }_{\mathbf{3}}·{\mathit{CB}}_{\mathit{g}}+{\beta }_{\mathbf{4}}·\mathit{ID}\hfill \\ & \hfill +f_1\left(AG\right)+f_2\left(HP\right)+f_3\left(AC\right)+f_4\left(WC\right)+f_5(LO,LA).\end{array}$$

(1)

The selected model only retains 10 of the 25 available explanatory variables. One binary variable, the bonus-malus level protection $BS$, is included in the model. Three categorical variables are included. They are the car body style $\mathit{CS}$ with four categories, the car brand group ${\mathit{CB}}_{\mathit{g}}$ with 17 categories, and the insurance deductible $\mathit{ID}$ with four categories. The continuous variables age of the policyholder $AG$, horsepower of the car $HP$, age $AC$, and weight $WC$ of the car are considered through the smoothing functions $f_1$ to $f_4$. Spatial information on the policyholder’s main residence enters through a smoothing function on the interaction between the longitude and latitude in $f_5(LO,LA)$. The other variables are excluded, since they do not improve the model under the BIC. When applying the model (1) on the data from the training sample, the values of the regression coefficients and the estimated degrees of freedom are reported in the first column of

Table A1. Regression results of the GAM (Equation (1)) and GLM (Equation (3)) calibrated on the training sample.

GAM (1)			GLM (3)
Intercept	$7.161$	***		$6.761$	***
$BS$ (baseline: Yes)
No	$-0.072$	***		$-0.072$	***
$\mathit{CS}$ (baseline: CL)
CH	$0.024$	*		$0.024$	*
CC	$-0.121$	***		$-0.126$	***
CO	$-0.195$	***		$-0.198$	***
${\mathit{CB}}_{\mathit{g}}$ (baseline: VW)
BMW	$-0.105$	***		$-0.116$	***
Daimler	$0.106$	***		$0.099$	***
Fiat	$0.062$	**		$0.067$	**
Ford	$-0.007$			$-0.007$
GM	$0.096$	*		$0.113$	*
Greely	$0.007$			$-0.004$
Honda	$-0.024$			$-0.019$
Hyundai	$-0.035$			$-0.026$
Independent	$0.082$			$0.086$
Mazda	$0.051$	*		$0.054$	*
PSA	$0.132$	***		$0.134$	***
Renault	$0.149$	***		$0.155$	***
Subaru	$0.038$			$0.037$
Suzuki	$0.207$	***		$0.212$	***
Tata	$0.088$	.		$0.102$	*
Toyota	$0.019$			$0.028$
$\mathit{ID}$ (baseline: 300)
500	$0.053$	***		$0.052$	***
1000	$0.218$	***		$0.221$	***
≥2000	$0.487$	***		$0.525$	***
${\widehat{f}}_1\left(\mathit{AG}\right)$	$7.009$	***	${\mathit{AG}}_{\mathit{c}}$ (baseline: 29–57)
			$18–21$	$0.293$	***
			$22–24$	$0.229$	***
			$25–28$	$0.057$	**
			$58–75$	$-0.033$	**
			$76–81$	$0.011$
			$>81$	$0.196$	***
${\widehat{f}}_2\left(\mathit{HP}\right)$	$3.927$	***	${\mathit{HP}}_{\mathit{c}}$ (baseline: >126)
			$41–125$	$0.070$	***
${\widehat{f}}_3\left(\mathit{AC}\right)$	$5.230$	***	${\mathit{AC}}_{\mathit{c}}$ (baseline: 0–3)
			4	$0.069$	***
			5	$0.097$	***
			6	$0.158$	***
			7	$0.178$	***
			$>8$	$0.269$	***
${\widehat{f}}_4\left(\mathit{WC}\right)$	$1.002$	***	$\mathit{WC}$	$0.0002$	***
${\widehat{f}}_5(\mathit{LO},\mathit{LA})$	$24.740$	***	${(\mathit{LO},\mathit{LA})}_{\mathit{c}}$ (baseline: 3)
			1	$-0.132$	***
			2	$-0.080$	***
			4	$0.068$	***
			5	$0.128$	***
			6	$0.254$	***
			7	$0.477$	***
BIC	201,748			201,516
N	65,950			65,950

Significance levels for p-values: *** p ≤ 0.001, ** p ≤ 0.01, * p ≤ 0.05, p ≤ 0.1.

in the Appendix A. The estimated degrees of freedom that are related to the smoothing functions correspond to the number of knots of the spline, where a higher degree relates to a higher number of inflection points. For example, for the age of the policyholder ${\widehat{f}}_1\left(AG\right)$, we find a value of $7.009$, which is higher than for the variables $HP$, $AC$, and $WC$ (see Figure 2 for a graphical illustration). The weight of the car variable has an estimated degree of freedom in ${\widehat{f}}_4\left(WC\right)$ that is almost equal to one ($1.002$), indicating a quasilinear impact of $WC$ on the response. Further, we observe that most regression coefficients (upper part of Table A1) yield high significance levels (p-values that are below 0.001) with only few sub-categories of the car brand group ${\mathit{CB}}_{\mathit{g}}$ not being significant. In the following, we test all interactions between the continuous variables $AG$, $HP$, $AC$, and $WC$, omitting the other (categorical, binary and spatial) factors as they would lead to more complex but not more explicit models. We do not consider the common age-gender interaction, since the gender variable is excluded from the model following the BIC. We find that none of the studied interaction terms improve the model under the BIC and we remain with the GAM that is given in Equation (1).

In Figure 2, we illustrate the effects of the continuous variables through the fitted smoothing functions ${\widehat{f}}_1\left(AG\right)$, ${\widehat{f}}_2\left(HP\right)$, ${\widehat{f}}_3\left(AC\right)$, and ${\widehat{f}}_4\left(WC\right)$. The dashed lines indicate the 95% confidence interval. We observe larger intervals, for example, at higher policyholder ages, where the number of observations is lower. In graph (a), ${\widehat{f}}_1$ indicates a higher effect for young customers, which relates to higher claim severity in that customer segment. This effect is decreasing through the age from 18 to 35 years, and they behave non-monotonically in the range between 35 and 75 years. From 75 years on, the effect on the claim severity increases again. Conversely, the smoothing function of the horsepower ${\widehat{f}}_2$ that is illustrated in the graph (b) is decreasing for values from 50 to 250 and then increasing. The age of the car effect ${\widehat{f}}_3\left(AC\right)$ in graph (c) is rather constant from zero to three, increasing from three to seven, and finally becoming non-monotonic. As mentioned earlier, the effect of the weight of the car $WC$ is rather linear (graph d). Further, we illustrate the spatial effect ${\widehat{f}}_5(LO,LA)$ in Figure 3. Customers living in the Italian-speaking region have the highest effect with ${\widehat{f}}_5(LO,LA)=0.4$, which corresponds to, on average, approximatively 40% more severe accidents than in regions with the baseline (zero) effect. Customers living in the region of the cities of Bern and St. Gallen have values of ${\widehat{f}}_5(LO,LA)=-0.2$. We observe that the effect is increasing from $-0.2$ to $0.0$ when moving from both cities to more rural and remote regions (e.g., Bernese Oberland and Appenzellerland). This indicates that the sole canton information commonly used today can be significantly enriched by integrating more detailed spatial information.

We cannot apply a linear model on these variables, as we observe nonlinear effects for the continuous variables $AG$, $HP$, and $AC$, as well as for the spatial information $(LO,LA)$. From the continuous factors, only the effect of $WC$ is suitable for linear modeling. To build an optimal GLM model, we use a data-driven method to classify the continuous variables $AG$, $HP$, and $AC$ into categories. In fact, evolutionary trees can help to bin consecutive values and such a method is, e.g., available in the package $\mathtt{evtree}$ in $\mathtt{R}$. We optimize the evolutionary trees through a tuning parameter $\alpha$ in the following penalty function,

$$\begin{array}{c}\hfill n·log\left(MSE\right)+4·\alpha ·(l+1)·log\left(n\right),\end{array}$$

(2)

where n relates to the size of the dataset and l is the number of classes. The number of classes l is decreasing with increasing $\alpha$ (Grubinger et al. 2014). This performance function is used to compare the models between each other. The model with the lowest value is considered the best one. However, the penalty (2) decreases with increasing $\alpha$ and it relates to simpler trees. The penalty measure will always choose the simplest model, which, following actuarial experience, is not necessarily the best model (Henckaerts et al. 2018). Thus, we cannot determine the optimal number of classes while exclusively using the penalty function (2) and we propose considering another way to determine the number of classes. At each step, for a given $\alpha$, we replace one smoothing function $f_i$ in Equation (1) by the obtained classes and hold the other variables fixed while determining the performance of the model using the BIC. This allows for us to assess the tradeoff between accuracy and complexity along the classes used. We choose the tuning parameter $\alpha$ out of the set $\{1,1.5,2,\dots ,9.5,10,15,20,\dots ,95,100,150,200,\dots ,950\}$. Figure 4 presents the values of the BIC for each $\alpha$. From graph (a), we read that, for $AG$, the BIC decreases until $\alpha$ takes the value of 350 and is increasing for higher values of $\alpha$. Thus, we use $\alpha =350$ for deriving the classes for $AG$. In the case of $HP$ and $AC$, the BIC decreases with $\alpha$ (see graphs b and c). In both cases, we remain with $\alpha =950$, leading to the simplest model.2 The classes that were obtained for $AG$, $HP$, and $AC$ are illustrated in the graphs (a) to (c) in Figure 2 with vertical lines indicating the boundaries of the bins. The dashed-pointed horizontal lines in the graphs indicate the predicted mean effects for each class. Following the above optimization of the penalization parameter, we find that the age of the policyholder is binned into seven classes ${\mathit{AG}}_{\mathit{c}}$, the horsepower of the car into two classes ${\mathit{HP}}_{\mathit{c}}$, and the age of the car into six classes ${\mathit{AC}}_{\mathit{c}}$.

From Figure 3, we observe that there are regions with similar effects on the claim severity. We do not only bin consecutive municipalities but aim to group regions with similar effects’ levels. The evolutionary tree method cannot be applied in this setup and we follow Henckaerts et al. (2018) who propose Fisher’s natural breaks algorithm. This algorithm maximizes the homogeneity within bins by classifying the variables that are close to their calculated average (Slocum et al. 2005). The method is readily available, e.g., in the $\mathtt{classInt}$ package in $\mathtt{R}$. The measures evaluating the performance of the created bins suffer from the same problem as the complexity measure shown in Equation (2). Thus, the best number of bins n is chosen by consistently measuring the final BIC performance. We let the parameter n take values between 2 and 15. The best classification (lowest BIC) is obtained while using seven spatial groups ${(\mathit{LO},\mathit{LA})}_{\mathit{c}}$, as can be seen from Figure 4d. Figure 5 presents the regional categories obtained.

While applying the above categories for the continuous variables, we derive the following GLM:

$$\begin{array}{cc}\hfill \mathbb{E}(log(S\left)\right)=& {\beta }_0+{\beta }_1·BS+{\beta }_{\mathbf{2}}·\mathit{CS}+{\beta }_{\mathbf{3}}·{\mathit{CB}}_{\mathit{g}}+{\beta }_{\mathbf{4}}·\mathit{ID}\hfill \\ & \hfill +{\beta }_{\mathbf{5}}·{\mathit{AG}}_{\mathit{c}}+{\beta }_{\mathbf{6}}·{\mathit{HP}}_{\mathit{c}}+{\beta }_{\mathbf{7}}·{\mathit{AC}}_{\mathit{c}}+{\beta }_8·WC+{\beta }_{\mathbf{9}}·{(\mathit{LO},\mathit{LA})}_{\mathit{c}}\end{array}$$

(3)

The second column in Table A1 reports the regression results.

3.2. Random Forests Models

In the following, we optimally calibrate a RF model on the training sample. Traditionally, the $RMSE$ optimization function is used (Hastie et al. 2009; Kuhn and Johnson 2013). However, this optimization function relies on the assumption that the claim severity S is normally distributed (Chai and Draxler 2014; Willmott et al. 2009). Because, from the GOF statistics in Table 2, we have concluded that S is best described through a log-normal distribution. Considering the log-transform of S, then $logS$ is normally distributed and the $RMSE$ can be applied. In this section, we propose tuning two RF models, one taking $logS$ (denoted by ${\mathrm{RF}}_{logS}$) and one taking S as the dependent variable (denoted ${\mathrm{RF}}_S$). In the numerical implementation, we use the available packages $\mathtt{ranger}$ and $\mathtt{caret}$ in $\mathtt{R}$.

A RF model is an aggregation of B regression trees, where each tree is built on a bootstrapped sample. While, for each feature/split, a sub-sample of m risk factors is chosen (Hastie et al. 2009; James et al. 2013). Each tree in the optimization is influenced by the node size. To balance the model between accuracy and complexity, we test five different minimum node sizes. Hence, three tuning parameters B, m, and the minimum node size need to be determined.

In a first step, we fix $B=1000$ and proceed with the choice of the optimal values for m and the minimum node size. We choose m within the set {1, 5, 10, 12, 15, 18, 20, 22, 25, 30, 35, 40, 45, 50} and the minimum node size is fixed to either 50, 100, 200, 500, or 1000 data points in each end node. We use a fivefold cross-validation to determine the best tuning parameters. This can be automatized with the $\mathtt{train}$ function in the $\mathtt{caret}$ package in $\mathtt{R}$(Kuhn 2008). In a fivefold cross-validation, the training sample is divided into five equally sized random sub-samples (without replacement). Four sub-samples of the five are used for training the model and the last one is used for validating the model (the validation sample, James et al. 2013; Kuhn and Johnson 2013). The performance of the model on the validation sample is measured with the help of the $RMSE$. The results that are illustrated in Figure 6 report the average value of the fivefold cross-validation samples’ $RMSE$, denoted by $\overline{RMSE}$, with the related standard errors $se$ for different values of m. The 95%-confidence intervals are calculated as $\overline{RMSE}\pm 1.96·se$(Nicholls 2014). In order to choose the best model, we remain with the simplest model having the lowest $\overline{RMSE}$ within the 95%-confidence interval of the best model. In the case of ${\mathrm{RF}}_{logS}$, the best results are obtained using $m=25$ and a minimum node size of 50. We retain the parameter value of $m=10$ and a minimum node size of 50, yielding a simpler model within the 95% confidence interval of the best model (solid horizontal lines in Figure 6). Figure 6b illustrates the results for ${\mathrm{RF}}_S$. We observe a minimum error for $m=10$ when the minimum node size equals 50, and we retain the values of $m=5$ and a minimum node size of 50 for further modeling. We note that a model with $m=1$ (one risk factor per regression) is still within the confidence interval of the ${\mathrm{RF}}_S$ model, but we exclude such a simple model. We are aware of the limitations of the tuning grid and the related results. We limited the tuning grid due to the available computational resources of the authors at the time of writing.

In a second step, we adjust for the optimal number of trees B with the above values for m and minimum node size. Using again fivefold cross-validation, we test the values of $B=500$, 1000, 1500, and 2000. While no important influence can be observed in both RF models (see Figure 7), we remain with $B=1000$. Thus, the ${\mathrm{RF}}_{logS}$ model is parameterized with $B=1000$, $m=10$ and a minimum node size of 50 and ${\mathrm{RF}}_S$ uses the parameters $B=1000$, $m=5$ and minimum node size of 50.

4. Comparison and Discussion of the Models

In this section, we compare the performance of the GAM (Equation (1)), GLM (Equation (3)), and both RF models. Because three of the models are based on $logS$, we consider the back-transformation to $\widehat{S}$ for the prediction of S. First, we make an overall comparison of the models while using different statistical measures. Second, for selected individual profiles, we focus on the smoothness of the predictions through contiguous values of the continuous risk factors.

4.1. Overall Model Comparison

For comparing the models, we take different perspectives and consider measures from the GOF (Willmott et al. 2009) and GOL (Denuit et al. 2019) statistics on the training and test samples. From the GOF statistics, we use the root mean squared error (RMSE), the mean absolute error (MAE), and the total relative error (RE). The total relative error evaluates the deviance of the prediction from the total claims amount, as follows:

$$\begin{array}{c}\hfill RE=\frac{{\sum }_i(S_i-{\widehat{S}}_i)}{{\sum }_i{S}_i}.\end{array}$$

(4)

Two GOL statistics are applied, namely the area between the concentration and Lorentz curve (ABC) and the integrated concentration curve (ICC). The concentration curve measures the share of true premiums that should have been collected for the portfolio, whereas the Lorentz curve measures the share of premiums that were collected for the portfolio. The smaller the area between both curves, the better the model in terms of premium income needs. The same interpretation holds for the ICC. Denuit et al. (2019) choose the optimal model by considering the combination of the lowest ABC and ICC. Further, we consider violin plots and the effective model predictions to waluate the models.

As we have log-transformed the claim severity S in the GAM, GLM, and ${\mathrm{RF}}_{logS}$ model, we transform the predictions from the log $\left(\widehat{logS}\right)$ to the claim severity level ($\widehat{S}$). By exponentiating the predictions $exp\left(\widehat{logS}\right)$, we lose some properties of the data (Longford 2009). Indeed, the sampling variance $\sigma$ needs to be included. Because traditional back-transformation methods can result in poor predictions, we use the non-parametric method described in the paper of Duan (1983),

$$\widehat{S_i}=exp\left(\widehat{log{S}_i}\right)·\frac{1}{n}\sum _{j=1}^{n}exp\left({\widehat{ϵ}}_j\right),$$

(5)

where n denotes the size of the dataset and ${\widehat{ϵ}}_j$ the residuals linked to the observation j. Duan’s smearing estimator demands that the error is homoscedastic (Ai and Norton 2000; Manning 1998; Manning and Mullahy 2001). The homogeneity in variance is traditionally controlled by representing the standardized residuals against the predictions. The homoscedasticity is given when the predictions are independent from the residuals, which means whether the points are randomly distributed along the standardized residuals and the predictions. We add a cubic spline estimator to the error visualizations of GAM, GLM, and ${\mathrm{RF}}_{logS}$ (see Figure 8) to test whether the distribution of the errors is homoscedastic. GAM and GLM residuals are homoscedastic, as the cubic spline is a line, which indicates that the homogeneity of the variance is given. However, $R{F}_{logS}$ standardized residuals against the predictions show a positive trend that we relate to heteroscedastic errors. Here, the smearing estimator from Duan (1983) is biased. Following Ai and Norton (2000), we replace the residuals by the standardized residuals in Equation (5), i.e.,

$$\begin{array}{c}\hfill \widehat{S_i}=exp\left(\widehat{log{S}_i}\right)·\frac{1}{n}\sum _{j=1}^{n}exp\left(\frac{r_j}{\frac{1}{n-1}{\sum }_{j=1}^n(r_j-\overline{r})}·\overline{r}\right),\end{array}$$

(6)

where $r_i$ is the residual of the ith observation and $\overline{r}$ is the residual average. In conclusion, we use Equation (5) to transform the predictions from the GAM and GLM, while Equation (6) is used in the case of ${\mathrm{RF}}_{logS}$.

We report the GOL and GOF results for the training sample in

Table 3. Comparison of the total predicted severity ${\sum }_i{\widehat{S}}_i$, goodness-of-fit statistics (GOF) and goodness-of-lift statistics (GOL) across the generalized additive models (GAM), generalized linear models (GLM), and random forests (RF) models in the training sample.

	Data	GAM	GLM	${\mathbf{RF}}_{log\mathbf{S}}$	${\mathbf{RF}}_{\mathbf{S}}$
Training sample (data from 2011–2014)
${\sum }_i{\widehat{S}}_i$ (in CHF)	149.3 mio	149.5 mio	148.4 mio	87.2 mio	149.0 mio
RE		$0.29\%$	$0.37\%$	$41.4\%$	0.04%
RMSE		3047	3049	3053	2860
MAE		1730	1732	1416	1650
ABC $(·{10}^{-3})$		4.4	5.1	112.9	96.7
ICC		0.456	0.456	0.324	0.364

, and for the test sample shown in

Table 4. Comparison of the total predicted claim severity ${\sum }_i{\widehat{S}}_i$, GOF and GOL across the GAM, GLM, and RF models in the test sample.

	Data	GAM	GLM	${\mathbf{RF}}_{log\mathbf{S}}$	${\mathbf{RF}}_{\mathbf{S}}$
Test sample (data from 2015)
${\sum }_i{\widehat{S}}_i$ (in CHF)	34.9 mio	34.8 mio	34.9 mio	20.3 mio	34.8 mio
RE		0.24%	0.08%	41.9%	0.16%
RMSE		2964	2965	3114	2957
MAE		1689	1692	1521	1701
ABC $(·{10}^{-3})$		5.643	7.252	5.396	9.915
ICC		0.458	0.459	0.458	0.454

. We observe that no model outperforms all others throughout the different. We observe that the ${\mathrm{RF}}_S$ has the smallest RE and RMSE value on the training sample. The ${\mathrm{RF}}_S$ model leading to the lowest RMSE was expected, as the model is optimized along this specific optimization function. For the other models, no important differences are found in the RMSE. Further, ${\mathrm{RF}}_{logS}$ has the lowest MAE, but, at the same time, the largest deviation from the total sum of claims (RE of 41.4%). Both of the RF models have high ABC values. A large ABC means that the RF models do not cover the individual claims. At the same time, these models have the smallest ICC, which means that the models cover the expected share of true premiums in the portfolio. By taking the combination of ABC and ICC, as described by Denuit et al. (2019), see Figure 9a, GAM is the best model. The violin plots in Figure 10a represent similar density plots for GAM and GLM. The violin plot of ${\mathrm{RF}}_S$ has a wider body and a similar median value than GAM and GLM. However, the one for ${\mathrm{RF}}_S$ has much more extreme predictions, better matching the larger claims. The ${\mathrm{RF}}_{logS}$ has a similar density plot than ${\mathrm{RF}}_S$, but with much lower values and not as high predictions as the other models. This explains why ${\mathrm{RF}}_{logS}$ has such a high RE, which indicates that this model is not well overlapping the total exposure of the portfolio. This is in accordance with the ABC, which tells us that the predictions do not cover the claim severity of the portfolio. The low MAE and ICC confirm that ${\mathrm{RF}}_{logS}$ is especially well performing on the lower part of the claims, which is due to the right-skewed data. The ${\mathrm{RF}}_S$ model has a much better coverage for the total exposure. Overall, the GAM performs well throughout all measures, with no extremely bad outcomes, and one can consider GAM to be the preferred model on the training sample.

Because such models are used in practice to evaluate the premiums for the following year, we focus now our study to see how the four models perform on unseen data. On the test sample, see Table 4, GLM covers the total sum of claims best. The deviation of the ${\mathrm{RF}}_{logS}$ model in terms of total claims is very high. The ${\mathrm{RF}}_S$ model remains with the lowest RMSE, closely followed by the GAM and GLM. The ${\mathrm{RF}}_{logS}$ model conforms best with the lowest MAE, i.e., it has on average the best cover of the claims. By considering the combination of ABC and ICC, see Figure 9b, GAM and ${\mathrm{RF}}_{logS}$ are similarly positioned. This means that GAM and ${\mathrm{RF}}_{logS}$ are best covering the portfolio expenses and the expected true premium. The violin plots, Figure 10b shows that ${\mathrm{RF}}_{logS}$ has much lower predictions when compared to the alternatives. The MAE and ABC measures confirm that the ${\mathrm{RF}}_{logS}$ model is well covering the right-skewed data, but it comes with issues in the higher claims. For the other models, we observe similar behaviors when considering the median. We note that the ${\mathrm{RF}}_S$ model yields higher predictions. When considering ABC, ${\mathrm{RF}}_S$ is less well covering the portfolio needs. Considering the test sample GAM and ${\mathrm{RF}}_{log}S$ seems to be the best alternatives. By combining the discussion from both training and test samples, an actuary would probably remain with the GAM model.

We analyze, in

Table 5. Comparison of the total predicted claim severity GOF and GOL across the GAM, GLM, and RF models in the training and samples for $logS$.

	GAM	GLM	${\mathbf{RF}}_{log\mathbf{S}}$
Training sample (data from 2011–2014)
RMSE	1.111	1.111	1.113
MAE	0.837	0.837	0.839
ABC ($·{10}^{-3}$)	0.250	0.107	2.150
ICC	0.497	0.498	0.498
Test sample (data from 2015)
RMSE	1.063	1.064	1.062
MAE	0.805	0.805	0.803
ABC ($·{10}^{-3}$)	0.255	0.414	0.2314
ICC	0.498	0.498	0.498

, if these differences remain on the log scale, as we observe high discrepancies for large claims in the ${\mathrm{RF}}_{logS}$ model. Therefore, we report the RMSE, MAE, ABC, and ICC measures. We observe no relevant differences in the RMSE, MAE, and ICC for the training sample. The only difference is observed in ABC, where the GLM is performing better and RF the worst. By considering the combination of ABC and ICC, see Figure 11a, the GLM is the best model in the training setup. When considering the results of the test sample, we do not measure the differences in the RMSE, MAE, and ICC. In the case of the ABC, GLM is worse than the alternatives. When considering Figure 11b GAM and ${\mathrm{RF}}_{logS}$ have similar results for ABC and ICC. This hints that the RF model and GAM can better generalize predictions when compared to the GLM.

4.2. Comparison on Individual Profiles

In the following, we compare the prediction performance of the models on the continuous variables age of the policyholder $AG$ and horsepower of the car $HP$ by fixing the other explanatory variables. Given the above performance statistics and because predictions are for $logS$, we do not consider the ${\mathrm{RF}}_{logS}$ model in the sequel. Nevertheless, one of our objectives is to understand why the ${\mathrm{RF}}_S$ model shows a lower performance on our data. In the following, we simplify the notation and write $RF$ for ${\mathrm{RF}}_S$. For this analysis, we perform predictions for four profiles along the age $AG$ (see Figure 12a–d) and for one profile along the horsepower $HP$ (see Figure 12e). For the profiles, we consider a baseline contract of a male policyholder having Swiss nationality, a bonus-malus level protection with a bonus-malus level of 30%, and using the car privately with regular commuter routes. We consider a customer driving a three-year-old limousine with a value of CHF 33 900, no accessories and a policy deductible of CHF 300. The other variables age of the policyholder $AG$, horsepower $HP$, age $AC$, and weight $WC$ of the car and the residence regions are varied along the values outlined in exhibit (f) of Figure 12. For profiles 1 to 4, the variations along ages are illustrated in graphs (a) to (d), while, for profile 5, the effect of horsepower is given in the graph (e). For all predictions, we provide 95%-confidence intervals: since these are point predictions, the confidence intervals are created with the help of the bootstrap ensemble technique, i.e., the variance of the predictions is obtained by creating 100 random training samples with replacements (Carney et al. 2003; Veaux et al. 2014).

Expectedly, we observe that the results from GAM exhibit smooth variations stemming from the smoothing splines used. The optimized bins that are created by the evolutionary trees and used in GLM are well visible along the age and horsepower variables (cf. Figure 2a,b). The predictions of the GAM and GLM conform with each other. RF presents partially significantly different predictions. For example, for ages below 30 years, the predicted claim severity $\widehat{S}$ from RF is decreasing with the age of the policyholder (see Figure 12a–d) comparably to GAM and GLM, but stays almost constant for ages from 30 to 80 years. The confidence intervals of the RF model often overlap the ones of the GAM and GLM, especially for younger ages from 18 to 30 years. While an overall agreement through the models exists along the age, we only find, in profile 1, that the RF confidence intervals do not overlap at higher ages. The predictions along the horsepower that are reported in Figure 11e are in agreement from 50 to 150. However, afterwards, RF results in higher predictions and the confidence intervals of the different models are distinct. The results of RF present many places where contiguous values of the risk factor lead to very different predictions. These findings may explain part of why the RF model yields high ABC and ICC values. In particular, we have observed that the results from the RF model do not vary along the ages from 30 to 80, which is leading to the RF model not covering the claims and the expected true premium of the portfolio.

For the same baseline profile underlying Figure 12, we illustrate the spatial claim severity predictions for GAM, respectively ${\mathrm{RF}}_S$ in Figure 13 and Figure 14. Thereby, we fix the age of the policyholder to 50 years, the horsepower of the car to 135, and the age and weight of the car to 3 years and 1 421 kg, respectively. We observe that the predictions that are illustrated in Figure 13 agree with the effects shown in Figure 3. As observed earlier while fitting the model, GAM predicts the highest claim severity for the Italian-speaking region and the lowest one for policyholders from the areas of Bern and St. Gallen. When considering Figure 14, we find that RF is not completely in line with the GAM results. In fact, certain policyholders from the Valais and Graubünden have a very high claim severity; higher variations are observed in the RF results. RF models seem to “react” much more importantly when there are differences between regions. Overall, we observe that the GAM is differentiating better along the longitude and latitude when compared to the RF model. This is in accordance with the results that were obtained in the figures above.

5. Conclusions

This paper compares the claim severity modeling and the predictions of GAM, GLM, and RF models when applied on the same car collision dataset from a Swiss insurer. In our application, based on a training sample, we build a forward and backward stepwise optimal GAM under the BIC for the severity S and derive from this model an optimized GLM with the help of evolutionary trees following the work of Henckaerts et al. (2018). The traditional regression models rely on a log-normal distribution implying an exponential back-transformation of the predictions. We also build two RF models using S and $log\left(S\right)$ with the root mean squared error (RMSE) as an optimization function. We compare the performance of GAM, GLM, and RF models on a test sample by taking several perspectives considering individual and total errors, violin plots, and comparing the model predictions along selected profiles.

We observe that the log-normal assumption for the claim severity implies a non-trivial back-transformation of the predictions when aiming to measure the model performance (Ai and Norton 2000; Duan 1983). We use the GOF and GOL measures to evaluate the performance of the models (Denuit et al. 2019). The GOF and GOL are not simultaneously improved in a given model, so that they provide valuable insights together with the violin plots and the selected profile representations. No model is outperforming the other ones throughout all the criteria, in accordance with the results of Denuit et al. (2019) and Henckaerts et al. (2020). Nevertheless, the GAM seems to be the one with the best overall performance (and no extremely bad results in any dimension) on both training and test samples. The GLM built along Henckaerts et al. (2018) is also a valid model. However, this model leads to bad predictions on unseen data. GAM and RF have a much better performance on the test sample. The ${\mathrm{RF}}_{logS}$ model does not cover the overall expenses, but it has a good coverage for the right-skewed part of the data. The weakness of the ${\mathrm{RF}}_{logS}$ model are the large claims. Instead, the ${\mathrm{RF}}_S$ model is well covering the overall expenses, but it iis not covering the premium needs for the claims and the true premium along the portfolio. For the application on profiles, we observe that the ${\mathrm{RF}}_S$ model leads to the results with the lowest variations, whereas the GAM and GLM show much more variations along the age, the horse power, and the location. The final choice of the model will depend on the particular preferences and requirements on the model, e.g., for usage in claims predictions or ratemaking.

Further, based on our data covering the whole Switzerland, to the best of our knowledge, we are the first to provide evidence for the importance of detailed spatial information along longitude and latitude in Switzerland. The traditional cantonal segmentation can be largely improved by integrating local aspects that are related to urbanicity (urban and rural regions) and specific characteristics of the terrain (cities, mountains). Indeed, Figure 4 highlights relevant differences in the claim severity that are not linked to the boundaries of the Swiss cantons. For practitioners and actuaries, this finding can serve to have a better understanding to define relevant risk factors and categories and to choose a model. A more fine-grained classification of customers for their expected severity levels along regions helps to make models more precise and, e.g., to offer more adequate premiums, thus making a company’s ratemaking more competitive, as shown in the papers of Denuit and Lang (2004) and Klein et al. (2014).

Finally, when considering the observations that were made in the present case study, several open research questions remain. While, in this paper, we focus on the log-normal distribution assumption for our data and analyze the available back-transformation, additional insights could be generated by choosing other distributions for the claims, like, for example, a Gamma or inverse Gaussian distribution (see e.g., Henckaerts et al. 2020; Staudt et al. 2001), and to compare these results with the prediction performance obtained here. The back-transformation is leading to high discrepancies on the severity, which are not observed on the log scale. These differences could be analyzed with more details (see also Henckaerts 2020). The deviance measure could be applied to evaluate the model and be compared to the ones that were used here. A thorough analysis of the impact of the tuning parameters on the prediction performance and on the size of the confidence intervals should be performed. Partitioning-out analysis (Belloni et al. 2014; Chernozhukov et al. 2015) could be used to analyze the impact of the different variables in combination with the representations of the selected profiles. While our research compares two families of model, further research could investigate other models, as, for example, penalized regressions and evolutionary trees.