Beyond Boundaries: The AHP-DEA Model for Holistic Cross-Banking Operational Risk Assessment

Abstract

Operational risk assessment has received considerable attention in bank risk management. However, current assessment methods are primarily designed to assess the risk profile of individual banks. To enable cross-bank operational risk assessment, we propose an integrated AHP-DEA (analytic hierarchy process–data envelopment analysis) method. This method determines the importance of assessment criteria by calculating the weighted sum of rank votes after obtaining the importance values for specific rankings with DEA. This procedure replaces the pairwise comparisons in AHP and addresses the challenge of traditional AHPs in determining appropriate importance values when dealing with a large number of indicators. We applied this method to assess the operational risks of three Chinese commercial banks, and the empirical results indicate that this integrated AHP-DEA method is simple and user-friendly, making it suitable for cross-bank operational risk assessment.

1. Introduction

As one of the three primary risks for commercial banks, operational risk has been incorporated into the risk management framework of the Basel II Accord [1]. The Basel II Accord defines operational risk as the risk of direct or indirect loss resulting from inadequate or failed internal processes, people, systems, or external events. This definition includes legal risk, but excludes strategic risk and reputational risk.

In the operational management of commercial banks, operational risk can be multifaceted, with characteristics that are difficult to quantify and predict. Given the potentially significant financial and reputational losses that operational risk can cause to banks, it is necessary to propose a comprehensive and systematic approach to assess operational risk.

The Basel II Accord introduced the measurement of operational risk for the first time [1]. The Accord prescribes three main measurement approaches: the basic indicator approach (BIA), the standardized approach (TSA), and the advanced measurement approach (AMA). With the release of the Basel III Accord, the basic indicator approach (BIA), the standardized approach (TSA), and the advanced measurement approach (AMA) have been integrated into a new approach known as the revised standardized approach [2]. At the same time, many researchers have been actively applying various models for the quantitative assessment of operational risk. According to a systematic review by Cornwell et al. [3], modern statistical and machine learning techniques dominate, including both supervised and unsupervised learning. Supervised learning methods, such as decision trees and artificial neural networks, are used for micro-level risk prediction, while unsupervised learning methods are primarily used for data organization and clustering. Traditional statistical approaches, such as the loss distribution approach (LDA) [4,5,6], extreme value theory (EVT) [7,8,9], copula functions [10,11], and Monte Carlo simulation [12], are widely used in the literature. These methods are mainly applied in the banking sector due to the Basel II operational risk capital requirements. Within the family of graphical probabilistic models, Bayesian networks play a critical role in data-driven operational risk management (ORM) research, which is used to identify factors and causal pathways of ORM events [13,14,15,16,17,18]. Expert systems, such as system dynamics and analytic hierarchy processes [19], are also widely adopted. Hybrid methods play a prominent role, sometimes spanning multiple model families, allowing researchers to flexibly select the most appropriate technology for the overall task, thereby enhancing the application effectiveness of each method [20,21,22].

While the above methods are effective in the assessment of operational risk levels in commercial banks, they do have some limitations:

• Most assessment methods are tailored to individual banks and lack a comprehensive understanding of operational risk across multiple banks or the entire banking industry. Consequently, these methods are primarily applicable to individual banks, with limited utility at the level of banking regulatory authorities.
• Although some methods, such as the analytic hierarchy process (AHP) and the analytic network process (ANP), can be applied to cross-bank risk assessment, they often face challenges in determining appropriate importance values due to the abundance of evaluation criteria.
• While existing methods can reasonably measure the magnitude of risk results, it is difficult to identify the underlying causes and types of risks. Providing useful and targeted recommendations to the bank management and regulatory bodies can be challenging.

Motivated by the above issues, this paper focuses on the assessment of operational risk across multiple banks. Specifically, we develop an integrated AHP-DEA method by combining the analytic hierarchy process (AHP) and data envelopment analysis (DEA). Then, we conduct an assessment of operational risks in three Chinese commercial banks using the proposed integrated AHP-DEA method, following a five-step process. Finally, we conduct a horizontal comparison of operational risks among these three banks and a detailed analysis of the risk-contributing factors associated with each bank, allowing us to make specific recommendations. The AHP-DEA method proposed in this paper can effectively address the limitations mentioned above for the following reasons:

• The integrated AHP-DEA method is a systematic approach used for multi-criteria multi-alternative optimization decision making. It enables a simultaneous horizontal comparison of operational risks among commercial banks.
• The method uses DEA to determine the importance values of specific rankings and then calculates the weighted sum of the rank votes to determine the importance values of the criteria, replacing the pairwise comparisons in the AHP. Therefore, it remains applicable even when faced with a large number of assessment criteria.
• Operational risk assessment is a multi-criteria decision problem that involves the evaluation of internal processes, people, systems, and external events. By analyzing the importance values obtained from DEA for each assessment criterion, we can clearly identify risk contributors and then propose recommendations accordingly.

With this method, we aim to provide a more comprehensive and accurate solution for cross-bank operational risk assessment, and we anticipate that this integrated method will help bank management and regulators make more informed decisions.

The rest of this paper is organized as follows. Section 2 reviews the literature on operational risk assessment criteria for commercial banks and previous studies on combining the AHP and DEA. Section 3 provides a detailed introduction to the preference voting DEA method and introduces a method that integrates the AHP and preference voting DEA. Section 4 presents a five-step procedure for evaluating operational risk levels in commercial banks. It demonstrates the application of the integrated AHP-DEA method to assess operational risk levels in three Chinese banks. Section 5 discusses and elaborates on the empirical results, and Section 6 presents the conclusion.

2. Literature Review

2.1. Operational Risk Assessment Criteria

The first step in assessing operational risk for commercial banks is to establish detailed operational risk assessment criteria, which include both main criteria and their corresponding sub-criteria. As an international financial regulatory body, the Basel Committee has provided a comprehensive classification of operational risk, which serves as an essential reference for scholars in formulating assessment criteria. According to the Basel II Accord [1], operational risk is categorized into four major types: internal process risk, people risk, system risk, and external event risk. These broad categories are further subdivided into seven specific events: internal fraud; external fraud; employment practices and workplace safety; clients, products, and business practices; damage to physical assets; business disruption and system failures; and execution, delivery, and process management. These 7 events are then further subdivided into 20 secondary categories and 70 tertiary categories.

In addition to traditional operational risk factors, many scholars have recognized the importance of ‘soft’ factors in researching risk management in commercial banks. Sanford and Moosa [23,24] argued that operational personnel’s internal and external states are crucial in driving operational loss events, thus incorporating social and organizational factors in operational risk assessment. Thakor [25] examined cultural issues in the banking industry and suggested that a strong corporate culture can foster internal trust within banks and positively influence ethical behavior and stability. Song and Thakor [26] proposed a simplified economic model to analyze bank culture systematically. They found that bank culture plays a role in aligning employees with the bank and influencing the bank’s focus on security. Barth and Monsouri [27] empirically analyzed how banks with different cultural orientations differ in compensation, stock returns, and bankruptcy risk. They highlighted the significance of corporate culture as a ‘soft’ governance factor in enhancing banking stability. These studies emphasize the importance of organizational culture in the operation of commercial banks and provide complementary insights to the reference criteria outlined in the Basel Accord, thus enriching the dimensions of operational risk assessment for commercial banks.

2.2. The Combination of the AHP and DEA

The analytic hierarchy process (AHP) is a hierarchical weighted decision analysis method used for addressing complex multi-criteria decision problems [28]. Data envelopment analysis (DEA) is a non-parametric testing method used for assessing the relative efficiency of comparable units of the same type [29]. The AHP and DEA each possess distinct advantages and disadvantages that have evolved independently over time. However, the idea of combining the AHP and DEA is not new. Many researchers have attempted to establish a relationship between the two methods in order to exploit their strengths and compensate for their respective limitations.

Scholars have utilized the strengths of DEA to address the shortcomings of the AHP using DEA to determine the relative importance values and local priorities within the AHP. Ramanathan [30] proposed the data envelopment analytic hierarchy process (DEAHP) method, which integrates DEA to generate local importance values in the AHP and demonstrated that no ranking reversal occurs when irrelevant alternative scenarios are added or removed. Sevkli et al. [31] successfully applied Ramanathan’s DEAHP method to select suppliers for a Turkish home appliance company and demonstrated the superiority of the DEAHP over the AHP in making more accurate decisions for high-value components. Liu and Hai [31] proposed a novel AHP method called the voting analytic hierarchy process (VAHP), which uses DEA to compute relative importance values of different criteria, replacing pairwise comparisons in the AHP to determine the overall ranking of suppliers. Hadi-Venchehand and Niazi-Motlagh [32] extended the VAHP, addressing some drawbacks of the original model, and successfully applied it to supplier selection problems. Wang et al. [33] proposed an integrated method that combines the AHP and DEA to assess the risk of hundreds or thousands of bridge structures. This method overcomes the limitations of the traditional AHP, which struggles to compare a large number of decision alternatives. Tavana et al. [34] combined the AHP with some weighted methods in multi-criteria decision making, effectively reducing the number of expert judgments and improving the acceptability and efficiency of the results.

Meanwhile, numerous scholars have utilized the advantages of the AHP to overcome the limitations of DEA. Farzipoor Saen et al. [35] proposed a method that integrates DEA and the AHP. This method uses the AHP to measure the relative importance values of decision-making units (DMUs), thereby addressing the issue of the relative efficiency of slightly non-homogeneous DMUs. Joblonsky [36] evaluated the efficiency of Czech Republic pension funds by combining standard DEA with the AHP. In this method, the AHP is used for interval pairwise comparisons to evaluate and classify efficient units. Lozano and Villa [37] introduced two new target-setting DEA methods and used the AHP to quantify the preferences of the decision-makers and to determine the relative importance. To assess the performance of DMUs, Pakkar [38] introduced a theoretical framework integrating DEA and the AHP. By employing the AHP to determine the priority importance values for inputs and outputs, Pakkar enhanced the evaluation within the parametric distance model, thereby improving the relative proximity to desired objectives. Tavana et al. [39] used pairwise comparisons in the AHP to overcome the limitations of traditional DEA and resolve the issue of low discriminative power inherent in conventional DEA methods.

The above literature review indicates that, apart from the voting AHP, all attempts to integrate the AHP and DEA require the construction of a comparison matrix. It becomes impractical when dealing with a large number of DMUs. In contrast, the voting AHP uses a DEA model with common importance values to determine the importance of each criterion and alternative, making it suitable for scenarios involving a considerable number of DMUs. It is suitable to employ the voting AHP for assessing operational risk since the operational risk assessment is essentially a challenging multi-criteria decision-making (MCDM) problem, involving various assessment criteria such as internal process risk, people risk, system risk, and external risk. However, this integrated AHP-DEA method proposed by Liu and Hai [23] has theoretical deficiencies. In this paper, we improve this method with the help of a linear programming model proposed by Wang et al. [40], which will be presented in the next section.

3. Methods

Before introducing the improved integrated AHP-DEA method, we first discuss the application of DEA in preference voting systems, called preference voting DEA in this paper.

3.1. The Preference Voting DEA Method

Data envelopment analysis (DEA) is a non-parametric testing method proposed by Charnes et al. [29] for assessing the relative efficiency of comparable units of the same type. As a mathematical programming approach, DEA can calculate the relative efficiency of decision-making units (DMUs) with multiple inputs and outputs, where the importance values assigned to each DMU are those which maximize the ratio between the weighted output and the weighted input. These importance values are endogenously determined by the DEA model, avoiding the subjectivity and arbitrariness associated with the manual determination of importance values. Currently, DEA is widely applied for efficiency and performance assessment in various fields, including banking [41,42], public policy [43], and economics and finance [44,45]. In addition, DEA is also used in preference voting systems to determine the most favorable importance values for each candidate object. DEA can ensure fair and accurate importance value allocation in evaluations, more precisely reflecting the relative advantages of candidate objects. Next, we will focus on explaining the application and evolution of the DEA model in preference voting systems.

In a preferential voting system, each voter selects $m$ candidates from a pool of $n$ candidates (where $n\ge m$) and ranks them from most preferred to least preferred. Each candidate may receive votes at various ranking positions. The total score for each candidate is the weighted sum of votes received at different positions. These importance values determine the relative importance of each ranking position, ultimately influencing the candidate’s overall score. Therefore, preference voting systems must assign significance values to ranking positions in a scientific manner to ensure a fair and accurate evaluation of candidates. Ultimately, the winner is the candidate with the highest total score.

Let $w_j$ be the relative importance value attached to the $j\mathrm{th}$ ranking place $(j=1,\dots ,m)$ and $v_{ij}$ be the vote of candidate $i$ being ranked in the $j\mathrm{th}$ place. The total score of each candidate is defined as the following:

$$Z_i={\displaystyle \sum _{j=1}^m{v}_{ij}{w}_j},i=1,\dots ,n$$

(1)

which is a linear function of the relative importance values. Once the values are given or determined, candidates can be ranked in terms of their total scores.

To determine the relative importance values, Cook and Kress [46] suggested the following DEA model, which determines the most favorable values for each candidate:

$$\begin{array}{cc}\hfill Maximize& Z_i={\displaystyle \sum _{j=1}^m{v}_{ij}{w}_j}\hfill \\ \hfill s.t.& {\displaystyle \sum _{j=1}^m{v}_{ij}{w}_j\le 1,i=1,\dots ,n}\hfill \\ & w_j-w_{j+1}\ge d(j,\epsilon ),j=1,\dots ,m-1\hfill \\ & w_m\ge d(m,\epsilon )\hfill \end{array}$$

(2)

where $d(.,\epsilon )$ is referred to as a discrimination intensity function that is non-negative and monotonically increasing in a non-negative discriminating intensity factor $\epsilon$ and satisfies $d(.,0)=0$. It has been found that the choice of the discrimination intensity functional $d(.,\epsilon )$ and the discriminating intensity factor $\epsilon$ has significant impacts on the winner. For example, Cook and Kress [46] investigated three special cases of the discrimination intensity function ($d(j,\epsilon )=\epsilon ,d(j,\epsilon )=\epsilon /j$, and $d(j,\epsilon )=\epsilon /j!$). Each of them leads to a different winner. Noguchi et al. [47] examined the six special discriminating intensity factor cases ($\epsilon :$$\epsilon =0,0.01,0.05,0.055,0.06$, and $0.07$). These cases also result in different winners.

To avoid the difficulties in determining the discrimination intensity function $d(.,\epsilon )$ and the discriminating intensity factor $\epsilon$, Noguchi et al. [47] suggested a strong ordering DEA model, which is shown below:

$$\begin{array}{cc}\hfill Maximize& Z_i={\displaystyle \sum _{j=1}^m{v}_{ij}{w}_j}\hfill \\ \hfill s.t.& {\displaystyle \sum _{j=1}^m{v}_{ij}{w}_j\le 1,}i=1,\dots ,n\hfill \\ & w_1\ge 2w_2\ge \dots \ge m{w}_m\hfill \\ & w_m\ge \epsilon =\frac{2}{Nm(m+1)}\hfill \end{array}$$

(3)

where $N$ is the number of voters. In this model, the strong ordering constraint $w_1\ge 2w_2\ge \dots \ge m{w}_m$ makes sense because it satisfies $w_1>w_2>\dots >m{w}_m$ and $w_1-w_2>w_2-w_3>\dots >w_{m-1}-w_m>0$. It also makes the choice of the discrimination intensity function $d(.,\epsilon )$ unnecessary. However, it is found that the choice of the discriminating intensity $2/Nm(m+1)$ [48]. In effect, $\epsilon$ can take any value within the interval $\left[0,1/Nm\right]$. In addition, to determine the value of $\epsilon$ in model (3), the number of voters needs to be known, but this is not always the case.

In what follows, three new models presented by Wang et al. [40] do not require any predetermination of parameters because the new models usually produce only one best candidate and there is no need to make any further choice with the help of any parameters. The new models are given as follows:

$$\begin{array}{cc}\hfill LP\text{-}1:& \\ \hfill Maximize& \alpha \hfill \\ \hfill s.t.& Z_i={\displaystyle \sum _{j=1}^m{v}_{ij}{w}_j}\ge \alpha ,i=1,\dots ,n\hfill \\ & w_1\ge 2w_2\ge \dots \ge m{w}_m\ge 0\hfill \\ & {\displaystyle \sum _{j=1}^m{w}_j=1}\hfill \end{array}$$

(4)

$$\begin{array}{ll}\hfill LP\text{-}2:& \\ \hfill Maximize& \alpha \hfill \\ \hfill s.t.& \alpha \le Z_i={\displaystyle \sum _{j=1}^m{v}_{ij}{w}_j}\le 1,i=1,\dots ,n\hfill \\ & w_1\ge 2w_2\ge \dots \ge m{w}_m\ge 0\end{array}$$

(5)

$$\begin{array}{ll}\hfill NLP\text{-}1:& \\ \hfill Maximize& Z_i={\displaystyle \sum _{j=1}^m{v}_{ij}{w}_j}\hfill \\ \hfill s.t.& w_1\ge 2w_2\ge \dots \ge m{w}_m\ge 0\hfill \\ & {\displaystyle \sum _{j=1}^m{w}_j^2=1}\end{array}$$

(6)

$LP\text{-}1$ and $LP\text{-}2$ are two linear programming models. Both of them maximize the minimum of the total scores of the $n$ candidates and determine a common set of importance values for all the candidates. The differences between the two models lie in that $LP\text{-}1$ requires the importance values to be summed to one, while $LP\text{-}2$ does not, and that $LP\text{-}2$ requires the total score of each candidate to be equal to or less than one, while $LP\text{-}1$ has no such a requirement. Once the importance values are determined, the total score of each candidate can be computed by Equation (7) and the winner can be selected.

$NLP\text{-}1$ is a nonlinear programming model which determines the most favorable importance values within the feasible region

$$\Omega =\left\{W=(w_1,\dots ,w_m)|w_1\ge 2w_2\ge \dots \ge m{w}_m\ge 0,{\displaystyle \sum _{j=1}^m{w}_j^2=1}\right\}$$

for each candidate.

3.2. The Improved Integrated AHP-DEA Method

The analytic hierarchy process (AHP) is a method developed by Saaty [28] to support multi-criteria decision making. Figure 1 illustrates how it decomposes a complex multi-criteria decision-making (MCDM) problem into hierarchical structures based on the decision objective, criteria (including sub-criteria if applicable), and alternative solutions in sequential order. This method determines the priority importance values of each lower-level element relative to a higher-level element by solving the characteristic vector of the decision matrices. It then utilizes a simple weighting method to calculate the total importance values of the decision alternatives for the overall decision objective. The alternative with the highest important value is considered the optimal solution.

The strongest feature of the AHP is its ability to express subjective knowledge by estimating pairwise comparison matrices and generating numerical priorities. However, when dealing with more general problems, the construction of the necessary pairwise comparison matrices becomes increasingly complex, cumbersome, and extensive as the number of selected indicators grows. In such cases, experts may find it challenging to accurately assess the importance of each pair of indicators during pairwise comparisons.

To address the above issue, Liu and Hai [23] introduced the voting analytic hierarchy process (VAHP). This method combines the AHP and DEA to derive collective preferences of criteria from the ordered preferences of individual group members. Group members express their preferences through a ranking voting system. The approach significantly reduces the workload compared to the AHP alone. Liu and Hai [23] specifically adopted the DEA method proposed by Noguchi et al. [47] to implement the VAHP method. However, as shown in Section 3.1, the theoretical foundation of this method has been questioned in the literature [48]. To enhance the method’s reliability, we decide to replace Noguchi et al.’s DEA method with a linear programming model proposed by Wang et al. [40] to implement the VAHP method. Next, we will provide a detailed explanation of the principles and procedures of this improved integrated AHP-DEA method.

We can assume that $C_1,\dots ,C_m$ represent $m$ decision criteria and $W={\left(w_1,\dots ,w_m\right)}^T$ is their normalized relative importance weight vector, satisfying the normalization condition ${\displaystyle {\sum }_{j=1}^m{w}_j=1},(w_j\ge 0,j=1,\dots ,m)$. When using the AHP method, the weight vector can be determined by comparing each of the $m$ decision criteria in pairs using the scale provided in

Table 1. The 1–9 scales for pairwise comparisons in the AHP.

Importance Intensity	Definition
1	Equal importance
3	Moderate importance
5	Strong importance
7	Very strong importance
9	Extreme importance
2, 4, 6, and 8	Intermediate values
Reciprocals	Reciprocals for inverse comparison

. An $m\times m$ matrix is formed based on these pairwise comparisons, which is commonly referred to as the pairwise comparison matrix. The matrix is defined below.

$$A={(a_{ij})}_{m\times m}=\begin{array}{c}{C}_1\\ C_2\\ ⋮\\ C_m\end{array}\left[\begin{array}{cccc}{a}_{11}& a_{12}& \cdots & a_{1m}\\ a_{21}& a_{22}& \cdots & a_{2m}\\ ⋮& ⋮& \cdots & ⋮\\ a_{m1}& a_{m2}& \cdots & a_{mm}\end{array}\right]$$

(7)

where $a_{ij}$ represents the result of the importance comparison between decision criteria $w_i$ and $w_j$, satisfying the characteristics of $a_{ij}=1$ and $a_{ij}=1/a_{ji}$. If the pairwise comparison matrix $A={(a_{ij})}_{m\times m}$ satisfies $a_{ij}=a_{ik}{a}_{kj}$ for any $i,j,k=1,\dots ,m$, $A$ is said to be perfectly consistent; otherwise, it is said to be inconsistent.

For the weight vector $W$, it can be determined by solving the following characteristic equation:

$$AW={\lambda }_{\max }W$$

(8)

where ${\lambda }_{\max }$ represents the maximum eigenvalue of the pairwise comparison matrix $A$.

When dealing with a large number of decision criteria, constructing pairwise comparison matrices and solving for the weight vector can become complex and time-consuming. To address this issue effectively, the preference voting DEA method can be used.

This method treats each decision criterion as a candidate, and experts only need to rank these candidates based on their professional knowledge and experience. The importance value of each decision criterion in respect to the decision objective can be defined as follows:

$$w_j={\displaystyle \sum _{k=1}^m{v}_{jk}}{u}_k,j=1,\dots ,m;k=1,\dots ,l$$

(9)

where $u_k$ represents the important value given to the $k\mathrm{th}$ ranking position and $v_{jk}$ represents the number of votes received by decision criterion $j$ at the $k\mathrm{th}$ position.

To determine the important value of each decision criterion with respect to the decision objective, we view each decision criterion as a decision unit (DMU), consider $u_k$ as the decision variable and also as the important value assigned to the output $v_{jk}$, and then construct the following DEA model with common importance values:

$$\begin{array}{rr}Maximize\alpha & \\ \hfill s.t.& \alpha \le w_j={\displaystyle \sum _{k=1}^m{v}_{jk}}{u}_k\le 1,j=1,\dots ,m\hfill \\ & u_1\ge 2u_2\ge \dots \ge l{u}_l\ge 0\hfill \end{array}$$

(10)

where $u_1,\dots ,u_l$ are decision variables, and $u_1\ge 2u_2\ge \dots \ge l{u}_l\ge 0$ represent the strong ordering conditions imposed on the decision variables.

The above model is derived from the DEA model developed by Wang et al. (model (5)) for preferential voting aggregation replacing Nougchi’s DEA model (model (3)) used by Liu and Hai [23]. By solving this model, we obtain the normalized relative importance weight vector for $m$ decision criteria.

To identify the importance of each decision alternative under each decision criterion, a same approach can be applied. After obtaining the importance values for decision criteria and alternatives through the preference voting DEA model, the overall important value for each alternative relative to the decision objective can be generated using the following simple additive weighting ($SAW$) method:

$$w_{A_i}={\displaystyle \sum _{j=1}^m{w}_{ij}{w}_j},i=1,\dots ,n$$

(11)

where $w_j(j=1,\dots ,m)$ is the important value of the decision criterion, $w_{ij}(i=1,\dots ,n)$ is the important value of the alternative under the decision criterion, and $w_{A_i}(i=1,\dots ,n)$ is the overall important value of the alternative relative to the decision objective. Based on these overall importance values, decision alternatives can be prioritized and decisions can be made. The decision alternative with the highest overall important value is considered the optimal decision.

4. Application of AHP-DEA in Operational Risk Assessment for Commercial Banks

In this section, we apply the proposed AHP-DEA method to assess operational risks in commercial banks. To measure the relative levels of operational risk among different commercial banks, we establish a model for evaluating and assessing operational risk levels in commercial banks based on the following five steps. The model is a hierarchical decision-making model, and the problem is to assess the operational risk profiles of three Chinese commercial banks. The first step involves structuring the problem into a hierarchy. The top level of the model represents the overall goal of assessing operational risk levels in commercial banks. The second level consists of five criteria contributing to this goal. The third level further divides these five criteria into nineteen sub-criteria. The bottom level comprises the three commercial banks participating in the assessment. Figure 2 illustrates the hierarchical structure.

4.1. Selecting Operational Risk Assessment Criteria

The Basel II Accord classifies operational risk as internal processes, people, systems, and external event risks [1]. Hence, these four indicators were selected as criteria for assessing operational risk. Additionally, as highlighted in Section 2.1, many scholars in the field of risk management in commercial banks consider ‘soft’ aspects. Therefore, we incorporated organizational culture into the assessment criteria for operational risk, resulting in a total of five indicators for evaluating operational risk levels in commercial banks.

4.2. Selecting Sub-Criteria under the Assessment Criteria

For the sub-criteria, we first referred to the operational categorization in the Basel II Accord (see Section 2.1) and selected 15 indicators as sub-criteria for internal process risk, people risk, system risk, and external event risk. These sub-criteria include internal process risk (error monitoring and reporting; settlement payment errors; document and contract defects; and execution errors), people risk (internal fraud; negligence and non-compliance; employee business capability; and personnel stability), system risk (system security and system failure), and external risk (external fraud; policy risk; external competition; legal risk; and external unforeseen event risk).

Regarding organizational culture risk, although the Basel Committee does not explicitly define this category, we referred to the scholars’ research mentioned in Section 2.1. We selected four indicators as sub-criteria for organizational culture risk, including organizational culture, governance structure, internal control system, and compensation system. Through precisely defined assessment criteria and sub-criteria, we constructed a hierarchically structured operational risk assessment model (Figure 2).

4.3. Prioritize the Order of Criteria or Sub-Criteria

In this study, 30 experts participated in the assessment. They included university professors, bank managers, and bank supervisors. We used a combined approach of online and offline data collection using a detailed questionnaire to invite experts to rank the criteria and sub-criteria for operational risk assessment. The online survey was conducted via email and online survey platforms, while the offline survey involved face-to-face interviews.

4.3.1. Prioritize the Order of Criteria

We established a total of five criteria: (1) internal process risk, (2) people risk, (3) system risk, (4) external risk, and (5) organizational risk. These criteria were treated as candidates, and each expert was tasked with providing an ordered list, indicating their priority ranking for these candidates. The distribution of votes for each candidate at different positions is illustrated in

Table 2. Priority votes of 5 criteria from 30 experts.

Criteria	1st	2nd	3rd	4th	5th	Total
Internal process risk	0	11	11	6	2	30
People risk	28	0	2	0	0	30
System risk	2	8	13	5	2	30
External risk	0	9	2	17	2	30
Organizational risk	0	2	2	2	24	30
Total	30	30	30	30	30	30

.

4.3.2. Prioritizing the Order of Sub-Criteria

Using the same approach, the nineteen sub-criteria were treated as candidates, and the votes received by each candidate are presented in

Table 3. Priority votes of internal process risk sub-criteria from 30 experts.

Sub-Criteria	1st	2nd	3rd	4th	Total
Error monitoring and reporting	2	4	4	20	30
Settlement payment errors	16	5	4	5	30
Document and contract defects	5	4	18	3	30
Execution errors	7	17	4	2	30
Total	30	30	30	30	30

,

Table 4. Priority votes of people risk sub-criteria from 30 experts.

Sub-Criteria	1st	2nd	3rd	4th	Total
Internal fraud	28	0	2	0	30
Negligence and non-compliance	2	28	0	0	30
Employee business capability	0	2	21	7	30
Personnel stability	0	0	7	23	30
Total	30	30	30	30	30

,

Table 5. Priority votes of system risk sub-criteria from 30 experts.

Sub-Criteria	1st	2nd	Total
System security	25	5	30
System failure	5	25	30
Total	30	30	30

,

Table 6. Priority votes of external risk sub-criteria from 30 experts.

Sub-Criteria	1st	2nd	3rd	4th	5th	Total
External fraud	20	4	4	2	0	30
Policy risk	2	19	2	5	2	30
External competition	4	0	19	2	5	30
Legal risk	2	2	3	19	4	30
External unforeseen event risk	2	5	2	2	19	30
Total	30	30	30	30	30	30

and

Table 7. Priority votes of organizational risk sub-criteria from 30 experts.

Sub-Criteria	1st	2nd	3rd	4th	Total
Organizational culture	20	0	4	6	30
Governance structure	2	24	2	2	30
Internal control system	8	4	18	0	30
Compensation system	0	2	6	22	30
Total	30	30	30	30	30

.

4.4. Calculating the Importance Values of Criteria or Sub-Criteria

We determined the importance values of different-level criteria in the analytic hierarchy process using model (10), which is derived from the LP-2 model proposed by Wang et al. [40]. In fact, Wang et al. [40] introduced three new models, but we chose the LP-2 model because it allows for the establishment of a universal normalized set of importance values. This ensures that all candidate factors are evaluated on a common basis and simplify the weighting process.

4.4.1. Calculating the Importance Values of Criteria

We used the data from Table 2 and calculated the importance values for the five criteria using model (10). As shown in

Table 8. Importance values of 5 criteria.

Criteria	Importance Values (Normal)
Internal process risk	4.847 (0.162)
People risk	12.555 (0.418)
System risk	5.248 (0.175)
External risk	4.299 (0.143)
Organizational risk	3.051 (0.102)

, the importance values assigned to internal process risk, people risk, system risk, external risk, and organizational risk were 4.847, 12.555, 5.248, 4.299, and 3.051, respectively. After normalizing the data, the results were 0.162, 0.418, 0.175, 0.143, and 0.102.

4.4.2. Calculating the Importance Values of Sub-Criteria

Using the same procedure, we solved for the data in Table 3, Table 4, Table 5, Table 6 and Table 7. We required the sum of the importance values for each sub-criterion under each criterion to be equal to 1. The results are presented in

Table 9. Importance values of 19 sub-criteria.

Sub-Criteria	Importance Value (Normal)	Sub-Criteria	Importance Value (Normal)
Error monitoring and reporting	4.960 (0.165)	External fraud	10.438 (0.348)
Settlement payment errors	10.120 (0.337)	Policy risk	6.051 (0.202)
Document and contract defects	6.600 (0.220)	External competition	5.182 (0.173)
Execution errors	8.320 (0.277)	Legal risk	4.182 (0.139)
		External unforeseen event risk	4.146 (0.138)
Internal fraud	13.760 (0.459)
Negligence and non-compliance	7.680 (0.256)
Employee business capability	4.680 (0.156)
Personnel stability	3.880 (0.129)	Organizational culture	10.960 (0.365)
		Governance structure	7.280 (0.243)
System security	18.333 (0.611)	Internal control system	7.680 (0.256)
System failure	11.667 (0.389)	Compensation system	4.080 (0.136)

.

4.4.3. Calculating the Global Importance Values of Sub-Criteria

The importance values calculated in Table 9 represent the local importance values of each sub-criterion under its respective criterion. To determine the global importance values of each sub-criterion with respect to the overall objective, we chose to multiply the sub-criterion importance values by their corresponding criterion importance values (as shown in Table 8). Taking internal fraud as an example, a sub-criterion under the people risk criterion, its normalized local importance value was 0.459. After multiplying this by the importance value of the people risk criterion, its global importance value was found to be 0.1921. Using this calculation method, we could derive the global importance values of all sub-criteria in relation to the overall objective, and the results are detailed in

Table 10. Global importance values of 19 sub-criteria relative to overall objective.

Sub-Criteria	Global Importance Value	Sub-Criteria	Global Importance Value
Error monitoring and reporting	0.0268	External fraud	0.0499
Settlement payment errors	0.0546	Policy risk	0.0288
Document and contract defects	0.0356	External competition	0.0246
Execution errors	0.0449	Legal risk	0.0199
		External unforeseen event risk	0.0197
Internal fraud	0.1921
Negligence and non-compliance	0.1071
Employee business capability	0.0653
Personnel stability	0.0540	Organizational culture	0.0372
		Governance structure	0.0248
System security	0.1069	Internal control system	0.0261
System failure	0.0680	Compensation system	0.0139

. Importantly, these global importance values could be directly applied to calculate the operational risk profiles of commercial banks. By calculating the sum of the products of the global importance value of each sub-criterion and the performance score under that criterion for each commercial bank, we could obtain the operational risk score for each commercial bank. In the following, we will provide a detailed explanation of this calculation method.

4.5. Calculating Performance Scores for Commercial Banks

4.5.1. Determining Performance Criteria

To determine the magnitude of operational risk for commercial banks, it is necessary to calculate the performance scores of each bank under each sub-criterion. To ensure consistency in the scoring criteria and avoid biases, we referred to the supplier performance scoring criteria developed by Lin and Hai [23] for supplier selection. After collaborating with 30 experts, we developed guidelines for determining performance scores for commercial banks (see

Table 11. Guidelines for scoring operational risk levels in commercial banks.

Grade	Very Low	Low	Medium	High	Very High
Score	0/1	2/3	5	7/8	9/10

). The guidelines use an 11-point grade scale, with each grade consisting of an adjective describing the bank’s performance under each sub-criterion and a corresponding score point or range. The experts evaluated the bank’s performance based on objective criteria, and we assigned performance scores to commercial banks accordingly. In summary, commercial banks received scores ranging from 0 to 10 for each sub-criterion.

4.5.2. Calculating Performance Scores

We selected three Chinese banks as alternatives to measure the relative levels of operational risk. The three banks were Bank A, a large state-owned traditional commercial bank; Bank B, a domestically listed joint-stock bank; and Bank C, a domestically listed urban commercial bank. The experts conducted qualitative assessments of the performance of the banks under each sub-criterion, referring to the guidelines in Table 11. Subsequently, based on the corresponding score points from these evaluations, we obtained performance scores for the three banks. Using these scores, we derived the operational risk scores for each bank. The bank with the highest score was considered to have the highest operational risk, and the other commercial banks were ranked accordingly. From a mathematical perspective, the operational risk score of a commercial bank was calculated as the sum of the products of each sub-criterion’s global importance value and the bank’s performance score under that criterion. The performance scores and operational risk scores for the three commercial banks are presented in

Table 12. The performance scores and operational risk scores for Bank A.

Criteria	Sub-Criteria	Importance Value	Score	Sub-Total
Internal process risk	Error monitoring and reporting	0.0268	4	0.1072
	Settlement payment errors	0.0546	4	0.2184
	Document and contract defects	0.0356	3	0.1068
	Execution errors	0.0449	4	0.1796
People risk	Internal fraud	0.1921	4	0.7684
	Negligence and non-compliance	0.1071	4	0.4284
	Employee business capability	0.0653	2	0.1306
	Personnel stability	0.0540	2	0.1080
System risk	System security	0.1069	4	0.4276
	System failure	0.0680	1	0.0680
External risk	External fraud	0.0499	2	0.0998
	Policy risk	0.0288	1	0.0288
	External competition	0.0246	2	0.0492
	Legal risk	0.0199	2	0.0398
	External unforeseen event risk	0.0197	2	0.0394
Organizational risk	Organizational culture	0.0372	3	0.1116
	Governance structure	0.0248	3	0.0744
	Internal control system	0.0261	3	0.0783
	Compensation system	0.0139	4	0.0556
Total risk scores				3.1199

,

Table 13. The performance scores and operational risk scores for Bank B.

Criteria	Sub-Criteria	Importance Value	Score	Sub-Total
Internal process risk	Error monitoring and reporting	0.0268	3	0.0804
	Settlement payment errors	0.0546	4	0.2184
	Document and contract defects	0.0356	3	0.1068
	Execution errors	0.0449	4	0.1796
People risk	Internal fraud	0.1921	4	0.7684
	Negligence and non-compliance	0.1071	4	0.4284
	Employee business capability	0.0653	2	0.1306
	Personnel stability	0.0540	4	0.2160
System risk	System security	0.1069	4	0.4276
	System failure	0.0680	1	0.0680
External risk	External fraud	0.0499	3	0.1497
	Policy risk	0.0288	2	0.0576
	External competition	0.0246	4	0.0984
	Legal risk	0.0199	2	0.0398
	External unforeseen event risk	0.0197	3	0.0591
Organizational risk	Organizational culture	0.0372	1	0.0372
	Governance structure	0.0248	2	0.0496
	Internal control system	0.0261	2	0.0522
	Compensation system	0.0139	1	0.0139
Total risk scores				3.1817

and

Table 14. The performance scores and operational risk scores for Bank C.

Criteria	Sub-Criteria	Importance Value	Score	Sub-Total
Internal process risk	Error monitoring and reporting	0.0268	2	0.0536
	Settlement payment errors	0.0546	3	0.1638
	Document and contract defects	0.0356	4	0.1424
	Execution errors	0.0449	3	0.1347
People risk	Internal fraud	0.1921	3	0.5763
	Negligence and non-compliance	0.1071	3	0.3213
	Employee business capability	0.0653	4	0.2612
	Personnel stability	0.0540	3	0.1620
System risk	System security	0.1069	3	0.3207
	System failure	0.0680	2	0.1360
External risk	External fraud	0.0499	4	0.1996
	Policy risk	0.0288	3	0.0864
	External competition	0.0246	4	0.0984
	Legal risk	0.0199	4	0.0796
	External unforeseen event risk	0.0197	5	0.0985
Organizational risk	Organizational culture	0.0372	3	0.1116
	Governance structure	0.0248	4	0.0992
	Internal control system	0.0261	4	0.1044
	Compensation system	0.0139	3	0.0417
Total risk scores				3.1914

.

5. Discussion

Table 8 and Table 10 display the importance values assigned by experts to the assessment criteria and sub-criteria for operational risk levels in commercial banks. The criteria are ranked as follows: people risk (0.418), system risk (0.175), internal process risk (0.162), external risk (0.143), and organizational risk (0.102). For the sub-criteria under each criterion, factors with global importance values equal to or greater than 0.05 are considered the most significant aspects of operational risk levels in commercial banks. The ranking for this is as follows: internal fraud (0.1921), negligence and non-compliance (0.1071), system security (0.1069), system failure (0.0680), employee business capabilities (0.0653), settlement payment errors (0.0546), and personnel stability (0.0540).

People risk is a crucial factor in operational risk for commercial banks, particularly in relation to internal fraud, negligence, and non-compliance. Therefore, effective management and considerable attention are required in order to address people risk, which remains the most significant factor in today’s operational risk management for commercial banks. Contrary to previous research conclusions, this study assigns a relatively high importance value to system risk, indicating that with the rapid growth of digital banking in China, system risk has become more prominent, highlighting the importance of system security in the digital era. Internal process risk, particularly in settlement and payment errors, is also a focal point due to the increasing scale and complexity of current banking operations. While external and organizational risks may be ranked lower in terms of importance value, they are equally important.

Table 15. Summary of performance scores for three banks.

Risk Type	Banks
	Bank A	Bank B	Bank C
People process risk	1.4354	1.5434	1.3208
Internal process risk	0.6120	0.5852	0.4945
System risk	0.4956	0.4956	0.4567
External risk	0.2570	0.4046	0.5625
Organizational risk	0.3199	0.1529	0.3569
Total risk	3.1199	3.1817	3.1914

presents a summary of the performance scores for three commercial banks based on different criteria. By combining data from Table 12 and Table 14, we can analyze the magnitude of operational risk and the variations in contributing factors among the three banks.

The study’s findings indicate that Bank C has an operational risk score of 3.1914, Bank B has a score of 3.1817, and Bank A has a score of 3.1199. Thus, this study asserts the operational risk profile of Bank C as the highest, followed by Bank B, with Bank A having the lowest operational risk profile.

In terms of people risk, Banks A and B have a higher level of people risk, while Bank C has a lower level of people risk. This is mainly due to the larger scale, extensive business scope, and complex financial transactions of Banks A and B, which expose them to higher risks of internal fraud, negligence, and non-compliance. Furthermore, as a joint-stock bank, Bank B may face greater competitive pressure, leading to a higher employee turnover rate, which further increases the risk. In contrast, Bank C is smaller in scale with simpler operations and lower employee turnover, thereby reducing people risk.

In terms of internal process risk, Banks A and B have a higher level of risk than Bank C due to their larger size and more complex operations. This complexity increases the likelihood of internal process errors, especially for Bank A, which is subject to more stringent regulatory and compliance requirements. In contrast, Bank C has simpler operations, resulting in lower internal process risk.

In terms of systemic risk, there is not much difference between the three banks. Modern banking institutions typically have similar levels of investment and management in information technology, resulting in relatively low risks related to system failure. Bank C, being smaller in scale with simpler operations and fewer customers, is less likely to be a primary target for cyber attacks, thereby reducing system security risk.

In terms of external risk, Bank C presents the highest level of risk, followed by Bank B, while Bank A presents the lowest level of risk. Bank C’s smaller scale may hinder its ability to conduct comprehensive credit assessments of borrowers, increasing the risk of external fraud. Furthermore, Bank C faces challenges in diversifying risks, making it more vulnerable to the impact of external events and increasing exposure to external competition and legal risks. In contrast, Bank A, being a state-owned bank, benefits from greater resources and government support, which reduces its external risk.

In terms of organizational risk, Bank B has the lowest organizational risk, while Banks C and A have relatively higher organizational risks. Bank B, being a joint-stock bank, places more emphasis on market orientation and competitiveness. It possesses a more flexible organizational culture and internal control system that enables better adaptation to market demands. In contrast, Bank A, being a state-owned bank, is susceptible to government intervention, with lower flexibility and incentive mechanisms. Bank C, being smaller in scale and facing intense competition, may encounter challenges in establishing a comprehensive internal control system and governance structure due to limited resources.

Therefore, customized recommendations can be made for different banks: for Bank C, strengthening preventive measures against external risks, especially unforeseen external events and external competitive risks, is recommended; for Bank B, the focus should be on mitigating people risks, especially personnel stability risks and internal fraud risks; and for Bank A, more attention should be paid to organizational risks, with an emphasis on cultivating organizational culture and making efforts to establish a flexible compensation system.

6. Conclusions

The scientific assessment of operational risk is crucial for the stable operation of commercial banks. However, most existing methods focus solely on risk assessment for individual banks and fail to elaborate on specific risk contributors. To address these challenges, this paper proposes an improved integrated AHP-DEA method. This method combines the strengths of the AHP and DEA, making it suitable for addressing the complexities of operational risk assessment, even when dealing with multiple assessment criteria and multiple commercial banks. We applied this method to assess the operational risk profiles of three Chinese commercial banks through a five-step process. Our analysis compares the operational risks of each company and provides detailed insights into their respective risk contributors, along with targeted recommendations. The following conclusions can be drawn: (1) people risk plays a crucial role in the operational risk assessment of Chinese commercial banks; (2) with the development of digital banking, the importance of system risk in the operational risk assessment has significantly increased; (3) different types of banks face different types and degrees of operational risk, which are closely correlated with factors such as their asset size and business complexity; and (4) political factors significantly affect the operational risk profiles of Chinese commercial banks, especially state-owned banks.

The main contributions of this paper are as follows. (1) This paper presents an integrated AHP-DEA method for cross-bank operational risk assessment, addressing the limitations of existing methods that primarily focus on risk assessment within individual banks. This fills a gap in the existing literature. (2) This paper improves the voting DEA method by replacing the original DEA model, increasing its reliability and applicability in commercial bank operational risk assessment. This enhancement improves the practicality and applicability of the method in addressing other multi-criteria decision-making problems. (3) This paper successfully applies the proposed model to assess operational risks in three Chinese commercial banks, revealing the specific risk characteristics faced by these banks. This analysis provides valuable decision support to bank management and regulatory authorities.

There are still limitations in this paper. We suggest some directions for future research. (1) Despite the adoption of widely accepted operational risk assessment standards, resource and time constraints prevented the coverage of all potential operational risk factors, which may lead to the neglect of certain indicators in the decision criteria. Therefore, future research could further investigate operational risk factors to ensure comprehensive assessment criteria. (2) The allocation of importance values may be affected by individual differences among experts. Therefore, it is necessary to establish a more authoritative team of experts to ensure the accuracy and reliability of assessment results. (3) The importance values of different factors may change over time and across regions. Therefore, it is worth considering the application of the proposed integrated method in other countries and regions in order to analyze the unique circumstances and dynamics that exist in different nations and geographic areas. Meanwhile, it is necessary to regularly update the conclusions to reflect industry changes, thereby enhancing the timeliness and reliability of the research’s practical application.