On the Assessment of e-Banking Websites Supporting Sustainable Development Goals

Abstract

The main aim of this article was to test the authors’ proprietary method (i.e., the conversion method applied to evaluate e-banking services that support sustainable development goals in households, communities, and society). The authors’ conversion method can be applied with the aim of maintaining a balance between households, producers, and public administration services in line with the principles of sustainable development of the information society in Poland. To achieve this goal, the authors identified the differences between the results obtained using the conversion method and the results produced by other methods such as TOPSIS, Promethee II, and PROSA involving the same group of respondents. A hypothesis was made about the existence of significant differences in the results obtained as part of the studies. The research was carried out on a sample of nearly 830 ratings concerning the 27 most popular electronic banks in Poland. As part of the survey, the respondents assessed 18 characteristics (attributes) of the selected banks using a simplified Likert scale. The study was conducted during the pandemic in Poland in 2020. The authors compared the results achieved in the case of the TOPSIS, Promethee II, and PROSA methods and the ones obtained with the application of the conversion method. Then, the ratings of the e-banking websites were arranged in descending order, and the distances between the positions in the rankings obtained by the conversion method and other methods were calculated. In addition, the R2 correlation coefficients were calculated for all combinations of the results received using individual methods. The results showed the greatest differences both in the absolute distances between the positions obtained in the ranking and the lowest value of the R2 correlation coefficient in the case of the conversion method in relation to the other methods. The limitation of the present research resulted from the fact that the study sample included respondents who were all members of the academic environment. The students analyzed in the study were part of a group supporting globalization processes where e-business solutions are widely used. However, the purchases of goods and services both local and foreign made by this group were often limited in scope and value due to a lack of funds. The research results indicate a potential need for improvement of the conversion method.

1. Introduction

The concept of a sustainable information society is interpreted by many researchers differently and sometimes even in contradictory ways [1,2,3]. This phenomenon has been broadly discussed by academics and researchers (e.g., [4]), and the concept of information society sustainability focuses on four basic spheres [5]:

• Environmental sustainability: environmental protection with the consideration of renewable eco-localization of the economy, ecological and economic efficiency, and the integrity and convergence of economic development and environmental protection (e.g., [6]);
• Economic sustainability: sustainable economic growth (associated with better decision making) under the conditions of high competition and free movement of goods, services, and capital (e.g., [7]);
• Sociocultural sustainability: solving sociocultural problems with a view of harmonious development in line with the principles of equal economic opportunities, justice, health protection, and ethics, among other factors (e.g., [8,9,10,11]);
• Political sustainability: stability of institutions and public administration in collaboration with other stakeholders, reflected in making public decisions in cooperation with the society and sustainable development of a civil society ([8,12]).

Numerous mutual relations between these spheres fall within the scope of sustainable development of an information society. These relations point to three major stakeholders of the information society: households, enterprises, and public administration bodies [13,14].

This article concerns the first of the spheres, namely increasing the chances of households engaging in sustainable development in relation to the other two spheres. The considerations presented in this paper are an extension of the research conducted by the authors to date [15,16]. Increased opportunities related to sustainability result from greater awareness of individual e-banking customers (i.e., the average stakeholders representing households) as to which banking service is best for them from the point of view of quality and usability. The authors also attempted to search for the “ideal” method to determine the reference of sustainable development in this area [17,18].

A few years ago, negative past experiences related to assessing the quality of websites using the AHP (I) [19] and Promethee II methods prompted the authors to create their own conversion method. This method is based on the distance to the average results obtained in the case of a particular bank, and it allows for eliminating the subjectivity of scores resulting from survey-based quantitative research. In addition, it facilitates the assessment of the quality of e-banking websites by a randomly selected user [20], for whom the tables of the AHP method [21,22] might appear too complicated and difficult to complete. Another problem which the new conversion method attempted to address was the fact that additional preferences assessed in other methods proved at times to be too abstract to be properly assessed by the respondents. The initially obtained results were so promising that the method began to be repeatedly verified, and it was applied along with other methods to evaluate e-banking and m-banking services [23,24].

The goal behind the implementation of the authors’ own conversion method was to address the issues appearing in assessing banking services through the use of simple methods (e.g., scoring method and scoring method with preferences) and the AHP method. The problems which the respondents indicated were mainly connected with the process of completing the tables correctly. At the same time, the authors did not want to lose the advantages of the AHP method (e.g., lower subjectivity of assessment). The use of scoring methods and the AHP and conversion methods in the research involving the same study sample placed the results obtained with the conversion method between the results of the scoring methods and the results obtained with the AHP method [24]. However, when other MCDA methods were used in later studies, the results of the conversion method started to be significantly different from the results of the Topsis, Promethee II, and Comet methods [25,26,27]. Therefore, the question arose of whether the conversion method should not generate results convergent with other methods if it were to be recommended as one of the methods ensuring a balance in society between the spheres of households, enterprises, and public administration. After finding out that the results of the conversion method differed from other methods selected for the study, the derivative question emerged: How can the abovementioned goal be obtained in the course of developing the conversion method?

Therefore, in this article, the H₀ hypothesis is formulated about the existence of significant differences between the results of the conversion method and the results obtained by the other selected methods—TOPSIS, Promethee II, and PROSA—in a study carried out on the same research sample.

Thus, the purpose of this study is to identify the differences between these results. Completing this task will lead to the next step (i.e., the use of a method that will allow for reducing the differences in the assessments). The advantages of the conversion method that have been tested and documented so far induced the authors to pursue this goal.

Thus, this study aims to present the disproportion between the results obtained with the authors’ own conversion method and the commonly applied methods of multi-criteria evaluation used to assess e-banking services during the pandemic in Poland in 2020.

In order to achieve this goal, Section 2 of the paper describes the MCDA methods used in the study. Section 3 provides a detailed description of the research method and the sample covered by the study. Section 4 presents the selected calculation results, with the position in the ranking and absolute distances between the calculation results and the calculated R2 correlation coefficients between the scores obtained as a result of the application of individual methods. The last section of the article presents the conclusions, research limitations, and directions for further studies aimed at solving the problem discussed in this work. It also recommends the possible next steps related to further studies to increase the usability and reliability of the conversion method when applied to different research problems.

2. Literature Review

Many situations related to decision-making processes tend to be so complex and ambiguous that they require the consideration of many decision variants. The variants consist of a number of previously distinguished and often contradictory attributes (criteria) of assessment [28]. The sum of the assessments of these criteria is applied to rate the quality of individual variants and their value for the decision maker in a situation where all available options are acceptable, and the main issue is to select the one which would best fit the intended purpose. At present, comparative research is underway to select the optimal method (if establishing the best universal method is not possible) that would be dedicated to specific applications or areas. However, most research is currently focused on developing and improving new MCDA methods [29,30,31,32]. The methods differ in terms of techniques and the determination of a set of criteria, the method of establishing the preference weights of individual criteria, the level of algorithm complexity, and the possibility of considering deterministic and non-deterministic data [33]. However, so far, despite the existence of several dozen MCDA methods, none of them can be considered ideal, and as such, they cannot be used to solve each and every potential problem in a decision-making situation [34,35,36]. Therefore, it is important to select the method which would be appropriate for a specific decision-making problem or the one which would ensure a correct assessment to be carried out by an expert or final decision maker [37,38,39].

The decision-making process may depend on the selection of a list of criteria and their significance for the decision maker in the first place. Individual decision makers may assign various weights to criteria, so there is no case in which a multi-criteria decision could be regarded as entirely objective. The choice related to a particular decision also depends on the method of calculating the decision variants and the resulting ranking. In the case of research involving large populations carried out as part of quantitative studies, the subjectivity of individual decision makers is averaged and thus partially eliminated. As a result, the final ranking for individual variants with or without the weights (equivalent weights) of the subsequent criteria is objective in a situation where the ranking is generated using a formalized algorithm.

At present, MCDA methods are undergoing dynamic development, and new methods with improved algorithms appear every year. However, applying an inappropriate method to a specific decision-making situation may lead to lower quality of the decision making itself, in particular when recommendations resulting from different methods concerning the same decision or dilemma are inconsistent.

The research problem related to multi-criteria decision-making analysis may focus on the following issues:

• Selecting the best option (decision), considering the evaluated criteria for individual customers;
• Arranging (ranking) variants into classes of acceptable alternatives;
• Sorting (classification) of decision variants into certain predetermined categories.

Research on multicriteria decision-making methods points to two approaches to this issue. The models include applying the utility function (differentiating only two straightforward decision-making forms: strict preferences and equivalences) and relational models extending the pointed decision-making scenario to comprise incomparability and a low preference for the considered alternatives. They vary significantly concerning the methodology related to applied attempts for given decision-making problems and the algorithm for indicating the most advantageous alternative. The relational model traces its origins from the European stream of decision making, and the relationship itself expresses the strictly defined priorities of the decision maker, referred to as the “surpassing/outranking relation”. This relationship is exploited in methods from the (French) Elimination Etchoin Traduisant la Realite (Electre) [21,28] or Preference Ranking Organization Methood for Enrichment Evaluations (Promethee) groups [40]. Promethee methods are applied to establish a synthetic ranking of the evaluated variants. It can be said that the methods included in this group combine the characteristics of most of the Electre methods regarding determining the preference (compliance or reliability) coefficients. In the Promethee II method, the decision maker can select one of six preference functions using a simple criterion, a quasi-criterion with an equivalence threshold, a criterion with a linear preference and a preference threshold, a level criterion with an equivalence and preference threshold, a criterion with linear preference and an area of indifference, and the Gaussian criterion.

Promethee II, PROSA [41], TOPSIS [42], and the conversion method were applied to prepare the final ranking of the assessed banks. The application of the Promethee II method was justified by the circumstance that, as opposed to most methods involving the utility function, this method is characterized by a reduced linear compensation effect of the criteria, and unlike other methods of the so-called “European school” (e.g., the family of Electre methods), a complete final ranking of the variants is obtained with their quantification.

The use of the PROSA method has a similar positive effect to that of the Promethee II method, but it also minimizes the effect of substitution (i.e., linear compensation of criteria).

The TOPSIS method in its original version [42] contains some disadvantages like, for example, input information needing to be provided in numerical form as well as definite, monotonically decreasing and increasing, and sharing a commensurable unit [43,44,45], which means that the technique might have some problems handling qualitative criteria, and it cannot tackle incompleteness and uncertainty of the assessment information [46,47]. Some propositions to solving this problem have been presented [46,48].

The application of the conversion method produced results that were different from the results obtained in the case of the use of other methods.

2.1. The Conversion Method

Here, we adopt the following assumptions for the conversion method [20]. After constructing the experts’ matrix of assessments of individual criteria for every website in the

Table 1. The experts’ table of assessments of particular criteria for each website.

		Websites
		${\mathit{a}}_{\mathbf{1}}$	${\mathit{a}}_{\mathbf{2}}$	…	${\mathit{a}}_{\mathit{m}}$
Criteria	$f_1$	$f_1\left(a_1\right)$	…	…	…
	$f_2$	…	…	…	…
	…	…	…	…	…
	$f_n$	…	…	…	$f_n\left(a_m\right)$

, we need to conduct the conversion with the determined vector, including the preferences of the superior level criteria [10,33]. Then, the authors transform the combined table containing scores into the vector with preferences (first converter).

The next steps are as follows:

• Building a matrix containing the distance values from the maximum value for every criterion in each website;

  ○

  Determining the maximum value:

  $$P_{i,max}=Max\left\{f_i\left(a_j\right),\dots ,f_n\left(a_m\right)\right\}\mathrm{for}i=1,\dots ,n\mathrm{and}j=1,\dots ,m$$

  (1)

  ○

  Determining the matrix including the distance values from the maximum value;

  $$\delta \left(f_i\left(a_j\right)\right)=P_{i,max}-f_i\left(a_j\right)\mathrm{for}i=1,\dots ,n\mathrm{and}j=1,\dots ,m$$

  (2)

  ○

  Calculation of the mean distance from the maximum value for every criterion;

  $$\overline{F_{i,j}}=\frac{{{\displaystyle \sum }}_{j=1}^m\delta \left(f_i\left(a_j\right)\right)}{m}$$

  (3)

• As an outcome of the operation mentioned above, building a matrix containing differences in the distance from the maximum value and the mean distance according to the criteria;
• Building conversion matrices which are modules of the relative distances of individual criteria for every bank website with the rest of the other criteria (the distance from the same criterion is equal to 0). The received distances below the diagonal are the conversed values located over the diagonal. Please refer to the

  Table 2. Conversion matrix.

  ${\mathit{a}}_{\mathit{j}}$		Criteria
  		${\mathit{f}}_{\mathit{i}\mathbf{,}\mathit{j}}$	${\mathit{f}}_{\mathit{i}\mathbf{+}\mathbf{1}\mathbf{,}\mathit{j}}$	${\mathit{f}}_{\mathit{i}\mathbf{+}\mathbf{2}\mathbf{,}\mathit{j}}$	…	${\mathit{f}}_{\mathit{n}\mathbf{,}\mathit{m}}$
  Criteria	$f_{i,j}$	0	${\alpha }_{i,j}-{\alpha }_{i+1,j}$	${\alpha }_{i,j}-{\alpha }_{i+2,j}$	…	…
  	$f_{i+1,j}$	${\alpha }_{i+1,j}-{\alpha }_{i,j}$	0	…	…	…
  	$f_{i+2,j}$	${\alpha }_{i+2,j}-{\alpha }_{i,j}$	…	0	…	…
  	…	…	…	…	0	…
  	$f_{n,m}$	…	…	…	…	0

  .
• Conversion matrices with averaging criteria: building one matrix with the average modules of the values for all criteria:

  $${\overline{A}}_{i,j}=\frac{{{\displaystyle \sum }}_{i=1,j=1}^{n,m}({\alpha }_{i,j}-{\alpha }_{i+2,j})}{n}$$

  (4)

  ○

  Transformation of the conversion matrix of the criteria into a superior preference matrix (performing calculation of the squared matrix, the addition of the rows, standardization of the received preference vector, repeated squaring, adding up the rows, standardization of the vector with preferences, and repeating this iteration until there are minimal differences in the sequential preference vectors);

  ○

  As an outcome of the mentioned operations, we obtain a criteria conversion matrix $T{a}_{mx1}$;

• Next, the authors transform the scores provided by the experts on the level of a matrix specifying the experts’ website assessments for individual criteria (second converter) [33]. The results were received by using an analogical strategy;
• Creating a matrix of distances from the maximum value for every criterion and every website;

  ○

  Determining the maximum value:

  $$P_{i,max}=Max\left\{f_i\left(a_j\right),\dots ,f_n\left(a_m\right)\right\}\mathrm{for}i=1,\dots ,n\mathrm{and}j=1,\dots ,m$$

  (5)

  ○

  Creating the matrix of distances from the maximum value:

  $$\delta \left(f_i\left(a_j\right)\right)=P_{i,max}-f_i\left(a_j\right)\mathrm{for}i=1,\dots ,n\mathrm{and}j=1,\dots ,m;$$

  (6)

• Computing the mean distance from the maximum value for every website:

  $$\overline{F_i}=\frac{{{\displaystyle \sum }}_{j=1}^m\delta \left(f_i\left(a_j\right)\right)}{m}$$

  (7)
• Creating a matrix of the differences of the deviations from the maximum value and the mean distance of the features from the maximum;
• For every criterion, creating a matrix of transformations (conversions) of the differences of the mean distance from the maximum value between the websites, identical to that presented above (the distance for a particular feature in the same website from the same website is 0), with values placed below the diagonal being the converse of the values located over the diagonal. Please refer to the

  Table 3. Matrix of transformations for each criterion.

  ${\mathit{f}}_{\mathit{i}}$		Websites
  		${\mathit{a}}_{\mathit{i}\mathbf{,}\mathit{j}}$	${\mathit{a}}_{\mathit{i},\mathit{j}\mathbf{+}\mathbf{1}}$	${\mathit{a}}_{\mathit{i}\mathbf{,}\mathit{j}\mathbf{+}\mathbf{2}}$	…	${\mathit{a}}_{\mathit{n}\mathbf{,}\mathit{m}}$
  Websites	$a_{i,j}$	0	${\alpha }_{i,j}-{\alpha }_{i,j+1}$	${\alpha }_{i,j}-{\alpha }_{i,j+2}$	…	…
  	$a_{i,j+1}$	${\alpha }_{i,j+1}-{\alpha }_{i,j}$	0	…	…	…
  	$a_{i,j+2}$	${\alpha }_{i,j+2}-{\alpha }_{i,j}$	…	0	…	…
  	…	…	…	…	0	…
  	$a_{n,m}$	…	…	…	…	0

  .
• Creating a module matrix of transformations of the differences of the mean distance from the maximum value between the websites for every criterion, please refer to the

  Table 4. A module matrix of transformations of the differences of the average distance from the maximum value between the websites for each criterion.

  ${\mathit{A}}_{\mathit{j}\mathbf{\dots }\mathit{m}}$		Criteria
  		${\mathit{f}}_{\mathit{i}\mathbf{,}\mathit{j}}$	${\mathit{f}}_{\mathit{i}\mathbf{+}\mathbf{1}\mathbf{,}\mathit{j}}$	${\mathit{f}}_{\mathit{i}\mathbf{+}\mathbf{2}\mathbf{,}\mathit{j}}$	…	${\mathit{f}}_{\mathit{n}\mathbf{,}\mathit{m}}$
  Criteria	$f_{i,j}$	0	${\overline{A}}_{i,j}$	…	…	…
  	$f_{i+1,j}$	$-{\overline{A}}_{i,j}$	0	…	…	…
  	$f_{i+2,j}$	…	…	0	…	…
  	…	…	…	…	0	…
  	$f_{n,m}$	…	…	…	…	0

  .

  $${\overline{A}}_{i,j}=\frac{{{\displaystyle \sum }}_{i=1,j=1}^{n,m}({\alpha }_{i,j}-{\alpha }_{i+2,j})}{n}$$

  (8)
• For every module matrix of transformation of the differences of the mean distance from the maximum value among the websites, squaring it, adding up the rows, standardization of the received ranking vector, and repeating this procedure until the received differences between the two ranking vectors for every criterion will be minimal.

As an outcome of the procedures demonstrated above, we acquired a conversion matrix of the websites’ assessments: $T{f}_{mx1}$ as follows:

• Utilizing the received vectors to create a combined ranking matrix by returning to the matrix where there are criteria in its side heading and suitable transfer of the acquired preference vectors for every criterion in the heading names of bank websites;
• Multiplying the matrix received in such a procedure by the previously computed preference vector:

  $$T^{\prime }=Tf⨂Ta$$

  (9)
• Reaching the final outcomes and conclusions. (It is important to notice that the lowest distances in this case are the most advantageous, and the comparability adjustments for other methods can be received by subtracting these values from 1 and their repeated standardization.)

The primary assumption for the developing presented method was establishing that it should be simple for application. The objective was achieved and proven in the number of advantages presented below. The only disadvantage of this method is the fact that the transformation of the results of the questionnaire is connected with the necessity to perform many complex operations.

The advantages of this method are as follows:

• The simplicity of usage (like in the performing of a scoring technique), which is caused by the fact that in the questionnaire form, there are questions regarding the subjective assessment of the element;
• In the situation of considering numerous assessment criteria or alternatives, the number of questions in the questionnaire does not increase significantly;
• The opportunity to apply the method with the involvement of people without special expert knowledge in a particular domain;
• There are no provided measures, as in the case of the Electre method, or a veto threshold, which may not be fully comprehensible to the respondent [49];
• The result of the computations, which takes the form of the importance of the assessments of the investigated objects.

2.2. The Promethee II Method

The second method applied in this study is Promethee II. The Promethee II method enables obtaining a full ranking of the alternatives. The outcome ranking is only partial for the previous version of this method [40]. Then, after establishing the compliance coefficients for every pair of variants, the dominance flows are designated for each of the variants:

• The leaving flow, describing how much the $a_i$ variant outranks the other variants:

  $${\varphi }^{+}(a_i)={\displaystyle \sum _{j=1}^n\pi (a_i,b_j)}$$

  (10)
• The entering flow, informing how much the $a_i$ variant is outranked by the other variants:

  $${\varphi }^{-}(a_i)={\displaystyle \sum _{j=1}^n\pi (b_j,a_i)}$$

  (11)

Subsequently, the decision maker can obtain a total ranking of the alternatives. In the Promethee II method, to generate a complete ranking of the alternatives, the net dominance flow described by Equation (3) should be calculated:

$$\varphi (a_i)={\varphi }^{+}(a_i)-{\varphi }^{-}(a_i)$$

(12)

In the Promethee II method, the equivalence and preference relations in a broad sense are defined as follows [11]:

• The $a_i$ variant outranks the $b_j$ variant ($a_i$ L $b_j$) if $\varphi (a_i)>\varphi (b_j)$;
• The $a_i$ variant is equivalent to the $b_j$ variant ($a_i$ I $b_j$) if $\varphi (a_i)=\varphi (b_j)$.

2.3. The PROSA Method

After establishing the value of ${\varphi }_{net}\left(a\right)$ i ${\varphi }_j\left(a\right)$ for j = 1 … n, the decision maker can define the compensation of criteria for individual decision variants as follows [41]:

• ${\varphi }_j\left(a\right)<<{\varphi }_{net}\left(a\right)$ denotes that for variant a, the performance of criterion j is compensated by other criteria (alternative a is not balanced with respect to criterion j);
• ${\varphi }_j\left(a\right)\gg {\varphi }_{net}\left(a\right)$ denotes that for alternative a, the performance of criterion j compensates other criteria (alternative a is not balanced with respect to criterion j);
• ${\varphi }_j\left(a\right)\approx {\varphi }_{net}\left(a\right)$ means that alternative a is balanced with respect to criterion j.

The operators >> and << denote the contractual relations “much greater than” and “much less than”, respectively. The mentioned relations reflect the subjective decision maker’s point of view as to whether the value on the right-hand side of the operator is much greater or much smaller than the value on the left-hand side and therefore whether the alternative a is balanced regarding criterion j or not.

In the following stage, the value of the average absolute deviation is determined in a weighted form, considering the balance (compensation) factor, as demonstrated in Equation (5):

$$WMAD\left(a\right)={\displaystyle \sum }_{j=1}^n\left|{\varphi }_{net}\left(a\right)-{\varphi }_j\left(a\right)\right|w_j s_j$$

(13)

where $s_j$ represents the equilibrium (compensation) factor for criterion j. It is clear that WMAD(a) is a specific weighted average distance of the solution ${\varphi }_{net}\left(a\right)$ from the solutions ${\varphi }_j\left(a\right)$ received for particular criteria.

The final evaluation of the considered variants (i.e., $PS{V}_{net}$ (PROSA Sustainable Value net)) is computed using Equation (6): $PS{V}_{net}\left(a\right)={\varphi }_{net}\left(a\right)-WMAD\left(a\right)$.

2.4. The Topsis Method

The TOPSIS method consists of six stages [37]. At the very beginning, the decision maker (DM) structures the decision problem (DP) using $n$ criteria and $m$ alternatives. A decision matrix $D\left[x_{ij}\right]$ is then created, with rows that represent the decision attributes of the evaluated alternatives and columns that represent the considered criteria:

$$D\left[x_{ij}\right]=\left(\begin{array}{ccc}{x}_{11}& \cdots & x_{1n}\\ ⋮& \ddots & ⋮\\ x_{m1}& \cdots & x_{mn}\end{array}\right)$$

(14)

After that, the decision matrix is normalized in the second stage using the formulae provided below for the benefit and cost criteria, respectively:

$$r_{ij}=\frac{x_{ij}-mi{n}_i\left(x_{ij}\right)}{ma{x}_i\left(x_{ij}\right)-mi{n}_i\left(x_{ij}\right)}$$

(15)

$$r_{ij}=\frac{ma{x}_i\left(x_{ij}\right)-x_{ij}}{ma{x}_i\left(x_{ij}\right)-mi{n}_i\left(x_{ij}\right)}$$

(16)

In the next stage, the normalized decision matrix is weighted using the provided vector with weights, giving a weighted, normalized decision matrix containing elements calculated using the formula provided below:

$$v_{ij}=w_j·r_{ij}$$

(17)

The aim of the fourth stage is providing the positive (PIS) and negative (NIS) ideal solutions ($V_j^{+}$ and $V_j^{-}$) received using the formulas presented below:

$$V_j^{+}=\left\{v_1^{+}, v_2^{+}, \dots , v_n^{+}\right\}$$

(18)

$$V_j^{-}=\left\{v_1^{-}, v_2^{-}, \dots , v_n^{-}\right\}$$

(19)

The most favorable alternative should be as close as possible to the PIS and as far as possible from the NIS. In order to calculate the distance from the PIS and NIS for each alternative, the Euclidean distance is employed in the fifth step:

$$D_i^{+}=\sqrt{{\displaystyle \sum }_{j=1}^n{(v_{ij}-v_j^{+})}^2}$$

(20)

$$D_i^{-}=\sqrt{{\displaystyle \sum }_{j=1}^n{(v_{ij}-v_j^{-})}^2}$$

(21)

The last step of the algorithm is performed to determine the values of relative closeness to the ideal solution:

$$C{C}_i=\frac{D_i^{-}}{D_i^{-}+D_i^{+}}$$

(22)

The received closeness coefficient $C{C}_i$ is the score value generated by the TOPSIS method and is applied to create the rankings of the evaluated variants [42].

3. Materials and Methods

Based on the authors’ experience related to the previous studies, the presented research method was involved in the study as follows:

• Creating a pilot version of the survey to verify the propriety and readability of the questions;
• Developing and improving the prototype of the survey based on the above-mentioned evaluation and subsequent construction of the final version of the questionnaire, adopting an unambiguous scale of attribute evaluation in the course of data collection;
• Random selection of groups of respondents and inviting them to complete the survey carried out with the computer-associated web interview (CAWI) method;
• Polling to receive data and initial verification of the procedure for filling in the questionnaire;
• Selection of methods for evaluating banking services to benchmark their quality to the authors’ own conversion method;
• Calculating the distance between the results obtained with the selected methods (i.e., the difference between the places in the ranking and between the results);
• Presentation of the correlation of the results of the conversion method in relation to other selected methods for evaluating websites;
• Analysis and discussion of the outcomes obtained as part of benchmarking;
• Concluding and preparing recommendations with regard to potential changes to the conversion method.

The selection of the research sample was a case of purposeful sampling. The study examined the opinions of students from the University of Warsaw aged 18–25 years in randomly chosen laboratory and lecture groups. Although this choice could have affected the results of the survey, it is important to note that 99% of the population in Poland are potential internet banking customers; over 50% actively use electronic banking, 33% of Poles are active mobile banking users, and the currently surveyed age group includes over 55% e-banking users [50]. Of the students, 38% declared they were not employed. The metrics of the survey are presented in

Table 5. Metrics of the survey.

Gender
woman			man
73%			27%
Age
18–25	26–35		36–45		45+
99%	2%		1%		0%
Place of Birth
city of 500,000+	100,000–500,000	10,000–100,000	1000–10,000	less than 10,000	village
28%	10%	5%	2%	9%	8%
Education
secondary education		bachelor’s		higher
94%		3%		2%
Status
student			working student
38%			62%

Source: own work.

.

The data utilized in this research were acquired and initially processed for analysis at the beginning of April 2020. A total of 1324 individuals took part in the survey, and 1008 respondents filled out the survey correctly, which was a 76% response rate. In general, there were 1074 ratings of online banking websites collected as part of the study (954 of them concerned ratings of 1 e-banking website, 42 evaluated 2 websites, and 12 assessed 3 websites).

A simplified, standardized Likert scale was used to evaluate the individual criteria in the banks selected by the e-banking clients [51].

The preliminary condition of the participation in the survey was the evaluation of at least one e-banking website from a selection of well-known e-banking services. This condition resulted from the need to collect responses from experienced e-banking clients using various banking services. Almost all respondents (95%) assessed only one bank, and more than 4% of the sample assessed two. Three banks were rated by less than 1% of the survey participants.

After another verification and considering the last comments received from the respondents, 18 attributes (criteria) were incorporated for the evaluation. They were divided into three dimensions: economic, technological, and anti-crisis criteria. A detailed list of attributes is presented in

Table 6. The average value of significance and preference indicators for individual attributes.

Cr	Criterion	Significance for the Respondent	Preference for the Respondent
C1	Nominal annual interest rate in personal accounts	74.32%	5.29%
C2	Account maintenance fee (PLN/month)	89.03%	6.34%
C3	Fee for a transfer to parent bank	88.51%	6.30%
C4	Fee for a transfer to another bank	90.24%	6.43%
C5	Direct debit	67.19%	4.78%
C6	Fee for issuing a debit card	69.58%	4.96%
C7	Monthly fee for a card (PLN/month)	88.27%	6.29%
C8	Interest on savings accounts	66.22%	5.58%
C9	Interest rate on deposits of PLN 10,000	82.64%	4.93%
C10	Interest rate on loans of PLN 10,000	92.72%	4.69%
C11	Additional services	69.39%	4.72%
C12	Account access channels	75.00%	5.89%
C13	Security	83.47%	6.60%
C14	Visualization	78.89%	4.94%
C15	Navigation	78.36%	5.34%
C16	Readability and ease of use	69.29%	5.94%
C17	The scope of functionality	65.92%	5.62%
C18	Anti-crisis measures	75.14%	5.35%

Source: own work.

.

Apart from the importance evaluation, the respondents defined their preferences concerning the share of individual attributes in assessing the quality of e-banking websites. As far as individual attributes are concerned, it turned out that they did not vary significantly from the average of 5.55%, and after dividing them into dimensions, the authors did not observe any significant differences.

In Table 5, the “significance for the respondent” parameter means how important the particular criteria was for the respondents in the procedure of assessment of e-banking websites. The higher the value, the more important the criteria were. Here, this parameter is presented as an average value of all responses. In addition, the “preference for the respondent” parameter means that there is a strong difference in ratings between the assessed websites. The higher the rating in a given criterion, the greater the perceived differentiation of ratings.

The respondents rated 27 (A1, A2, …, A27) e-banking websites for the following banks: Alior Bank (Alior Bank SA), Bank BPS, Grupa BPS (Bank Polskiej Spółdzielczości SA), Bank Gospodarstwa Krajowego, Bank Millennium (Bank Millennium SA), Bank Pekao (Bank Polska Kasa Opieki SA), Bank Pocztowy (Bank Pocztowy SA), BGŻ BNP Paribas (Bank BGŻ BNP Paribas SA), BOŚ Bank (Bank Ochrony Środowiska SA), Citi Handlowy (Citi Handlowy, Banku Handlowego w Warszawie SA), Credit Agricole (Credit Agricole Bank Polska SA), Deutsche Bank in Poland (Deutsche Bank Polska SA), EnveloBank, Eurobank (Euro Bank SA), Get In Bank (Getin Noble Bank SA), Idea Bank (Idea Bank SA), ING Bank Śląski (ING Bank Śląski SA), INTELIGO, PKO Bank Polski (Bank Polski SA), iPKO, PKO Bank Polski (Bank Polski SA), mBank (mBank SA), Nest Bank (Nest Bank SA), Raiffeisen POLBANK (Bank BGŻ BNP Paribas SA), Santander Bank Polska, Santander Consumer Bank, SGB Spółdzielcza Grupa Bankowa (Spółdzielcza Grupa Bankowa SA), T-Mobile Usługi Bankowe (Alior Bank SA), Toyota Bank Polska SA (Toyota Bank Polska SA), and Volkswagen Financial Services, Volkswagen Bank (Volkswagen Bank GmbH Sp. Z o.o. Polish Branch).

4. Analysis of the Results

The calculation results for each method are presented below. All calculations were carried out based on the same database including the responses obtained from a survey conducted among e-banking users.

4.1. The Results of the Application of the Conversion Method

In all cases, the empirical research was carried out for 27 banks which were evaluated with the application of 18 criteria. In the case of the use of the conversion method, the ranking of variants was created based on the conversion of the data obtained in the study, in line with the assumptions of the conversion method described in Section 3. The highest results were achieved for A22 with a score of 0.1515, followed by A23 with a score of 0.1366. A26 (0.0859) took the third position. The worst result was obtained for A1 with a score of 0.0001. Thus, it emerged that there was a significant difference between the best and the worst results. A1 accounted for only 0.06% of A22. A clear division may be noticed, and a group of very good scores included A22, A23, A26, and A24, and among the worst results, there were A1, A2, A7, and A4.

The sensitivity analysis shows that the solution was resistant to criteria changes up to only 2% for the best and worst variants. The remaining variants showed much greater resistance to changes in terms of priorities.

4.2. The Results of the Application of the Promethee II Method

In the authors’ empirical research, 27 banks were evaluated with the application of a total of 18 criteria. The criteria, including all attributes, are presented in Table 3. Moreover, the weights of the significance assigned to each criterion are also shown in Table 3.

The ranking of variants was established using the ordinary (real) criterion as a preference function. The results of the variants’ evaluation and their positions in the ranking are presented in Table 4. According to the adopted criteria weights, A14 was recognized as the best variant. When analyzing the elements in Table 4, it should be noted that individual variants formed specific groups of solutions regarding the applied criteria and their weights. They included the best (A14 and A19), good (A13, A16, and A18), and bad (A25 and A26) solutions. Naturally, the presented ranking of decision variants is not final. The subjective nature of the vector of priorities applied in the study and the need to examine the strength of the positioning of individual decision variants in the ranking constitute the basis and direction for subsequent studies. The studies include sensitivity analysis of the presented decision model. Taking into consideration a large number of criteria (18), the authors decided against conducting geometrical analysis for interactive assistance (GAIA) analysis. GAIA aims to deliver a complete graphic representation of the decision problem, enables the analysis of the “goodness” of the obtained solution, and indicates the directions of its possible improvement. In the GAIA methodology, the information on the k-criterion decision problem presented in the k-dimensional Euclidean space is displayed in one plane, so with the current number of criteria (18), it was found that it was burdened with an interpretation error. The error appeared to be too significant to produce reliable findings to be interpreted by the authors.

The next step in examining the obtained solution was the sensitivity analysis that allowed the authors to determine the stability of the obtained solution in terms of changes in the weights of the criteria. Considering the fact that we may have looked at the problem from different perspectives (represented by the decision makers), and the significance of individual sets of criteria may have been different, the present analysis took into account its own weights of the sets of criteria (obtained in the current research).

The sensitivity analysis shows that the solution was resistant to changes in the weights of the criteria sets, and the tendency was assessed to be 4%. It is worth noting at this point that the dominating variants in the rankings (A14 and A19) were characterized by very high resistance to changes in priorities (solution robustness). The latter constitutes a strong basis for recommending these alternatives as strongly dominant remaining variants. This situation is similar to the sensitivity analysis of the “worst” variant. Here, the A6 alternative was characterized by high resistance to changes in priorities (about 30% of the changes in the priority vector). The results of the variants’ evaluation and their positions in the rankings are presented in the collective table (Table 4).

4.3. The Results of the Application of the PROSA Method

The ranking of decision variants was determined using the PROMETHEE II and PROSA methods. During the preference modeling, an ordinary (real) criterion was used as a preference function. The value s = 0.3 was assumed as the maximum degree of criteria compensation. The results of the variants’ evaluation and their positions in the rankings are presented in Table 4. With the adopted criteria weights, W14 was considered to be the best location variant. When analyzing the results, it should be noted that the individual variants created specific groups of solutions due to the applied criteria and their weights. They can be divided into the following groups: the best solutions (W14 and W19), good solutions (W6, W8, and W20), and bad (W24 and W25) solutions.

Naturally, the presented rankings of the decision variants is not final. The subjective nature of the introduced priority vector and the need to examine the strength of the positioning of individual decision variants in the rankings constitute the basis for further studies, including sensitivity analysis of the presented decision model. As in the case of the Promethee II assessment model, due to the significant number of criteria (18), the authors decided to not perform GAIA analysis.

The subsequent step in examining the obtained solution was the sensitivity analysis that allowed the authors to determine the stability of the obtained solution in terms of changes in the weighting of the criteria. During the sensitivity analysis of a set of solutions, it was decided that each time the significance of the analyzed criterion was linearly modified (increased), the remaining values of the total vector of priorities were distributed proportionally among the other criteria.

The sensitivity analysis shows that the solution was resistant to changes in the case of the weights of the sets of criteria in the range from 5% to 10%. It is worth noting at this point that the variants dominating in the rankings (A19 and A14) were characterized by very high resistance to changes in priorities (solution robustness). This constitutes a sound basis for recommending these alternatives as the strongly dominant remaining variants. The situation was similar in the case of the sensitivity analysis concerning the “worst” variant. Here, the A6 alternative was characterized by high resistance to changes in priorities (from approximately 40% to 100% of the changes in the case of the priority vector).

4.4. The Results of the Application of the TOPSIS Method

In the empirical research, 27 banks were evaluated with the application of a total of 18 evaluation criteria. The score assigned to each criterion was obtained as a result of the subsequent steps described in Section 2.3 of the paper.

The evaluation of banks based on the computed weights was first presented by an evaluation where all the weights of the criteria were equal. The results of the evaluation are presented in Table 5. The top rank was assigned to the bank designated as A14 with a score of 0.7674, followed closely by the bank referred to as A4 with a score of 0.7641 and A19 with a score amounting to 0.7515. The worst bank, referred to as A6, scored only 0.3208, a value which was over twice as low as the best bank. The difference in scores between the banks represented by A16, A5, A13, and A23, which ranked from sixth to ninth, was minute, and therefore, these banks could change their positions in the rankings depending on subjective perceptions. As a consequence, they showed largely similar characteristics in terms of the quality of e-banking websites, and from the point of view of their clients, they could be used interchangeably. The results of the assessment are presented in

Table 7. The results of calculations of the assessment of banking services using the conversion, Promethee II, PROSA, and TOPSIS methods for 27 banks in Poland in 2020.

		Conversion		Promethee II		PROSA		TOPSIS
Bank	Alt.	Score	Rank	Score	Rank	Score	Rank	Score	Rank
Alior Bank (Alior Bank SA)	A1	0.0001	27	0.1964	10	0.5404	10	0.6854	10
Bank BPS Grupa BPS, (Bank Polskiej Spółdzielczości SA)	A2	0.0003	26	−0.4156	23	0.4829	22	0.4901	22
Bank Gospodarstwa Krajowego	A3	0.0006	25	−0.3623	22	0.4753	23	0.5176	20
Bank Millennium (Bank Millennium SA)	A4	0.0020	24	0.5580	3	0.5534	3	0.7641	2
Bank Pekao (Bank Polska Kasa Opieki SA)	A5	0.0138	16	0.3474	9	0.5523	5	0.7050	8
Bank Pocztowy (Bank Pocztowy SA)	A6	0.0143	15	−0.7961	27	0.4210	27	0.3208	27
BGŻ BNP Paribas (Bank BGŻ BNP Paribas SA)	A7	0.0038	23	−0.1066	16	0.5181	14	0.5863	16
BOŚ Bank (Bank Ochrony Środowiska SA)	A8	0.0076	22	−0.1060	15	0.5118	18	0.5865	15
Citi Handlowy (Citi Handlowy, Banku Handlowego w Warszawie SA)	A9	0.0166	14	−0.1540	17	0.5067	16	0.5672	17
Credit Agricole (Credit Agricole Bank Polska SA)	A10	0.0185	12	0.0141	13	0.5263	12	0.5991	13
Deutsche Bank in Poland (Deutsche Bank Polska SA)	A11	0.0125	18	−0.4600	24	0.4765	24	0.4919	21
EnveloBank	A12	0.0116	20	−0.0125	14	0.5210	15	0.6022	12
Eurobank (Euro Bank SA)	A13	0.0124	19	0.3713	8	0.5576	9	0.7091	5
Get In Bank (Getin Noble Bank SA)	A14	0.0099	21	0.6412	1	0.5806	2	0.7674	1
Idea Bank (Idea Bank SA)	A15	0.0127	17	0.0858	11	0.5395	13	0.5925	14
ING, ING Bank Śląski (ING Bank Śląski SA)	A16	0.0184	13	0.3908	6	0.5576	6	0.7071	7
INTELIGO, PKO Bank Polski (Bank Polski SA)	A17	0.0323	10	−0.1762	19	0.5078	17	0.5473	18
iPKO, PKO Bank Polski (Bank Polski SA)	A18	0.0668	6	0.4773	4	0.5473	4	0.7083	6
mBank (mBank SA)	A19	0.0652	7	0.6370	2	0.5528	1	0.7515	3
Nest Bank (Nest Bank SA)	A20	0.0296	11	0.4079	5	0.5669	8	0.7193	4
Raiffeisen POLBANK (Bank BGŻ BNP Paribas SA)	A21	0.0817	5	−0.1633	18	0.4715	20	0.4018	25
Santander Bank Polska	A22	0.1515	1	0.0719	12	0.5517	11	0.6262	11
Santander Consumer Bank	A23	0.1366	2	0.3747	7	0.5517	7	0.7019	9
SGB Spółdzielcza Grupa Bankowa (Spółdzielcza Grupa Bankowa SA)	A24	0.0835	4	−0.6301	26	0.4420	26	0.3982	26
T-Mobile Usługi Bankowe (Alior Bank SA)	A25	0.0588	8	−0.5736	25	0.4657	25	0.4313	24
Toyota Bank Polska SA (Toyota Bank Polska SA)	A26	0.0859	3	−0.2640	20	0.4968	19	0.5438	19
Volkswagen Financial Services, Volkswagen Bank (Volkswagen Bank GmbH Sp. Z o.o. Polish Branch)	A27	0.0529	9	−0.3535	21	0.4854	21	0.4814	23

Source: own work.

.

The sensitivity analysis allowed the authors to observe that in the case of seven criteria, no matter how much their weights changed, the bank A14 would still take the winning position in the rankings. In the remaining cases, the weight of the criterion would need to be changed by at least 10% to observe changes regarding the leading position in the evaluation scores. These two observations confirm that the selection of bank A14 as the leading bank was quite stable. However, this could not be said of the remaining banks, especially those which ranked from 5th to 10th. For example, for criterion C1, depending on how much its weight changed, the change in the ranking positions may have amounted to 15 rank places (the latter concerned the ranking positions from 5th to 10th). Contrary to criteria C1–C2 and C15–C18, the criteria C3, C10, and C14 were the factors characterized by the least significant changes in rank regardless of their importance. Therefore, their impact on the overall ranking was the lowest.

5. Discussion of the Results

The issue of comparing the results obtained with different methods to increase the possibility or potential of achieving sustainability in an information society has not been considered in the literature so far. Extensive comparative studies [52] focused on the assessment of the characteristics of individual methods [53], mathematical “sophistication”, mathematical argumentation of the method of website evaluation [37], adaptation to a particular industry and its needs [54,55,56], or the creation of new, complex methods [57,58], rather than the adaptation of the applied methodology for the objective assessment of websites by the client [59].

In the last step of this research, the results from the conversion method were compared with the results obtained with the TOPSIS, Promethee II, and PROSA evaluation methods. Promethee II was used with the ordinary criterion ranking and PROSA method with s = 0.3. The results are presented in Table 4.

The analysis of the results allowed the authors to observe that although almost all the rankings were similar, the actual precise order of individual banks was slightly different for each of them. For example, in the case of the Promethee rankings, the winning alternative was A14, similar to the application of the TOPSIS method. However, in the case of the PROSA rankings, A19 outranked A14 by around 0.01. All rankings were identical in terms of the worst alternatives, which included A6, A24, and A25. The slight differences in the rankings demonstrate the importance of the selection of the MCDA aggregation method for the final results of the evaluation of alternatives. It turned out, however, that even at first glance, the results of website evaluation obtained using the conversion method differed significantly from the results of the evaluations carried out using the remaining methods. A detailed analysis is presented in

Table 8. Differences in absolute scores between the ratings of places in the rankings obtained using the conversion method and the results obtained using the Promethee II, PROSA, and TOPSIS methods (city distance).

Bank	Alt.	Conversion-PROSA	Conversion-Promethee II	Conversion-TOPSIS	PROSA-Promethee II	PROSA-TOPSIS	Promethee II-TOPSIS
Alior Bank (Alior Bank SA)	A1	17	17	17	1	1	0
Bank BPS Grupa BPS, (Bank Polskiej Spółdzielczości SA)	A2	4	3	4	1	1	1
Bank Gospodarstwa Krajowego (BGK)	A3	2	3	5	0	3	2
Bank Millennium (Bank Millennium SA)	A4	21	21	22	2	3	1
Bank Pekao (Bank Polska Kasa Opieki SA)	A5	11	7	8	2	1	1
Bank Pocztowy (Bank Pocztowy SA)	A6	12	12	12	0	0	0
BGŻ BNP Paribas (Bank BGŻ BNP Paribas SA)	A7	9	7	7	1	1	0
BOŚ Bank (Bank Ochrony Środowiska SA)	A8	4	7	7	2	1	0
Citi Handlowy (Citi Handlowy, Banku Handlowego w Warszawie SA)	A9	2	3	3	2	1	0
Credit Agricole (Credit Agricole Bank Polska SA)	A10	0	1	1	1	0	0
Deutsche Bank in Poland (Deutsche Bank Polska SA)	A11	6	6	3	2	1	3
EnveloBank	A12	5	6	8	1	2	2
Eurobank (Euro Bank SA)	A13	10	11	14	6	2	3
Get In Bank (Getin Noble Bank SA)	A14	19	20	20	1	0	0
Idea Bank (Idea Bank SA)	A15	4	6	3	1	2	−3
ING, ING Bank Śląski (ING Bank Śląski SA)	A16	7	7	6	2	3	1
INTELIGO, PKO Bank Polski (Bank Polski SA)	A17	7	9	8	0	1	1
iPKO, PKO Bank Polski (Bank Polski SA)	A18	2	2	0	6	4	2
mBank (mBank SA)	A19	6	5	4	5	3	1
Nest Bank (Nest Bank SA)	A20	3	6	7	6	2	1
Raiffeisen POLBANK (Bank BGŻ BNP Paribas SA)	A21	15	13	20	4	1	7
Santander Bank Polska	A22	10	11	10	2	2	1
Santander Consumer Bank	A23	5	5	7	1	1	2
SGB Spółdzielcza Grupa Bankowa (Spółdzielcza Grupa Bankowa SA)	A24	22	22	22	0	0	0
T-Mobile Usługi Bankowe (Alior Bank SA)	A25	17	17	16	0	1	1
Toyota Bank Polska SA (Toyota Bank Polska SA)	A26	16	17	16	0	0	1
Volkswagen Financial Services, Volkswagen Bank (Volkswagen Bank GmbH Sp. Z o.o. Polish Branch)	A27	12	12	14	1	3	2
		248	256	264	50	40	30

Source: own work.

. The sum of the absolute differences between the results obtained by the conversion method and the remaining methods was 5–8 times greater than in the case of the application of a city’s distance between the other methods.

So far, no one has studied the distances between results and simultaneously between places in the rankings, as was performed in this paper. This did not seem relevant in light of obtaining the results which, by way of assumption, could be objective [37,59]. However, the situation where the application of two different methods leads to different results is becoming more complex and ambiguous. A question arises: Which of the methods should be recommended to clients to improve their circumstances in terms of assessment and selection of online and mobile bank services?

Although there was a very strong correlation between each pair of the rankings (strongest for Promethee vs. PROSA, followed by TOPSIS vs. Promethee II and TOPSIS vs. PROSA in descending order), the actual results differed depending on the MCDA method used. It is important to indicate that large differences in the results obtained by the conversion method were also confirmed by the calculated correlation matrix (

Table 9. Correlation matrix between the results obtained with the conversion, TOPSIS, Promethee II and PROSA methods (R²).

Methods	Conversion	Promethee II	PROSA	TOPSIS
Conversion	1	8.34%	6.58%	24.66%
Promethee II	8.34%	1	99.31%	95.97%
PROSA	6.58%	99.31%	1	96.62%
TOPSIS	24.66%	95.97%	96.62%	1

Source: own work.

). There was a very low correlation between the results obtained by the conversion method and the remaining methods, and the value did not exceed 25%.

When analyzing the research findings presented in this paper, it is important to state that a marked divergence occurred between the results obtained by the conversion method and the other methods used in this study. Thus, it emerged that the H₀ hypothesis was verified positively. The next questions which arise in this situation are the following: How do we interpret and assess such differentiation of these results? Is the difference a cause for concern or a negative phenomenon?

6. Conclusions

In the research examining the electronic banking sector in Poland conducted in 2020, there occurred significant differences between the results of commonly used methods and the results obtained with the application of the authors’ own conversion method. Naturally, this does not necessarily mean that this method is qualitatively inferior to the other methods used in the research. However, the difference appeared to be significant and rather high.

The latter induced the authors to examine the research problem further. Their goal was to identify the differences between the conversion method and other methods. The indicated differences described above, as well as the similarities occurring in the case of the results obtained with the application of the remaining methods, led the authors to re-examine the assumptions and calculations received with the use of the conversion method. In the next steps, the authors intend to verify their method, with a possible result that the calculations from the conversion method will be more correlated with the remaining scores obtained using other methods.

A promising direction for modifying this method may be the use of the penalty scalarizing function in the scalarizing conversion method, reducing the range of obtained results [60].

The basic limitations of this research were twofold. The first limitation resulted from the selection of the research sample. Examining the opinions expressed by the academic community had a strong impact on the results in the case of all methods. However, it is important to note that the study sample included a group of individuals who are most active in terms of internet use. Furthermore, the representatives of this population are open to new solutions related to the use of electronic banking and electronic payments. The second limitation of the study was the choice of specific methods for comparison. Some of them could be considered incomparable. Bearing this in mind, the authors decided to compare the distances between the places in the rankings apart from carrying out a general assessment of the standardized calculation results.

Thus, in the next steps, the authors will focus on further studies aimed at extending the research samples to include other groups of e-banking users and applying other methods of evaluating websites. They also intend to address the issue of the actual consequences of the differentiation in the results obtained with the application of different evaluation methods. The questions which need to be considered in the next stages of research concern the two basic research problems, namely, (1) whether it is wrong that the results of the conversion method differ significantly from other methods used in this study, and (2) assuming that a systemic method reducing this difference could be developed, would it change the ranking of e-banking websites, and could it be recommended to bank clients?