Next Article in Journal
Some New Sufficient Conditions on p-Valency for Certain Analytic Functions
Next Article in Special Issue
Density-Distance Outlier Detection Algorithm Based on Natural Neighborhood
Previous Article in Journal
Nonparametric Directional Dependence Estimation and Its Application to Cryptocurrency
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Learning the Parameters of ELECTRE-Based Primal-Dual Sorting Methods that Use Either Characteristic or Limiting Profiles

1
Faculty of Informatics, Universidad Autónoma de Sinaloa, Culiacán 80040, Mexico
2
Faculty of Accounting and Administration, Universidad Autónoma de Coahuila, Torreon 27000, Mexico
3
Bussiness School, Tecnológico de Monterrey, Monterrey 64849, Mexico
*
Author to whom correspondence should be addressed.
Axioms 2023, 12(3), 294; https://doi.org/10.3390/axioms12030294
Submission received: 31 January 2023 / Revised: 3 March 2023 / Accepted: 8 March 2023 / Published: 11 March 2023
(This article belongs to the Special Issue Combinatorial Optimization & Applications)

Abstract

:
Two multicriteria-sorting methods that generalize the relational paradigm have been recently presented in the literature. One uses objects representative of classes, the other uses objects in the limiting boundaries of classes; both can use either a reflexive or an asymmetric preference relation. However, defining the parameters of relation-based methods is not straightforward. The present work operationalizes those methods with a methodology that takes examples provided by the decision-maker and, using an accuracy measure that specifically fits the characteristics of the methods, exploits an evolutionary algorithm to determine the parameters that best reproduce such examples. The assessment of the proposal showed that (i) it can achieve considerably high levels of out-of-sample effectiveness with only a few decision examples; (ii) the inference process is more effective learning the parameters of the method based on representative objects; (iii) it tends to be more effective with a reflexive relation; (iv) the effectiveness decreases while increasing the number of classes, which is not always the case when increasing the number of criteria. Theoretical properties of the proposed methodology will be investigated in future works.

1. Introduction

For more than three decades, the multicriteria ordinal classification (also called multicriteria sorting) problem has captured the attention of the multicriteria decision-making research community. In multicriteria sorting, decision actions (alternatives, objects), which are described by multiple assessment criteria, must be assigned to predefined and ordered classes (or categories). In this type of decision problem, we must pay attention to two fundamental issues: (i) the way in which the preferences of the decision-maker (DM) are modelled, and (ii) the way in which the classes are characterized.
There are three main paradigms for modelling the preferences of the decision-maker regarding his/her preferences for an action over another:
-
Using a value or utility function (the functional paradigm; e.g., [1]), such that the function provides a numeric representation of the DM’s desirability toward the alternatives.
-
Building a binary preference or outranking relation (the relational paradigm; e.g., [2]), where the preference relation between pairs of alternatives can be determined.
-
The symbolic paradigm, mainly related to the use of Rough Sets to create a system of decision rules (e.g., [3]).
On the other hand, classes can be characterized through reference actions, known as profiles, in one of the following ways:
-
Using limiting profiles that describe the boundaries between classes [4,5,6,7,8,9,10,11,12,13], such that the profiles can be exploited to discern two consecutive classes (e.g., [4,5,6,7,8,9,10,11,12,13]).
-
Through representative or characteristic profiles used to symbolize a typical action in the corresponding class (e.g., [8,14,15,16,17,18,19]).
-
Using assignment examples as in [3,20,21,22,23,24].
The profiles (either limiting or characteristic) may be seen as actions, since they are characterized by their impacts on the criteria. The set of assignments examples provided by the DM may not only be used to characterize classes, but also to infer the parameters of a functional or relational decision model (e.g., [20,21]). This approach to obtain parameters indirectly is also known as preference disaggregation analysis (PDA). PDA methods use regression-like techniques to “learn” the decision model that represents the underlying assignment policy, which is manifested in the assignment examples given or approved by the DM [25].
In this paper, our interest is restricted to the relational paradigm. Within this, the most popular multicriteria-sorting methods belong to those in the ELECTRE family. The first, ELECTRE TRI (later renamed ELECTRE TRI-B), describes classes through a single limiting profile. The authors in [16] proposed ELECTRE TRI-C, in which categories are described by a single characteristic (“central”) profile. Both methods use an outranking relation to compare actions with profiles. Subsequently, the previous ELECTRE TRIs were extended to handle several profiles in ELECTRE TRI-nC and ELECTRE TRI-nB [11,17]. These methods were also extended by [10].
Relational-based ordinal classification methods were recently generalized by [26,27]. Using either a reflexive or an asymmetric general preference relation, Fernández et al. proposed methods that characterize classes through either limiting or representative profiles. These methods fulfill the fundamental properties previously discussed by [5] for ELECTRE TRI-B and revisited by [16] for ELECTRE TRI-C. The proposals by [26,27] are a wide generalization of the relational paradigm applied to ordinal classification.
Using ELECTRE methods, the direct parameter elicitation can be a difficult task [28]. Typically, the DM finds severe difficulty in defining parameter values whose meaning is confusing to her/him. This is particularly true when setting veto thresholds, as the veto concept is unfamiliar to most DMs. The direct elicitation task is even more complex in the case of ordinal classification methods where categories are described by several limiting profiles, the existence of which is subject to question in many real problems [15,16], and where several preference-based separability conditions involving parameter values must be fulfilled. The strong requirements in the proposal by [26] contribute to making the direct elicitation of parameters and profiles a very big concern.
Indirect elicitation is a suitable alternative. In this approach, the DM typically uses his/her holistic judgments to provide/accept a set of assignment examples that inherently contain his/her underlying assignment policy. This can be less cognitively demanding for the DM due to any of the following reasons: (i) the DM often feels more comfortable making decision assignments than justifying/explaining them; (ii) the DM can provide decision assignments that were previously provided/accepted by her/him; (iii) the DM can provide assignments of a set of fictitious examples that can be easily classified; (iv) the DM can make decision assignments on a subset of actions, for which the she/he feels comfortable.
Under some strong simplifications, the inference of outranking model parameters in ELECTRE TRI-B was approached through classical mathematical programming techniques in [29]. However, such an indirect way of obtaining parameter values becomes a very complex optimization problem when veto thresholds must be inferred. These thresholds become real-valued decision variables. Inferring all the parameters of ELECTRE-based methods simultaneously involves addressing non-linear optimization problems with non-convex constraints [29,30]. There are some works that have assessed the effectiveness of different optimization methods to infer the parameters of ELECTRE-based methods simultaneously (e.g., [31]); their general conclusion is that in such cases, evolutionary algorithms should be used as in [25]. The authors in [32] proposed an evolutionary algorithm to infer the whole ELECTRE TRI-nB model, including preference parameters and limiting profiles. Less sophisticated heuristic approaches may be used when the preference model does not include veto, as in [33]. A satisfiability-based approach to learn the parameters of a non-compensatory sorting model was proposed by [34].
With respect to ELECTRE TRI-nC, obtaining representative profiles directly is less cognitively demanding than obtaining the limiting profiles in ELECTRE TRI-nB. The same happens when the methods with limiting and representative profiles proposed by [26,27] are compared. As a consequence, in a direct elicitation framework, most DMs should prefer a method based on representative profiles. However, in an indirect elicitation framework, the cognitive effort required by the DM consists basically of the creation of the set of assignment examples. Therefore, the DM should prefer the method that provides more “learning” capacity. Such capacity might depend on (i) the way that it is used to characterize the ordered classes; (ii) the number of assignment examples provided by the DM and (iii) the type of preference relation, namely, reflexive or asymmetric, that is used to compare actions with profiles.

Contributions

This paper presents a method to infer the parameter values and profiles of the methods in the proposals of both [26,27]. The first of these proposals is based on limiting profiles while the second one is based on characteristic profiles. In both cases, the outranking relation of ELECTRE TRI is used [2]; both methods can be used with either a reflexive preference relation or its asymmetric version. [32] The non-linear optimization problem maximizes an agreement measure between the model and the assignment examples and is addressed using a genetic algorithm. Variants of the genetic algorithm have been successfully used in several works by the authors [30,31,32,35], and have shown to be considerably more effective than other metaheuristics [31] in similar contexts (that is, in terms of the elicitation procedures of ELECTRE-inspired model parameters). Therefore, one of our main interests in this work is to analyze if the genetic algorithm with specific characteristics for the current problem continues to be as effective as in those works. We intend to answer the following research questions:
(1)
What are the learning capabilities of the proposal for each method, and for which of these methods is the inference process more effective?
(2)
Similarly, how does the effectiveness of the proposal behave regarding the type of relation (reflexive or asymmetric)?
(3)
To what extent is the “learned” model able to represent the underlying DM’s assignment policy when new actions are classified?
(4)
How does this capacity depend on the number of assignment examples provided by the DM?
(5)
How does this capacity depend on the number of classes and the number of criteria?
The first two questions are specially interesting. One could hypothesize that the method based on limiting profiles can make the proposal to learn with higher levels of effectiveness because of its ability to identify the boundaries of each class; however, the experiments showed that this is not the case: the highest levels of effectiveness were achieved in the context of the method based on characteristic profiles. Similarly, since an asymmetric relation has more inherent information than a reflexive relation, one might think that using the former should yield better results. However, the experiments also showed that it was actually in the context of the reflexive relation that the proposal achieved the best effectiveness.
In this paper, extensive computational experiments are performed to respond to the above questions.
The structure of the paper is as follows: Section 2 provides a brief background to the proposal, including the description of the multicriteria-sorting methods. Section 3 explains the generalities of the proposed methodology, while Section 4 presents the details for inferring the parameter values of the ELECTRE-based methods proposed in [26,27]. Finally, Section 5 assesses the robustness of the proposal and Section 6 concludes this paper.

2. Some Background

2.1. A Theoretical Insight

The family of ELECTRE (ELimination Et Choix Traduisant la REalité, ELimination and Choice Expressing the REality) methods exploit the so-called outranking relation [36,37,38] that represents the assertion “action x is at least as good as action y”. These family of methods is particularly useful when the decision problem includes at least three criteria to assess the actions and when either the actions are evaluated on an ordinal scale or there is a lot of heterogeneity or compensation among the criteria [37]. Depending on the type of decision problem (choosing, ranking, sorting), the family of ELECTRE methods offers subsets of methods able to address the problem; the ELECTRE TRI methods are within the subset of multicriteria-sorting methods.
The ELECTRE TRI methods share several consistency properties, namely, homogeneity, unicity, independence, monotonicity, conformity, and stability [5,16,39]. There are, however, some important differences. ELECTRE TRI-nC and its extensions are based on two (ascending and descending) rules, which are symmetric via the transposition operation. Such an operation consists of simultaneously inverting the order of the categories and the sense of preference in all the assessment criteria. According to [9,40], all relational-based multicriteria-sorting methods should fulfill symmetry in terms of the transposition operation, as carried out by ELECTRE TRI-C, ELECTRE TRI-nC and its extensions [16,39]. However, ELECTRE TRI-B and its extensions do not fulfill this symmetry property [9,40]. As [9] state, this lack of symmetry is a consequence of the way ELECTRE TRI-B defines the categories as closed from below.
ELECTRE TRI-B and ELECTRE TRI-nB are composed of two procedures, namely, the pseudo-conjunctive and pseudo-disjunctive rules. In the pseudo-conjunctive procedure, an outranking relation S is used to compare actions against limiting profiles (xSy denotes “action x is at least as good as action y”), while the pseudo-disjunctive procedure uses the asymmetric preference relation P, which is the asymmetric part of S. Ref. [9] (respectively, [12]) proposed to replace the conjoint use of the pseudo-conjunctive and pseudo-disjunctive procedures of ELECTRE TRI-B (resp., ELECTRE TRI-nB) by descending and ascending rules, which use S and are symmetric in terms of the transposition operation. However, in these proposals, the limiting profiles are fictitious actions, which do not belong to any category; to a large extent, this contradicts the conformity requirement, which states that the limiting profiles have to be assigned to the categories to which they belong.
The conflict between conformity and correspondence through the transposition operation was solved by [26]. They proposed to describe the limiting boundary Bk between classes Ck and Ck+1 by two “layers”; namely, BUk and BLk. Ck+1 is closed from below by BLk, while Ck is closed by BUk. Based on a general reflexive (respectively, asymmetric) binary preference relation S (resp. P), the method in [26] fulfills the whole set of consistency properties required by [5] and is symmetric in terms of the transposition operation. However, this theoretical advance required imposing strong conditions on the set of limiting profiles, which constitutes the main obstacle for the method in [26].
The outranking relation of the ELECTRE family and its extensions are particular cases of the relation on which the method in [26] is based. In the following, we call ELECTRE TRI-nB-2 to this method when S is the outranking relation as in ELECTRE TRI-B or ELECTRE III [2].
Regarding the relational methods that characterize classes through representative profiles, a similar generalization was proposed by [27]. Any reflexive preference relation S (or its asymmetric part P) can be used by two assignment rules that are equivalent through the transposition operation. If S is the outranking relation as in ELECTRE TRI-nC, the rules suggest assigning actions to ranges of possible categories containing the classes suggested by that former method.
Thus, both proposals in [26,27] fulfill the desirable properties. As discussed in the introduction, in a direct elicitation framework, the use of “characteristic” or representative profiles requires less effort from the DM. However, in a PDA approach, the cognitive demand depends only on the number of assignment examples. Given several assignment examples, it is interesting to illustrate (i) which way describing classes should be preferred and (ii) which preference relation (S or P) provides better results.

2.2. A Brief Description of the ELECTRE TRI-nB-2

ELECTRE Tri-nB-2 is a particular case of the method proposed in [26] when S is the outranking relation of ELECTRE TRI. The boundary Bk that separates the classes Ck and Ck+1 is described by a set of limiting profiles bk,j (j = 1, …, card (Bk)), Bk = {bk,j}, and is made up of two “layers”, called BLk and BUk, which are disjoint sets. BLk is composed of profiles belonging to Ck+1, while BUk by other limiting profiles belonging to Ck. So, Ck+1 is closed from below by BLk, and Ck is closed from above by BUk.
Let us denote by P the asymmetric part of S and by D the Pareto dominance relation. In ELECTRE TRI-nB-2, the profiles should fulfill the following requirements:
  • There is no (w,z) belonging to BLk × BLk such that wPz;
  • There is no (w,z) belonging to BUk × BUk such that wPz;
  • There is no (w,z) in BUk × BLk fulfilling wSz;
  • There is no (w,z) in Bk × Bh (h > k) fulfilling wSz;
  • For all z in BUk there is y in BLk−1fulfilling zSy;
  • For all z in BLk there is y in BUk + 1 that fulfills ySz;
  • For all z in BUk there is w in BUk + 1that fulfills wDz;
  • For all z in BUk there is w in BUk−1that fulfills zDw;
  • For each z in BLk there is y in BLk−1that fulfills zDy;
  • For all z in BLk there is y in BLk + 1 that fulfills yDz.
The authors in [26] proposed two assignment rules, namely, primal and dual, which are symmetric in terms of the transposition operation; these rules should be used conjointly. They are based on a relation S between actions and boundaries, defined as: (i) xSBk if, and only if, there is yBLk such that xSy and there is no wBk fulfilling wPx; (ii) BkSx if, and only if, there is yBUk such that ySx and there is no wBk fulfilling xPw.
The primal rule is a descending procedure that assigns the action x to Ck+1, where k is the subscript of the first limiting boundary fulfilling xSBk. On the other hand, the dual rule is an ascending procedure that assigns x to Cj, where j is the first subscript fulfilling BjSx. All the categories in the range between Cj and Ck (or vice versa) are possible assignments for x.

2.3. The Methods Based on Comparing Actions against Representative Profiles

The authors in [27] proposed alternative multicriteria-sorting methods, one based on S and one based on P.
In both methods, each category Ck is characterized by a subset Rk of representative profiles, rk,j, j = 1,…, card (Rk), which must fulfill several demands. For the S-based method, these demands are the following:
  • For all ordered pairs (k, h) (h > k), for k = 1, …, M-1, for each y in Rh, there is no z in Rk, fulfilling zSy;
  • For all ordered pairs (k, h) (h > k), for k = 1, …, M-1, for each y in Rk, there is a z in Rh, fulfilling zSy;
  • For all ordered pairs (k, h) (k > h), for k = 2, …, M, for each y in Rk, there is a z in Rh fulfilling ySz.
  • For all ordered pairs (k, h) (h > k), for k = 1, …, M-1, for each y in Rk, there is a z in Rh, fulfilling zDy;
  • For all ordered pairs (k, h) (k > h), for k = 2, …, M for each y in Rk, there is a z in Rh, fulfilling yDz.
The S-relation between actions and subsets of profiles is defined as
  • xSRk if and only if there is a y belonging to Rk, that fulfills xSy;
  • RkSx if and only if there is a y in Rk, fulfilling ySx;
Like ELECTRE TRI-nC, [27] proposed two assignment rules, which are symmetric via the transposition operations. The descending rule finds the first subscript k for which xSRk, and x is assigned to Ck or Ck+1. In the ascending rule, the first h such that RhSx is identified, and then x is assigned to Ch−1 or Ch. The conjoint assignment rule suggests assigning x to a class within the range Ch-Ck (or vice versa).
For the P-based method, the requirements are the same as above—replacing S by P, but adding (vi). For k = 1,…, M, there is no pair (z, w) in Rk × Rk that fulfills zPw. This requirement forces profiles to be central in order to achieve a good characterization of their related category (a central profile is a characteristic action representative of a class). The relation P between actions and subsets of profiles is defined as the relation S, but with P instead of S. The descending and ascending rules are similar, but replacing xSRk with xPRk, and RhSx with RhPx. The conjoint assignment rule is identical to that of the S-based method.

3. The Proposed Methodology

The proposed methodology is based on a set of reference examples. This set is built by the decision-maker according to his/her own system of preferences/decision policy; then, based on a given decision model able to assess xSy (from which, xPy can be derived), the proposed methodology intends to define the model’s parameter values such that the decision model can reproduce the reference examples as accurately as possible.
Even when this work exploits the proposed methodology in the context of the ELECTRE TRI models [26,27], the methodology is general enough to work with virtually any decision model (not only those based on ELECTRE methods) whose complexity requires the optimization stage to be an evolutionary algorithm. Other examples of decision models that can fulfill such a condition are those based on value functions with veto conditions (e.g., [41]), the ELECTRE methods with interactions between criteria [42], the extended ELECTRE method to handle reinforced preferences and counter-veto effects [43], the hierarchical version of the ELECTRE methods [44], and the interval outranking approach by [45,46].
Figure 1 shows an insight into the proposed approach.

3.1. Input Data

As shown in Figure 1, the input data of the proposed approach are composed by a coherent family of criteria, a set of decision examples, a set of preferentially ordered classes, and a set of (limiting or characteristic) profiles for each class.

3.1.1. A Coherent Family of Criteria

The set of criteria must be established based on three characteristics: (1) no redundancy (each criterion is considered only once); (2) completeness (the criteria characterize all the significant objectives for the decision problem); and (3) consistency, which involves the DM’s preference to be consistent with the comprehensive assessment.

3.1.2. A Set of Assignment Examples

Let x and y be decision alternatives, each of them characterized by its scores on the family of criteria. The assignment examples provided by the DM are of the form “x should be assigned to class Ci”. These decisions may be past decisions or new easy-to-make decisions (that can even include fictitious alternatives). In each of these situations, the examples provide instances about the judgment policy of the DM as well as his/her system of preferences; furthermore, the examples provided can influence the preference model defined by the proposal. Therefore, it is important for the DM to carefully express his/her holistic preference on the basis of all the criteria defined in Section 3.1.1.

3.1.3. A Sequence of Preferentially Ordered Classes

Let C1, C2, …, CM, be a sequence of classes such that each class denotes intensity of preference, thus the decision alternatives assigned to Ci are not worse than those assigned to Cj for i > j.

3.1.4. The Number of (Limiting or Characteristic) Profiles

As stated in the introduction, classes may be characterized by limiting profiles that describe the boundaries between classes or by representative (central) profiles. Each of these profiles is characterized through scores on the family of criteria (as decision alternatives). The proposal expects the DM to provide the number of profiles to be defined by the approach.

3.2. A Genetic Algorithm to Select the Most Convenient Set of Parameter Values

A genetic algorithm is used to infer representative profiles and parameter values compatible with the decision examples provided. The canonic version of a genetic algorithm was used to define the most convenient parameter values. The description of this genetic algorithm is provided in Section 4.3.

3.3. Interaction with the DM and Final Selection of the Parameter Values

Finally, the approach identifies the set of parameter values and profiles most compatible with the decision examples provided by the DM; if the DM is not satisfied with this set, then he/she should adjust the information provided as the input. This information is always consistent, and there should always be at least one set of parameter values and profiles compatible with the decision examples. If the DM is comfortable and approves the recommendation, the inference process ends; otherwise, the input information should be revised and adjusted.

4. Eliciting the Preference Models

As explained above, the proposed methodology can work with virtually any decision model. Here, we describe the components used to infer the sets of parameter values and profiles of either ELECTRE TRI-nB-2 or the method based on comparing actions against representative profiles. In both cases, the outranking relation of ELECTRE TRI is used (cf. [2]). The degree of credibility of the outranking relation, σ(x, y), is built on A × A, where A is a set of actions described on the basis of N criteria (without loss of generality, we assume that the performance on these criteria should be maximized). The output of σ(x, y) denotes the degree of credibility of the assertion “x is at least as good as y” from the perspective of the DM. The steps to calculate σ(x, y) can be followed from [2].
The relative importance of each criterion is called weight and reflects its voting power; the weight wi of each criterion gi must fulfill wi > 0 and, for convenience, ∑wi = 1. The procedure described in [2] to calculate σ(x, y) uses an indifference threshold, qi(gi(x)) ≥ 0, and a preference threshold for each criterion gi fulfilling pi(gi(x)) ≥ qi(gi(x)). It also uses veto thresholds, vi(gi(x)), to reflect the veto power of some criteria toward the hypothesis that the outranking relation is met. The authors in [47] advanced this procedure to also consider pre-veto thresholds, ui(gi(x)). For readability purposes, we use qi, pi, vi, ui to denote these concepts. Finally, if there is enough evidence to accept that “x is at least as good as y” and there is no enough evidence that opposes the assertion, then it is accepted that “action x outranks action y”, which is denoted by xSy. xSy can be assessed using the outranking relation and a credibility threshold, λ. Formally,
xSy ⇔ σ(x, y) ≥ λ.
Note that these parameters depend on the decision policy of the decision-maker; thus, decision models with different sets of parameter values can lead to different decisions. Our proposal consists of inferring the parameter values that best restore the reference decisions provided by the DM.

4.1. Inferring an ELECTRE TRI-nB-2 Model

According to previous discussions, the information that must be inferred to fully operationalize the ELECTRE TRI-nB-2 method is the following:
  • Weights: wi for i = 1, …, N;
  • Veto thresholds: vi for i = 1, …, N;
  • Pre-veto thresholds: ui for i = 1, …, N;
  • Majority threshold: λ.
Limiting profiles: Bk = BLkBUk, where each element in BLk and each element in BUk are characterized by their assessments on the N criteria. The set of profiles Bk (made up by the disjoint sets BLk and BUk) that separates the classes Ck and Ck+1, and whose cardinality is given by the DM, must fulfill the conditions described in Section 2.2.
Let C = {C1, C2, …, CM} be the set of classes defined by the DM, such that Ci + 1 is preferred over Ci, and T be the set of actions used in the decision examples provided by the DM, such that each x T has been assigned by the DM to a class CjC. Each of these decision examples contains holistic information about the preferences of the DM. Therefore, the goal of the inference process is to assign values to the parameters so the ELECTRE TRI-nB-2 method can reproduce the decision examples.
The inferred information that is most appropriate to fit the examples provided by the DM, n B i n f * , is the one that minimizes the number of inconsistencies with respect to the expressed decisions. Each x T is assigned by the DM to a range of classes through his/her own system of preferences, say n B D M ; similarly, the ELECTRE TRI-nB-2 method can use n B i n f to assign each x T to a range of classes. Thus, let χ D M be the set of classes to which the DM has assigned x and let χ i n f be the set of inferred classes. Since x is not necessarily assigned to only one class but to a range of classes, the so-called F1-score [48] is exploited here. The authors in [49] use the concepts of precision, Q, and recall, R, to define this measure: F1-score = 2QR/(Q + R). The authors in [33] use such a procedure to define A c x , n B D M , n B i n f in the following non-linear optimization problem:
M a x i m i z e w i , v i , u i , λ , B k F B
Subject to
wi > 0,
wi = 1,
vi > ui > pi,
0.5 < λ < 1,
Bk fulfills constraints i–x in Section 2.2.
  • where
i = 1, …, N,
F B = A c n B D M , n B i n f c a r d ( T )
A c n B D M , n B i n f = x D A c x , n B D M , n B i n f
and
A c x , n B D M , n B i n f = 2 χ D M χ i n f χ D M + χ i n f
Note that the decision variables in Problem (1) are those listed at the beginning of this Section.

4.2. Inferring a Model for the Representative-Profiles-Based Methods

A set of assignment examples is also used to define the information required by the methods that use representative profiles. The information that must be inferred in this case is the same as in the case of the ELECTRE TRI-nB-2 model except for the case of the profiles since now the profiles that must be inferred are representative of each class.
Let W be the set of assignment examples such that each x W is assigned by the DM to a range of elements of the set of classes C = C 1 , , C k , , C M . Since both the DM and the method described in Section 2.3 assign each action x W to a range of classes, the F1-score is also used in the context of representative profiles to determine the most convenient set of profiles and parameter values, n C i n f . Following the intuition of the notation used in the previous section, the fitness function is defined as follows:
M a x i m i z e w i , v i , u i , λ , R k F C
Subject to
wi > 0,
wi = 1,
vi > ui > pi,
0.5 < λ < 1,
Rk fulfills constraints i–v in Section 2.3.
  • where
i = 1, …, N,
F C = A c n C D M , n C i n f c a r d ( T )
A c n C D M , n C i n f = x D A c x , n C D M , n C i n f
and
A c x , n C D M , n C i n f = 2 χ D M χ i n f χ D M + χ i n f
Note that the decision variables in Problem (1) are those listed at the beginning of Section 4.1, except for Rk that substitutes Bk.

4.3. A Genetic Algorithm to Address Equations (1) and (2)

Note that the optimization problems resulting from Equations (1) and (2) are non-linear optimization problems with non-convex constraints given that the whole sets of parameters mentioned in Section 4.1 are being inferred simultaneously [29]. Therefore, following evidence from other works [25,31], we use an evolutionary algorithm to address these problems. Particularly, we use a genetic algorithm inspired by those described in works that are similar to a certain extent [27,30,31,32,35]; we use similar genetic operators with different representations of individuals.
Evidently, using a decision model different to that used here (an ELECTRE-based decision model) implies some changes to the procedure described below; particularly, different parameters require a different structure of the individuals.
In the case of Problem (1), the algorithm uses a real-valued vector composed of n 3 + K M 1 + 1 genes to denote each individual (where K is the number of profiles used to separate each pair of classes and n is the number of criteria), as shown in Figure 2; in this figure, gi(bk,j) is the impact of profile bk,jBk on the ith criterion. In the case of Problem (2), the algorithm also uses a real-valued vector but is now composed of n 3 + O M + 1 , where O is the number of profiles used to characterize each class, as shown in Figure 3.
The initial population of the genetic algorithm contains individuals that are generated randomly but that fulfill the constraints established by the DM/analyst pair. The selection of the parents in the algorithm is performed using binary tournament, while the crossover is at one point of the individuals. To facilitate consistency, the genes related to the weights form an inseparable unit. Only one offspring individual is created from each pair of parent individuals; then, a given probability is assigned to the possible mutation of the offspring individual. This mutation involves the random generation of a randomly selected gene (or set of genes, in the case of the weights) in such a way that the mutated offspring fulfills the constraints. This way, a number of offspring individuals equal to the number of individuals in the population is generated (say, pop_size), and all the (parent and offspring) individuals are inserted into a pool from which pop_size-1 individuals are randomly selected to create the population in the next iteration of the algorithm. Each individual’s fitness is calculated using Problems (1) or (2). When the stopping criterion is met, the best individuals found so far compose the set of best solutions in the current execution of the algorithm. A centroid of such a set is calculated using the average values of the parameters in the set. Such a centroid is considered as the best solution when it achieves the best-known fitness value, otherwise, the solution closest to the centroid is chosen (in terms of the Euclidean distance). Furthermore, to ensure robustness, twenty executions of the algorithm are performed.
The control parameters of the genetic algorithm are defined following the results of previous works by the authors [30,31,32,35,50]. In those works, similar optimization problems have been addressed through the genetic algorithm; its parameters had been previously determined using a classical configuration technique, ParamILS [51]. Assessing the neighborhood of such values helped us to determine the specific control parameters used in this work in preliminary experimentation. The control parameters used by the algorithm are size of the population, number of generations, probability of crossover, and probability of mutation. The values are, respectively, 200, 200, 60% and 2%.
This procedure is formalized in Algorithm 1.
Algorithm 1. Genetic Algorithm proposed to address Problems (1) and (2).
Require: A set of reference examples, T
Ensure: ρ f i n a l , individual representing the population with the best fitness values
   1:    i 1
   2:    ρ n u l l
   3:                g 0
   4:                P g create-Initial-Population ()
   5:               for g < 200  do
   6:                     H g    create-Offspring   ( P g , selection, crossover, mutation)
   7:                     P g + 1 generate-Population ( P g H g )
   8:                     g g + 1
   9:               end for
10:          b e s t k n o w n find-Best   ( P g )
11:          ρ    find-Centroid ( b e s t k n o w n )
12:      if    ρ    is-best ( b e s t k n o w n )
13:                   ρ f i n a l ρ
14:      else
15:                   ρ f i n a l find-closest ( ρ )

5. Assessing the Robustness of the Inference Procedure

This section details the experimentation performed to assess the robustness of the inference approach.

5.1. Experimental Design

The basic experimentation consists of the following steps:
  • Simulate the preference model of a decision-maker and some limiting (respectively, characteristic) profiles that are all compatible with the model that uses limiting (respectively, characteristic) actions; this way, both the parameter values of the preference model and the set of profiles are known.
  • Use the known preference model to assign a set of reference actions to ordered classes.
  • Exploit Algorithm 1 to infer the parameter values of the preference model.
  • Simulate actions different to the ones used as reference.
  • Assign the new actions to ordered classes using the simulated preference model; also assign them using the inferred parameter values.
  • Obtain an out-of-sample effectiveness, measuring the proportion of coincidences between the assignments made on points 4 and 5.
To improve the soundness of the experimentation, the procedure described above is performed twenty times and, for each of these, the experiment setup uses a wide variety of values, as shown in Table 1.
Four profiles were used (per class/per boundary) in all scenarios. The mean values of the results are used below and, since there are large numbers of experiments being carried out, following the Central Limit theorem, it can be assumed that such mean values are normally distributed. Therefore, two-sample t-tests with a significance level of 0.05 are used to assess the null hypothesis of “two mean values are equal”.

5.2. Results

First, we identify the type of method (based on central or limiting profiles) that displays higher performance using the reflexive S relation (since it presents milder requirements); then, we make a deeper analysis of the results provided by such a method.
In Table 2, the fourth column compares the out-of-sample effectiveness of the classification method based on central objects in terms of a different number of training objects per class (2, 4, and 6). For each number of criteria (N) and classes (M), the effectiveness is compared according to the number of reference objects per class. The group that showed a statistically significant difference with respect to the others is marked in red. If there are two rows marked in red (for example, in the row with N = 3 and M = 2), it means that those values are better than the rest, but they do not have a significant difference between themselves.
The sixth column reflects the same process but for the method based on limiting profiles. The blue color indicates the row that shows a statistically significant difference with respect to the others.
Remarkably, all the values in column 4 are significantly higher than the corresponding values in column 6. The method based on representative profiles is clearly superior to the method based on limiting profiles for all N, M, and the number of training objects.
Table 3 integrates the information considering the number of criteria N as a separation variable. In column 2 of this table, the value that shows a statistically significant difference with the others in the same column is marked in red. In column 4, the highest value (with a significant difference) is marked in blue. Table 4 shows similar information in the context of the number of classes M.
Since it is evident that the method based on representative profiles was more effective than the one based on limiting profiles, we proceed to deepen in the results provided by the former.
Table 5 compares the effectiveness obtained using the method based on representative objects first with the reflexive relation S and then with the asymmetric relation P. Only the effectiveness in the triples (N, M, nclass) is shown in this table, where the difference between the treatment with S and the treatment with P showed significant differences. The treatment that was superior is marked in red.
Table 6 reproduces the results of Table 5 but with a focus on the number of criteria. A significant difference is marked in red.
Similarly, Table 7 focuses on the number of classes to compare the effectiveness of the method based on representative profiles using the relation S and using the relation P.

5.3. Discussion

The proposed methodology is very general; it can be exploited to infer virtually any multicriteria decision model that is complex enough that the optimization process must be performed through metaheuristics. An important advantage of this work consists of the practical value of the experiments. Given a multicriteria-sorting problem, the experiments answer the following questions: which of the considered sorting methods can the decision-maker choose? Should he/she use representative or limiting profiles to identify the classes? Should he/she use an asymmetric preferential relation, or a reflexive relation is enough to reach high levels of effectiveness? How many reference objects/assignment examples should he/she provide?
The experiments assessed the robustness of the proposed methodology by using many diverse scenarios, including the number of criteria, the number of classes, the number of reference objects (assignment examples) per class, the decision policy of the simulated decision-maker providing the examples, and the type of sorting method and preference relation. It is easy for the decision-maker to provide the reference examples required by the proposed methodology. According to the results of the experiments, only a limited number of examples can be enough to obtain sufficiently high levels of effectiveness in the definition of the parameter values.
It can be seen from Table 2 that, when there are only two classes and the method based on characteristic profiles is used, then two or four objects per class is enough to reach maximum effectiveness; so, little cognitive effort is required from the DM in this case. However, this effect is not appreciated with the method based on limiting profiles, where, even with the minimum number of criteria (three) and classes (two), the highest level of effectiveness is only achieved using the maximum number of objects per class (six). It can be seen from this table that, when there are more than two classes, increasing the number of objects per class most of the time improves the capacity to learn the reference examples.
The range of effectiveness values in Table 2 for the method based on characteristic profiles is 0.82–0.97, with an average value of 0.89. As expected, the lowest effectiveness levels are seen in the highest complexities of the problem, where nine criteria and five classes are considered; although, even in this case, increasing the number of objects per class almost always increased the effectiveness. All this is also true for the method based on limiting profiles, whose range of effectiveness in Table 2 is 0.57–0.89 (with significative lower values than those produced using characteristic profiles, respectively). Another interesting result from this table is that effectiveness seems to be more affected by the number of classes than by the number of criteria; for example, for the method based on reference profiles, the average effectiveness when N = 3 and M = 5 is 0.88, while the average effectiveness when N = 5 and M < 4 is greater than 0.88. Very similar results can be seen for the method based on limiting profiles and for other combinations of N, M. This assertion is confirmed by Table 3 and Table 4; the former shows that increasing the number of criteria does not necessarily decrease effectiveness, while the latter clearly shows that increasing the number of classes decreased effectiveness.
Unequivocally, Table 2, Table 3 and Table 4 show higher learning abilities when using representative profiles over using limiting profiles, both with the reflexive relation S. This is an outstanding result since the method based on comparing actions against representative profiles demands the fulfillment of fewer conditions compared to those of the method based on limiting profiles. Therefore, future works relying on the proposed inference process should follow a methodology based on representative profiles.
Table 5, Table 6 and Table 7 show that, in most cases, the effectiveness of the method based on representative profiles using the reflexive relation S is significatively greater than that when the asymmetric relation P is used; this is particularly true when there are three and five criteria (Table 6) and when there are four and five classes (Table 7).
On the other hand, it appears that using S provides higher effectiveness in the presence of lower numbers of objects per class, since Table 5 shows that using P was better only once when there were two objects (in the case N = M = 3); however, further experiments are required to provide deeper insights in this regard.

6. Conclusions and Future Work

The full operationalization of recent multicriteria-sorting methods has been achieved. These methods can use either a reflexive or an asymmetric general preference relation and either limiting or representative profiles; furthermore, they fulfill all the fundamental properties commonly required from multicriteria-sorting methods (see [5,16]); therefore, they constitute some of the widest generalizations of the relational paradigm applied to ordinal classification. However, the methods require the definition of many parameter values (common in outranking-based methods), which is arduous work for the decision-maker that can lead to counterintuitive results.
This paper has described a complete methodology to infer the most convenient parameter values based on sets of decision examples provided by the decision-maker. The proposed inference methodology addresses the specific characteristics of the sorting methods through an accuracy measure that defines the effectiveness of the inference process. The optimization of this measure is performed through evolutionary algorithms given the non-convexity of the search space, which is caused by inferring all the parameters of the methods simultaneously.
The main conclusion of the results is that, for all the tested scenarios in the context of the multi-criteria aggregation of preferences from ELECTRE III, the highest effectiveness values of the inference process were obtained when using the method based on representative profiles. Since such a method is also the one with the mildest requirements, the decision-maker–decision-analyst pair should concentrate on this sorting method. On the other hand, using such a method with the reflexive relation S was apparently more effective than using it with the asymmetric relation P; however, this result was not conclusive, since using the relation P seems to outperform the former in presence of lower numbers of classes; so, further experiments should be performed to test this hypothesis. The good performance of the reflexive relation regarding its asymmetric counterpart could be a result of milder conditions. Note that the condition imposed to the method when it is using the relation P implies that the profiles must be central. By not constraining the method to such a condition when it is using S, perhaps the profiles can better represent the class. Theoretical (more than practical) studies will be performed in future work to discover the real causes.
The results shown in Table 2 indicate that, when two classes are used in the sorting process, then only two and sometimes four objects per class are enough to reach the highest effectiveness. However, when there are more than two classes in the problem, then, regarding a number of criteria and a number of classes, the highest effectiveness can be achieved with the highest number of objects per class (six). Therefore, an interesting future research line is to determine if this is the highest achievable effectiveness, or if such an effectiveness can be increased by increasing the number of objects per class. Of course, if the latter is true, deciding to increase the effectiveness of the inference process is dependent on the decision-analyst–decision-maker pair, and it should be considered that the effectiveness of most scenarios of Table 2 is considerably high. Other results indicate that increasing the number of classes decreased the effectiveness of the methodology, but increasing the number of criteria does not necessarily provoke this effect.
It is important to remark that these conclusions are valid for models based on ELECTRE III. Analyses with other ways to aggregate preferences are pending and will be addressed in future work. More research is needed to compare our results and validate our conclusions using other metaheuristic approaches.

Author Contributions

Conceptualization, E.F.; methodology, E.F., J.N. and E.S.; software, J.N.; validation, E.F. and E.S.; formal analysis, E.F. and J.N.; investigation, J.N. and E.S.; resources, J.N., A.F. and R.D.; data curation, J.N.; writing—original draft preparation, E.S.; writing—review and editing, E.F., A.F. and R.D.; visualization, A.F. and R.D.; supervision, E.F.; project administration, E.S.; funding acquisition, A.F. and R.D. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

Data are contained within the article.

Acknowledgments

Jorge Navarro is grateful for the support from the Autonomous University of Sinaloa. Eduardo Fernández, Efrain Solares and Abril Flores acknowledge the support from the Autonomous University of Coahuila. Efrain Solares thanks the Mexican Council for Science and Technology (CONACYT) for its support to project no. 321028.

Conflicts of Interest

The authors declare no conflict of interest.

Nomenclature

Aset of actions described on the basis of N assessment criteria
bk,jjth element of Bk
Bkset of limiting profiles separating classes Ck and Ck+1
BLksubset of limiting boundaries that closes Ck+1 from below
BUksubset of limiting boundaries that closes Ck from below
Ccollection of ordered classes
Ckkth element of C
DPareto dominance relation
ξ n B set of models containing the information necessary to operationalize the ELECTRE TRI-nB-2 method
ξ n C set of models containing the information necessary to operationalize the methods that use characteristic profiles
giith assessment criterion
K number of profiles used to separate each pair of classes
λcredibility threshold for the outranking relation such that xSy ⇔ σ(x, y) ≥ λ
Mnumber of classes
n B D M assignments made by the decision-maker through his/her own system of preferences in the context of profiles in the limiting boundaries
n B i n f element of ξ n B
n B i n f * inferred information that is most appropriate to fit the examples provided by the decision-maker in the context of profiles in the limiting boundaries
n C D M assignments made by the decision-maker through his/her own system of preferences in the context of profiles that characterize the classes
n C i n f element of ξ n C
n C i n f * inferred information that is most appropriate to fit the examples provided by the decision-maker in the context of profiles that characterize the classes
n c l a s s number of assignment examples per class
n D M s number of simulated decision-makers
n O S number of out-of-sample actions
Nnumber of assessment criteria
O number of profiles used to characterize each class
pi(gi(x))indifference threshold of criterion gi
Pasymmetric part of S
qi(gi(x))indifference threshold of criterion gi
rk,jjth element of Rk
Rkset of representative profiles
ρ f i n a l individual representing the population with the best fitness values
Soutranking relation such that xSy denotes “action (or set of limiting profiles) x is at least as good as action y (or set of limiting profiles)”
σ(x, y)degree of credibility of the outranking relation
Tset of actions used in the decision examples provided by the decision-maker in the context of profiles in the limiting boundaries
uipre-veto threshold of criterion gi
viveto threshold of criterion gi
wiweight (relative importance) of criterion gi
χ D M set of classes to which the decision-maker has assigned action x
χ i n f set of classes inferred by the proposal to assign action x

References

  1. French, S. Decision Theory: An Introduction to the Mathematics of Rationality; Halsted Press: Ultimo, Australia, 1986. [Google Scholar]
  2. Roy, B. The Outranking Approach and the Foundations of ELECTRE Methods. In Readings in Multiple Criteria Decision Aid; Springer: Berlin/Heidelberg, Germany, 1990; pp. 155–183. [Google Scholar]
  3. Greco, S.; Matarazzo, B.; Slowinski, R. Conjoint Measurement and Rough Set Approach for Multicriteria Sorting Problems in Presence of Ordinal Criteria. In A-MCD-A: Aide Multi Critère à la Décision: Multiple Criteria Decision Aiding; Office of Official Publications of the European Communities: Luxembourg, 2001; pp. 117–144. [Google Scholar]
  4. Yu, W. Aide Multicritère à La Décision Dans Le Cadre de La Problématique Du Tri: Concepts, Méthodes et Applications. Ph.D. Thesis, Dauphine Université Paris, Paris, France, 1992. [Google Scholar]
  5. Roy, B.; Bouyssou, D. Aide Multicritère à La Décision: Méthodes et Cas; Economica Paris: Paris, France, 1993. [Google Scholar]
  6. Araz, C.; Ozkarahan, I. Supplier Evaluation and Management System for Strategic Sourcing Based on a New Multicriteria Sorting Procedure. Int. J. Prod. Econ. 2007, 106, 585–606. [Google Scholar] [CrossRef]
  7. Nemery, P.; Ishizaka, A.; Camargo, M.; Morel, L. Enriching Descriptive Information in Ranking and Sorting Problems with Visualizations Techniques. J. Model. Manag. 2012, 7, 130–147. [Google Scholar] [CrossRef]
  8. Ishizaka, A.; Nemery, P. Assigning Machines to Incomparable Maintenance Strategies with ELECTRE-SORT. Omega 2014, 47, 45–59. [Google Scholar] [CrossRef] [Green Version]
  9. Bouyssou, D.; Marchant, T. On the Relations between ELECTRE TRI-B and ELECTRE TRI-C and on a New Variant of ELECTRE TRI-B. Eur. J. Oper. Res. 2015, 242, 201–211. [Google Scholar] [CrossRef] [Green Version]
  10. Corrente, S.; Greco, S.; Słowiński, R. Multiple Criteria Hierarchy Process for ELECTRE Tri Methods. Eur. J. Oper. Res. 2016, 252, 191–203. [Google Scholar] [CrossRef] [Green Version]
  11. Fernández, E.; Figueira, J.R.; Navarro, J.; Roy, B. ELECTRE TRI-NB: A New Multiple Criteria Ordinal Classification Method. Eur. J. Oper. Res. 2017, 263, 214–224. [Google Scholar] [CrossRef]
  12. Bouyssou, D.; Marchant, T.; Pirlot, M. A Theoretical Look at ELECTRE TRI-NB; ffhal-02898131v1; Peter Lang International Academic Publishing Group: New York, NY, USA, 2020. [Google Scholar]
  13. Kadziński, M.; Martyn, K.; Cinelli, M.; Słowiński, R.; Corrente, S.; Greco, S. Preference Disaggregation for Multiple Criteria Sorting with Partial Monotonicity Constraints: Application to Exposure Management of Nanomaterials. Int. J. Approx. Reason. 2020, 117, 60–80. [Google Scholar] [CrossRef]
  14. Perny, P. Multicriteria Filtering Methods Based Onconcordance and Non-Discordance Principles. Ann. Oper. Res. 1998, 80, 137–165. [Google Scholar] [CrossRef]
  15. Köksalan, M.; Özpeynirci, S.B. An Interactive Sorting Method for Additive Utility Functions. Comput. Oper. Res. 2009, 36, 2565–2572. [Google Scholar] [CrossRef]
  16. Almeida-Dias, J.; Figueira, J.R.; Roy, B. Electre Tri-C: A Multiple Criteria Sorting Method Based on Characteristic Reference Actions. Eur. J. Oper. Res. 2010, 204, 565–580. [Google Scholar] [CrossRef] [Green Version]
  17. Almeida-Dias, J.; Figueira, J.R.; Roy, B. A Multiple Criteria Sorting Method Where Each Category Is Characterized by Several Reference Actions: The Electre Tri-NC Method. Eur. J. Oper. Res. 2012, 217, 567–579. [Google Scholar] [CrossRef] [Green Version]
  18. Buğdaci, A.G.; Köksalan, M.; Özpeynirci, S.; Serin, Y. An Interactive Probabilistic Approach to Multi-Criteria Sorting. IIE Trans. 2013, 45, 1048–1058. [Google Scholar] [CrossRef]
  19. Kadziński, M.; Tervonen, T.; Figueira, J.R. Robust Multi-Criteria Sorting with the Outranking Preference Model and Characteristic Profiles. Omega 2015, 55, 126–140. [Google Scholar] [CrossRef]
  20. Jacquet-Lagreze, E. An Application of the UTA Discriminant Model for the Evaluation of R & D Projects. In Advances in Multicriteria Analysis; Springer: Berlin/Heidelberg, Germany, 1995; pp. 203–211. [Google Scholar]
  21. Zopounidis, C.; Doumpos, M. PREFDIS: A Multicriteria Decision Support System for Sorting Decision Problems. Comput. Oper. Res. 2000, 27, 779–797. [Google Scholar] [CrossRef]
  22. Greco, S.; Mousseau, V.; Słowiński, R. Multiple Criteria Sorting with a Set of Additive Value Functions. Eur. J. Oper. Res. 2010, 207, 1455–1470. [Google Scholar] [CrossRef] [Green Version]
  23. Błaszczyński, J.; Greco, S.; Słowiński, R.; Szelag, M. On Variable Consistency Dominance-Based Rough Set Approaches. In Proceedings of the International Conference on Rough Sets and Current Trends in Computing, Kobe, Japan, 6–8 November 2006; pp. 191–202. [Google Scholar]
  24. Dembczyński, K.; Greco, S.; Słowiński, R. Rough Set Approach to Multiple Criteria Classification with Imprecise Evaluations and Assignments. Eur. J. Oper. Res. 2009, 198, 626–636. [Google Scholar] [CrossRef]
  25. Doumpos, M.; Marinakis, Y.; Marinaki, M.; Zopounidis, C. An Evolutionary Approach to Construction of Outranking Models for Multicriteria Classification: The Case of the ELECTRE TRI Method. Eur. J. Oper. Res. 2009, 199, 496–505. [Google Scholar] [CrossRef]
  26. Fernández, E.; Figueira, J.R.; Navarro, J. A Theoretical Look at Ordinal Classification Methods Based on Comparing Actions with Limiting Boundaries between Adjacent Classes. Ann. Oper. Res. 2022, 1–25. [Google Scholar] [CrossRef]
  27. Fernández, E.; Figueira, J.R.; Navarro, J.; Solares, E. A Generalized Approach to Ordinal Classification Based on the Comparison of Actions with Either Limiting or Characteristic Profiles. Eur. J. Oper. Res. 2023, 305, 1309–1322. [Google Scholar] [CrossRef]
  28. e Souza Rodrigues, B.; Martins Floriano, C.; Pereira, V.; Costa Roboredo, M. An Algorithm to Elicitate ELECTRE II, III and IV Parameters. Data Technol. Appl. 2021, 55, 82–96. [Google Scholar] [CrossRef]
  29. Mousseau, V.; Slowinski, R. Inferring an ELECTRE TRI Model from Assignment Examples. J. Glob. Optim. 1998, 12, 157–174. [Google Scholar] [CrossRef]
  30. López, J.C.L.; Solares, E.; Figueira, J.R. An Evolutionary Approach for Inferring the Model Parameters of the Hierarchical ELECTRE III Method. Inf. Sci. 2022, 607, 705–726. [Google Scholar] [CrossRef]
  31. Cruz-Reyes, L.; Fernandez, E.; Rangel-Valdez, N. A Metaheuristic Optimization-Based Indirect Elicitation of Preference Parameters for Solving Many-Objective Problems. Int. J. Comput. Intell. Syst. 2017, 10, 56. [Google Scholar] [CrossRef] [Green Version]
  32. Fernández, E.; Figueira, J.R.; Navarro, J. An Indirect Elicitation Method for the Parameters of the ELECTRE TRI-NB Model Using Genetic Algorithms. Appl. Soft Comput. 2019, 77, 723–733. [Google Scholar] [CrossRef]
  33. Meyer, P.; Olteanu, A.-L. Handling Imprecise and Missing Evaluations in Multi-Criteria Majority-Rule Sorting. Comput. Oper. Res. 2019, 110, 135–147. [Google Scholar] [CrossRef]
  34. Belahcene, K.; Labreuche, C.; Maudet, N.; Mousseau, V.; Ouerdane, W. An Efficient SAT Formulation for Learning Multiple Criteria Non-Compensatory Sorting Rules from Examples. Comput. Oper. Res. 2018, 97, 58–71. [Google Scholar] [CrossRef] [Green Version]
  35. Fernández, E.; Navarro, J.; Solares, E.; Coello, C.A.C.; Díaz, R.; Flores, A. Inferring Preferences for Multi-Criteria Ordinal Classification Methods Using Evolutionary Algorithms. IEEE Access 2023, 11, 3044–3061. [Google Scholar] [CrossRef]
  36. Basilio, M.P.; Brum, G.S.; Pereira, V. A Model of Policing Strategy Choice: The Integration of the Latent Dirichlet Allocation (LDA) Method with ELECTRE I. J. Model. Manag. 2020, 15, 849–891. [Google Scholar] [CrossRef]
  37. Figueira, J.; Mousseau, V.; Roy, B. ELECTRE Methods. In International Series in Operations Research and Management Science; Figueira, S.G., Ehrgott, M., Eds.; Springer Science + Business Media, Inc.: New York, NY, USA, 2005; Volume 78, pp. 133–162. [Google Scholar]
  38. Tam, C.M.; Tong, T.K.L.; Lau, C.T. ELECTRE III in Evaluating Performance of Construction Plants: Case Study on Concrete Vibrators. Constr. Innov. 2003, 3, 45–61. [Google Scholar] [CrossRef]
  39. Fernández, E.; Navarro, J.; Solares, E. A Hierarchical Interval Outranking Approach with Interacting Criteria. Eur. J. Oper. Res. 2022, 298, 293–307. [Google Scholar] [CrossRef]
  40. Roy, B. Présentation et Interprétation de La Méthode ELECTRE TRI Pour Affecter Des Zones Dans Des Catégories de Risque. Doc. Du LAMSADE 2012, 124, 1–25. [Google Scholar]
  41. de Almeida, A.T. Additive-Veto Models for Choice and Ranking Multicriteria Decision Problems. Asia-Pac. J. Oper. Res. 2013, 30, 1350026. [Google Scholar] [CrossRef]
  42. Figueira, J.R.; Greco, S.; Roy, B. ELECTRE Methods with Interaction between Criteria: An Extension of the Concordance Index. Eur. J. Oper. Res. 2009, 199, 478–495. [Google Scholar] [CrossRef] [Green Version]
  43. Roy, B.; Słowiński, R. Handling Effects of Reinforced Preference and Counter-Veto in Credibility of Outranking. Eur. J. Oper. Res. 2008, 188, 185–190. [Google Scholar] [CrossRef] [Green Version]
  44. Corrente, S.; Figueira, J.R.; Greco, S.; Słowiński, R. A Robust Ranking Method Extending ELECTRE III to Hierarchy of Interacting Criteria, Imprecise Weights and Stochastic Analysis. Omega 2017, 73, 1–17. [Google Scholar] [CrossRef] [Green Version]
  45. Fernández, E.; Figueira, J.R.; Navarro, J. Interval-Based Extensions of Two Outranking Methods for Multi-Criteria Ordinal Classification. Omega 2020, 95, 102065. [Google Scholar] [CrossRef]
  46. Fernández, E.; Figueira, J.R.; Navarro, J. An Interval Extension of the Outranking Approach and Its Application to Multiple-Criteria Ordinal Classification. Omega 2019, 84, 189–198. [Google Scholar] [CrossRef]
  47. Mousseau, V.; Dias, L. Valued Outranking Relations in ELECTRE Providing Manageable Disaggregation Procedures. Eur. J. Oper. Res. 2004, 156, 467–482. [Google Scholar] [CrossRef] [Green Version]
  48. van Rijsbergen, C. Information Retrieval: Theory and Practice. In Proceedings of the Joint IBM/University of Newcastle upon Tyne Seminar on Data Base Systems, Newcastle upon Tyne, UK, 4–7 September 1979; Volume 79. [Google Scholar]
  49. Sasaki, Y. The Truth of the F-Measure. Teach Tutor Mater 2007, 1, 1–5. [Google Scholar]
  50. Fernandez, E.; Navarro, J.; Solares, E.; Coello, C.C. Using Evolutionary Computation to Infer the Decision Maker’s Preference Model in Presence of Imperfect Knowledge: A Case Study in Portfolio Optimization. Swarm Evol. Comput. 2020, 54, 100648. [Google Scholar] [CrossRef]
  51. Hutter, F.; Hoos, H.H.; Leyton-Brown, K.; Stützle, T. ParamILS: An Automatic Algorithm Configuration Framework. J. Artif. Intell. Res. 2009, 36, 267–306. [Google Scholar] [CrossRef]
Figure 1. General scheme of the proposal.
Figure 1. General scheme of the proposal.
Axioms 12 00294 g001
Figure 2. Individuals used in the genetic algorithm to represent n B i n f .
Figure 2. Individuals used in the genetic algorithm to represent n B i n f .
Axioms 12 00294 g002
Figure 3. Individuals used in the genetic algorithm to represent n C i n f .
Figure 3. Individuals used in the genetic algorithm to represent n C i n f .
Axioms 12 00294 g003
Table 1. Configurations of the experiments.
Table 1. Configurations of the experiments.
Aspect of the Experiment’s ConfigurationNotationValues Used
Number of criteria N 3, 5, 7, 9
Number of classes M 2, 3, 4, 5
Number of assignment examples per class n c l a s s 2, 4, 6
Cardinality of the set of assignment examples c a r d T M n c l a s s
Number of out-of-sample actions n O S 400
Number of simulated decision-makers n D M s 20
Table 2. Out-of-sample effectiveness of both sorting methods using a reflexive relation S, considering a different number of criteria, classes, and reference actions per class.
Table 2. Out-of-sample effectiveness of both sorting methods using a reflexive relation S, considering a different number of criteria, classes, and reference actions per class.
NMnclassEffectiveness with Characteristic ProfilesStandard DeviationEffectiveness with Limiting ProfilesStandard Deviation
3220.93550.02610.85000.0559
40.94540.01970.86060.0653
60.94760.02170.88890.0495
3320.89400.04210.74570.0859
40.91760.03200.79690.0672
60.92490.03110.80840.0620
3420.88410.03770.69730.0921
40.88960.04150.74120.0744
60.91810.02920.75130.0729
3520.85650.04900.67140.0848
40.88200.03140.69630.0811
60.88940.03160.73610.0728
5220.94570.02020.83080.0602
40.94980.01790.82580.0659
60.95150.01550.83890.0566
5320.87480.03060.70780.0801
40.89090.02890.72230.0792
60.90190.02380.74770.0658
5420.84520.03380.63440.0716
40.85590.02800.66540.0700
60.86600.02900.68450.0708
5520.82190.03930.59710.0712
40.83580.03220.62880.0740
60.86140.03180.63620.0684
7220.95830.01730.83200.0650
40.95690.02590.82800.0596
60.95680.01450.84110.0576
7320.86560.03650.68740.0739
40.88980.02380.73000.0683
60.89050.02310.72440.0629
7420.84600.03220.63280.0697
40.85200.02600.61830.0803
60.85710.02480.64520.0934
7520.81830.02900.58000.0755
40.82820.02540.61860.0696
60.83560.02200.58460.0631
9220.96960.01830.85020.0649
40.97030.01400.85320.0633
60.96830.01550.83750.0640
9320.87440.02770.68500.0835
40.87740.02470.68520.0709
60.88680.02160.69100.0637
9420.84550.03090.61440.0764
40.85340.02440.63600.0661
60.85380.02660.66020.0787
9520.81620.02970.56900.0803
40.82300.02800.57660.0761
60.83160.02640.60330.0700
Table 3. Out-of-sample effectiveness of both types of methods regarding the number of criteria.
Table 3. Out-of-sample effectiveness of both types of methods regarding the number of criteria.
NEffectiveness with Characteristic ProfilesStandard DeviationEffectiveness with Limiting ProfilesStandard Deviation
30.90710.04340.77040.0992
50.88340.05170.71000.1071
70.87960.05570.69350.1166
90.88090.06020.68850.1226
Table 4. Out-of-sample effectiveness of both types of methods regarding the number of classes.
Table 4. Out-of-sample effectiveness of both types of methods regarding the number of classes.
MEffectiveness with Characteristic ProfilesStandard DeviationEffectiveness with Limiting ProfilesStandard Deviation
20.95460.02190.84480.0631
30.89070.03380.72760.0826
40.86390.03740.66510.0882
50.84170.03990.62480.0894
Table 5. Comparison between the effectiveness of the method based on representative profiles using the reflexive relation S and using the asymmetric relation P.
Table 5. Comparison between the effectiveness of the method based on representative profiles using the reflexive relation S and using the asymmetric relation P.
NMnclassEffectiveness Using SEffectiveness Using P
3240.93550.9398
3320.89400.9043
3340.91760.9124
3420.88410.8553
3460.91810.9029
5220.94570.9414
5520.82190.8025
5560.86140.8344
7240.95690.9617
7260.95680.9618
7340.88980.8852
7420.84600.8366
7520.81830.8104
7540.82820.8201
7560.83560.8303
9220.96960.9663
9240.97030.9651
9260.96830.9703
9320.87440.8698
9340.87740.8815
9440.85340.8447
9460.85380.8633
9540.82300.8144
9560.83160.8265
Table 6. Comparison of effectiveness using the reflexive relation S and using the asymmetric relation P in the context of the number of criteria.
Table 6. Comparison of effectiveness using the reflexive relation S and using the asymmetric relation P in the context of the number of criteria.
NEffectiveness Using SStandard DeviationEffectiveness Using PStandard Deviation
30.90710.04340.90420.0451
50.88340.05170.87810.0567
70.87960.05570.87750.0604
90.88090.06020.87860.0616
Table 7. Comparison of effectiveness using the reflexive relation S and using the asymmetric relation P in the context of the number of classes.
Table 7. Comparison of effectiveness using the reflexive relation S and using the asymmetric relation P in the context of the number of classes.
MEffectiveness Using SStandard DeviationEffectiveness Using PStandard Deviation
20.95460.02190.95470.0212
30.89070.03380.89060.0349
40.86390.03740.85900.0382
50.84170.03990.83410.0439
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Navarro, J.; Fernández, E.; Solares, E.; Flores, A.; Díaz, R. Learning the Parameters of ELECTRE-Based Primal-Dual Sorting Methods that Use Either Characteristic or Limiting Profiles. Axioms 2023, 12, 294. https://doi.org/10.3390/axioms12030294

AMA Style

Navarro J, Fernández E, Solares E, Flores A, Díaz R. Learning the Parameters of ELECTRE-Based Primal-Dual Sorting Methods that Use Either Characteristic or Limiting Profiles. Axioms. 2023; 12(3):294. https://doi.org/10.3390/axioms12030294

Chicago/Turabian Style

Navarro, Jorge, Eduardo Fernández, Efrain Solares, Abril Flores, and Raymundo Díaz. 2023. "Learning the Parameters of ELECTRE-Based Primal-Dual Sorting Methods that Use Either Characteristic or Limiting Profiles" Axioms 12, no. 3: 294. https://doi.org/10.3390/axioms12030294

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop