A Multiple Criteria Decision Making Approach to Designing Teaching Plans in Higher Education Institutions

Abstract

The involvement of competences in the teaching–learning planning process in Higher Education is essential for their success in the European Higher Education Area. This study presents a participatory multi-criteria model based on Voting Analytic Hierarchy Process (VAHP) analysis, focusing on the attainment of competences that permits consensus between lecturers and students in the design of teaching plans using two assessments: the assessment of competences by students and the lecturers’ assessment of the contribution of teaching strategies to the attainment of each competence. To validate the methodology, a survey was carried out to determine the preferences of 211 students on the competences of a quantitative subject in several business degrees and a survey of 11 lecturers to assess the contribution of the teaching strategies in the acquisition of each competence. The results show that practical lessons and group work should receive more importance in the teaching plans of the subject Financial Management according to the defined competences, the students’ preferences and the criterion of each lecturer for each teaching strategy. The results show the applicability of the participatory methodology proposed to formally agree on the design of teaching plans in higher education organizations between lecturers and students.

1. Introduction

Higher education organizations are involved in a changing environment, and their position in global contexts increasingly depends on their ability to adapt to the environment, together with the continuous improvement of teaching and research activities [1]. In this context, external stakeholders require transparency and accountability and the main challenge for higher education institutions lies in rethinking their role in the higher education landscape to consider and improve their performance profiles, i.e., how good higher education institutions are at performing their activities [2].

The modern concept of the university as a “stakeholder university” vs. “republic of scholars” implies the need to adapt to new government policies that address how higher education institutions should be run and organized [3]. On the other hand, the orientation towards human capital formation, that is, professionals from an innovative perspective and oriented to the current labor market is a reality that cannot be overlooked [4]. This requires paying attention to three issues: planning, the teaching–learning process and learning outcomes.

Due to increased global competitiveness and increased expectation and challenges, higher education organizations need to carefully define their institutional profiles [2]. Using this approach, strategic plans and governance processes can support branding policies to define the institutional profile from a management approach [3]. Both topics could respond perfectly to two key questions regarding management: how and when.

However, the essence of universities lies in the quality of these three issues: planning, the teaching–learning process and learning outcomes. Thus, to define the profile of the higher education organizations completely, it is necessary to add tools able to respond to a third question related to education planning: why. This question is directly related to the goals of the teaching–learning process. At present, in some European universities, for example in Spain, educational plans are defined principally by the lecturers, and students are present in the Curriculum Commissions in a very minor way. Student participation is defined by each university [5]. Therefore, occasionally the preferences of the main stakeholder involved in the teaching–learning process, the students, are not considered. However, to ensure the quality of the teaching–learning process it is necessary for the goals of the lecturers and students to be aligned [6,7,8]. In this work quality is understood to be “the result of the interaction between lecturers, students, and the learning environment in which program content, learning opportunities, and resources are tailored to their purposes” [9].

Concern about including the participation of students in higher education governance, has increased in the last decade [10]. Although some authors remark that there is a lack of a rigorous definition of what quality means in higher education and that quality does not normally consider the preferences of the students [11]. Furthermore, some studies show the utility of students’ preferences to evaluate services in order to improve their quality and the overall perception of lecturers by the students. In fact, Ref. [7] highlight the importance of higher education institutions to “have a deep understanding of shared responsibility between participants and further emphasize the importance of consumer responsibility to play an active role in shaping the value of their experiences”.

Moreover, there are some studies that have focused on connecting lecturers and students in order to discuss the most effective teaching and learning methods [11,12]. Specifically, Ref. [11] stress the relevance of flexibility in teaching plans and the participation of students to ensure the quality of these plans. The goal in the long term is to offer support to the higher education institutions in order to build participative teaching and learning plans without losing rigor, effectiveness and efficiency. This participative approach permits the closing of the gap between the goals and teaching methodology, and the expectations of the students [11].

In spite of this, and the fact that the legislation [5] provides for this participation, the reality is that usually students have a low level of involvement as co-decision makers for education planning. This can result in divergences between the teaching objectives and the expectations of the students, decreasing the motivation of the students and reducing the quality of the learning process [13,14].

In the complex context of higher education, competence-based teaching has been implanted successfully in a large number of universities. In fact, “the majority of the institutions designed their education through a mix of traditional and competence-based teaching and assessment methods” [15]. A consolidated example of the strength of competence-based-learning is the European Higher Education Area (EHEA). The EHEA is a dynamic, complex, student-focused system that replaces the traditional learning system by focusing on linear learning using a multi-approach learning system based on an interdisciplinary context [16,17]. With this new approach, it is necessary to include the preferences of all the stakeholders to ensure the efficiency of the teaching policies [18]. In this context, the student assumes a very important role in the design and assessment of the teaching plans [19]. In this line, the references of quality of the European model in higher education recommend prioritizing the development of the methodologies for assessment that include the participation and autonomy of the students in the teaching process [20]. Therefore, it is necessary to highlight the concern of European universities about innovation. Specifically in [21], issues related to the future of the EHEA were discussed, and the relevance of “supporting higher education institutions in their efforts to start or maintain a structured dialogue on innovation and enhancement of learning and teaching, involving students, lecturers and also relevant external stakeholders” was stressed emphasizing the orientation towards learning outcomes and the consideration of skills to address the current and future challenges of society [21].

The students themselves could be the best referees to assess the achievement of the competences and it seems essential to include their perceptions relating to the goals for the teaching–learning process defined by teaching plans [22,23,24,25]. The loss of communication between students and other stakeholders linked to the education system, such as lecturers but also the institutions themselves, can create gaps that decrease the accuracy of defining competence goals. A good way to compare teaching units to the study programs of higher education institutions is to analyze the results in terms of the achievement of the competences. Therefore, it is necessary to define the results of learning correctly and establish criteria related to the content and global competences of each subject [26]. Moreover, it is important not to lose sight of the goal of preparing students for impact-focused careers [27]. However, this is only possible when an optimal combination of the competences for each subject and level occurs [28] together with a continuous assessment of teaching strategies to ensure the attainment of the academic goals by the students [29].

The main goal of this paper is to provide a Multiple Criteria Decision-Making (MCDM) methodology that permits the avoidance of divergences between the expectations of the students and both the competence/goal focus, related to some subjects, and the teaching strategies used by the lecturers to reach these goals.

2. Literature Review

The aggregation of individual preferences is a complex process, and problems frequently appear in relation to consistency with the consensus of group assessment and with social legitimacy problems related to ethical topics. From a rational approach, considering the different preferences and interests of the different individuals in society, the huge problem of social choice is based on how to relate social preferences and joint decisions to individual values and interests [30]. On one hand, this could be a problem associated with the manipulability of the social decision function. On the other hand, when decision-making problems need to consider several alternatives from different criteria, the complexity of the aggregation process increases, and it is essential to include the individual valuations with rigor [31]. In this context, a multi-criteria analysis can provide rigor to participative decision-making models when it is essential to ensure minimum objectivity, representativity and transparency in a global and democratic context [32,33].

In recent years, the growing relevance of participative processes related to teaching policies in higher education has driven the development of new methodologies pinpointed by the optimization of group consensus [18]. Specifically, in higher education, MCDM has been efficient in obtaining joint assessments in this context [22]. In fact, these techniques have worked well in the planning and assessment of complex projects and have been used to design public policies when it is necessary to obtain a global assessment from multiple individuals and a high number of stakeholders [34].

Some reviews present the frequent use of Analytic Hierarchy Process (AHP) in the area of education [35]. AHP considers individual preferences and assessments of value judgements about the relative importance of the analyzed criteria constructing pairwise matrices [35]. The combined use of the AHP with other MCDM techniques such as Goal Programming (GP), Fuzzy sets (FST) or the Analytic Network Process (ANP) is normal. GP is a linear programming technique that helps to solve complex problems through an optimization process that looks for compromise solutions that may not fully satisfy all the goals but do reach certain satisfaction levels set by the decision-maker [36]. Fuzzy logic was originally developed by [37] and permits representing vague knowledge to model decision-making problems [38]. The ANP [39] applies a similar logic to the AHP method but using network relations instead of pairwise relationships.

Ref. [40] carried out a more in-depth review in which they analyzed more than 42 papers on the application of multi-criteria methods to assess the efficiency of e-learning showing that the most used methods are AHP and AHP combined with other methods, usually fuzzy methods.

So far, AHP has not been applied to assess classroom higher education, but some studies have applied it to assess user satisfaction in online courses. Ref. [41] considers four main dimensions to assess the support system in an e-learning context: Learner Interface, Learning Community, System Content and Personalization. Ref. [42] assesses the applicability of an e-learning system in a computer course, combining AHP with another multi-criteria method: Goal Programming. Ref. [43,44] also proposes an evaluation model based on AHP and Fuzzy methods. On the other hand, Ref. [21] uses AHP combined with Analytic Network Process (ANP) in higher education to analyze the preferences of the stakeholders and they present a case study, with the prioritization of five projects, about the use of the resources in universities.

The use of voting methods in education has been studied from different points of view. Some studies propose digital methods as a way to obtain student votes in a classroom [45]. Ref. [46] analyzes the assessment by the lecturer and the students of different alternatives in the content of a subject. On the other hand, some research has shown the utility of combined analysis based on voting and multi-criteria methods [47,48,49].

3. Methods

3.1. Description of the Method and Justification

The proposed methodology focuses on the acquisition of competences and provides a consensus between lecturers and students in the design of the teaching plans as a result of a double assessment: the assessment of the teaching strategies and the assessment of the objectives in terms of the achievement of competences.

This methodology uses a mixed multi-criteria analysis: the voting analytic hierarchy process (VAHP). Multi-criteria analysis provides rigor to the participative decision-making processes [50,51]. On the other hand, voting methods allow aggregation in a simple way, regarding individual preferences in only one global assessment [52].

AHP deals with subjective preferences and allows the researcher to quantify trade-offs using a paired comparison between criteria [53], and to prioritize different alternatives. Therefore, AHP obtains the individual preferences and assessments through opinions about the relative importance of the paired criteria and alternatives. In the educational context, AHP has been used frequently [36], although it presents some limitations. Firstly, when it uses subjective valuations it is necessary to validate the consistency of the primary results. In addition, when the number of criteria is high, the survey becomes too complex and the process can become tedious and difficult as it results in a very high number of paired comparisons to evaluate.

The combination of the voting methods with hard multi-criteria techniques like the AHP method can make it easier to obtain consistent results when the participants are not experts, as in the case of students. Additionally, it allows the researcher to deal with a higher number of criteria without losing the operativity of the process [54].

The students are the main stakeholders involved in the teaching–learning process, although considering the preferences of the students is not common when designing teaching plans. Nevertheless, lecturers should use teaching strategies and other tools in the classroom with one objective: the acquisition of skills as defined in the teaching plan of any given subject. This objective is related to the global and transversal competences and the specific competences, but it is not usual to establish priorities between them. On the other hand, students have perceptions about the importance of these competences but as of yet a formal model that obtains their preferences does not exist. In addition, the teaching strategies used by lecturers might not meet the expectations of the students in this sense.

The proposed model is capable of identifying preferences about the relevance of particular competences, related to one of the student’s subjects, using a voting method. It simultaneously permits the addition of criteria from the lecturers about the contribution of each resource in relation to each competence. This provides the lecturers with more flexibility to fit the resources to the expectations of the students in relation to the learning process of the subject, the acquisition of the key competences and should help improve motivation.

The methodology is based on two assessments: (1) the students assess the competences related to a subject and (2) the lecturers assess the contribution of the teaching strategies available to help students obtain these competences.

The model Voting Analytic Hierarchy Process (VAHP) proposes using a voting method based on the Borda count combined with AHP to aggregate the priorities of the students with the global and specific competences related to a subject or grade in a simple way. The Borda count aggregation method can be explained as follows: by giving each candidate or alternative, a number of points corresponding to the number of candidates ranked lower. The candidate that obtains the most points is the Borda winner [55].

The hierarchy of the model is defined on two levels. The first level shows the global objective: to identify the best teaching strategies needed to obtain the competences as defined in a subject teaching plan. The second level is formed by the dimensions or competences that need to be assessed. Therefore, to achieve the global objective, this model is defined in three steps:

I. Assessment 1: this is based on an assessment by the students about the relevance of the competences related to a subject. The model allows the analyst to obtain normalized weights that represent the most preferred competences by the students.

II. Assessment 2: this is based on an assessment by the lecturers of the contribution of each teaching resource to achieve one competence related to a subject. The model allows the analyst to obtain normalized weights that represent the most relevant teaching strategies for each competence.

III. Final result: this is the result after the aggregation of assessment 1 and assessment 2 and represents a final score for each resource related to their contribution towards the acquisition of the most preferred competences by the students.

Figure 1 shows a visual description of the process.

3.2. Assessment 1: Assessment of the Competences by the Students

To obtain information about the preferences of the students in relation to one particular subject, we presented a survey to each student that aimed to determine their opinion about the relevance of each competence. The objectives of the study and the criteria were explained to the students. The survey provided the individual with the opportunity to vote and these results were dealt with using a Borda count. As a result of the participative process, a conjoint ranking for the individual preferences about the competences was obtained.

Voting methods provide different options to choose the most preferred alternative from a group. However, it is not possible to get a measure of the robustness of each option, i.e., it is not possible to obtain the weight of preference for each of the ranked items.

This model provides a ranking of the competences based on the relevance of the related criteria. The ranking and weights are calculated as follows:

• Criteria Rank: the students assign marks to each competence using a number from 1 to V (V ≤ C), where C is the number of competences or criteria in order of relevance.
• Conjoint Result: the calculation of the weight is based on [36]. For the C criteria and competences, n_s students and $V_s$ votes, the largest score ${\eta }_{cc}$ (for the criteria c when you get a score $vs$th in the preferences) is defined as a result of the maximum of the sum of the votes for the $c$th criteria.

  $${\eta }_{cc}=máx{\displaystyle \sum _{vs=1}^{Vs}}{u}_{cvs}{x}_{cvs}$$

  (1)

  where x_cvs is the total number of the votes for the criteria c with a vote $v_s$ of the $n_S$ students, $c=\left\{1,2,\dots ,C\right\}$, $v_s$ = $\left\{1,2,\dots ,C\right\},$

  $$\begin{gathered} s.t. \\ {\eta }_{cp}={\displaystyle \sum _{vs=1}^{Vs}}{u}_{cvs}{x}_{pvs}\le 1,p=\left\{1,2,\dots .,C\right\} \end{gathered}$$

  (2)

  $$u_{cvs}>0,u_{cvs}-u_{cvs}+1\ge d\left(v_s,\delta \right)=\delta >0,i.e.,u_{c1}>u_{c2}>\cdots >u_{cv}>\cdots \ge \delta$$

  (3)

  $$u_{c1}\ge 2u_{c2}\ge \cdots >V_s{u}_{cvs} \mathrm{and} u_{cvs}\ge \delta$$

  (4)

  where

  $$\delta =\frac{2}{V_s\left(V_s+1\right)n_s}$$

  where constraint (3) refers to a weak order, constraint (4) refers to a strong order and constraint (2) is necessary to obtain normalized weights.

The Equation (1) is applied in the majority of the voting methods based on score rules. Specificaly, the desirability of the criteria c defined in (1), where $x{c}_{vs}$ is the number of vsth places that the criteria c obtains from the students. This is applied when:

• Rule of priority: the weights $u_{cvs}$ obtain the value $u_{c1}=1,u_{cvs}=0,\forall v_s\ge 2$, i.e., only the first option of each participant is considered $v_s$ = 1.
• Borda count: each participant ranks their preferences of the C criteria and competences, assigning $n_s$ votes to the preferred candidate, $n_s-1$ to the second choice and, finally 1 to the least preferred candidate.

3.3. Assessment 2: Assessment of the Contribution of Teaching Strategies for Each Competence by the Lecturers

The assessments of the lecturers are obtained through a survey that includes some questions about the contribution of the teaching strategies in the acquisition of the competences.

The application of Equations (1)–(4) in r teaching strategies and t lecturers, provides a ranking of the most competent teaching strategies for any one competence. The ranking and the weights are calculated in the same way as assessment 1.

3.4. Final Results: Provide a Rank That Highlights the Best Fitting Teaching Strategies Capable of Obtaining the Competences Most Preferred by the Students

Finally, to aggregate the assessments of the lecturers and the preferences regarding the competences of the students, this model uses a linear aggregation method (Equation (5)).

$${\eta }_{rc}={\displaystyle {\sum }_c^C}{\eta }_{cc,c} {\eta }_{rr,c}$$

(5)

where ${\eta }_{rc}$ is the final score for each resource related to their contribution to the acquisition of the C competences analyzed, ${\eta }_{cc}$ is the weight for the competence c obtained in assessment 2 and represents the largest score for each competence voted on by the students, and ${\eta }_{rr}$ is the weight for the resource r obtained in assessment 2 and represents the largest score of the contribution of each resource about each competence voted on by the lecturers.

Linear aggregation methods are useful when all individual criteria have the same measurement unit and the analyst wants compensability between them [56,57]. Therefore, this method allows us to obtain a ranking of the most important teaching strategies related to a subject, based on the importance, in the opinion of the students, of each competence. This importance is quantified by the weights obtained in assessment 1. Consequently, the conjoint result merges the know-how of the lecturers with the expectations of the students in relation to one particular subject and provides a guide to designing participative teaching plans and the improvement of existing plans.

4. Case Study

The model was validated on the basis of two different samples. Sample 1 was formed by 211 students from the University Rey Juan Carlos. The students were divided into four groups from different subjects: Business Administration and Management (BAM); Law and BAM; Tourism and BAM; and Advertising and Public Relations (APR) and BAM, in the third and fourth years (

Table 1. Degree and course of the students.

Degree	Participants	Stage
BAM	76	Third Year
Law and BAM	43	Fourth Year
Tourism and BAM	50	Fourth Year
APR and BMA	36	Fourth Year
Others	6	-

). The ages were between 20 and 38, they started university between 2002 and 2017, and the first year of the degree was between 2010 and 2017. 61.61% of the participants were women and 38.39% men. This survey was carried out through the platform Google Forms in classroom.

Sample 2 was formed by 11 lecturers of the subject Financial Management at the same university. This second survey was carried out in a lecturers’ meeting.

These two questionnaires can be found as Supplementary Material Q1 and Q2.

We partially followed the University Students’ Expectations of Teaching (USET) survey [12] and the Socrates funded CEESOC project (Comparison of Expectations of European Students on ODL Courses) to develop two surveys that allowed us to obtain the preferences of students regarding the most important competences for them and information about the combination of teaching strategies that best fit these expectations. Survey 1 allowed us to obtain the relevance of 9 competences (

Table 2. Generic and subject specific competences (based on [26]).

Competences	Type	Description
GC1—Capacity for analysis and synthesis	Generic	Identify and describe the elements of a reality and organize the most significant information based on pre-defined criteria.
GC2—Information management skills	Generic	Ability to retrieve and analyze information from different sources and use it efficiently to solve a problem, make a decision and be able to communicate it
GC3—Decision making	Generic	Analyze a problem and choose between different alternatives to optimize the expected result as a result of the action
GC4—Capacity to learn	Generic	Assimilation of new information and being able to apply it efficiently
GC5—Ability to work autonomously	Generic	Work autonomously without support
GC6—Capacity to apply knowledge practically	Generic	Contextualize and adapt theoretical contents in real situations
GC7—Problem solving	Generic	Identify difficulties in the development of a project and choose an alternative way to solve it
SC1—Comprehension of financial operations in the business environment	Specific	Select the best information to understand the financial operations in a business framework
SC2—Problem solving regarding financial valuation	Specific	Learning, understanding and applying knowledge and contributing towards intellectual autonomy in a framework of financial valuation.

) using a score from 1–10. The survey was presented to sample 1, formed by students of the subject Financial Management. Survey 2 allowed us to obtain opinions about the contribution of five teaching strategies for each of the nine competences analyzed. Survey 2 was presented to sample 2, consisting of lecturers.

The criteria in the survey were: 7 generic competences and 2 subject specific competences (Table 2) related for the subject Financial Management. A description of each is shown in the Grade Memory in Business Administration and Management.

The criteria valuated by the lecturers were: theoretical lessons, practical lessons, group activities, group presentations and related activities (

Table 3. Teaching strategies available for lecturers.

Teaching Strategies	Description
R1—Theoretical lessons	Traditional classes to explain theoretical contents
R2—Practical lessons	Traditional classes to solve problems, practical exercises and case studies.
R3—Group activities	Group homework or classroom based group work
R4—Group presentations	Presentations of the projects to a classroom audience
R5—Activities related to the subject	Conferences, seminars.

).

The model was applied in the case study as follows; firstly, Equation (1) was applied considering constraints (2), (3) and (4), where $x_{cvs}$ is the total number of votes for competence c with a vote $v_s$ from the $211$ students, therefore, $c=\left\{1,2,\dots ,9\right\}$, $v_s$ = $\left\{1,2,\dots ,9\right\}.$ Secondly, we reapplied the model with $x_{rvt}$ number of votes for resource r with a vote $v_t$ from the $11$ lecturers, therefore $r=\left\{1,2,\dots ,5\right\}$, $v_t$ = $\left\{1,2,\dots ,5\right\}.$ We obtained the final score using Equation (5) where ${\eta }_{cc}$ is the weight for each competence obtained in assessment 2 and represents the largest score of the 9 competences valuated by the 211 students, and ${\eta }_{rr}$ is the weight for each resource obtained in assessment 2 and represents the largest score of the 5 strategies valuated by the 11 lecturers.

5. Results and Discussion

The results of the surveys of the analyzed subject are described as follows.

Table 4. Result of the voting process for the students in relation to the importance of the competences.

Competences	1º	2º	3º	4º	5º	6º	7º	8º	9º	Total
SC1	33	30	36	13	16	17	15	31	20	211
SC2	28	35	22	26	20	19	25	20	16	211
GC1	1	10	20	22	18	32	32	30	46	211
GC2	4	6	10	24	37	34	34	36	26	211
GC3	23	17	33	30	46	26	19	9	8	211
GC4	25	20	18	25	25	33	26	24	15	211
GC5	21	18	26	16	18	19	30	22	41	211
GC6	50	37	21	26	15	13	11	21	17	211
GC7	26	38	25	29	16	18	19	18	22	211
Total	211	211	211	211	211	211	211	211	211

presents the results of the voting process of the 211 students in relation to the relevance of each competence analyzed.

These results, applying Equation (1), present a ranking of the preferences and their normalized weights: GC6(0.157), SC1(0.130), SC2(0.125), GC7(0.125), GC3(0.116), GC4(0.111), GC5(0.101), GC2(0.068) and GC1(0.066). Therefore, the best valuated competences were capacity to apply knowledge practically (GC6), comprehension of financial operations in the business environment (SC1) and problem solving regarding financial valuation (SC2) and problem solving (GC7). The worst valuated competences were: capacity for analysis and synthesis (GC1), information management skills (GC2) and ability to work autonomously (GC5).

However, some studies obtained divergent results regarding the perception of the acquisition of some of these competences. For example, Ref. [58] collected the perceptions of 2556 students in ten Spanish public universities about their level of competence in ten competences, and found that, for students in this same branch of knowledge, the competence Autonomous Learning was among the three lowest rated competences, that is, with a poorer level of acquisition. However, in this study, the students scored it as one of the three least important competences. This suggests the need to incorporate lecturers’ knowledge to ensure that gaps do not occur in the acquisition of certain skills.

To identify which teaching strategies fit better with the preferred competences as stated by the students, we asked the lecturers about the contribution of the strategies for the acquisition of the competences considered. The results of the assessments in relation to the teaching strategies are shown in Supplementary material (Tables S1–S5).

Finally,

Table 5. Result of the votes for the teaching strategies. The figures in brackets show the normalized weight of each teaching strategy.

Students	Lecturers (Teaching Strategies)
Competences	R1	R2	R3	R4	R5
SC1 [4 (0.102)]	1 (0.264)	3 (0.146)	9 (0.041)	7 (0.045)	4 (0.101)
SC2 [8 (0.087)]	2 (0.138)	1 (0.276)	8 (0.067)	9 (0.041)	8 (0.086)
GC1 [5 (0.092)]	5 (0.100)	6 (0.064)	7 (0.082)	1 (0.243)	5 (0.091)
GC2 [7 (0.087)]	7 (0.073)	7 (0.060)	2 (0.157)	2 (0.211)	7 (0.088)
GC3 [9 (0.040)]	6 (0.083)	8 (0.058)	3 (0.143)	3 (0.133)	9 (0.055)
GC4 [3 (0.110)]	3 (0.123)	5 (0.067)	6 (0.100)	6 (0.086)	3 (0.109)
GC5 [2 (0.171)]	4 (0.111)	9 (0.048)	5 (0.115)	4 (0.108)	2 (0.167)
GC6 [1 (0.222)]	8 (0.062)	2 (0.171)	1 (0.169)	5 (0.091)	1 (0.216)
GC7 [6 (0.089)]	9 (0.046)	4 (0.109)	4 (0.125)	8 (0.042)	7 (0.088)
Final score	0.108174	0.115573	0.116242	0.106058	0.132343

summarizes the results of the votes of the 11 lecturers regarding teaching strategies (columns 2–6) and the valuations of the students for each competence (column 1) and shows the final score. Columns 2–6 present the normalized weights that represent the contribution of each teaching resource to the acquisition of each competence. The first column shows normalized weights in relation to the preferences of the students to each competence. The last row shows the final result obtained after applying Equation (5).

Applying Equation (5) we obtained the conjoined results, which consider the assessments of the students and lecturers. As a result of the aggregation of the student preferences in relation to the competences and the assessment of the lecturers in relation to the contribution of each resource of these competences, we obtained the final weights for each teaching resource: R5 (0.132343), R3 (0.116242), R2 (0.115573), R1 (0.108174) and R4 (0.106058). With the normalized weights, the final results about the distribution of the teaching strategies to cover the competences of the subject Financial Management were: activities related to the subject (0.229), group activities (0.201), practical lessons (0.2), theoretical lessons (0.187), and group presentations (0.183), respectively.

On the one hand, the results show a strong preference from the students for the competences linked to the most applied approach of the subject analyzed. As a result, the highest valuations refer to the competences: capacity to apply practical knowledge, comprehension of financial operations in the business environment and problem solving regarding financial valuation.

When the assessment of the teaching strategies of the lecturers were aggregated with the preferences of the students about the competences considered, the results recommended an increase in the efforts to develop activities, group work and practical classes. This is aligned with the quantitative approach of this subject and is probably aligned with the needs of the students regarding the intensity of the empirical work and the analytical explanations by the lecturers. These strategies are strongly related to applied classroom work. However, the teaching strategies that are linked to theoretical content seem less important for the students to achieve the competences of the subject Financial Management.

An important role of practical work is to help students develop links between observations and ideas. However, these ideas must be introduced [59]. Similar results regarding the perceived usefulness by students of quantitative subject matter strategies associated with exercise resolution have been found in other studies. Ref. [60] collected the perceptions of 55 first-year students of a semi-quantitative subject about the contribution of nine teaching strategies and found that students gave a higher score (84%) to the degree of usefulness of the strategies associated with numerical exercise resolution and problems for the achievement of the general competences compared to other strategies. This greater practical emphasis was also found by [61] for the development of generic skills by undergraduates.

Although the case study only considers the competences defined in the teaching plan of the subject Financial Management and the teaching strategies used by the lecturers of this subject, the model is flexible enough to add, remove or change other competences related to other subjects or related to the same subject but not considered until now.

Through the decision process it is possible to identify the most appropriate tools for the achievement of the most relevant skills and a flexible and participative assessment method that connects the preferences of the lecturers and the students. This permits the alignment of the teaching–learning goals and the teaching methods with the preferences of the students and the most relevant competences.

Likewise, this methodology is applicable when analyzing the suitability of new teaching strategies to apply in the teaching–learning process, to assess their utility and usability and quantify their contribution to the achievement of the competences related to a subject.

It would be interesting to complete this study with individual and group feedback from the students in order to validate their agreement with regard to the conjoint results. When identifying conflicts, it is recommendable to quantify them and develop some type of negotiation process to minimize the conflicts between lecturers and students to achieve a consensus.

The main limitation found in the case study is related to the continuity of the study with the same participants to define control mechanisms in order to validate the agreement between the individual assessment of the students and the conjoint results. This could be a problem if it were necessary to develop future negotiation processes.

However, this is the first study that proposes the participation of students in relation to the relevance of competences and the assessment of lecturers in a structured model and we tested the utility of the multi-criteria analysis for this proposal. Furthermore, it would be interesting to continue in this direction in order to improve the applicability of other mixed multi-criteria methods, to increase the operativity of the assessment system and to design mechanisms to pinpoint conflicts and develop negotiation processes between lecturers and students to ensure consensus.

6. Conclusions

The proposed model provides a formal methodology to aggregate the preferences of a group of students and lecturers about the competences related to a subject and the available teaching strategies. As a result, the methodology allows the analyst to obtain a combined student–lecturer opinion related to the use of each teaching resource for any one subject in order to achieve the necessary competences. Thereby, lecturers obtain useful information in order to lead the teaching–learning process that agrees with the expectations of the students and improves their motivation about any given subject.

In addition, this model not only allows us to obtain a ranking of the strategies that fit best with the expectations of the students, but it can also be useful to assess new “candidate-competences” or to analyze the elimination of others. Furthermore, it provides a methodology to design participative teaching plans based on competence-based learning, which treats the students as key stakeholders and participants in the teaching–learning process.

The case study allowed us to identify the most preferred competences of a group of students in relation to the subject Financial Management. The three competences most valued by the students were: capacity to apply knowledge practically, comprehension of financial operations in the business environment and problem solving regarding financial valuation. Additionally, it identified the best fitting teaching strategies: activities related to the subject, group activities and practical lessons.

The proposed methodology provides information about the best fitting teaching strategies for the teaching–learning process of any subject by aggregating the assessment of students and lecturers and permitting the identification of conflicts between them. This information could be used by management in higher education institutions to improve the quality of teaching plans. In addition, the model can be applied to different subjects and levels providing a formal method to obtain comparable information between institutions and improve quality in higher education organizations.

7. Limitations and Recommendations

The main limitations found in the realization of this study were associated with data collection. Conducting student surveys is not without its difficulties. On the one hand, one must find the moment when students are most receptive to answering the survey. On the other hand, monitoring is extremely difficult, since groups of students change from one course to another. From a methodological perspective, the number of competencies and teaching strategies to be evaluated should not be excessively large. Although the proposed model is more flexible than the traditional AHP, if the number of criteria to be analyzed is very large, the evaluations can be excessively complex for non-expert participants.

Based on the results obtained that support previous studies, such as [62,63], a greater effective incorporation of students in the design of teaching learning process planning seems recommendable. However, to ensure the quality of the process it would also be advisable to integrate the knowledge of experts, in this case, lecturers and other key stakeholders. These processes should be designed following the ethical principles agreed in 2015 by the Academy of Social Sciences [64] to ensure that the integration of social agents is carried out in an integral, fair and responsible way, respecting the privacy, autonomy, diversity and values of individuals and communities.

Finally, reviewing and updating both the competences and the teaching strategies on a continuous basis is recommended, with the aim of aligning the teaching plans with innovations in the field of Higher Education. In addition, it is necessary to review the degree of agreement of the students with the teaching plans on a regular basis in order to detect possible negative effects such as the lack of motivation of some students.

This updating work should be transferred to the regulatory field. During the development of this work, it was found that the competences identified in the official documents of the teaching plans for the subjects should be reviewed according to the objectives of the subject and aimed at training students, not only from an academic perspective, but also from a labor perspective [25]. The university must be prepared to provide a quality, multidisciplinary education that forms professionals based on excellence, with broad knowledge, but also as citizens of future society.