Using Competency Maps for Embedding and Assessing Sustainability in Engineering Degrees

Abstract

This paper features a methodology for embedding and assessing a competency in an academic curriculum using competency maps. This methodology enables embedding and assessment of any competency in any curriculum, regardless of the educational level, as long as the competency is correctly described by means of a competency map. As an example of the application of this methodology, a proposal for embedding and assessing sustainability in engineering degrees is presented. A competency map embodies the set of learning outcomes of the competency that students should have acquired upon completion of their studies. This information allows the designers of the curriculum to determine the learning outcomes that should be developed in the degree and to distribute them appropriately among the subjects. The presence map can be constructed from the competency map. It contains information regarding the extent to which each learning outcome of the competency map is being developed in the degree. This paper proposes the construction of a presence map in two steps: (1) perform a survey and (2) conduct a semi-structured interview with professors. The interview, which is conducted by one or several experts in the competency, allows the different criteria used by the professors when filling out the questionnaire to be unified, whereas the presence map shows whether a particular competency is correctly embedded in the curriculum and the aspects that could be improved. Finally, to validate that the students are achieving the learning outcomes of the competency map, we propose a survey to measure the students’ perception about their own learning in the competency. These results can be compared with the presence map to help determine whether, from the students’ point of view, the expected learning outcomes are being achieved in the corresponding subjects. The aim of this process is to provide the information necessary to indicate any changes in the curriculum that may improve the embedding of the competency.

1. Introduction

A quality university education must ensure that students acquire the skills related to their degree [1,2,3]. The concept of competency can be understood as the combination of knowledge, skills and attitudes appropriate to the context [4]. At the end of the 20th century, a revolution took place in the educational methodologies employed to achieve this goal [5,6] fundamentally due to the use of new technologies. At the beginning of the 21st century, changes were made in the teaching–learning process in all educational institutions throughout the world. Some of these institutions led the process [7] by integrating in the curriculum not only the technical competencies but also the professional competencies that, in the second half of the 20th century, had waned in most of the world’s universities, giving way to technical competencies [8]. This was because some educators argued that “the skills-based approach was pedagogically dangerous because it encourages bad teaching” [9]. However, other authors maintained that if a curriculum is focused on solving current problems with current technologies, this program quickly becomes outdated, especially in engineering degrees [10]. In the same way, Hu [11] stated that it is more important for a curriculum to integrate knowledge and competencies for their application in new situations rather than teaching vanguard technology, as it may become obsolete after a few years.

Professional competencies refer to the transversal skills and experiences applicable to a wide variety of jobs, situations and tasks in most areas of knowledge [12]. Some examples of professional competencies include teamwork, communication, interpersonal skills, critical thinking, ethics, social commitment and sustainability. Different authors have advocated the need to introduce these competencies in all educational curricula. Taraman and Tovar [13] claimed that success or failure in the educational process should be measured according to the set of competencies students should acquire. Newrock and Tovar [14] interviewed industry leaders in order to identify the requirements of employers regarding professional competencies. They concluded that graduates require much more than excellent technical knowledge; they also need to be able to communicate their ideas, understand the meaning of corporate ethics and personal ethics, develop social skills and manners and learn how to respect their peers.

Considering all of the above, different initiatives have contributed to defining the training of engineering students in different countries around the world. Since the beginning of the 21st century, an increasing number of universities have incorporated professional competencies in the context of the European Higher Education Area [8]. Accreditation systems have defined the professional competencies in engineering education curricula in the USA [15], Canada [16] and Europe [17]. As an example of a project aimed at fostering personal and interpersonal skills in students, we highlight the CDIO project (conceive, design, implement and operate) [1,18] promoted by the Department of Aeronautics and Astronautics at MIT, with extension to many other universities around the world. Initially aimed at Europe and later extended to other continents, the Tuning Project is led by the University of Deusto in the Basque Country, with the aim of standardizing engineering curricula in terms of professional competencies. The Tuning Project defines 30 professional competencies classified into three categories (instrumental, interpersonal and systemic). Together with the University of Groningen, it currently coordinates the Tuning Academy (http://tuningacademy.org/ (accessed on 19 July 2022)) and has projects in more than 58 countries in America, Africa, Asia and Europe [2].

Competency-based education continues to pose a challenge [19]. Each professional competency can be deployed throughout the curriculum by means of different strategies above beyond their occasional introduction in certain subjects. The development of a competency must be included in a planned way in different subjects of the curriculum, thereby forming a competency itinerary. Coordination between the subjects is essential to establish an itinerary for each competency to which the different subjects contribute in accordance with their characteristics, such as the level they occupy in the degree, typology (mandatory or elective), degree of specialization, etc. In this way, the subjects of the competency itinerary cover the learning outcomes students are expected to acquire according to different approaches and domain levels throughout the entire curriculum.

Figure 1 provides an example of the itinerary for a given competency. The boxes correspond to all the subjects of a bachelor’s curriculum structured in eight semesters (five subjects per semester). The red squares represent the subjects belonging to the competency pathway, namely the subjects that contribute to the development of this competency. Students reach the different domain levels of the competency as they progress through the curriculum. To ensure that students acquire the competency, planning must be meticulous, and the subjects must be well coordinated.

Competency maps [8] are an essential tool for facilitating this coordination and for distributing the learning outcomes related to the competency among the different subjects of the itinerary. A competency map defines a competency in terms of several competency units, which may be regarded as the different issues to be developed in the competency. Each competency unit is precisely defined by means of learning outcomes, which must be developed in the subjects of the competency itinerary. The learning outcomes are drawn up using a learning taxonomy. Figure 2 shows the generic format of a competency map containing n competency units and using a three-level learning taxonomy, referred to as level 1, level 2 and level 3.

Competency maps are a tool that can help curriculum designers to satisfy the principles of constructive alignment [20,21], as by making the learning outcomes of the competency explicit, the definition of learning objectives in the subjects is more clearly defined, and their subsequent assessment more accurate.

The competency map may contain information regarding in which subject of the competency itinerary the different learning outcomes should be developed, as shown in Figure 3. The competency in this figure is defined using eight competency units, and the competency itinerary consists of five subjects (S1...S5). As an example, the learning outcomes assigned to subject S1 are shown in red. For simplicity, in this figure, all the learning outcomes from a cell of the map are assigned to the same subject (or to a set of subjects, such as level 2 of competency unit 2, which is assigned to subjects S1 and S3). However, it is possible to assign learning outcomes from the same cell to different subjects.

This paper features a methodology for embedding and assessing a competency in an academic curriculum using competency maps. This methodology enables embedding and assessment of any competency in any curriculum, regardless of its educational level, as long as the competency is correctly described by means of a competency map. As an example of the application of this methodology, a proposal for embedding and assessing sustainability in an engineering degree is presented.

In general, engineering teachers have not received a complete training in all professional competencies, specifically in sustainability competencies. This increases their difficulty when it comes to embedding learning outcomes for education for sustainable development (ESD) in their subjects [22,23,24].

The EDINSOST projects have developed several tools to answer the following questions regarding ESD in each of the degrees analyzed:

• What learning outcomes should graduates have acquired upon completion of their university studies?
• To what extent is ESD present in the curriculum?
• To what extent do students perceive that they have acquired the ESD-related learning outcomes foreseen in the curriculum?

To answer the first question, the EDINSOST projects developed a sustainability competency map (SCM) for each degree program [25]. This map contains the set of ESD-related learning outcomes to be achieved by students in the degree program. The SCM is the key element for the design of the curriculum and makes it possible to distribute the different learning outcomes among the subjects of the ESD itinerary, as well as to determine how the Sustainable Development Goals (SDGs) are being appraised in the curriculum. It also makes it possible to determine what ESD training teachers should receive. As a result of the projects, three maps were developed, one for each of the participating knowledge areas (education, engineering, and business administration and management). The three maps are very similar, and thanks to the transversality of ESD, the learning outcomes are easily extrapolated to other degrees in other areas of knowledge.

The answer to the second question is provided by the degree’s sustainability presence map [26]. This map identifies how much work has been done on each of the learning outcomes of the SCM in the degree program. For this purpose, all the subjects of the ESD competency pathway are analyzed.

Finally, to answer the third question, a questionnaire was drawn up for students in order to determine what they perceived to have learned [27] in relation to ESD.

ESD aims to create citizens who contribute to achieving the Sustainable Development Goals (SDGs) [28] as determined by the United Nations 2030 agenda [29]. The SDGs consist of 17 goals to be achieved globally by 2030. The United Nations has defined a set of targets [29] for each SDG that define precisely what each goal aims to achieve. Each goal has a different number of targets, depending on the idiosyncrasies of the goal itself. Moreover, to facilitate the task of embedding ESD in curricula at all educational levels, UNESCO has defined fifteen learning objectives for each SDG [28] using a three-level taxonomy (cognitive, socioemotional and behavioral). For each SDG, UNESCO defines five learning objectives at each level of the taxonomy.

In order to facilitate the task of universities of introducing ESD in their degree curricula, the EDINSOST2-SDG project selected the targets [29] and learning objectives [28] related to each area of knowledge, in addition to generating a table for each knowledge area that relates the SDG learning outcomes with the goals and learning objectives of the area itself. Thus, when embedding the SDG learning outcomes in a degree program, the UN targets and UNESCO learning objectives related to the SDGs are also taken into consideration. This allows each degree program to assess how the SDGs are appraised in each curriculum, an issue which currently poses a challenge for all universities around the world.

This paper presents a methodology for embedding and assessing a competency in an academic curriculum using competency maps. As an example of the application of the methodology, a proposal for embedding and assessing ESD in engineering degrees is presented. This work was developed within the framework of the EDINSOST projects. Two projects were carried out between 2016 and December 2022, EDINSOST1 and EDINSOST2-SDG, lasting for three and four years, respectively, and coordinated by the University Research Institute for Sustainability Science and Technology of the Universitat Politècnica de Catalunya–BarcelonaTech. Both projects propose a methodology for introducing and evaluating sustainability in the curricula of undergraduate and master’s degree programs at Spanish universities. Fifty-nine researchers from eleven Spanish universities participated in these projects.

2. Materials and Methods

2.1. Sustainability Competency Map (SCM) for Engineering Curricula

In EDINSOST2-SDG, a common SCM was designed for all engineering degrees based on the SCM developed in EDINSOST1. The SCM design and validation process can be found in [30]. The engineering SCM of the EDINSOST2-SDG project is a matrix design based on the four sustainability competencies defined by the Conference of Rectors of Spanish Universities (CRUE) [31].

• C1: Critical contextualization of knowledge by establishing interrelations with social, economic, environmental, local and/or global problems;
• C2: Sustainable use of resources and prevention of negative impacts on the natural and social environment;
• C3: Participation in community processes that promote sustainability;
• C4: Application of ethical principles related to the values of sustainability in personal and professional behavior;

Each competency is further defined by one or more competency units, which are described using as a starting point the three dimensions of sustainability (environmental, social and economic) plus a holistic dimension. Whenever possible, only the holistic dimension was used in each competency to achieve the simplest possible map. Thus, competencies C1, C3 and C4 have a single competency unit that is defined holistically. However, competency C2 consists of four competency units—one for each dimension. Each Competency unit is described in the form of learning outcomes using the simplified Miller pyramid [32] as a taxonomy. The taxonomy has three domain levels: know, know-how and ‘demonstrate + do’ (the original Miller pyramid consists of four levels, as the ‘demonstrate’ and ‘do’ levels are treated independently). Miller’s taxonomy was designed in a medical setting, where differentiating between “demonstrate” and “do” is important because people’s lives are at stake. In the engineering environment, however, these levels can be treated together. Using a taxonomy with a small number of domain levels is important because it reduces the size of the SCM. The fewer the domain levels, the smaller the number of map cells and, therefore, the fewer the learning outcomes. The fewer the learning outcomes, the easier it is to embed the SCM in the degree curriculum. Each competency unit is described by one or more learning outcomes at each level of the taxonomy. The engineering SCM of the EDINSOST2-SDG project described in [30] contains 53 learning outcomes. Figure 4 shows a schematic of this map.

To relate the engineering SCM to the SDGs, a table was created in which the learning outcomes of the map are related to the SDG targets [29] and the UNESCO learning objectives [28]. Details on the process of creating and validating this table can be found in [30].

2.1.1. Simplification of the Sustainability Competency Map for Engineering Curricula

In 2021, the Universitat Politècnica de Catalunya–BarcelonaTech decided to use the EDINSOST model to incorporate ESD in the curricula of all its degrees. After analyzing the tools designed by the EDINSOST2-SDG project [30], the UPC asked the project researchers to simplify the tools to facilitate their use for professors and management teams unfamiliar with ESD. The EDINSOST team then began a process to create a simplified version of the SCM by reducing the number of learning outcomes from the original map. For this purpose, a goal was defined such that each cell of the simplified map should, whenever possible, correspond to a single learning outcome.

To achieve this objective and validate the process, the following steps were taken:

• The EDINSOST engineering team was divided into two groups of the same size (seven researchers): one to perform the first approximation (working group) and the other to validate the result (validation group).
• The working group analyzed the learning outcomes in each cell and grouped those that were most closely related and could be written coherently as a single learning outcome. The new learning outcomes were initially drafted keeping the original formulation; from this phrasing, a simplification process was initiated with the aim of obtaining complete but shorter texts. This process resulted in an engineering SCM with fewer learning outcomes.
• The validation group analyzed the new learning outcomes and compared them with the initial ones to ensure that all the information described by the original learning outcomes was included in the new learning outcomes. Some deficiencies were identified in this regard.
• The working group adopted the validation group’s suggestions and drafted a new proposal containing a total of 29 learning outcomes.
• The validation group reviewed the map again to verify that the original learning outcomes were correctly represented in the new proposal.

Once the simplified engineering SCM was obtained, the tables that relate the learning outcomes to the UNESCO learning objectives and targets defined by the UN for the SDGs were updated. To this end, each learning outcome of the simplified engineering SCM was assigned the learning objectives and targets that were assigned to the related learning outcomes of the original SCM.

2.1.2. Algorithm for Creating an SCM for a Specific Engineering Degree

The simplified engineering SCM contains the ESD-related learning outcomes common to all engineering degrees. These learning outcomes include the SDGs, thanks to the relationship established with the UNESCO learning objectives and UN targets.

To define the SCM of a specific engineering degree (IT, civil, etc.), it is necessary to add to the engineering SCM the learning outcomes in ESD specific to that engineering degree. This process is conducted by analyzing the UN targets and UNESCO learning objectives related to engineering. The methodology proposed for this process is as follows:

1.

Start from the engineering SCM that contains the learning outcomes common to all engineering degrees;

2.

Analyze UN targets and UNESCO learning objectives and select those related to the specific engineering degree;

3.

For each target and learning objective selected, check if there is any learning outcome that covers it;

a.

If available, update the table of relationship between learning outcomes, UN targets and UNESCO learning objectives;

b.

If not, define a new learning outcome in the SCM of the specific engineering degree and update the relationship table.

This process should be conducted with the aim of adding as few learning outcomes as possible to the SCM. This implies that a new learning outcome may cover several UN targets or UNESCO learning objectives.

2.2. Sustainability Presence Map

The sustainability presence map of a degree contains information on the extent to which each learning outcome of the degree’s SCM is present in the curriculum. In this paper, we propose the construction of the sustainability presence map based on semi-structured interviews conducted with the professors responsible for the courses of the ESD itinerary. These professors reply beforehand to a questionnaire, which the interviewers use as a starting point to conduct the interview. The interviewers must have good knowledge of the degree’s SCM in order to complete the sustainability presence map carefully and accurately. The fact that the sustainability presence map is completed by a single person (or a small group working in a coordinated manner) ensures that the same criteria are followed in all the subjects of the itinerary and that the contribution of each subject to the sustainability presence map does not therefore depend on the experience and knowledge of the professor. The “Discussion” section describes in detail why this methodology was chosen and explains other methodologies that were considered.

The professor questionnaire used by the interviewers is a simplification of the questionnaire presented in [30], which was modified in order to adapt it to the simplified engineering SCM. The following methodology was used to simplify the questionnaire:

• A group of four researchers from the project’s engineering group (working group) reviewed the original questionnaire and drafted the new questions in accordance with the same criteria used in the original questionnaire but adapting them to the simplified engineering SCM. This process was undertaken so that the validation process conducted for the original questionnaire would be valid with respect to the new questionnaire;
• The remaining researchers in the project’s engineering group (validation group) reviewed the questionnaire individually (specifically, the relevance of each question based on its relation to the simplified engineering SCM) and suggested possible changes to the phrasing;
• A meeting of all members of the project engineering group was subsequently held to discuss the proposals and generate a new version of the questionnaire;
• This new version underwent a further revision, this time conducted by the education and business administration and management groups in order to complete the final questionnaire. Like the SCM, this questionnaire is highly transversal and was therefore used with minor modifications in the three knowledge areas of the EDINSOST2-SDG project.

3. Results

3.1. Engineering SCM Simplified

As detailed in Section 2, the EDINSOST2-SDG team simplified the engineering SCM to make it easier to handle for professors and management teams unfamiliar with ESD. As a result, the simplified engineering SCM consists of only 29 learning outcomes, compared to 53 in the original one.

Table 1. Simplified engineering SCM scheme.

Engineering SCM
Competency	Dimension	Competency Unit	Domain Levels
			Level 1 Know	Level 2 Know How	Level 3 Demonstrate + Do
C1. Critical contextualization of knowledge, establishing interrelations with social, economic, environmental, local and/or global problems.	Holistic	CU1.HO. Learners have a historical and contemporary perspective and understand the systemic nature of environmental, social and economic problems, as well as their interrelationships and future challenges, both locally and globally.	C1.HO.1.1	C1.HO.2.1	C1.HO.3.1
C2. Sustainable use of resources and prevention of negative impacts on the natural and social environment.	Environmental	CU2.EN. Learners are able to detect and analyze the environmental impact of their professional activity and to propose sustainable solutions.	C2.EN.1.1 C2.EN.1.2	C2.EN.2.1	C2.EN.3.1
	Social	CU2.SO. Learners are able to detect and analyze the social impact of their professional activity and to propose sustainable solutions.	C2.SO.1.1 C2.SO.1.2	C2.SO.2.1 C2.SO.2.2	C2.SO.3.1
	Economic	CU2.EC. Learners are able to manage the material, financial and human resources of the projects in their professional field with sustainability criteria to ensure their economic viability.	C2.EC.1.1	C2.EC.2.1	C2.EC.3.1
	Holistic	CU2.HO. Learners are able to detect and analyze the environmental, social and economic impact of their professional activity and to propose, design, organize and carry out sustainable actions.	C2.HO.1.1 C2.HO.1.2 C2.HO.1.3	C2.HO.2.1 C2.HO.2.2	C2.HO.3.1
C3. Participation in community processes that promote sustainability.	Holistic	CU3.HO. Learners are capable of participating in inclusive reflection and decision-making processes with a global citizenship perspective and of working from their professional field on interdisciplinary and transdisciplinary projects that guide society towards sustainable transitions.	C3.HO.1.1 C3.HO.1.2	C3.HO.2.1	C3.HO.3.1
C4. Application of ethical principles related to the values of sustainability in personal and professional behavior.	Holistic	CU4.HO. Learners act in accordance with ethical and deontological principles related to the values of sustainability.	C4.HO.1.1 C4.HO.1.2	C4.HO.2.1	C4.HO.3.1

provides an outline of the simplified engineering SCM with the final definition of the competency units and the coding of the learning outcomes, whereas

Table 2. Precise definitions of the simplified engineering SCM learning outcomes.

C1.HO.1.1	Learners know the main causes, consequences and proposed solutions to sustainability problems (social, economic and/or environmental), both local and global, especially in their professional field, for example, Sustainable Development Goals from Agenda 2030 and IPCC reports.
C1.HO.2.1	Learners reflect critically about sustainability in their professional field.
C1.HO.3.1	Learners are able to relate a sustainability problem of a product or service in their professional field with the methods and strategies to face them.
C2.EN.1.1	Learners know metrics (or tools) to measure the environmental impact of products and services related to their professional field (for example, environmental footprint, pollutant emissions, resource/energy consumption, biodiversity impact, waste generation, Directive 2014/95/UE for non-financial reporting, etc.).
C2.EN.1.2	Learners know strategies and/or technologies for reduction, reuse and recycling of natural resources and waste related to products and services in their professional field.
C2.EN.2.1	Learners know how to use appropriate metrics (or tools) to measure the environmental impact of products and services related to their professional field throughout their life cycle (extraction, production, use and end of life).
C2.EN.3.1	Learners take into account environmental criteria in projects related to their professional field and include indicators to estimate/measure environmental impact.
C2.SO.1.1	Learners know metrics (or tools) to measure and describe the social impact of products and services related to their professional field (for example, social life cycle assessment, ISO 26000, Directive 2014/95/UE for non-financial reporting, etc.).
C2.SO.1.2	Learners know the basic concepts of health, security and social justice related to their professional field (for example, ergonomics, accessibility, user experience, equity, diversity, common good, transparency, human rights, gender perspective, needs of the most vulnerable groups, discrimination, dignity, anticorruption, etc.).
C2.SO.2.1	Learners know how to use appropriate metrics (or tools) to measure the social impact of products and services related to their professional field.
C2.SO.2.2	Learners understand the direct and indirect consequences for security, health and social justice of products and services related to their professional field.
C2.SO.3.1	Learners take into account security, health and social justice criteria in their projects and actions and include indicators to measure social impact.
C2.EC.1.1	Learners know the basic concepts of resource management applicable to the management of projects in their professional field and methods (or tools) to estimate their economic viability (for example, fixed and variable costs, amortization, budgets, Gantt diagrams, externalities analysis, CANVAS analysis, SWOT analysis, business plans, strategic plans, cost–benefit analysis, etc.).
C2.EC.2.1	Learners understand the economic viability plan of a project in their professional field and identify the economic consequences it will have on society.
C2.EC.3.1	Learners are able to plan a project in their professional field, design an economic viability plan and follow-up the economic management throughout its useful life.
C2.HO.1.1	Learners know the strategic role of their profession in sustainability and the direct and indirect consequences of the use of products and services related to their professional field on society, the economy and the environment.
C2.HO.1.2	Learners know different economic approaches that promote sustainable development (for example, circular economy, economy of the common good, social economy, ecological economy, etc.).
C2.HO.1.3	Learners know the roles, rights and duties of the different stakeholders (professionals, companies, legislation, clients, consumers, etc.) in the production and consumption of products and services related to their professional field.
C2.HO.2.1	Learners know how to analyze the alternatives to products or services in their professional field to decide which is the most sustainable.
C2.HO.2.2	Learners know how to apply different sustainability approaches to production, consumption (responsible consumption) and recycling.
C2.HO.3.1	Learners are able to bring new ideas and solutions to a project in their professional field to make it more sustainable, to propose sustainable projects, to follow-up and dismantle appropriately and to select which indicators will be used to measure sustainability.
C3.HO.1.1	Learners know the main economic and environmental stakeholders related to their professional field.
C3.HO.1.2	Learners know techniques and/or tools to promote, in processes and projects in their professional field, their collaboration, the consideration of needs and expectations (information processes, consultation, participation and integration) and cooperation among them (scenario-building techniques, cocreation of knowledge, etc.).
C3.HO.2.1	Learners know how to collaborate with the different stakeholders involved in a project in their professional field, to identify their needs and expectations and to assess the implications they may have on the sustainability of the project.
C3.HO.3.1	Learners are able to use techniques and/or tools to promote collaboration and cooperation in interdisciplinary and transdisciplinary contexts in their professional field, participating in processes of reflection and decision making as agents of change towards sustainable transitions.
C4.HO.1.1	Learners know the code of ethics of their profession, the main ethical issues, and the laws and regulations related to sustainability.
C4.HO.1.2	Learners know the concepts of social commitment and corporate social responsibility, as well as their possibilities and limitations.
C4.HO.2.1	Learners are capable of identifying and critically assessing the implications of ethical and deontological principles related to the values of sustainability in their professional field and of critically assessing the responsible action of companies.
C4.HO.3.1	Learners are capable of exercising their profession and of actively participating in responsible action in the entities in which they develop their profession, taking into account ethical principles related to the values of sustainability (for example, equality; justice; the precautionary principle; prevention of damage; responsibility towards present and future generations; protection and restoration of a healthy environment; and social, economic and environmental human rights).

contains the precise definition of each learning outcome.

To update the table relating the SDGs according to the SCM learning outcomes, an analysis was performed to determine which UNESCO learning objectives [28] and UN targets [29] are related to engineering degrees. Figure 5 presents the results for the UNESCO learning objectives, and Figure 6 presents the results for the UN targets. The columns of both tables represent the 17 SDGs, and the rows represent the learning objectives (15) and targets (between 5 and 19) for each SDG.

The next step consisted of relating the SCM learning outcomes to the UNESCO learning objectives and the UN targets that are related to all engineering degrees (those marked in green in Figure 5 and Figure 6). Details of this mapping can be found in [30]. The learning objectives and targets marked in yellow were not taken into consideration because the engineering SCM is a map common to all engineering degrees. As described in Section 2.1.2, each engineering degree should add to its specific SCM its own learning outcomes and relate them to some of the learning objectives and targets marked in yellow (those related to the degree).

Finally, the relationship of the learning outcomes of the engineering SCM (Table 2) with the UNESCO learning objectives (Figure 5) and the UN targets (Figure 6) was established.

Table 3. Specific relationship between the learning outcomes (LOu) of the simplified engineering SCM and the SDGs through (i) UNESCO learning objectives (LOb) and (ii) UN targets (T).

	SDG
	1		2		3		4		5		6		7		8		9		10		11		12		13		14		15		16		17
	LOb	T	LOb	T	LOb	T	LOb	T	LOb	T	LOb	T	LOb	T	LOb	T	LOb	T	LOb	T	LOb	T	LOb	T	LOb	T	LOb	T	LOb	T	LOb	T	LOb	T
C1.HO.1.1	C3					3.9	C4 C5	4.4 4.7	C5	5.1 5.5 5.b		6.3 6.4	C1	7.1 7.3	C1	8.4	C4	9.2 9.4 9.b		10.2	C1		C2	12.2 12.6 12.8	C3	13.3		14.1		15.1		16.10	C2
C1.HO.2.1						3.9		4.4 4.7				6.3 6.4					B2				C1			12.6		13.3				15.1		16.10
C1.HO.3.1						3.9		4.4 4.7				6.3 6.4	B2	7.2		8.4	C4 B1	9.2 9.4 9.b	B3					12.6		13.3				15.1		16.5
C2.EN.1.1						3.9		4.4 4.7				6.3 6.4		7.2		8.4		9.2 9.4 9.a						12.2 12.4 12.5	C3	13.3		14.1		15.1			C5
C2.EN.1.2						3.9		4.4 4.7				6.3 6.4	C3	7.2		8.4	C1	9.2 9.4					C2	12.2 12.4 12.5				14.1		15.1
C2.EN.2.1						3.9		4.4 4.7			C4	6.3 6.4	C3	7.2		8.4	B2 C1 C2	9.2 9.4					C2	12.2 12.4 12.5		13.3		14.1		15.1				17.7
C2.EN.3.1						3.9		4.4 4.7				6.3 6.4	B2	7.2		8.4	B2	9.2 9.4 9.b			B5		B1	12.2 12.4 12.5	B1	13.3		14.1		15.1			S3
C2.SO.1.1						3.6 3.9		4.4 4.7		5.1 5.5 5.b						8.2		9.2	C2	10.2													C5
C2.SO.1.2					C1	3.6 3.9		4.4 4.7								8.2	C5	9.2					C2
C2.SO.2.1	C4				C1	3.9	C3	4.4 4.7	C5							8.2	C2 C3	9.2	B1 C1 C4				C2								S4
C2.SO.2.2						3.6 3.9		4.4 4.7	C5	5.1 5.5 5.b						8.2	C1 C2	9.2	C2 S1 S5	10.2											C4 C5 S4
C2.SO.3.1	B1 B5					3.6 3.9		4.4 4.7	B2	5.1 5.5 5.b					B1 B5	8.2		9.2	B2 B4	10.2			B1										S3
C2.EC.1.1								4.4 4.7						7.2		8.2		9.2 9.4
C2.EC.2.1								4.4 4.7						7.2		8.2 8.4		9.2 9.4						12.2
C2.EC.3.1								4.4 4.7						7.2	B4	8.2 8.4		9.4
C2.HO.1.1						3.9		4.4 4.7		5.b	C4	6.3 6.4	C3			8.4	C1 C3	9.4 9.b				11.a		12.4 12.5 12.6 12.8				14.1				16.10	C4
C2.HO.1.2						3.9		4.4 4.7						7.2	C1 B1 B3 B5		C5	9.2 9.4				11.a		12.2 12.6									C5
C2.HO.1.3						3.9		4.4 4.7						7.2		8.4	C5	9.2 9.b				11.a	C3	12.4 12.6
C2.HO.2.1						3.9		4.4 4.7				6.3 6.4		7.2		8.4	C2 C3 C4 C5 B2	9.2 9.4					C5 B4	12.2 12.4 12.5 12.6				14.1		15.1
C2.HO.2.2						3.9		4.4 4.7				6.3–6.4				8.4		9.2 9.4					C4 C5	12.2 12.5 12.6				14.1
C2.HO.3.1	B5					3.9		4.4 4.7				6.3 6.4	B2	7.2	B1 B3 B5	8.4	B1 B2 C3	9.2 9.4	B4	10.2	B1 B4	11.a	B1 B3	12.2 12.4 12.5 12.6	B5	13.3		14.1		15.1			B1
C3.HO.1.2								4.4 4.7		5.1 5.5 5.b										10.2		11.a		12.8								16.10
C3.HO.1.1								4.4 4.7		5.1 5.5 5.b										10.2				12.8								16.10	C4
C3.HO.2.1								4.4 4.7		5.1 5.5							B2			10.2		11.a		12.3 12.8								16.10
C3.HO.3.1								4.4 4.7		5.1 5.5 5.b			S1				B5	9.b		10.2	B2 B4 S1 S2	11.a	S2	12.6 12.8	S3					15.1		16.10	B1
C4.HO.1.1						3.9		4.4 4.7		5.1 5.5 5.b		6.3 6.4						9.4 9.b	C5	10.2				12.6–12.4						15.1
C4.HO.1.2								4.4 4.7		5.1 5.5 5.b										10.2												16.10
C4.HO.2.1	S5 B1					3.9		4.4 4.7		5.1 5.5 5.b		6.3 6.4					B2 B5	9.4		10.2			C5 S5	12.6	B1 B5							16.10	B1 B5
C4.HO.3.1	B1 B5					3.6 3.9		4.4 4.7		5.1 5.5 5.b		6.3 6.4					B1 B3 B5	9.4 9.b		10.2	B1 B4		B1 B4 S5	12.4 12.6	B5					15.1		16.10	S3 B1 B5

shows this relationship. Table 3 presents the specific relationship between the learning outcomes (LOu) of the simplified engineering SCM and the SDGs through (i) UNESCO learning objectives (LOb) and (ii) UN targets (T). The UNESCO learning objectives are expressed using the same coding as in Figure 5 (C, cognitive; S, socioemotional-S; B, behavioral), and the UN targets are coded using the same numbering scheme as in Figure 6.

The next step consisted of relating the SCM learning outcomes to the UNESCO learning objectives and the UN targets that are related to all engineering degrees (those marked in green in Figure 5 and Figure 6). Details of this mapping can be found in [30]. The learning objectives and targets marked in yellow were not considered because the engineering SCM is a map common to all engineering degrees. As described in Section 2.1.2, each engineering degree should add to its specific SCM its own learning outcomes and relate them to some of the learning objectives and targets marked in yellow (those related to the degree).

3.2. Presence Map

The design of the professors’ questionnaire was based on the engineering SCM, with the purpose of determining which learning outcomes are developed in each subject and to what extent the teacher considers that they are developed in the subject. The questions are answered using a four-point Likert scale with the following meaning:

• Nothing: the learning outcome is not developed in the subject;
• A little: the learning outcome is developed superficially or partially;
• Enough: the learning outcome is developed more than superficially but not in depth;
• A lot: the learning outcome is developed in depth.

The questionnaire consists of 18 questions (compared to the 29 learning outcomes of the SCM). This reduction in the number of questions compared to the number of learning outcomes makes it easier for professors to answer the questionnaire. In each question, keywords are highlighted to facilitate understanding.

The questions included in the questionnaire are presented in

Table 4. Items related to the category “Critical contextualization of the concept of sustainability”.

Critical Contextualization of the Concept of Sustainability
C1.HO.1.1.	Knows the principal causes, consequences and solutions proposed to solve sustainability problems (social, economic and/or environmental), both local and global, especially in their professional area, for example, the 2030 Agenda Sustainable Development Goals, IPCC reports, etc.
C1.HO.2.1.	Think critically about sustainability in their professional field.
C1.HO.3.1.	Is able to relate a sustainability problem of a product or service of their professional field with the methods and strategies required to address it.

,

Table 5. Items related to the category “Sustainable use of resources and impact prevention (environmental, social and economic)”.

Sustainable Use of Resources and Prevention of Impacts (Environmental, Social and Economic)
C2.EN.1.1./2.1./3.1.	Knows about, knows how to use and includes metrics (and/or tools) in their projects in order to measure the environmental impact of the products and services related to their professional field (for example, environmental footprint, polluting emissions, consumption of resources/energy, damage to biodiversity, generation of waste, 2014/95/EU directive for non-financial reports, etc.).
C2.EN.1.2	Knows the strategies and/or technologies for reducing, reusing and recycling natural resources and waste related with the products and services of their professional field.
C2.SO.1.1./C2.SO.2.1./C2.SO.3.1.	Knows, knows how to use and includes in their projects metrics (and/or tools) that measure and describe the social impact of products and services related to their professional field (e.g., social life cycle analysis, ISO 26000 on social responsibility, Directive 2014/95/EU for non-financial reporting, etc.).
C2.SO.1.2./C2.SO.2.2.	Knows the concepts of health, safety and social justice related to their professional field (e.g., ergonomics, accessibility, user experience, equity, diversity, common good, transparency, human rights, gender perspective, needs of the most vulnerable groups, discrimination, dignity, anticorruption, etc.) and understands the direct and indirect consequences regarding safety, health and social justice of products and services related to their professional field.
C2.EC.1.1./C2.EC.2.1./C2.EC.3.1.	Knows the basic concepts of resource management applicable to the management of projects in his/her professional field and methods (and/or tools) to estimate their economic viability (e.g., fixed and variable costs, amortization, budgets, Gantt charts, externality analysis, CANVAS analysis, SWOT analysis, business plans, strategic plans, cost–benefit analysis, etc.) and is able to plan a project in his/her professional field, draw up an economic viability plan and monitor the economic management throughout its useful life.
C2.HO.1.1.	Knows the strategic role of their profession in sustainability and the direct and indirect consequences of the use of products and services related to their professional field on society, the economy and the environment.
C2.HO.1.2.	Knows different approaches that promote sustainable development (e.g., circular economy, economy of the common good, social economy, green economy, etc.) and considers the roles, rights and duties of the actors involved.
C2.HO.1.3.	Knows the roles, rights and duties of the different agents (professionals, companies, legislation, clients, consumers, etc.) in the production and consumption of products and services related to their professional field.
C2.HO.2.2.	Knows how to apply sustainability approaches in production, consumption (responsible consumption) and recycling.
C2.HO.2.1./C2.HO.3.1.	Knows how to analyze the different alternatives to products or services in their professional field in order to decide which is the most sustainable and is able to contribute new ideas and solutions in a project in their professional field in order to make it more sustainable, to select indicators to measure its sustainability, to propose sustainable projects and to carry out adequate follow-up and decommissioning.

,

Table 6. Items related to the category “Participation in integrative processes of reflection and decision making”.

Participation in Integrative Processes of Reflection and Decision Making
C3.HO.1.1./C3.HO.2.1.	Knows the main stakeholders and social, economic and environmental agents related to the activity of their professional field and knows how to collaborate with them to identify their needs and expectations and assess the implications they may have on the sustainability of the project.
C3.HO.1.2./C3.HO.3.1	Knows and is capable of using techniques and/or tools to promote, in processes and projects in their professional field, the collaboration of the main stakeholders and social agents, taking into consideration their needs and expectations (information processes, consultation, participation and integration) and their mutual cooperation (scenario-building techniques, cocreation, etc.).

and

Table 7. Items related to the category “Application of ethical and deontological principles”.

Application of Ethical and Deontological Principles
C4.HO.1.1./C4.HO.1.2.	Knows the code of ethics of their profession, the main ethical issues, the laws and regulations related to sustainability, the concepts of social commitment and corporate social responsibility and their possibilities and limitations.
C4.HO.2.1.	Is capable of identifying and critically assessing the implications of ethical and deontological principles related to the values of sustainability in their professional field and of critically assessing the responsible action of companies.
C4.HO.3.1.	Capable of exercising their profession and of actively participating in responsible action in the entities in which they develop their profession, taking into account the ethical principles related to the values of sustainability.

and grouped into four categories:

• Critical contextualization of the concept of sustainability;
• Sustainable use of resources and prevention of impacts (environmental, social and economic);
• Participation in integrative processes of reflection and decision making;
• Application of ethical and deontological principles.

These categories correspond to the CRUE competencies in sustainability [31], although in the questionnaire, they are referred to as categories to facilitate understanding for professors unfamiliar with competency-based learning.

In the last category, the questionnaire specifies that the following ethical principles related to sustainability values are considered: equality; justice; the precautionary principle; prevention of damage; responsibility to present and future generations; protection and restoration of a healthy environment; and social, economic and environmental human rights (this text is not included in Table 7).

In Table 4, Table 5, Table 6 and Table 7, the column on the left represents the coding used in the SCM for the learning outcomes related to the question, and the right-hand column presents the question.

4. Discussion

Competency maps are a fundamental tool for curriculum designers and subject professors. They enable curriculum designers to create training itineraries for each competency in the degree and to distribute the learning outcomes of the competencies among the subjects in the itinerary. If this work has not been done by the curriculum designers, competency maps allow those responsible for the subjects to select which learning outcomes of the competencies assigned to the subject fit best and to build on this the learning objectives of the subject. In the latter case, it is necessary to carry out a coordination task to ensure that all the learning outcomes of each competency are developed in at least one subject of the competency itinerary.

Many degrees have been designed without using competency maps. In these degrees, presence maps are especially useful for identifying which learning outcomes are being developed sufficiently and those that are not developed enough. This information indicates how degree courses may be modified in order for the competencies to be fully developed.

Presence maps can be drawn up (1) on the basis of the official documents that describe the curriculum (in the case of the Spanish university system, these documents are called Verifica), (2) from the learning guides of the subjects or (3) from the information provided directly by the professors. Although the three sources of information should be mutually consistent and lead to very similar presence maps, this was far from the case in practice. García-Gallofré and Segalàs [33] analyzed sustainability competency in the UPC “Product Development Design Engineering” degree from three points of view (Verifica, learning guides and teacher questionnaires) and concluded that the information from the three sources was not mutually consistent, thereby producing three different maps. The Verifica documents contained information that had not been used accurately by the professors when designing the courses (in some cases, due to lack of knowledge of the contents of the Verifica document), and the learning guides of the courses published on the website of the degree were not up-to-date. The information provided by the faculty was extracted from the questionnaire for professors of the EDINSOST2-SDG project [30] (which corresponds to the engineering SCM of 53 learning outcomes). After interviewing a representative set of professors, García-Gallofré and Segalàs [33] determined that the most reliable of the three maps was the one generated from the information provided by professors. However, based on the interviews conducted, the authors realized that the criteria used by individual professors to answer the survey questions depended heavily on each professor’s knowledge of ESD. In many cases, García-Gallofré and Segalàs found that the Kruger–Dunning effect [34] occurred, whereby professors who were less familiar with ESD overestimated their own knowledge, as well as the learning achieved by students in ESD-related activities in their subjects. This work motivated the EDINSOST2-SDG researchers to design and conduct semi-structured interviews with professors who had previously answered the questionnaire so that it is the experienced interviewer who constructs the sustainability presence map using the same criteria for all subjects; therefore, the presence map is not automatically constructed from the results of the questionnaire (using the professors’ criteria).

Sánchez-Carracedo et al. [35] studied the extent to which the sustainability presence map of a degree was completed as a function of the number of subjects in the ESD itinerary across a broad set of degrees in the Spanish university system. They concluded that the greater the presence of sustainability in the presence maps, the greater the number of subjects in the ESD itinerary. This led them to conjecture that because, in general, the professors of the different subjects had not acted in a coordinated manner when designing ESD-related activities, the sustainability map contained no set of learning outcomes that was more generic than the rest. In other words, the teachers’ knowledge of ESD is not concentrated in a specific group of learning outcomes but depends entirely on the characteristics of the teaching staff. This fact justifies that similar degrees taught at different universities have very different sustainability presence maps, as described in the same study. Figure 7, taken from [35], presents as an example the presence of the four CRUE competencies [31] in bachelor’s degrees in primary education at seven Spanish universities. The vertical axis indicates the percentage of presence. As shown in the figure, the presence of each competency depends on the university where the degree is taught.

The presence map shows which competency learning outcomes are not being developed in the curriculum. In the case of the professional competencies, it is very likely that this is due to a lack of training of the faculty in said competencies. Therefore, the presence map also provides information about the issues for which the faculty of a given degree program requires training (it may be that professors of the same degree program from different universities have different training needs). In light of this, EDINSOST2-SDG project researchers developed an ESD course for university faculty. This course focuses on helping professors to integrate some of the SCM learning outcomes into the training activities they undertake in their respective subjects. In addition, the project developed and/or collected different open educational resources [36] that help teachers to develop the different SCM learning outcomes in their subjects.

Finally, to complete the process of embedding ESD in the curriculum, it is necessary to determine whether students are achieving the expected learning outcomes upon completion of their studies. In other words, whereas the presence map of a competency indicates how the competency is integrated into the curriculum, it is also necessary to determine whether the subjects are achieving their learning objectives and whether the students are reaching the learning outcomes of the competency map. To obtain this information, the EDINSOST2-SDG project developed a questionnaire to determine students’ perceptions about their own learning in ESD [30]. This questionnaire corresponds to the non-simplified version of the engineering SCM (53 learning outcomes). Project researchers are currently working on the design of a questionnaire for the simplified engineering SCM (29 learning outcomes).

The results of this questionnaire do not enable us to determine whether the students have achieved a given learning outcome—only their perception of their achievement of the learning outcome. This constitutes one of the main limitations of this methodology. However, given a sufficient number of responses, the results of the questionnaire can be very useful. Obtaining realistic information about students’ ESD training would entail the performance of some evaluation tests, which, in turn, would present significant logistical problems.

Sánchez-Carracedo et al. [37] surveyed first- and final-year students in nine engineering degrees and nine education degrees in the Spanish university system. The results showed that the mean percentage of learning perceived by students is higher in engineering degrees (33%) than in education degrees (27%), despite the fact that the mean increase in learning declared by students upon completion of their studies in both areas of knowledge is similar (66%). Engineering students report higher learning gains than education students in all sustainability competencies, with the exception of ethics. Figure 8, extracted from the paper by Sánchez et al. [37], shows the overall results of this study for each CRUE competency [31]. On a scale of 0 to 3, Figure 8a shows the incremental learning value for first- and fourth-year students in the two knowledge areas. The white bars in the graph indicate the reported learning differences between first- and fourth-year students in each case. On a scale of 0 to 1, Figure 8b shows the value of the percentage of learning achieved by the students with respect to expected learning.

These data should be compared with the sustainability presence map in each specific degree in order to determine to what extent students perceive that they are achieving the expected results. This comparison was made by Valderrama et al. [38] to analyze the bachelor’s degree in primary education teaching at the University of Seville. The sustainability presence map of the degree was constructed from the teaching guides of the subjects, and the student survey was answered by 104 first-year students and 86 fourth-year students. Valderrama et al. [38] found that students perceived an improvement in the learning of some outcomes, although these outcomes were not apparently developed in the sustainability presence map. On the other hand, no improvement was detected for other learning outcomes contained in the sustainability presence map. The authors of the paper concluded that the subjects are failing to achieve their ESD learning objectives and that the students are either trained in sustainability outside the university or the subject learning guides do not reflect the work done by the students throughout their studies—a finding which is compatible with the results obtained by García-Gallofré and Segalàs [33]. Responses to some questions of the questionnaire revealed that first-year students reported greater knowledge than fourth-year students, confirming a Kruger–Dunning effect [34]. The results obtained by Valderrama et al. support the proposal made in this paper to implement presence maps based on semi-structured interviews with teachers after they have answered the corresponding questionnaire.

The experience acquired during the EDINSOST projects has enabled the design and development of a training course for teachers to help them to embed SCM learning outcomes in the training activities of their subjects. The course has been implemented in all the universities belonging to the EDINSOST2-SDG project, as well as in some foreign universities. A database of open educational resources for ESD that is being developed within the framework of the project will make it easier for teachers to introduce educational activities for ESD in their subjects. One of the universities in the consortium, UPC, is currently developing a pilot database of open educational resources for ESD.

5. Conclusions

Although in the context of the European Higher Education Area, many universities incorporate professional competencies, competency-based education remains a challenge. Strategies for embedding professional competencies in the curriculum may be diverse, but coordination of the subjects of a degree is essential for the competency to be deployed correctly throughout the competency itinerary. In this way, students acquire the different domain levels of the competency as they progress through the curriculum.

The methodology presented in this paper is based on the use of two inter-related and complementary tools, the objective of which is to deploy a competency in the curriculum of any degree: 1) a competency map to facilitate embedding and assessment of the competency in the curriculum and 2) a presence map to analyze the extent to which the competency is integrated in a curriculum. The analysis of the level of embedding of a competency in the curriculum can be completed by using a third tool: a student questionnaire to determine students’ perception about the acquisition of the competency (not presented in this paper). As an example of application of the methodology, in this paper, we investigated the sustainability competency in engineering degrees.

Using a learning taxonomy, competency maps describe the learning outcomes that the curriculum should develop and enable them to be precisely distributed among the subjects of the competency itinerary according to the characteristics and circumstances of each subject. Each subject of the competency itinerary contributes to the students’ acquisition of the competency. However, it is difficult to acquire a comprehensive perception of what happens in the classroom and the competency level achieved by the students. Each teacher adapts to the demands of the curriculum according to his or her own knowledge, experience and motivation.

Presence maps enable us to determine the extent to which the competency is developed in the curriculum, as well as which learning outcomes are developed therein and in which subject. A presence map therefore represents a suitable tool for identifying gaps and/or overlaps between subjects, thereby modifying them and balancing their contributions to the acquisition of the competency. In addition, by highlighting the aspects of the competency that remain undeveloped, presence maps furnish information about the areas in which the faculty responsible for each degree program requires training. A presence map can be constructed from (1) the official documents of the degree program, (2) the teaching guides of the subjects or (3) the information provided directly by the faculty. In this paper, we argue that presence maps constructed from information provided by the professors, sifted through a semi-structured interview, is the method that produces presence maps that are closest to reality.

Finally, a questionnaire is also required in order to determine the students’ perception of their own learning of the competency. A comparative study between students’ perception of their sustainability learning and the sustainability presence map of their degree conducted in the context of the EDINSOST1 project indicates that teaching and learning in sustainability are unaligned. In a situation such as the current one, it is of urgent importance to improve ESD teacher training and ensure that efforts are reflected in the learning of students, who will be responsible for making decisions in the future.