Second Phase of the Adaptation Process of the Mathematics Self-Efficacy Survey (MSES) for the Mexican–Spanish Language: The Confirmation

Abstract

There are great economic benefits and qualities of life when a country invests in the development of professionals in STEM areas. Unfortunately, there is a gender gap, as women are lagging behind their peers, as well as minority groups such as Hispanics, who are grossly underrepresented in these careers. Therefore, it is a priority to generate assessing instruments that are adapted to the cultural context of Latino students in their language to attract a more diverse population to STEM areas. This study presents a thorough validation process of the adaptation of the Self-Efficacy Survey (MSES) to the Spanish language and Mexican engineering context. Exploratory and confirmatory factor analyses were conducted with data collected from 683 Mexican engineering students to analyze its validity. The results highlight that the original three dimensions of the MSES still show a sound structure to assess math self-efficacy, and the confirmatory factor analysis eliminated items that were outdated and out of the context of this specific population. As a result, this study presents a 12-item adaptation that could help Latino researchers to collect reliable math self-efficacy data to better understand how their students feel when they learn and practice mathematics.

1. Introduction

1.1. Cultural and General Context of Mathematics in STEM Education

There is a prevailing need for countries all over the world to look for constant development by training their human capital in the areas of science, technology, engineering, and mathematics (STEM) [1]. This need has placed countries such as the United States or Mexico to work on initiatives within their national plans to promote their economic development [2,3], making extra efforts to increase the quantity and quality of their professionals in STEM areas, as well as trying to implement and develop new technology aiming to solve their most urgent needs. However, developing professionals in STEM areas in different regions and countries has been a challenge, one of the reasons could be that there are minority groups such as African Americans and Latinos who are lagging behind in representation in these careers [4]; likewise, there is a gender gap where women are not represented in equal proportion to men in the rate of graduates in these careers, and they are not in the labor field in a proportion equal to their male peers [5,6,7,8]. Therefore, educational institutions at all levels must be prepared to promote all types of students’ interest in studying STEM careers, as well as to improve their curricular plans and pedagogical strategies to help their students to successfully complete their majors despite their possible cultural differences [9].

The current trends in educational innovation in upper secondary education and higher education contemplate several options that may help to improve the promotion of STEM careers within their students [10], considering strategies such as applying different didactic methods, improving assessment processes, involving technology in the classroom, training instructors and administrative staff, or developing both technical and transversal skills [11,12,13,14]. The success of these strategies would be of great help to fulfill the labor world market demands, as well as the UNESCO and the OECD goals of developing and enhancing young students’ creativity by performing designs or using technology that is an important part of the STEM areas [15].

Although there are many factors that may influence students’ decisions to pursue a STEM-related major [14,16,17,18], the literature suggests that students who have developed good skills in mathematics [19,20] and have taken more math courses at the high school level are the ones who most frequently enroll in STEM-related majors [21,22,23]. Having a good set of mathematical skills could also increase students’ probability of successfully completing their studies in STEM majors due to the confidence that they develop in their math abilities [24].

This study focuses on students’ perceptions about their math abilities and their feelings when they are learning and practicing math. According to math self-efficacy literature, students that reported high math self-efficacy feelings are more likely to be involved in math-related activities and use their math knowledge to solve problems using advanced calculations [25,26,27,28]. Conversely, students with low math self-efficacy feelings are usually less interested in taking math-related courses that could help them to develop their math abilities [29], making them more likely to have negative experiences while learning math and to avoid math-related activities in school and in their daily activities [30,31]. The self-efficacy concept was developed by Bandura [32], establishing that: “the perception of people about their capacities to organize themselves and carry out the actions necessary to achieve a certain level of performance” (p. 391). Self-efficacy beliefs are a core concept in Bandura’s Social Cognitive Theory, playing a determinate role in people’s activities selection and their effort, persistence, and emotional reactions to performing such activities [32].

In the end, students’ math self-efficacy feelings can influence their decision to pursue and successfully complete a STEM major, as these majors have many courses that use math as a tool to solve problems and design technological solutions. Hence, the importance of this type of research, where an instrument that could help STEM educators and professors to better understand their students’ math self-efficacy feelings, is presented, and the relevance of cultural and context adaptations is discussed.

1.2. Purpose

The goal of this study was to continue the validation process of the adaptation of the mathematics self-efficacy survey to the Spanish language and Mexican engineering context. This study is an extension of the exploratory analysis conducted by Morán-Soto et al. [33], and more factorial analyses were conducted to confirm previous findings aiming to provide sound evidence that the instrument adaptation could be used to assess engineering students’ math self-efficacy for future research projects. The thorough validation process presented in this study is important due to the relevance of current research analyzing the effects of math self-efficacy on students around the world.

Despite findings highlighting the connection between students’ math self-efficacy feelings and their interest and motivation to select and complete a STEM major [34,35], there are methodological flaws in the way that self-efficacy data are collected by some researchers. These methodological issues are usually presented when math self-efficacy is measured using well-recognized instruments without considering the differences created by students’ educational level, major, language, and cultural context [36,37,38]. If researchers collect new data sets using instruments that are not adapted and tested for validation for the target population, then the findings and conclusion of that study could be biased by a different interpretation of the items and cultural and context misunderstandings [39], jeopardizing the trustworthiness of the results [40]. Developing awareness of this trustworthiness issue could help future researchers to improve the quality of their work since there are many examples of studies that collected math self-efficacy information using the Mathematics Self-Efficacy Survey (MSES) without considering any context and cultural adaptation or thorough validation tests to confirm the suitability of this instrument in the context of their research [36,41,42].

The MSES was selected for this adaptation to the Spanish language and Mexican engineering context due to its global relevance and constant use during four decades of math self-efficacy research since its creation [43]. The MSES is frequently used in self-efficacy research despite the multiple instruments that have been created by other researchers around the world, including instruments designed to measure primary and secondary students’ math self-efficacy [44,45] and others aiming to measure this variable for college students in different contexts [46,47]. Although these tailormade instruments are well developed and are usually based on the Bandura’s guide for constructing self-efficacy scales [48], most of the math self-efficacy research found in the current literature trusts the MSES or any of their three subconstructs to assess certain levels of math self-efficacy in different contexts, including studies with adult students [49], nursing students [50,51], gifted students [52], and pre-service teachers [53], among others.

In the end, having more reliable assessment tools to collect math self-efficacy information in different contexts, languages, and cultures will help the international research community to feel more welcome and included in the current math education research agenda, which ultimately will provide engineering educators and professors with information to develop a better understanding of their students’ needs while learning and practicing math. This adaptation in particular will be of great value for Latino researchers interested in math self-efficacy research conducted in Hispanic countries since there is a lack of suitable instruments that could yield reliable assessments of this construct in the Spanish language and Latin American context and culture.

2. Materials and Methods

2.1. Participants

The data for this study were collected in two different semesters, with a sample of 396 participants from the fall 2021 semester and 287 participants from the spring 2022 semester. All participants (n = 683) were first-year students enrolled in a mandatory math course in different engineering programs at a public university in Mexico that is part of the largest technological university system in the country. All participants voluntarily agreed to answer a paper-based survey during their class time.

Data from 396 participants were collected for the first sample with 251 males (63%) and 145 females (37%) during the fall 2021 semester. The age of the participants ranged between 17 and 41 (mean = 18.87), and 7% were enrolled in biochemistry, 3% in civil, 9% in electrical, 38% in industrial, 20% in mechanical, 17% in chemistry, and 6% in computing systems engineering. Likewise, the second sample was collected from 287 participants with 150 males (52%) and 137 females (48%) during the spring 2022 semester, with 20% enrolled in biochemistry, 17% in civil, 5% in electronic, 27% in industrial, 10% in mechatronics, 4% in chemistry, and 17% in computing systems engineering. All participants from the spring 2022 sample were different from the ones who answered the survey during the fall 2021 data collection.

These samples were used in two different validation tests, with the fall 2021 sample used for an exploratory factor analysis and the spring 2022 sample for a confirmatory factor analysis. Literature suggests more than six participants for each item in the factorial analysis [54,55,56]. Therefore, the size of these two samples was in line with the minimum criterion ratio suggested in psychometrics for structural validation analysis, with a subject-item ratio of 11 for the fall 2021 sample and 8.4 for the spring 2022 sample.

Additionally, the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy [57] was checked for both samples, with a KMO = 0.94 for the fall 2021 sample (for the exploratory factor analysis) and a KMO = 0.95 for the spring 2022 sample (for the confirmatory factor analysis). These KMO values were close to 1, indicating that the patterns of correlations are relatively compact, and factor analysis should yield reliable results [57]. These results showed that both sample sizes were adequate for the factorial analyses [58].

2.2. Instrument

The instrument that was adapted in the previous study [33] was the “Mathematics Self-Efficacy Survey” [43]. This instrument was selected due to its consistent results assessing students’ mathematical self-efficacy perceptions in high school [59] and college [60,61]. The MSES was originally written in English language and consists of 52 items presented on a Likert scale form asking participants to rate their level of confidence from 0 (“no confidence at all”) to 9 (“complete confidence”). In the previous research conducted by Morán-Soto et al. [33] the MSES items were translated from the English to Spanish language, and some cultural adaptations were made to adjust and test the validity of the data collected from Mexican engineering students. Additionally, the Likert scale was modified to a 1 to 10 scale aiming to avoid possible issues coming from considering “0” as a value, keeping the original 10-point Likert scale and helping Mexican students to rate their math self-efficacy level.

The 37 items resulting from the MSES adaptation in the previous research [33] were divided into three subconstructs, 13 items for everyday math activities (e.g., calculate the total of your grocery bill in your head while you take the items one by one), 8 for math-related courses (e.g., obtaining a B or higher grade in your geometry course), and 16 for math problem solving (e.g., The average of three numbers is 30. If the fourth number is at least 10. Which is the average of the four numbers?). Continuing with the validation process of the MSES and following the suggestion made by the previous adaptation and validation phases described by Morán-Soto et al. [33], the eight math-related courses items of the MSES adaptation were removed and replaced with six math courses that were more appropriate for the engineering students’ context. The six math courses were selected from the engineering curriculum that most engineering students need to complete for their major in Mexico [62]: (1) precalculus, (2) differential calculus, (3) integral calculus, (4) linear algebra, (5) vectorial calculus, and (6) differential equations. This way, the instrument adaptation was used with 35 items to collect information for the two samples (fall 2021 and spring 2022).

2.3. Exploratory Factor Analysis

The multivariate normality properties of the data were tested aiming to determine the suitability of conducting factor analyses according to the nature of the data. Two separate multivariate normality analyses were performed using the two collected samples, the fall 2021 sample with 396 participants and the spring 2022 sample with 287 participants from the spring 2022. These samples showed coefficients in the ranges from −2.3834 to −0.4661 for skewness, from 2.4292 to 9.2376 for the kurtosis index for the fall 2021 sample, from −2.0786 to −0.4878 for skewness and from 2.4207 to 8.5523 for the spring 2022 sample. According to Kline [63], items that reach absolute values greater than 3.0 for skewness and 10.0 for kurtosis could be problematic for large samples. Hence, these two samples were considered within the acceptable range of multivariate normality.

Continuing with the validation process, an exploratory factor analysis (EFA) was conducted with the fall 2021 sample (n = 396) looking for replicability of the factorial model established in the previous work by Morán-Soto et al. [33]. This EFA was carried out aiming to confirm the previous evidence of the structural model stability, as well as to analyze how the six new items load in the math-related course subconstruct.

Before conducting the EFA, a scree-plot test was performed aiming to identify the ideal number of factors to consider when performing the EFA. The graphical results of the scree-plot are shown in Figure 1 (elbow graph). These results helped to determine that three factors were optimal for the EFA, which was in line with the factors established by the original instrument and the previous results adapting the MSES to the Spanish language [33,43].

Following the scree-plot test results, a three-factor EFA was conducted using an oblique (promax) rotation [64]. This rotation is normally used when the multivariate data normality is previously established by the skew and kurtosis analysis results [65]. Performing an additional EFA was necessary to check the stability of the suggested model structure. The structure of any instrument is the baseline to collect reliable data for variables with multiple subconstructs [65], hence the importance of conducting EFA and replications aiming to provide enough evidence that the instrument structure remains stable when data from different populations are analyzed. For this particular case, the EFA was necessary due to the modification of the six math courses aiming to evaluate the math self-efficacy of engineering students in a better context and looking for more reliable subconstructs to better explain math self-efficacy [66]. As a result, the three-factor EFA was conducted before moving to the following factorial analysis tests (confirmatory factor analysis).

The cutoff value to evaluate the items using the EFA was a loading factor of 0.32 or higher, which helped to determine if such items could be considered as part of a factor [67]. Although other researchers suggest different cutoff factor loadings such as Hinkin suggesting 0.40 [68] or Costello and Osborne suggesting 0.30 [69], this study used the 0.32 or higher suggested by Tabachnick and Fidell [67] to allow the data to define the most appropriate structure for the items assessing math self-efficacy, as there is not a global consensus about the right cutoff value for “good” factor loadings [70]. Using EFA is a common strategy when social science researchers are testing new or adapted instruments aiming to validate the appropriateness of the data collected with such instruments and the way that certain items group in multidimensional variables [71].

2.4. Confirmatory Factor Analysis

Following the EFA results, a confirmatory factor analysis (CFA) was conducted with the spring 2022 sample (n = 287). The CFA is used in social science research to analyze if the items hypothesized to measure a single underlying latent construct are occurring [72]. Unlike the EFA, the CFA explicitly tests a previously specified model aiming to determine if the items load onto a specific factor as expected by the researcher [73]. The latent factors were allowed to co-vary during the CFA, which is consistent with the oblique rotation that was previously used in the EFA [71,74].

The results of the CFA were tested with several indexes of model fit and significance of the paths between latent variables for the three subconstructs using the remaining items following the EFA results. These fit indexes included chi-square, which should be non-significant at the p > 0.05 value [75]; Comparative Fit Index (CFI), expecting acceptable values above 0.95 [72,76]; Tucker Lewis Index (TLI), expecting acceptable values above 0.95 [72,76]; and root mean square error of approximation (RMSEA), expecting values less than 0.08 for moderate fit, 0.05 for a good fit, or 0.01 for excellent fit [77]. All these fit indexes were used to evaluate the CFA results because each index provides a different facet of how the model fits the data. The fit indexes and chi-square statistics resulting from the CFA were adjusted using the Satorra–Bentler correction aiming to find a robust solution due to sample size and the nature of the data [78]. At the end of the CFA, some items were deleted one by one to conduct an additional CFA after each item deletion aiming to optimize model fit indexes and statistical significance in the CFA results [79]. These deleted items were selected from the ones that showed the lowest EFA factor loadings with the goal of reaching an acceptable model to assess math self-efficacy.

2.5. Internal Consistency

At the end of the factorial analyses, the reliability indexes for each of the three resulting math self-efficacy subconstructs were calculated using Cronbach’s Alpha (α), expecting values of 0.80 or higher to validate the internal consistency of each subconstruct [80]. All the quantitative analyses including the descriptive, factorial, and reliability analyses were performed using the statistical software R [81].

3. Results

The results of the EFA are presented in

Table 1. Loading factors resulting from the exploratory factor analysis.

Item	Factor Loadings
	Everyday Math Activities	Math-Related Courses	Math Problem Solving
Q1 *	0.59
Q2 *	0.48
Q3	0.70
Q4 *	0.62
Q5 *	0.64
Q6	0.69
Q7	0.76
Q8 *	0.64
Q9 *	0.49
Q10 *	0.55
Q11 *	0.64
Q12	0.65
Q13 *	0.38
Q14 *		0.72
Q15		0.80
Q16		0.82
Q17		0.77
Q18		0.74
Q19 *		0.80
Q21 *			0.62
Q22 *			0.51
Q23 *			0.55
Q24 *			0.55
Q25 *			0.58
Q26			0.67
Q27 *			0.88
Q28 *			0.64
Q29 *			0.61
Q30			0.77
Q31			0.75
Q32 *			0.69
Q33			0.77
Q34 *			0.63
Q35 *			0.38

Items indicated with a * were ultimately removed from the CFA to strengthen the model fit.

. Only one item (Q20: “In Starville, a calculation with Ө using two variables a & b is defined by a Ө b = a × (a + b). Therefore, 2 Ө 3 equals to __________”) was deleted for not reaching the minimum loading factor criterion of 0.32 to be considered part of a factor with a factor loading of 0.27 in the math problem-solving subconstruct. The remaining 34 items were grouped into three factors, explaining a total cumulative variance of 45%. These items presented a factorial load range from 0.38 to 0.88 (see Table 1), which is in line with the guidelines for optimal practice when performing exploratory factor analyses where a loading factor greater than 0.32 is recommended to interpret the EFA results [69].

3.1. Confirmatory Factor Analysis

The model fit resulting from the CFA using the spring 2022 sample (n = 287) with the 34 remaining items after the EFA (see Table 1) did not achieve acceptable standards. Hence, items with the lowest factor loadings from the EFA and the CFA results of every time that this test was conducted were systematically removed (see Table 1), aiming to achieve an acceptable model fit [72,76]. Although the EFA and the CFA factors loadings are usually similar, the CFA factor loadings change each time such that an additional CFA is conducted after deleting one item. Therefore, the deleted items are not always the ones with the lowest EFA factor loadings, but a combination of both factors, the new CFA loading factors and the EFA loading factors. According to this approach, 22 items were removed (see Table 1) from the three subconstructs of the adapted instrument to assess math self-efficacy: nine items from everyday math activities (Q1, Q2, Q4, Q5, Q8, Q9, Q10, Q11, and Q13; see

Table 2. Items deleted from the instrument to improve CFA model fit.

	Items
Q1	Determine how much interest you will pay on a $675 USD loan over two years at 14.75 % per year.
Q2	Calculate the amount of wood you need to buy to build two bookcases 2 m high and 1 m wide.
Q4	Calculate how much fabric you need to buy to make curtains for two equal square windows 1.5 m on each side.
Q5	Calculate how much interest you will earn on your savings account in 6 months, and analyze how that interest is calculated.
Q8	Determine the amount of tip corresponding to your share of a restaurant bill divided by eight people.
Q9	Set a monthly budget for yourself.
Q10	Balance your expenses and your weekly income without mistake.
Q11	Determine which of the two summer jobs is the best offer, one with a higher salary but no benefits, the other with a lower salary but this covers the costs of lodging, maintenance and transportation.
Q13	Calculate the quantities of a recipe for a dinner for 41 people when the original recipe is for 12 people.
Q14	Precalculus
Q19	Differential equations
Q21	Sally needs three pieces of cardboard for a school project. If the pieces are represented by rectangles A, B, and C, help Sally to arrange their areas in increasing order (assumes b > a).
Q22	The average of three numbers is 30. The fourth number is at least 10. What is the minimum average of the four numbers?
Q23	To build a table, Michele needs 4 pieces of wood that are 2.5 feet long for the legs. He wants to determine how much lumber he needs for 5 tables. Please reason as follows: 5 × (4 × 2.5) = (5 × 4) 2.5. What property of real numbers is Michele using?
Q24	Five points lie on a line. T is next to G. K is next to H. C is next to T. H is next to G. Determine the order of appearance of these five points on the line.
Q25	There are three numbers. The second is twice the first, and the first is a third of the other number. The sum of the three is 48. Find the largest number.
Q27	The hands of a clock make an obtuse angle when it marks ____ o’clock. (Consider that there is more than one correct answer).
Q28	Bridget buys a package containing 9-cent and 13-cent stamps for $2.65 USD. If there are 25 stamps in the package, how many 13-cent stamps are in the package?
Q29	Write an equation that expresses the condition “the product of two numbers R and S is one less than twice the sum of both”.
Q32	The formula to convert temperature from Celsius to Fahrenheit is F = $\frac{9}{5}$ C + 32. How many degrees Fahrenheit is 20 degrees Celsius?
Q34	If 3X – 2 = 16, what is the value of X?
Q35	Fred’s bill for some household items was $13.64 USD. If he paid with a $20 USD bill, how much change should they give him?

), two from math-related courses (Q14 and Q19; see Table 2), and eleven from math problem solving (Q21, Q22, Q23, Q24, Q25, Q27, Q28, Q29, Q32, Q34, and Q35; see Table 2).

In the end, the final CFA model was tested with 12 items, with four items for each of the three subconstructs, resulting in good overall fit indexes, with a Comparative Fit Index of 0.954, a Tucker Lewis Index of 0.950, and a root mean square error of approximation value of 0.056. The chi-square statistic was significant with a p < 0.001, which was an expected result due to the sample size (n = 287); as such, this value is still considered within the range of a good model fit. All these fit index values suggest that this model is appropriate for future data collection with similar populations in the Spanish language. The results of the CFA are shown in Figure 2 with acceptable loading factors ranging from 0.69 to 0.88.

3.2. Internal Consistency

The factor reliability analyses of the three math self-efficacy subconstructs are presented in

Table 3. Cronbach’s alpha results of the first and second sample.

Subconstruct	Cronbach’s Alpha
Everyday math activities	0.86
Math-related courses	0.88
Math problem solving	0.85

, showing acceptable Cronbach’s alpha values.

4. Discussion

The EFA was conducted with 35 items measuring self-efficacy in three subconstructs: (a) everyday math activities (13 items), (b) math-related courses (6 items), and (c) math problem solving (16 items). Conducting this EFA was an important step to provide further evidence that the MSES adaptation to the Spanish language clustered as the original MSES three-dimension model [43] and can replicate the model suggested by previous EFA [33].

Conducting this EFA was necessary due to the substitution of the items related to the original math-related courses in the data collection of this study. The eight items related to math-related courses were replaced by six college-level math courses that are part of the engineering curricula in most of the Mexican engineering majors [62]. These six items were not included in the previous EFA, hence the importance of conducting additional EFA including these new items. Additionally, to the analysis of these new items, the EFA was conducted aiming to evaluate the replicability of the structural solution obtained in previous factorial analyses [65].

Providing further evidence that the three MSES subconstructs are conserved as established by the original theory (everyday math activities, math-related courses, and math problem solving; see Table 1) was important for the validation process of the data collected with this instrument adaptation. These results are especially relevant since other authors have suggested that more subconstructs could be necessary for the MSES structure in their EFA validation tests. A study from Kranzler and Pajares [82] suggested that four factors would describe the MSES subconstructs in a better way, separating the math-related courses into two different subconstructs: one with the math and statistics courses, and the second one with courses where math is used but is not the main focus. Another study from Langenfeld and Pajares [83] suggested that five factors would facilitate a better assessment of math self-efficacy, separating the math-related courses subconstruct into two in a similar way to that of Kranzler and Pajares [82] and forming an additional problem-solving subconstruct with only two items.

The previous EFA results presented by Morán-Soto et al. [33] could not be fully replicated with this study EFA due to the elimination of the item Q20. Although one item did not reach a loading factor greater than 0.32 for any subconstruct, this EFA replication showed that the three-dimension model for measuring math self-efficacy was highly stable. These results provide certainty that the three-factor model proposed by the original version of the MSES and this study still work in a stable way for math self-efficacy data collection.

This study was able to conduct the confirmatory phase of the validation process trough CFA due to the positive results of the structural tests of the MSES adaptation to the Spanish language and Mexican engineering students’ context using exploratory analyses [66]. The confirmatory phase of the validation process of an instrument helps the researchers to test the belonging of each item to the factors established by the exploratory factor analyses [72]. Therefore, the validation process moved to a confirmatory modeling stage where the 34 remaining items were tested to confirm their belongness within each of the three previously established subconstructs (everyday math activities, math-related courses, and math problem solving, see Table 1).

The confirmatory phase of the validation process is stricter than the exploratory phase due to the existing hypothesis proposing that each item loads into a specific factor [72]. Therefore, deleting more items during the confirmatory process is usually necessary to fulfill an acceptable model fit for the CFA, losing items that are lacking a meaningful connection with the responses of the other items previously grouped into each factor [73]. During the confirmatory phase of the adaptation process of the MSES to the Spanish language and Mexican engineering context, 22 items were deleted before reaching an acceptable model fit for this test. Although losing 22 items is not ideal in validation tests, it is important to fulfill the minimum model fit for the CFA [76] to provide evidence that the items of each subconstruct are reliable in assessing all the dimensions of the variable; and the importance of these model fit indexes was clearly established by Hu and Bentler after adjusting this model fit from 0.90 [84] to 0.95 in their most recent analysis of the CFA results [76].The loss of these items resulted in a 12-item model assessing three subconstructs of math self-efficacy with four items for each subconstruct (everyday math activities, math-related courses, and math problem solving; see Figure 2). From the 22 items that were deleted following the CFA results, nine were from the everyday math activities, two from the math-related courses, and eleven from the math problem-solving subconstructs (see Table 1).

A detailed analysis of the deleted items after the CFA showed that most of the nine items deleted from the everyday math activities subconstruct were related to activities that are no longer part of current students’ common activities (see Table 2). These items were asking students to perform math calculations with activities that were common in students’ lifestyles several years ago, such as making your own bookshelves and curtains, or opening a savings account and investing your savings. The lack of experience in performing these kinds of activities could generate a disconnection from the math calculations of the activity, creating an extra cognitive charge for students that are not related to their confidence performing the calculations [85]. The elimination of this type of item was something expected, as the original version of the MSES was developed four decades ago, and students’ lifestyle have changed drastically during that period [86]. These findings suggest that some items of this MSES adaptation to the Mexican engineering context need to be updated. This way, students could be asked about activities that are related to their actual context and require the use of technologies, generating a better understanding of their math self-efficacy in their current daily activities [87].

Likewise, by analyzing the deleted items from the math-related courses subconstruct, there is a difference between the two math courses that were removed from the instrument adaptation and the four math courses that were kept in the instrument (see Table 2). The precalculus and differential equations courses are not a requirement for all engineering students in the Mexican engineering curricula [62]. For instances, differential equations are required only for certain majors that use these advanced math applications to solve problems such as electrical, mechatronics, and chemistry. Conversely, precalculus is only taken by students that did not reach the minimum requirement of math knowledge during the selection test; therefore, they need to take an extra semester of math courses to level up their math abilities. This context may help to understand the reasons why these two courses were not kept in the math-related courses subconstruct after the CFA, generating different responses from students that experienced the precalculus course and the ones that started in differential calculus as their first college math course, as well as students from different majors that are familiar with the topics that they will face in the differential equations course.

The subconstruct that lost more items during the CFA was the math problem solving with eleven items (see Table 2). The issue with these items was similar to what happened with the everyday math activities subconstruct where the items were outdated or out of context. This issue was greater for the math problem-solving items since engineering students usually work with advanced math topics where they used the help of technology such as calculators and software to solve all kinds of technological problems. Hence, the items presented by the original version of the MSES were not challenging enough, or they were perceived by the engineering students as different from the way they practice and develop their mathematical skills in their college math courses [28,88,89,90]. This idea is supported by the four items that this subconstruct kept, where students are asked about trigonometry applications, rates involving distances in a map, and equations that could be solved using a calculator or a model. These findings pinpoint the importance of the context when developing or adapting new instruments [48]. Proposing adaptations to the problem-solving items using college-level math problems in the context of engineering applications may present a better challenge for engineering students, resulting in a better assessment of this math self-efficacy subconstruct.

The results and discussion about the outdated items and the way that Mexican engineering students responded to these items helped to bring to light the importance of conducting thorough validation tests and considering cultural and context aspects that may affect the assessment of any variable. If researchers decide to mutilate an instrument using only one section of a multidimensional variable, then their results could be incomplete or biased due to the lack of enough information; likewise, if researchers take the decision to modify any item because they consider that it is outdated or out of context without collecting data from a sample of the target population to conduct the necessary tests, then their results and conclusions would be biased due to the lack of evidence about the way that the target population response to the items. Hence, conducting EFA and CFA will help researchers to enhance the trustworthiness of their findings, helping them to avoid rushed decisions such as deleting or modifying items without enough evidence coming from the data collected from the target population.

This study presents a thorough validation process using exploratory and confirmatory factor analyses. This detailed process may help researchers to avoid trustworthiness issues due to the lack of validation tests and may prevent them from rushing this validation process by conducting only basic content validity tests [36,91]. All the methodological steps followed by this validation process could help math educators to validate data collected with new or adapted instruments, especially if they want to assess math self-efficacy using the MSES without appropriate validation tests to adapt it to the culture, language, and context of their target population [37,53].

The adaptation of the MSES to the Mexican engineering context and Spanish language that is described in this study (see

Table 4. Validated items of the instrument after adjustment of the CFA model.

	Items
	Everyday math activities
Q3	Calcular los impuestos que tendrías que pagar en un año de trabajo dependiendo de tu ingreso anual (30% de ISR).
Q6	Calcular cuánto interés ganarás con tu cuenta de ahorros en 6 meses, y analizar cómo ese interés es calculado.
Q7	Estimar el costo total de tu mandado en tu cabeza conforme tomas los artículos.
Q12	Calcular cuánto ahorrarías si hay un 15% de descuento en un artículo que deseas comprar.
	Math-related courses
Q15	Cálculo Diferencial
Q16	Cálculo Integral
Q17	Cálculo Vectorial
Q18	Álgebra Lineal
	Math problem solving
Q26	En cierto triángulo, el lado más corto es de 6 pulgadas, el lado más largo es el doble de largo que el más corto, y el tercer lado es 3.4 pulgadas más corto que el lado más largo. ¿Cuál es la suma de los tres lados en pulgadas?
Q30	Formular el problema que debe resolverse para encontrar el número que se pide en la expresión “seis menos que el doble de 4 5/6”.
Q31	En cierto mapa, 7/8 de pulgada representan 200 millas. ¿Qué tan separadas están dos ciudades cuya distancia de separación en el mapa es de 3 1/2 pulgadas?
Q33	Resuelve 3 3/4 − 1/2 =

) highlights the importance of the possible context differences that different populations may face [92,93]. Most of these context and cultural issues are difficult to evaluate without the help of data collection and confirmatory factor analysis, which provides evidence of the belongingness of the items in the predicted factors based on how the target population perceives and responds to the items. In the end, performing confirmatory analysis provided certainty that future data collections using this instrument adaptation will be more accurate and will help researchers to better understand students’ experiences learning math in Spanish-speaking populations [72]. The final version of this validation process is presented in Table 4, showing the 12 items (four items for each of the three subconstructs) that have been confirmed by all the validation tests for collecting data to assess the math self-efficacy of Spanish-speaking populations in Mexico.

4.1. Implications for Practice

Developing better instruments to assess math self-efficacy in different contexts and languages will help researchers to improve the quality of their studies, facilitating the design of better strategies to motivate the interest of diverse populations in STEM majors and helping them to successfully complete their studies [94]. Having a sound assessment of engineering students’ math self-efficacy levels can contribute to a better understanding to how they learn, practice, and apply math topics to solve problems in STEM areas, which ultimately could help educators to generate better learning environments to develop students’ mathematical skills [95]. Additionally, a better understanding of the psychological aspects that may affect students’ math learning could have a positive impact on underrepresented groups in STEM-related majors [96,97], helping educators around the world to find ways to motivate them to pursue a STEM career. This way, groups with different genders and backgrounds could feel more welcome and understood, which ultimately may foster a greater diversity of people’s interest in learning and practicing mathematics to apply to STEM topics [23].

4.2. Limitations and Future Work

A limitation of this study was that the sample size was limited because this study was a follow-up of the exploratory and replication phase of the MSES adaptation to the Spanish language [33]. Therefore, the sample selection was limited to first-year students of the same institution and with similar characteristics that have not previously participated in the data collection, generating a smaller sample of students for both the fall 2021 and spring 2022 samples.

This study presents a thorough adaptation process of the MSES to the Spanish language and Mexican engineering students’ context. Although this process will facilitate data collection aiming to expand the literature about mathematics self-efficacy in Spanish-speaking communities, it is advisable to perform further cultural adaptations and validation tests if this instrument is going to be used to collect information in other Spanish-speaking countries in Latin America, Europe, or even among Spanish-speaking students in the United States. The adapted version of the MSES presented in this study will provide a good starting point to conduct other adaptation projects to other contexts and cultures, and the validation process followed in this study could be used as a guide to continue refining an instrument that will facilitate a thorough assessment of the math self-efficacy levels of students in different cultures and contexts.

5. Conclusions

This study highlights the importance of performing thorough validation tests before collecting data with new or adapted instruments. Conducting the EFA and replicating the results with additional EFA was a relevant step for providing sound evidence that the structure of the MSES adaptation to the Spanish language was stable with three dimensions as suggested by the original instrument 40 years ago. Performing some replicability tests aiming to confirm a stable structure of an instrument should be considered a necessary step by researchers looking for data validation, and this replication step should be considered before proposing a CFA for more reliable results.

This study presented evidence of the replicability of the instrument structure with three math self-efficacy subconstructs; hence, the CFA was conducted with the 34 remaining items. The 22 items eliminated according to the CFA results are clear evidence that instruments need to be adapted to the context of the population that is going to be assessed. The original MSES was developed four decades ago, and most of the items were outdated and out of context of current students’ lifestyles. Although the MSES is a well-recognized test used in math self-efficacy studies around the globe, this study suggests that many cultural and context adaptations need to be conducted before using it to assess math self-efficacy in future studies.

Although this study presented an exhaustive validation process through exploratory and confirmatory factor analyses, the data validation should be considered an ongoing process that needs to be conducted to address the needs and experiences of diverse populations. However, this detailed process can help researchers to avoid reliability issues due to the lack of validation tests and can prevent them from speeding up this validation process by performing only basic content validity tests. As the 12-item instrument (see Table 4) presented in this study went through a rigorous process using confirmatory and exploratory factor analyses to adapt the MSES to the Spanish language and Mexican engineering context, it could be used to collect data with similar populations, and it could also be used as a guide to continue with instrument refinement through further adaptations and validation tests. In the end, having better instruments to assess math self-efficacy in the Spanish language would help Latino researchers become more involved in the math education agenda with the goal of improving Latino students’ math learning.