Building a Model for Observing the Educational Practice of Mathematics Teachers

Abstract

This study relies on a theoretical perspective to provide a model that can be used to observe the educational practice of mathematics teachers. To this end, various existing observation instruments are studied, which, in the case of mathematics, aim to observe the teaching practice employed in the classroom, without considering the before and after of the implementation, which is what characterizes the professional task of teaching. Indicators that emerge from the teaching practice are proposed, together with the teacher’s knowledge and reflection constructs, based on three phases: for, in, and on the educational practice. As a result, understanding the educational practice of mathematics teachers would allow various educational stakeholders (teachers, administrators, instructors of teachers, and others) to focus their attention on what elements develop so as to improve how students are taught, and consequently learn, mathematics.

1. Introduction

Educational practice (EP) includes both the before and after of the actual practice of giving classes [1]. Consequently, its analysis cannot be undertaken solely by observing what happens in the classroom. A comprehensive approach would require taking into account elements that are external to the class proper [1], whose origin lies in other areas and levels, such as, for example, those related to class planning, or the reformulation of the class once it is taught. However, research in this regard is scarce and shows the need for clear approaches to a good EP that allows teachers to enhance their performance and transfer their practice to new situations [2]. This study contributes to advancing this line of research, providing a theoretical model that visualizes the elements that should be considered in the observation of EP, highlighting the importance of before and after the practice, which has not yet been considered in the current literature [2].

Given this need, we are faced with the task of designing an instrument for observing the educational practices of mathematics teachers, independent of the academic level they teach, the goal being to identify the fine differences and similarities in each educational level. In this way, we propose to answer questions such as: What elements should be considered when observing the EP of a mathematics teacher? How are these elements related or linked to one another? These are approached with the main goal of building a model that allows us to observe educational practice of mathematics teachers.

By determining this model, we can advance in the construction of an instrument to identify and showcase good EPs such that they can be used by other stakeholders in the educational system (such as educators of teachers, ministerial directors, and others) and new generations of teachers. Thus, this model intends to contribute to the construction of instruments for the observation of PE, for example, for teachers’ self-evaluation of professional competencies or for teachers’ peer evaluation, which can serve as guidelines in the education of teachers, both initial and continuing.

2. Frame of Reference

The elements that comprise the frame of reference of this study help to answer the question, “What elements characterize the EP of mathematics teachers?”, that is, those that aid in observing the EP of mathematics teachers, as indicated below.

First, we distinguish between practice and educational practice (EP), teaching practice and pedagogical practice. In doing so, we detail those elements that, according to the literature, describe a good EP, and specifically a good EP in mathematics, in order to continue defining the construct of a good mathematics teacher and its relationship with EP. We also consider reflection, which leads to the conceptualization of a reflective teacher, and to differentiating between reflection for, in and on practice. We also address components of EP based on the knowledge of the teacher and conclude by presenting the existing models in the literature that are used to observe the teaching practice.

2.1. Teaching, Pedagogical and Educational Practice

Within the field of education, the notions of teaching practice, educational practice, learning practice, and others, are sometimes used interchangeably. Therefore, for this study, we have to underscore their differences and take a theoretical stance in order to define them clearly.

Teaching practice [3] or teaching action [4] will be understood as that which is carried out in classrooms, in which a teacher does what is necessary to achieve learning in their students and where an effort is made to plan a class based on a context. Its analysis is complex, and its characteristics are [5]:

-

Multidimensionality, in relation to the existence of various simultaneous events.

-

Immediacy, relative to fast-moving events that are often difficult to understand, control and direct.

-

Unpredictability, meaning unexpected situations occur that shift the class away from the initial plan, which is favorable on certain occasions.

-

The notion of EP adopted herein is one that boils down not just to teaching, but that includes three phases: pre-active, interactive and post-active, each one corresponding to the actions of the teacher—for teaching—before, during and after the class [1,6]. These practices refer to the set of tasks that comprise the teacher’s work; that is, the actions that the teacher takes—before, during and after the teaching activity—to convey the content to be taught [4].

Pedagogical practice considers teaching and educational practice, but also encompasses the practice of collective work with other teachers, engaging with parents and practices with collaborators [7].

The diagram in Figure 1 illustrates the relationship between the three aforementioned constructs—educational, teaching and pedagogical practice—in which the latter encompasses the first two. This work will focus on EP, which considers the before and after of the classroom activity, and the classroom activity proper.

2.2. Towards the Ideal of Good Educational Practice

The goal of observing EP leads us to scrutinize its ideal form. However, as with the previous constructs, various uses are assigned to the terms effective teaching, good teaching, good class or model class, and best practices, so some clarification is needed in this regard.

A distinction between the constructs “good teaching” and “successful teaching” was made by [8]. The authors note that successful teaching focuses on achieving an expected objective, such as providing students with skills to obtain high scores on standardized tests. By contrast, good teaching is consistent with morally defensible and rationally sound principles. This means, for example, that if a child who is taught to kill then carries out this action, this is considered successful teaching, but obviously not good teaching. By contrast, teaching a child to approach modeling situations based on his age might not be successful, since he might approach them without a critical view of reality. Even so, that process can be considered good teaching. The following characteristics of the concept of “good practices” are attributed by [9]:

(a) It helps improve the performance of a process; (b) it reflects a systematized, documented and experienced occurrence; (c) it applies methods of excellence based on innovation; and (d) the category of “best practices” means it can be extrapolated to other contexts.

In this study, we consider the definition of “good teaching” proposed by [8] to extend this notion to that of a good EP, considering the intent of good teaching in the pre-active and post-active phases.

There are also certain considerations that must be taken into account in relation to best practices. These include the strict meaning of the term “best” practices, since no practice can be properly and entirely qualified as “best”. A “best universal practice” is impossible to find, at least not in the field of education, since it is utopian to think that every component of a practice is good. Given their human nature, there is no teacher who achieves that attribution in a compendium of best practices. In this sense, within our educational actions, there will be elements that deserve to be regarded as best educational practices. Every teacher satisfies, in one way or another, certain descriptors of best educational practices.

Another important consideration when studying best EP is that they be defined according to a specific context, meaning their “quality” depends on the conditions of the context in question [2]. For example, a best educational practice in a rural sector may imply assigning relevance to the tasks typical in the area, while in a vulnerable populated sector, teaching may focus on inculcating habits of desirable and lasting behavior. Therefore, there must be some consensus on objective criteria that can be used to identify these “best practices” in education, which must be based on the official regulations and guidelines of the context and research [2].

Best EP in the Mathematics Classroom

In the case of mathematics, in keeping with [10], a good EP includes problem solving, where certain care must be taken, since good teaching can lead students to better solve exercises, transforming them, over time, into easier, routine exercises, by definition [11]. This ability of the teacher to properly manage problem-solving is another element that the model we seek to develop should take into account.

Good teaching must include various aspects. Authors such as [12] mention that, in relation to the discussion in the classroom, a mathematics teacher with good EP: must provide enough waiting time for his students, must try to include as many students as necessary, must control the behavior, and must be mathematically correct (the teaching must be free from mathematical errors, it should be clear and complete). These indicators could be part of our model to observe the EP of a mathematics teacher.

The role of the mathematics teacher in the classroom can be observed by comparing discussions between peers and those led by the teacher. The latter are more effective for reaching higher levels of mathematical reasoning and include higher quality explanations and links between current and new knowledge [12].

In turn, regarding good teaching in mathematics, the National Council of Teachers of Mathematics of the United States [13] proposes eight practices of effective teaching that research indicates should be regular components in every mathematics class (Figure 2), which can be part of the indicators of the model designed in this study.

Since these elements must be present in all mathematics classes, they should also be present in planning and subsequent reflection activities; that is, in the entire mathematics EP.

2.3. Another Ideal: The Good Teacher

The study of EP can lead to a preliminary dilemma that needs clarification: What is the goal of observing EP: to find best practices or the educational practices of good teachers? The answer is not simple, since one might tend to select teachers classified as experts to see if they exhibit good EP, which in many cases do not meet the specified criteria [2]. In other words, it is important to realize that studying the educational practice of expert teachers does not imply having good EP. Even so, one criterion for selecting teachers to study educational practices is usually their expertise, so keeping their characteristics in mind is also relevant [2].

In research, the terms best practices, good teachers and teachers with successful practices sometimes allude to the same thing, exhibiting certain differences [14]. As [12] states, if we want mathematics teachers to improve their practices, the discussion should focus more on what effective teachers do, on what sets them apart from ineffective teachers, and less on the results of standardized assessments.

A valuable quantity of studies on the expertise of mathematics education from high-performing countries in education was analyzed by [15], specifically, China, Japan, Singapore, South Korea, and Taiwan. This aspect was subsequently addressed in PME 36 [16], where the first Research Forum was dedicated to the conceptualization and development of expertise in mathematics instruction in different countries [17]. In this PME, ref. [18] notes that a teacher of excellence is one who is capable of:

(i)

Selecting and, if necessary, adjusting the appropriate tasks, especially exploratory tasks, that actively involve students in mathematical work and stimulate them to develop their own strategies, concepts, and representations.

(ii)

Leading discussions that create opportunities to negotiate meaning, develop mathematical reasoning, and institutionalize knowledge.

Elsewhere, ref. [14] reviews the literature on these concepts, observing two types of characteristics to identify mathematics teachers of excellence: primary and secondary characteristics. The primary characteristics refer to aspects of teaching and knowledge, while the secondary ones consider professional experience [19]. The primary characteristics are:

(a)

Understanding the disciplinary contents of student learning and teaching strategies.

(b)

Selecting and, if necessary, adjusting the appropriate tasks—taking into consideration those of greater difficulty, especially exploratory tasks, which actively involve students in mathematical work, stimulating them to develop their own strategies, concepts and representations.

(c)

Leading discussions in class that create opportunities to negotiate meanings, develop mathematical reasoning, and institutionalize new knowledge, so students can relate content to different situations.

The secondary characteristics involve general aspects of professional experience, such as:

(a)

In-service teacher, with five or more years of classroom teaching experience.

(b)

Outstanding results in institutional and national evaluations (if applicable).

(c)

Recommended by peers and managers at the school.

(d)

Involvement in processes to update their knowledge, such as training courses, postgraduate studies (Master’s, PhD), participation in education research and innovation processes.

(e)

Having been nominated for or received prizes in teaching contests.

(f)

Outstanding performance by his/her students in local, national, and international evaluations.

Many of these readily “observable” characteristics in a teacher, such as experience and previous training, do little to explain the variation in the effectiveness of a teacher [20]. Therefore, much of the variability that cannot be explained can be attributed, in part, to the classroom practices of teachers [12].

2.4. Reflection and Practice

When investigating the criteria used to observe (or, in some cases, evaluate) EP, several elements can be identified. For example, university-level teaching in Spain [21] considers three dimensions: class planning, class implementation, and results (in terms of the objectives and of reviewing and improving the teaching). Reference [6] develops another proposal to evaluate the EP of teachers based on three dimensions: (1) the didactic thinking of the teacher and lesson planning; (2) the educational interaction inside the classroom; and (3) reflection on the results achieved. A common thread seems to guide the work in relation to the study of educational practices, and in which three dimensions emerge: the before, now and after of the teaching practice. The relationship between these three dimensions is interdependent, meaning each one affects and is affected by the others, which is why it is essential to address them holistically [6].

Accordingly, ref. [22] suggest that reflecting for, in and on the practice can be an element that helps to define a good EP. In other words, a good EP is evident in a teacher who acts professionally, where reflection becomes important in their professional development [23,24]. In general terms, good professional practices depend less on the scientific and technical knowledge available for immediate use than on the ability to reflect before making a decision [1].

An approach to the training of professionals is carried out by [22], aimed at developing this capacity for “reflection-in-and-on-action”, which is the most appropriate, in his opinion, to help future and in-service professionals to confidently and proficiently confront the issues and problems that, in a complex and unpredictable way, arise in everyday practice (issues that characterize the complexity of teaching practice). These ideas of reflection in and on practice have had a great impact on education, where the notions of reflection for, in and on practice stand out. These three concepts are related to EP through its pre-active, active and post-active phases, as shown in Figure 3.

In the pre-active phase, the teacher is expected to reflect on the practice, inquiring about potential errors and problems that may arise in the classroom. In the active phase, reflection in action is expected, where the teacher brings their expertise to bear in order to cope with contingencies. In the post-active phase, the teacher can make decisions for another intervention or for subsequent classes. Given the relationship between professionalism and reflection [23,24], observing a good EP in mathematics requires observing the characteristics of a reflective mathematics teacher. In this regard, ref. [25] posit that reflective mathematics teachers are capable of:

1.

Perceiving situations in the environment that require personal action on their part;

2.

Distancing themselves from situations in their environment so as to analyze their elements;

3.

Specifying and eliminating elements that condition how these situations are perceived, including those derived from their beliefs or implicit schemes;

4.

Resorting to other sources to find other ways of interpreting and responding to situations;

5.

Forcing themselves outside their comfort zone;

6.

Becoming aware of the complexity of their students’ practice and learning;

7.

Adapting their practical action to the conditions of the context;

8.

Being open towards mathematics and willing to transform their conceptions of it;

9.

Being aware of the complexity of learning mathematical knowledge.

While it seems that these skills cannot be fully demonstrated from an observational standpoint, it is feasible to articulate them by observing the EP, through the self-assessment process that the teacher has to carry out in the three phases of their EP involving the indicators established by the model proposed.

2.5. Teacher Knowledge and Practice

A teacher’s practice is contingent upon the knowledge they possess. Therefore, it is worth asking if there is a relationship between a good practice and the professional knowledge of the teacher who experiences it.

Various authors have alluded to this relationship from varying perspectives. One of them, ref. [26], notes that as we learn more about teaching (and good teaching in particular), it will be possible to recognize what baseline knowledge a teacher must have in order to teach. This way, we can be sure that expert teachers (with full knowledge of the content to be taught) are capable of defining, describing and reproducing good teaching.

There are a wide variety of theoretical models to conceptualize the professional knowledge of teachers, promoted mainly in the wake of Shulman’s [27] contributions. In the area of mathematical education, these include Mathematical Knowledge for Teaching (MKT) [28] and the Mathematics Teacher’s Specialized Knowledge (MTSK) model. These models can also be used to analyze the teacher’s practice. In this regard, ref. [29] state that there is a significant, strong and positive association between the domains of MKT and the mathematical quality of instruction (MQI). They are composed of several dimensions that characterize the rigor and richness of mathematics in the classroom, including the presence or absence of mathematical errors, mathematical explanation and justification, and the use of mathematical representation [29]. The MQI observation protocol is intended to capture “the disciplinary integrity of the mathematics presented to students, including the degree to which teacher errors occur during mathematics class”. It also captures, for example, precision in mathematical language and whether correct generalizations are being made in class. References [30,31] mention that the MTSK model can be used to identify good teaching practices based on solid repertoires of knowledge.

For this study, the MTSK model [32] is considered, given its specificity to mathematics, and its contribution as a model for observing the specialized knowledge of teachers who teach mathematics. In our proposal, the MTSK model provides an overview of the teaching knowledge that a teacher has, so that indicators can be selected for observing EP, which can be classified according to the different knowledge domains presented by this model. This makes it necessary to specify its main elements below.

The conceptualization of the MTSK (Figure 4) encompasses two large domains: Mathematical Knowledge (MK) and Pedagogical Content Knowledge (PCK). The teacher’s beliefs are also present, which play a central role in their entire practice, and are thus a central component of their knowledge.

2.6. Indicators for Observing EP in Mathematics Teachers

In order to construct a model to observe the EP of mathematics teachers, it is necessary to consider what other investigations suggest in this regard. Unfortunately, in the field of mathematics education, there are no studies that seek to observe EP in all its phases (before, during and after); rather, they observe the classroom in situ, that is, during the teaching practice. Even so, the models that underlie the observation of this practice can provide baseline elements for proposing a model to observe the EP of mathematics teachers. Accordingly, we present some background in this area and then, as a result of this search and of the foregoing, we propose a model that can be used to observe the EP of mathematics teachers.

Given the non-existence of explicit models that guide the observation of the teaching practice of mathematics teachers, there are instead instruments to observe EP. The dimensions present in these observation instruments have been identified, which allows us to deduce the underlying model. In this regard, ref. [34] conducted a thorough review, presenting a detailed summary of the elements that are present in the instruments for observing mathematics classes, which we complement with instruments designed in light of the context of this study, in Chile or those supported by the MTSK model, as shown in

Table 1. Dimensions present in instruments for observing mathematics.

Instrument	Purpose	Dimensions
MateO [35]	Observe mathematics classes, provide teacher feedback and design professional development plans for teachers.	- Organization and planning - Classroom interactions - Presentation of the mathematical content - Mathematical management
Guideline for the Observation of Good Teaching Practices (PBPD) [36]	Identify episodes during class in which good teaching practices are observed.	- Strategies for teaching mathematics - Classroom environment
POEMat. ES [37]	Observe the teaching practices of secondary education mathematics teachers taped on video.	- Mathematical content - Didactics of mathematical content - Classroom management

.

As stated by [34], we see that there is no consensus on the dimensions underlying the existing proposals for observing the teaching practices of mathematics teachers. Even so, we do note that the main purpose is to holistically observe the interactions between the teacher, the mathematical content, and the students. Therefore, the interaction between the teacher and mathematical content (related to the teacher’s knowledge of the discipline) is a key dimension in all the proposals. Many of them also emphasize the teacher–student interaction based on the management of or environment in the classroom. In addition, dimensions related to the implementation of teaching or work with students and mathematics are considered by connecting classroom practices with mathematics (related to the teacher’s didactic knowledge). Thus, based on the foregoing, we can identify four key dimensions: D1: content knowledge; D2: pedagogical knowledge; D3: climate or environment for learning, and D4: reflection. These will be considered in the proposed model that is presented in the results.

3. Research Method

This research follows a mixed approach, whose qualitative component is developed during the process of constructing the model for observing EP in mathematics, while the quantitative component is developed by validating the content of the resulting model through the judgment of experts in this area. The study consists of two phases. In the first, a qualitative approach is used to specify the proposal for a model to observe EP in mathematics, which includes its dimensions and indicators.

Before the content is validated using the judgment of experts, the model (with its dimensions and indicators) is evaluated by three experts in didactics, two of whom are experts in the MTSK model, and the third is an expert in designing instruments to observe the mathematics classroom. In this evaluation, the experts unanimously express the complexity of observing the fourth dimension, reflection, due to the difficulties involved in observing it before, during and after the practice, without having various data collection instruments. They also make suggestions to improve the indicators. The proposed model that arises from the reformulations carried out based on this process is then validated using an expert judgment analysis.

Once this version of the model to observe EP in mathematics was established, it was subjected to a content validation process through expert judgment. A series of selection criteria were established for the experts, according to their training and area of specialization in mathematics education. These criteria were: (1) PhD with more than 5 years of experience in the area; (2) specialists in the MTSK model or in the construction of teaching observation models; (3) be Spanish-speaking (Spanish and/or Portuguese). With this, we hope to avoid selection bias in the sample of experts consulted in this research.

A total of 15 experts who met the established criteria were identified, who were contacted electronically to participate in the study, establishing a minimum of eight experts to ensure the convergence of the statistical procedures used in the analysis of results (asymptotic confidence intervals for Aiken’s V [38]). These eight experts come from different countries (one from Colombia, four from Chile, two from Spain and one from Mexico), their areas of research are the didactics of mathematics, all of them are specialists in the MTSK model, and some have worked on the design of classroom observation guidelines.

After selecting the group of collaborating experts, each of them was sent a document electronically explaining the purpose of the study, which included a guide to assess the different indicators considered in the model developed, as per the components and levels described in Table 1.

Based on the quantitative paradigm, to ensure greater objectivity in the model, an expert validation process was carried out in which different researchers in the field were asked about the clarity, consistency and relevance of each indicator, on a scale from 1 to 4, as illustrated and detailed in

Table 2. Criteria and levels for the expert assessment.

Criterion/Description	Level	Indicator
Clarity The indicator is easy to understand, meaning its syntactic and semantics are adequate.	1. Does not meet the criterion	The indicator is not clear.
	2. Low	The indicator requires several modifications or one very large modification in terms of the words used and their meaning, or their order.
	3. Moderate	A very specific modification of some of the terms of the indicator is required.
	4. High	The indicator is clear, with adequate semantics and syntax.
Consistency The indicator is logically related to the dimension it is measuring.	1. Does not meet the criterion	The indicator has no logical relationship to the dimension.
	2. Low	The indicator is tangentially related to the dimension.
	3. Moderate	The indicator is moderately related to the dimension it is measuring.
	4. High	The indicator is completely related to the dimension it is measuring.
Relevance The indicator is essential or important, that is, it must be included.	1. Does not meet the criterion	The indicator can be eliminated without affecting the measurement of the dimension.
	2. Low	The indicator has some relevance, but another indicator may already include what it measures.
	3. Moderate	The indicator is relatively important.
	4. High	The indicator is highly relevant and must be included.
Sufficiency The indicators that belong to the same dimension are enough to measure it.	1. Does not meet the criterion	The indicator is not sufficient to measure the dimension.
	2. Low	The indicators measure some aspect of the dimension, but they do not cover it entirely.
	3. Moderate	Some indicators must be enhanced in order to fully assess the dimension.
	4. High	The indicators are sufficient to measure the dimension.

.

Regarding Table 2, the experts evaluated the clarity, consistency, and relevance of each of the indicators considered in the model, while the sufficiency evaluated the degree to which the indicators considered are sufficient to describe the dimension for which they were proposed.

Finally, regarding the analyses conducted of the evaluations resulting from the expert judgment, we began with an exploration of each dimension, in which we used the average scores of the indicators that comprise each one. For example, for dimension 1 (D1), seven indicators are considered, so we initially analyzed the average score between them, as with dimensions 2 and 3 (D2 and D3). We subsequently used the Aiken V index [38] to quantify the degree of agreement or concordance between the scores assigned by the experts. In practice, ref. [38] recommends as adequate or acceptable point values of V higher than 0.7, whereas when considering estimates in 95% confidence intervals [39], thresholds greater than 0.5 are acceptable at the lower limits [40]. These results are presented mainly through the use of tabular and graphic representations that allow describing the trends of the assessments based on the various dimensions and criteria considered.

4. Results

The findings are presented in three stages. First, the model for observing EP in mathematics is shown, which includes the dimensions and descriptors. We then present the analysis of the expert judgment and explain the degree of to which the experts agreed in their assessments. The changes made to the original version are explained with examples. Finally, the model used to observe EP that results from the study is presented.

4.1. Result 1, Initial Proposal for a Model to Observe the EP of Mathematics Teachers

Based on the background provided, in order to observe EP in mathematics, the diagram in Figure 5 is considered, which takes three aspects into account: (a) the existing models for observing the teaching practice, (b) the intention to look not only at the interactive phase, but the pre-active and post-active phases as well, and (c) the relationship and relevance of the characteristics of reflective mathematics teachers and good teachers.

The goal of the study is to provide a model that can be used to observe the educational practice of mathematics teachers. To do this, we propose dimensions that emerge from those identified in the previous section aimed at observing the teaching practice. The dimensions considered involve “classroom climate or learning environment” and “knowledge”, with the latter materializing through the MTSK model and its “mathematical knowledge” and “didactic knowledge” dimensions.

These dimensions seem insufficient to observe the before and after of the practice, so it is necessary to complement the proposal with the reflection construct through Schön’s reflection for, in and on the practice [22], as materialized in the characterization of a reflective teacher proposed by [25]. These dimensions are combined in a model that has been called the MathTEP Model, an acronym for Mathematics Teacher Educational Practice (the English abbreviations are used for ease of understanding), which is shown in Figure 6.

The model revolves around the EP, and reinforces the relationship between reflection for, in and on the practice. In addition, it proposes four dimensions for observing EP. Two of them are at the same level (in the middle) and have to do with the specialized knowledge of the mathematics teacher. The model also considers the “environment for learning” dimension, which allows connecting elements of classroom management. Finally, the model proposes a fourth dimension, reflection, given its role in the relationship between education and professionalism. The dimensions of the model are detailed below.

Dimension 1—mathematical knowledge: This extension focuses on the disciplinary knowledge that the teacher must bring to bear in their educational practice, considering its entire breadth, meaning the knowledge of mathematical concepts, phenomenology, representation registers, and more. It likewise involves bringing to bear knowledge that explains the students’ reasoning processes and the connections to expand upon or simplify content.

Dimension 2—pedagogical knowledge: This dimension focuses on knowledge related to teaching methods, using mistakes, learning difficulties, and the creative use of resources. It also considers the way the teacher structures the class and collaborative work, as well as the curricular elements, among other aspects.

Dimension 3—environment conducive to learning: This dimension focuses on observing whether the teacher considers recognition and stimulus for and in their students, their way of participating in the teaching–learning process and the spontaneous participation of the students. This dimension also includes the extent to which the teacher considers the non-verbal communication and diversity of the students.

Dimension 4—reflection for, in and on the practice: This dimension considers the various skills a reflective teacher must possess, including being able to: perceive environmental situations, identify and eliminate conditioning elements (such as beliefs), resort to external sources, go beyond the limits of their comfort zone, and be aware of the complexity of the practice.

4.2. Result 2, Analysis of the Expert Judgment

As mentioned in the methods section, before validating the content using an expert judgment process, the model (with its dimensions and indicators) was evaluated by three experts in didactics (pilot application), where they conducted an in-depth review of the presentation and content of the proposal. Based on this evaluation, it was decided to consider an analysis of three dimensions for this validation process: D1, content knowledge; D2, didactic knowledge; and D3, learning environment.

We begin by exploring the distribution of the mean scores assigned by the experts to each dimension (D1, D2 and D3), differentiating based on the four criteria considered (Table 2). In this regard, as shown in Figure 7, note that the median of the global evaluations for each dimension exceeds 3.5 points in most cases (except for D1, on clarity, which is 3.3 points). This reflects a high degree of agreement between the judgment of the experts involved.

Figure 7 shows that the highest-rated criteria were those related to relevance (green) and consistency (red), with at least 75% of the mean scores exceeding 3.3 points for consistency and 3.7 points for relevance, with the maximum score being 4 points. Regarding sufficiency, the minimum score was 3 points, while at least 40% of the judges gave the first two dimensions (D1 and D2) 4 points, and more than 70% assigned this maximum score to dimension 3 (D3). In turn, with regard to clarity, there is greater variability in the responses, although in every case, the minimum mean scores were close to 3 points. In decreasing order, the median scores were 3.7 (D3), 3.5 (D2), and 3.3 (D1) points.

In terms of dimensions, we see that the scores were higher with respect to the four criteria in dimension 3 (D3), while the other two (D1 and D2) reflect the same pattern for sufficiency, but in terms of clarity, consistency and relevance, the scores in the second dimension were higher than the first. However, in general terms, in every case, the 25th percentile is at least 3 points, meaning that the majority of experts consulted (at least 75%) have a favorable view of the indicators in each dimension.

Subsequently, we determined the Aiken V index for each of the indicators that comprise the three dimensions in order to quantify the degree of agreement between the experts’ assessments, and to identify those indicators that need some adjustment based on the comments included with the assessment.

Table 3. Indices and CI (95%) for Aiken’s V by dimension and evaluation criterion.

Dimension	Indicator	Clarity		Consistency		Relevance		Sufficiency
		V	CI (95%)	V	CI (95%)	V	CI (95%)	V	CI (95%)
D1	1.1	0.63 *	[0.43–0.79]	0.67 *	[0.47–0.82]	0.96	[0.8–0.99]	0.83	[0.64–0.93]
	1.2	0.67 *	[0.47–0.82]	0.83	[0.64–0.93]	0.96	[0.8–0.99]
	1.3	0.88	[0.69–0.96]	1	[0.86–1]	1	[0.86–1]
	1.4	0.79	[0.6–0.91]	1	[0.86–1]	1	[0.86–1]
	1.5	0.75 *	[0.55–0.88]	1	[0.86–1]	1	[0.86–1]
	1.6	0.83	[0.64–0.93]	0.92	[0.74–0.98]	1	[0.86–1]
	1.7	0.79	[0.6–0.91]	0.92	[0.74–0.98]	0.92	[0.74–0.98]
D2	2.1	0.92	[0.74–0.98]	0.96	[0.8–0.99]	1	[0.86–1]	0.83	[0.64–0.93]
	2.2	0.83	[0.64–0.93]	1	[0.86–1]	1	[0.86–1]
	2.3	0.88	[0.69–0.96]	0.88	[0.69–0.96]	1	[0.86–1]
	2.4	0.71 *	[0.51–0.85]	0.92	[0.74–0.98]	1	[0.86–1]
	2.5	0.83	[0.64–0.93]	0.96	[0.8–0.99]	0.92	[0.74–0.98]
	2.6	0.92	[0.74–0.98]	1	[0.86–1]	1	[0.86–1]
D3	3.1	0.92	[0.74–0.98]	0.96	[0.8–0.99]	0.96	[0.8–0.99]	0.96	[0.8–0.99]
	3.2	0.88	[0.69–0.96]	0.96	[0.8–0.99]	1	[0.86–1]
	3.3	0.79	[0.6–0.91]	0.96	[0.8–0.99]	1	[0.86–1]
	3.4	0.83	[0.64–0.93]	0.88	[0.69–0.96]	0.88	[0.69–0.96]
	3.5	0.96	[0.8–0.99]	1	[0.86–1]	1	[0.86–1]
	3.6	0.88	[0.69–0.96]	1	[0.86–1]	1	[0.86–1]
	3.7	0.96	[0.8–0.99]	0.96	[0.8–0.99]	1	[0.86–1]

shows the point values of the V index, along with their respective 95% asymmetric confidence interval (CI), for the four criteria considered.

Table 3 shows that, in general, the V index was higher than the recommended limit of 0.7 in most of the indicators analyzed according to all the criteria considered, except in five cases (shown with an asterisk in Table 3), where this value was close to (above or below) the cutoff value. This is reinforced when observing the CIs associated with these indices, which only in these five cases had a lower limit close to 0.5. We thus view the high degree of agreement between the expert evaluations as a positive.

Regarding the criteria considered, the assessments of the clarity of the indicators yielded the greatest variability and is where we found four cases with decreased indices. These correspond to the indicators 1.1 “Relate to everyday life (phenomenology)”, 1.2 “Explicit reasoning processes” and 1.5 “Explicit mathematical concepts at the end of the session”, of the first dimension related to mathematical knowledge, and 2.4 “Formative/process evaluation” of the second dimension, which considers didactic knowledge. Therefore, we recommended reviewing these cases and implementing possible modifications to their content based on the comments included in the report associated with each expert’s assessment. Regarding the consistency of the indicators, we find the remaining case with an index below the expected value, 1.1 of D1, which coincides with one of the highlights in the clarity criterion, while for the other indicators, this indicator yielded a good result. As for the relevance criterion, it is the one with the highest degree of agreement, where the indicator with the lowest score corresponds to 3.4 of D3, with 0.88 and a lower CI limit of 0.69. This highlights the adequate agreement in the expert evaluations when considering the relevance of the indicators proposed for each case. Finally, regarding the sufficiency criterion, the indices are acceptable for the three dimensions analyzed, which allows us to conclude that the number of indicators considered as descriptors of each dimension are sufficient in the model developed.

In what follows, we detail the adjustments and modifications made to the dimensions and indicators of the model based on the results of the expert judgment described above.

4.3. Result 3, Model to Observe EP in Mathematics

When analyzing the results produced by the expert judgment, we observed that the indicators performed well in terms of relevance and coherence, but that clarity needed to be reviewed in many cases. Initially, its analysis led us to reformulate those indicators where the Aiken V index was less than 0.83. Subsequently, based on the evaluators’ observations, it was possible to show that those indicators in the “environment conducive to learning” dimension have to do with the “didactic knowledge” dimension, from the point of view of the MTSK model, within the KFML domain in the category “emotional aspects of mathematics learning”. The analysis carried out led us to reduce the dimensions from three to two, materializing in the dimensions and indicators shown below.

DIMENSION

D.1 Mathematical knowledge

•

D1.1 Relate to the everyday (KoT): The teacher coherently relates the content of the class to phenomena or occurrences that give meaning to the mathematical object.

•

D1.2 Explicit reasoning processes (KPM): The teacher explains their mathematical reasoning behind the topics,

•

D1.3 Diversity of representations (KoT): The teacher uses different semiotic representations over the course of the mathematical activity and establishes relationships between them.

•

D1.4 Connections (KSM): The teacher connects class contents with others that may be simpler or more complex, transversal or auxiliary, in a pertinent, coherent and explicit way.

•

D1.5 Explanation of mathematical concepts (KoT): The teacher provides a mathematical explanation of the content that considers definitions, procedures and properties.

•

D1.6 Formal language (KPM): The teacher uses symbols and formal and technical mathematical language appropriate to the grade level.

•

D1.7 Specific practices of the mathematical task (KPM): The teacher uses specific practices of the mathematical task (for example, mathematical modeling, mathematical argumentation, problem solving).

DIMENSION

D.2 Pedagogical Knowledge

•

D2.1 Use of the mathematical output of students (KMT): The teacher uses the mathematical output (written or verbal responses) of the students, interpreting and using their reasoning, to make decisions about the teaching process.

•

D2.2 Use of errors and difficulties (KFLM) (KMT): The teacher addresses the errors (or mathematical difficulties) of the students as a strategy to enhance their learning process, guiding them towards the expected learning.

•

D2.3 Use of resources (KMT): The teacher shows knowledge of the limitations and scope of the resources (manipulatives, digital or others) that they use in order to teach the content of the class, in keeping with the learning objective.

•

D2.4 References and theoretical perspectives used to teach the content (KMT): During class, the teacher presents different theoretical perspectives for teaching the mathematical content, such as types of representation, conceptual/procedural aspects, the Van Hiele model, concrete, pictorial, and symbolic.

•

D2.5 Relationship with previous or subsequent content (KMLS): The teacher ascertains the students’ knowledge in order to relate the new content and other content from later years.

•

D2.6 Language and tasks appropriate to the grade level, curriculum, content and objective of the class (KMLS): The teacher proposes language and mathematical tasks that are clear and appropriate to the school level, curriculum, content, and objective of the class.

•

D2.7 Recognition and encouragement (KFML): The teacher provides feedback, stimulating the students to work, promoting a positive attitude in the students, one of effort and perseverance towards learning mathematics.

•

D2.8 Teacher-student link (KFML): The teacher is empathetic and available to the students, exhibits a flexible and democratic stance, and is able to recognize and manage the emotions of the students in relation to learning mathematics.

•

D2.9 Classroom climate (KFML): The teacher creates an environment of respect and trust that motivates and promotes the learning of mathematics, and encourages spontaneous, collaborative, and active participation by the students.

•

D2.10 Non-verbal and body language (KMT): The teacher uses gestures, tone of voice, or the body as a scaffold for learning mathematics.

•

D2.11 Time for participation and learning (KFML): The teacher efficiently uses the time to teach mathematics, adapting to the learning pace of the students.

•

D2.12 Managing context and diversity (KFML): The teacher considers the context of the class and its relevance to the teaching and learning of mathematics

•

D2.13 Collaborative work (KMT): By managing the mathematical tasks, the teacher promotes collaborative work in the classroom to engage in learning mathematics.

•

D2.14 Selection of tasks for the development of mathematical practice (KMT): The teacher considers mathematical tasks, taking into account the specific practices of the mathematical task (for example, mathematical modeling, mathematical argumentation, problem solving).

Now, considering that in this study, while we were unable to investigate how to observe this third dimension, we maintain that it can be evaluated through a self-assessment carried out by the teacher before, during, and after the practice. This would have to be the subject of discussion and analysis in subsequent studies. Therefore, the model of Figure 8 shows how the three dimensions revolve around the EP at its three moments: before, during, and after the practice.

In the model, the didactic and mathematical knowledge of the mathematics teacher have been placed as baseline dimensions and at the same level, thus emphasizing their importance in the teaching and learning process of this subject. At the top of the model is the reflection dimension, as this element can be used to articulate the three moments of PE, giving strength to the ideas of [22] of looking at practice through reflection before, during, and after the practice, as a powerful and necessary implement in the education professional’s toolkit.

5. Discussion and Conclusions

We have proposed a model for observing the educational practice of mathematics teachers. The result is a model whose dimensions revolve around the teacher’s knowledge and reflection. This model is a first approach for observing the educational practices of mathematics teachers. It reflects the results of a study where the time before, during, and after the class becomes relevant, and in which the teacher’s reflection and knowledge play a key role.

The model identifies the role that some elements already recognized by other authors play in the EP of mathematics teachers. These include the role that problem solving plays in EP [10,11], from the point of view of both the mathematical knowledge of the content dimension (D1.7) and the pedagogical knowledge dimension (D2.14), where the teacher uses and exhibits specific practices of the mathematical task (for example, mathematical modeling, mathematical argumentation, problem solving) and considers mathematical tasks by taking into account the practices specific to the mathematical task.

Along the same lines, the model considers the ideas of [12] on EP in mathematics education by recognizing elements of the content knowledge dimension, such as being clear and complete in the way mathematical ideas are conveyed (D1.6). Moreover, in keeping with this author, the model also highlights aspects of the dimension of pedagogical knowledge, such as the importance of the time that the teacher must provide in the classroom (D2.11), the relevance of managing the context and diversity (D2.12), the teacher’s role in generating discussions between peers and those guided by the teacher (D2.9), and the teacher’s role in linking new knowledge with the students’ current knowledge (D2.5).

However, the model also highlights the eight actions of effective teaching in mathematics classes proposed by [13]. It thus accounts for the importance of establishing mathematical goals focused on learning, by considering that the teacher must present clear mathematical tasks suited to the grade level, the curriculum, the content, and objective of the class (D2.6). It also considers the implementation of tasks that promote reasoning and problem solving and the development of procedural fluency based on conceptual understanding, where the teacher presents mathematical tasks while taking into account the specific practices of mathematical work, such as mathematical modeling, mathematical reasoning and argumentation, and problem solving (D2.14). The model relies on using and linking mathematical representations, both from the dimension of mathematical knowledge (D1.3) and from the dimension of didactic knowledge (D2.4). The model also considers fostering meaningful mathematical discourse by generating an environment of respect and trust that motivates and promotes the learning of mathematics and results in the spontaneous, collaborative, and active participation of the students (D2.9). The model considers the posing of deliberate questions and obtaining and using evidence of student thinking by the teacher, who must use the mathematical output (written or verbal responses) of the students, interpreting and using their reasoning, to make decisions involving the teaching process (D2.1). Encouraging a productive effort in learning mathematics is another element considered in the model, this one based on recognition and encouragement (D2.7), where the teacher gives feedback to stimulate the students in their work, promoting a positive attitude in the students, one of effort and perseverance towards learning mathematics.

A further consideration of the model is the conception of a mathematics teacher of excellence [18], meaning one who selects and, if necessary, adjusts the appropriate tasks (D2.6), actively involves students in mathematical work (D2.9 and D2.13), and stimulates them to develop their own strategies, concepts, and representations (D2.4 and D2.13). In addition, this teacher should engage in discussions that create opportunities to negotiate, develop mathematical reasoning, and institutionalize knowledge (D2.9 and D2.14).

The model also considers the primary characteristics of an expert mathematics teacher—proposed by [14]—where the teacher must: understand the disciplinary contents (D1) and student learning and teaching strategies (D2); be able to select and, if necessary, adjust the appropriate tasks, actively involve students in mathematical work, stimulating them to develop their own strategies, concepts and representations (D2.9 and D2.14); and be able to engage in class discussions that create opportunities to negotiate meanings, develop mathematical reasoning, and institutionalize new knowledge for relating the content to different situations (D2.9 and D2.14).

The role that the MTSK theoretical model [32] played in building the dimensions and indicators of the MathEP model is evident. The two large domains of knowledge (mathematical and didactic knowledge) and their categories were key to defining two of the dimensions and their indicators. This also allows differentiating the dimensions of this model with those underlying the classroom observation instruments that have been studied [36,37,38], which often separate the didactic elements into various components. In this sense, this study contributes to mathematics education because its findings are based mainly on a robust theoretical reference of mathematics education, namely, the MTSK model.

About the limitations of this study, we highlight the difficulty to delve into the teacher’s reflection dimension, due to the complexity of being able to observe it during practice. It has not been possible to establish indicators that lead to interpreting teachers’ reflections abilities while carrying out their work in the classroom, since these are not usually expressed explicitly (which can emerge from a subsequent interview and writing questions related to the class implementation, raising an appreciation for the comments made by teachers at the time of being interviewed).

Even so, we maintain that this dimension can be evaluated through a self-assessment carried out by the teacher before, during, and after the practice. This would have to be the subject of discussion and analysis in subsequent studies, to explore how they respond to this reflective dimension. We stress the need to continue exploring this aspect, given the relationship between professionalism and reflection [23,24], where observing a good EP in mathematics requires observing the characteristics of a reflective mathematics’ teacher, which seems not to be directly observable in the classroom, as we mentioned earlier. In this line, it is proposed as a projection of the study to investigate how to materialize the observation of the nine skills of a reflective mathematics teacher put forth by [25].

A study may also be proposed to observe what other elements can be incorporated into the model, especially at the pre-active and post-active stages, in order to move towards a comprehensive and complete instrument for observing the EP of mathematics teachers.

This study contains various implications for mathematics education. The most relevant is that the construction of the model described will contribute to advancing the construction of an instrument to identify good EP and give visibility to these, so that they serve other agents of the educational system (such as teacher trainers, ministerial directors, among others) and to the new generations of teachers. It is intended to contribute to the construction of instruments for the observation of EP, for example, for teachers’ self-assessment of professional competences or for teachers’ co-evaluation among peers, which can serve as guidelines in the education of teachers (both initial and continuing). This model would allow different stakeholders in the educational system (teachers, managers, educators of teachers) to focus on what elements to develop in order to improve how students are taught, and consequently learn, mathematics. Finally, combining the “environment for learning” dimension with the “teacher’s pedagogical knowledge” based on the MTSK model raises another implication of the study by opening a discussion into what our takeaways should be when observing a mathematics class (and the moments before and after).

On the other hand, although the context of the pandemic did not affect the development of the research, since it was possible to carry out the theoretical and experimental study without close contact with people, it can be noted that, due to it, the use of technological tools for data collection and communication with experts was an aspect used positively in this study. Finally, the model built in this study allows observing the EP of the mathematics teacher regardless of the type of classes implemented (face-to-face, virtual, or hybrid). Consequently, we can affirm that the model presented is not influenced by the context where the practice is situated and the agents that intervene in it. This represents a strength of the model, because in the face of emerging scenarios such as the context of virtually for teaching that has emerged as a result of COVID-19, the different dimensions that make up the model provide inputs to teachers to adapt their teaching practices. At the same time, it can be useful for the development of research that focuses on observing the impact of the pandemic in educational settings [41,42], particularly in the EP of the mathematics teacher.