Formative Assessment of Pre-Service Teachers’ Knowledge on Mathematical Modeling

Abstract

This document reports how formative assessment strategies promote the knowledge of modeling of pre-service mathematics teachers. This knowledge is understood from content and vehicle points of view. Formative assessment strategies were designed and experimented with 14 participants in a mathematical modeling course offered to pre-service teachers in a Colombian university. Thematic analysis was conducted on lesson plans built by pre-service teachers. In those plans, they evinced knowledge of class management, mathematics teaching, problem solving, and modeling teaching. Finally, the collective construction of assessment rubrics is highlighted. Its contributions and limitations as a formative assessment tool are reported. The role played by the advisors’ feedback and support to pre-service teachers is also presented.

1. Introduction

Research on mathematics teachers’ knowledge has produced models regarding characteristics, dimensions, components, and facets of teachers’ teaching knowledge have emerged. Pino-Fan, Assis, and Castro [1] explored some dimensions and theoretical-methodological tools suggested by the didactic-mathematical knowledge (DMK) model for the analysis, characterization, and promotion of teacher’s knowledge, intended to efficiently develop their teaching practices. Carrillo-Yañez and his team [2] presented the mathematics teacher specialized knowledge model (MTSK); the authors proposed a framework that considers mathematical-knowledge specialization as a model-inherent property which extends to all subdomains. Such models are ways to investigate, understand, analyze, and evaluate teachers’ mathematics knowledge. Some models transcend a descriptive dimension and offer tools for intervention in training programs that promote teacher knowledge development. In those cases, continuous evaluation of teachers’ knowledge becomes a tool to study and promote the evolution of such models.

In a complementary perspective, assessment of teachers’ knowledge is associated with the knowledge they have developed to accredit, certify, or get promoted in their profession. To this end, research methods have been developed to measure teachers’ knowledge and produce valid and useful results for policy formulation [3]. Mesa and Leckrone [3] offer an overview of six types of processes, methods, and components to be assessed regarding mathematics teachers’ knowledge.

In another perspective, training programs are concerned not only with determining teachers’ knowledge, but also have the objective of promoting it. In this regard, assessment of teachers’ knowledge can be considered both summative and formative. Accordingly, a course to promote teachers’ mathematical modeling knowledge was developed and related formative-assessment strategies were implemented. To analyze the contribution of these strategies, a study was developed to answer the question: how can pre-service teachers’ knowledge on mathematical modeling be assessed in a formative way?

2. Theoretical Background

2.1. Teacher’s Knowledge on Mathematical Modeling in Mathematics Education

International research on modeling in mathematics education has revealed the opportunities that modeling offers for learning and development of students’ competencies, supporting of institutional needs, and fostering of teacher training (ICTMA collection). Blum [4] pointed out that the integration of modeling in school implies open and demanding environments which require complex teaching abilities and, consequently, ways of evaluation capable of facing those requirements. Certainly, mathematical and extra-mathematical knowledge is also required, as well as some familiarity with the selected modeling tasks. Research has also highlighted that teachers require experiences to transcend the use of routine and stereotyped tasks, so they can promote in their students’ critical views and help them to solve real-life problems, to use mathematics in society [5,6,7], and connect mathematics and other STEM areas [8]. In their research, Romo-Vázquez, Barquero, and Bosch [5] point out that teachers require to transcend the rigidity of the curriculum, strict time schedules, lack of adapted assessment devices, problems in the use of ICT, multidisciplinary challenges, among other aspects.

Cetinkaya, Kertil, Erbas, Korkmaz, Alacaci, and Cakiroglu’s literature review [6] reported that teachers have limited professional knowledge about the nature of mathematical modeling and about how to use it in mathematics teaching and learning. These authors suggested to pay greater attention to modeling-related learning opportunities for pre-service and in-service teachers through training programs. In their research, theses authors grouped a significant part of the modeling research into the following topics: (i) knowledge of the cognitive demands of certain modeling activities in order to select tasks and appropriate curricular materials for promoting specific concepts in students; (ii) knowledge about how to manage tasks and organize speech during modeling activities; (iii) knowledge on how to promote adaptive activities, make strategic interventions and foster independence as a form of scaffolding and promotion of the principle of minimal teacher assistance; (iv) knowledge of productive modeling ways (contrasted with less productive ones) to help students differentiate between more and less useful ideas, as well as to make connections between them; (v) recognition of unexpected solving approaches to modeling and development of strategies to deal with crises in the modeling process; (vi) mathematical and extra-mathematical knowledge and abilities to use information and communication technologies (ICT) effectively during the modeling processes.

Teacher’s knowledge on mathematical modeling must also include at least two intersecting dimensions, namely: conceptions about the nature of modeling and students’ training purposes [7]. In this study, the nature of modeling involves a conception of the object and the tool [9]; regarding training purposes, it is assumed that future teachers should not only learn mathematics, they should also learn to use modeling in their professional practice; that is, teachers should promote mathematical thinking as well as modeling skills and competencies. In Figure 1, this perspective of the teacher’s modeling knowledge is represented.

In this framework, the intersection between the conception of the tool and the purpose of mathematics training implies the design of learning environments that allow future teachers, through modeling, to conceptualize, to solve problems, and to generalize mathematical concepts. The intersection between the conception of modeling as a teaching tool and as a professional tool suggests the need to promote the development of knowledge in which the (future) teacher uses mathematical modeling in the design of tasks, classes, and environments for mathematics learning, considering all the facts that this implies (students learning, curriculum, context, among others). The intersection between the object and mathematics teaching perspectives implies the design of environments in which (future) teachers can “learn to do modeling”; this also implies the development of a sensitivity to identify and delimit problems, to select relevant variables, techniques, procedures and ways to build models, to solve problems using mathematics, to validate the results, etc. Finally, in the intersection between modeling as an object and as a professional practice, knowledge about the nature of modeling for teaching can be considered, including the type of tasks, type of environments according to contextual and institutional needs, among others.

It is expected that for each of the abilities that the literature suggests for mathematics teachers, these perspectives and purposes can be identified, so each one can fit in some of the intersections shown in Figure 1. Figure blocks do not represent disjoint compartments in the teacher’s knowledge, but analytical categories for the design of learning environments for those professionals. Due to the nature of the question that motivated this study, the intersection between tool and object perspectives will be used to train future teachers.

2.2. Formative Assessment for the Teacher’s Knowledge

For Black and Wiliam [10,11] formative assessment or assessment for learning demands from teachers and students an active interpretation and use of evidence about their performance to make decisions during the processes. This is a practice that seeks a constant improvement of teaching and learning, tracking students’ development in order to make decisions and reformulate tasks according to the observed results [10,11].

In this study, teacher educators and pre-service mathematics teachers were considered key actors in the process of formative assessment. According to Black and Wiliam [11], formative assessment involves several stages, namely: the establishment of training goals or purposes, information gathering about students’ thinking and knowledge, and a plan proposal (methods, strategies, environments). Pre-service teachers were allowed to participate in the stage planning, that is, they participated in the delimitation of the evaluation criteria and the procedures and strategies to achieve compliance with this purpose. Black and Wiliam [11] argued that five principles can be recognized in the design of environments for formative assessment, namely: (i) clarify and share learning intentions and criteria for success; (ii) design effective classroom discussions and other learning tasks that provide evidence of student understanding; (iii) provide feedback to helps students progress; (iv) promote students interaction to improve learning; (v) mobilize students to empower themselves in their learning.

Formative assessment, as a means of supporting the development of teachers’ knowledge, considers the strategies, media, environments, and roles of teachers as learners and of teacher educators as teachers. In a synthesis of contributions from a special issue on formative assessment and professional learning of teachers (Teachers and Teaching, Vol 19, No 2), Tigelaar and Beijaard [12] found that in the context of professional learning, teachers can be considered as learners, given that the evidence of learning that is being collected during formative assessment processes provides them with an idea of how are they performing, where do they need to move, and what can they do to get there. Regarding strategies, the authors highlight the presence of heuristic diagrams, self-evaluations combined with co-evaluation, formative feedback, negotiated evaluation, among others.

3. Methodology

3.1. Context and Participants

This study was carried out during the second semester of 2019, in a mathematical modeling course for pre-service teachers. The course was part of a Bachelor program offered by a school of Education at a public university in Medellín, Colombia. In Colombia, mathematics teachers are prepared in Bachelor programs offered either schools of Education or Mathematics Sciences or both (more details about the Colombian Mathematics teacher preparation system see Guacaneme-Suárez et al. [13]).

Throughout the course, the pre-service teachers had to develop modeling tasks [14] and analyze their own experience based on theoretical and empirical constructs studied during the process. They also participated in workshops and discussed with modeling researchers and with in-service teachers who had modeling experience. During the course, students had to develop a modeling project [14] and design a lesson plan.

The course was distributed in 16 sessions of 4 h each. In the first session, the objectives of the course, the methodology, and the evaluation products were presented. The meaning of mathematical modeling and their experience in previous courses were also discussed. In sessions 3, 8, 15, and 16 oral presentations about their progress in the projects and lesson plans were developed. Both the teachers and the pre-service teachers could comment, suggest and argue about the progress of their classmates. Based on the approach of Black and Wiliam [11], the course followed the phases and roles for teacher educators and pre-service teachers. The main aspects of formative assessment during the course can be found in

Table 1. Aspects of formative assessment adapted to this research.

	Training Purposes	Where Is the Student Now	How to Get There
Teacher Educator	1. Clarify learning intentions for success. The teacher educators specify at the beginning of the course the objectives, methodologies, and tasks to be carried out. A “class by class” is created where the objectives and purposes of each session are specified.	2. Design effective classroom discussions and other learning tasks that provide evidence of student understanding The teacher educator designed training environments for pre-service teachers. The task involved assessment and selection of relevant tasks, actions, interactions, class, and extra-class strategies, and class management.	3. Provide feedback that makes students move forward. The teacher educator offered continuous advice in order to promote reflection and problematization/continuous questioning; teachers also offered feedback. All this was done both in class and in extra-class spaces.
Peers	Understand the intentions and participate in the construction of assessment criteria and learning expectations. (Collaborative rubric)	4. Students actions as training resources for others. Students participated in the development of tasks and reflections on what they know, why they know it, and why what they know is useful for their future professional practice.	4. Students actions as training resources for others. Students actively participated in joint sessions; they commented, criticized, and made suggestions to other classmates’ actions. They also participated in the construction of criteria for the rubrics of products of the course.
Pre-Service Teachers	Understand the intentions and participate in the construction of assessment criteria and learning expectations; design paths and strategies to meet these commitments collaboratively (Development of projects, reports, and lesson plans)	5. Empower students as responsible for their own learning They participated in readings, discussions, and workshops on “what should be known” and why it is important to know it.	5. Empower students as responsible for their own learning They got continuous advice for the development of the proposed professional tasks (projects, design of class plans).

.

Fourteen pre-service teachers (11 female and 3 male) participated in the course and were informed of the ethical protocols, signing an informed consent. The names used in this article are pseudonyms. The mathematics education program was a five-year BSc program, the students (pre-service teachers) were selected according with their scores from university entrance examination. 7 of the participants in their fourth year, and 7 were in the fifth year of the program. The pre-service teachers’ ages ranged from 19 to 23 years. All participant had completed mathematics courses (e.g., geometry, arithmetic, mathematical analyses), mathematics education courses (e.g., Didactics of algebra, geometry, statistics), and a part of pedagogical courses (e.g., curriculum, educational politics, culture and education). Only six participants reported that they were coursing practicum. None of them reported work experiences as teacher.

3.2. Data

The pre-service teachers committed themselves to the development of the modeling tasks, the projects, and the lesson plans. The collective construction of the rubrics was made around the seventh-class session, after studying theoretical aspects of mathematical modeling and developing related tasks. Each session of the course was videotaped, therefore, for the lesson plans developed by the students, videos of the discussion sessions and of the evaluation rubric agreements were recorded.

Each workgroup participated in at least one advisory space with the teachers. A video that records the interaction between the trainers and the pre-service teachers was recorded. There were four work teams in the course. Each one developed a class plan that was reported in a written document and video-recorded while presented to classmates and teachers.

3.3. Data Analysis

To analyze the data (videos and documents), a category system with its respective coding was developed in an iterative process of going back and forth between predefined concepts (see the second section of this article) and data. The three researchers reached a common understanding on the codes and categories, later, the second author of this article organized and coded the data. He performed the first analysis of each lesson plan separately. The three researchers were regularly meeting to discuss and negotiate agreements and disagreements about the evidence, and data interpretations in light of the theory.

With the data from each lesson plan, a thematic analysis was carried out [15,16], the information was organized by themes, and points of convergence and divergence were sought. This allowed the emergence of other categories of analysis in light of the theoretical aspects described above. Then, the entire team of researchers conducted a cross-sectional analysis of the four lesson plans. The final system of topics and categories is detailed in

Table 2. Category and code system.

Themes	Categories	Codes
Knowledge in the design of lesson plans.	Knowledge on modeling as a vehicle Knowledge on modeling as a content or object	Use.
		mathematical concepts
		Sub-processes.
		Abilities (others)
		Simplification.
		Experimentation.
		Delimitation of problems.
		Abstraction.
		Context.
		Mathematization.
		Communication (others).
Formative assessment of pre-service teacher’s knowledge on mathematical modeling.	Contributions from rubrics construction. Contributions from advise sections	Orientations.
		Share goal.
		Limitations.
		Feedback
		Questioning.
		Reflection.
		Limitations.

. In the results section, the meaning of the categories is illustrated in greater detail with fragments of conversations extracted from the videos and the lesson-plans documents.

4. Results

The results of this study are presented in two sections: the first one presents the results of the analysis of each lesson plan; in the second one, an analysis of the formative assessment of the knowledge of pre-service teachers is made from a joint interpretation of the four lesson plans.

4.1. Analysis of the Four Lesson Plans

4.1.1. Lesson Plan 1: Clash Royale. Mathematical Modeling Experience in the Classroom

This team designed a class based on the use of the Clash Royale video game. The objective of the class was “To record and interpret numerical data from the environment offered by the Clash Royale video game” (Document 1—Class Plan). In their report, the students argued their design on the need for learners to build and compare representations, and to solve arithmetic problems that involve calculation and estimation strategies [17].

Pre-service teachers argued that the need to know a game and build winning strategies enables students to face a challenge. The class design was structured in three stages, each one one-hour long. The first stage was based on the recognition of the video game, its components, rules, players, etc. The second stage involved the delimitation, collection, and organization of data; according to the pre-service teachers “the students will have to extract different numerical data from the game environment: elixir production, cost (in elixir), attack speed, resistance and damage produced by the characters of the cards. The data obtained will be recorded in tables…” (Document 1—Class Plan). The third stage was organized through questions about the strategy to play the game efficiently.

This team proposed an evaluation of the class with scores according to the following game criteria: exploration and systematization of numerical data (10 points), analysis of situations (10 points), development and implementation of strategies (20 points), and communication of proposals by the students, during the dialogue spaces in each stage (10 points).

An analysis of this lesson plan allows to infer students’ understanding of mathematical modeling as the solution of problems using mathematics; in the context of the video game, mathematical modeling was represented by the construction of a strategy to improve performance. Despite this, aspects such as mathematical work and validation of results were absent. During the modeling process, pre-service teachers took into account elements such as data collection and its organization, identification of variables to reach the solution, reasoning, and communication. In a broad understanding of mathematical modeling, these processes are part of modeling learning. Additionally, considering Colombian curricular guidelines, this team proposed to promote in students the creation of representations to solve problems. These aspects are key in modeling processes as a tool to achieve some curricular goals.

The lesson plan included considerations about assessment related to professional knowledge. For the team, the assessment was present in the three stages of the class. It was based on criteria to assess what students can do; however, it was not in line with the proposed objective or with the stated standards of the class. In this case, knowledge on the assessment during the modeling process is a key aspect in the professional training of pre-service teachers and is related to the intersection between this component and the modeling-as-an-object perspective presented in Figure 1.

4.1.2. Lesson Plan 2: Impacts on a Person’s Life Expectancy Caused by Tobacco Use

The team designed a class to promote reflection on the consequences of tobacco use and the understanding of linear functions. In this case, pre-service teachers relied on Colombian curricular guidelines [17]. From this document, they extracted the notion of “learning evidence” that guided the assessment proposal.

The lesson plan was structured in four stages. In the first one, students became familiar with the context, identified a smoker, and interviewed her/him to obtain data on their age, habits, and motivations for smoking. In the second stage, students were invited to deepen in the context understanding; To do this, teachers proposed to observe a video and to answer three questions about the consequences of tobacco use, life expectancy, and its decrease due to tobacco. In the third stage, the students used the rates of change and percentages included in the video (years of life per amount of tobacco use) and, based on the data obtained in the interview, they concluded on the life expectancy of the interviewed person. In the fourth stage, students constructed tables of values and other representations of the obtained data set. After constructing Cartesian graphs, students were asked to “Show your model below, and explain how you got there” (Document 2—Lesson Plan).

An analysis of this lesson plan shows the intention of pre-service teachers to design a modeling task to promote reflections on health care. This purpose is within the scope of the socio-critical perspective of modeling that was studied during the course. In the class plan, there is also an interest in delimiting stages and tasks that students perform, which are gradually designed for the development of the activity. There is an interest in using change ratios to interpret data tendencies and construct linear functions; all of this describes a perspective of mathematical modeling as a tool to understand a situation, to mathematize it through linear functions, and to reflect on the impact of tobacco consumption.

On the other hand, the ordering of data, its organization in tables, and the identification of trends in generated graphs was encouraged. These elements are important for the learning of modeling as an object. Aspects such as experimentation, delimitation of a context, validation, and communication of the results were not observed in this lesson plan. Nor was it observed the creation of a space for reflection on the learning process by students or the promotion of actions or campaigns for health care, aspects that could strengthen the socio-critical scope of the modeling process.

4.1.3. Lesson Plan 3: Get Oriented and Take Tours inside the University of Antioquia

This team proposed a class to study spatial location, including direction, distance, position in space, and representation of space. These themes were based on Colombian curricular guidelines [17].

The class plan was designed based on a fictitious situation in which school children would visit the university facilities, the place where pre-service teachers carry out their studies. The tasks were organized in four stages. The first stage consisted on tracing a path through a 6 × 6 squared mesh; only horizontal and vertical displacements were allowed. The second stage involved a tour of several places of the University. In the third stage, in the classroom, students must mark on a map the most significant places during the tour. Finally, in the fourth stage, a plenary session was proposed in which they describe what they learned about the more meaningful, faster, and shorter routes. This team considered that evaluation should be used at every stage. They consider, as pre-service teachers, to be attentive to what children do and say, so that they could make timely recommendations. They would pay attention to the way they communicate, during the fourth stage, their actions, and recommendations to other classmates.

In the analysis of this lesson plan, knowledge on modeling was identified as a vehicle to promote spatial location skills in students. Although it was a possible scenario for mathematical work, the activity was not conceived to build mathematical models as representations, but to use notions of laterality and their mental representation. Students supported their choice in the course bibliography. In the class plan, modeling in primary school was described differently as conceived in higher grades; modeling was understood as “a mathematization of reality”, according to Parra-Zapata and Villa-Ochoa [18]. Stages were planned so the children gradually gained experience, represented their knowledge on maps, and communicated them to their peers. Regarding modeling as an object, opportunities to explore, position one-self, and move inside the environment are worth noting.

Unlike the first two teams, in this lesson plan, no evaluation rubrics were identified, but there was a continuous effort to be attentive to students’ actions and reflections to offer feedback; this evinces comprehension of formative assessment as a permanent activity throughout the modeling process.

4.1.4. Lesson Plan 4: Mobile Operators in Colombia

Unlike the previous ones, this lesson plan focused on solving a problem through an authentic context, supporting students to understand the phenomenon of mobile phone consumption in the country. The design was supported by the course bibliography. The pre-service teachers determined the topics that would include the process, namely: directly proportional magnitudes, conversion of measurement units, collection, and interpretation of data; however, they reported that such topics should emerge as part of the solution, but they were not the main objective of the designed task. Like the other teams, design criteria were justified in the Colombian curricular guidelines [17]. Unlike the other teams, in this lesson plan, the pre-service teachers provided information about what they considered a classroom environment should be: they described the way they conceived the active role of the students, the role of the teachers as helpers, and how to promote collaborative work and good use of resources by the students.

The lesson plan included five class sessions. In the first session, they created a fictional character (Carlos) who needed a mobile phone and wanted to purchase a plan. To help him, the team proposed to the students to inquire about operators, plans, costs, and other relevant facts. They would also assess Carlos’ needs and determine how each plan could or could not satisfy his needs. In the second session, students were invited to fill out a table containing information about Gigabytes, prices, duration, among others. Based on the table, students should generate proposals to solve Carlos’ needs. The third session focused on Carlos’ need to use the internet to upload photos. Students should offer responses according to the number of files to upload and the number of messages received and sent. The fourth session was called “decision making”, students were invited to determine Carlos’ internet consumption and, based on that, offer him recommendations to make a decision.

In the analysis of this lesson plan, the pre-service teachers created a fictitious character as a way of delimiting the activity so that it became semi-open, that is, it had intentionality and facilitated the knowledge of the phenomenon, the identification of variables, and some simplification according to the initial intention. It is worth noting the effort of pre-service teachers to create not only a working guide for students but also to consider criteria to consolidate a participatory learning environment. That way they, as teachers, could regulate their actions while following and supporting students’ performance. This course of action is related to what Cetinkaya et al. [6] call spaces that promote adaptive interventions.

In this lesson plan, opportunities offered by “experimentation” with the phenomenon are highlighted. Pre-service teachers propose to students to identify variables, obtain and organize data, and make inferences about them. The construction of models was guided by the identification of patterns in the data and inductive reasoning. Nevertheless, little emphasis was put on promoting communication of the results to the fictitious character and offering mathematical generalization of the generated algebraic model. All these elements are related to the perspective of modeling as an object.

In these four lesson plans, pre-service teachers show their knowledges on mathematical modeling. These knowledges include several understandings about modeling (process, problem solving) and purposes (introduce a content or developing critical and other skills) [19,20]. It also notes several of types and uses of contexts for the development of modeling (e.g., realistic, authentic [14,21]). The inclusion of tasks and phases was a common aspect in the lesson plans; assessment strategies were also included in all plans. The following section reports how the formative assessment strategies of the course promote knowledge on mathematical modeling.

4.2. Analysis of Formative Assessment of Pre-Service Teacher’s Knowledge on Mathematical Modeling

The lesson plans provided information about the knowledge that future teachers developed about teaching (through) modeling, that is, modeling as a teaching vehicle and modeling as a teaching content or object [9]. However, in the context of a teacher-training course, it is not only interesting to identify the generated knowledge, but also how it was promoted; in other words, it is important to consider a formative assessment.

As showed in the previous section, the four lesson plans were guided by a similar framework. This framework included title, class objective, alignment with Colombian curricular guidelines, class development, assessment, and bibliographic references. Additionally, lessons included student’s work guides and a justification of the design based on the course’s theoretical references. This structure of the four lesson plans included a guide for the student. The similarity in the structure of the lesson plans is due to the agreements reached for the construction of the rubric.

As reported in the methodological section, the pre-service teachers participated in the construction of the rubric, where the components of the lesson plans and evaluation criteria were established. As an example, Amelia pointed out that “A class must have a clear objective, which is expected to be achieved in one or more sessions. In every class that we have had, they presented an objective, the development of the class and the methodology, and, well, the evaluation” (Video, 4 July 2019, negotiation of the guide). Also, Carlos pointed out that “In the tasks that we have read, we see that the authors always state their purpose and establish the tools to measure the achievements of the modeling tasks” (Video, 4 July 2019, negotiation of the guide).

An analysis of the video of the rubric-construction session allowed us to infer the main guidelines on which students relied to consolidate the rubric and the structure of the lesson plans. These results are presented in

Table 3. Lesson-plans elements and supports.

	Lesson-Plan Elements	Theoretical Support
I	State a theoretical approach to the way of modeling assumed in the lesson plan.	Villa-Ochoa, Castrillón-Yepes y Sánchez-Cardona [14]
II	In the Colombian context, relate or support your lesson plan according to theoretical and methodological guidelines defined by the Ministry of National Education.	Ministry of Education [17]
III	Define the materials, resources and times necessary to achieve the lesson plan objective.	Bassanezi [22], Biembengut y Hein [23]
IV	Describe how the evaluation process is carried out in the lesson plan.	Aydogan Yenmez et al. [24], Diefes-Dux et al. [25]

.

Rubrics are instruments designed to help assessors, teachers, and students to judge the quality and progress in student’s performance [26]. These instruments are used for both summative and formative assessment. The participation of pre-service teachers in the design of the rubric was intended to promote formative assessment about their modeling knowledge. This participation produced the structural components of the lesson plans (components to be evaluated) and detailed criteria for evaluating them (descriptions of student’s performance). The consolidated rubric is presented in Appendix A.

The participation of pre-service teachers in the construction of the lesson-plan structures and its corresponding rubric offered them opportunities to anticipate what would be the evidence of their learning about the use of modeling in teaching; in the words of Black and Wiliam [10,11], this participation contributed to the principle of “clarifying learning intentions”. As shown in the previous section, in the lesson plans, certain knowledge became evident: knowledge about the management of the class (lesson plan 4); knowledge on the use of modeling to teach mathematical content (lesson plan 1, 2, 3) and knowledge on problem solving (lesson plan 4). To a lesser extent, knowledge about the teaching of modeling was evidenced, including subjects such as: knowledge of the context (lesson plans, 1, 2, 3, and 4); exploration of conditions and variables (lesson plans, 1, 2, 3, and 4); construction of a model (lesson plans 1 and 4) and use of mathematical information to understand the implications of a situation (lesson plan 2). Despite this, processes such as reasoning and communication, which are fundamental in modeling, were not noticeable in all the designed plans.

Table 4. Knowledge in the lesson plans.

Lesson Plan	Class Management	Teaching of Mathematical Content	Problem Solving	Modeling Teaching
				Context	Variables	Model Construction	Information Use
1		☑		☑	☑	☑
2		☑		☑	☑		☑
3		☑		☑	☑
4	☑		☑	☑	☑	☑

summarizes the knowledge evidenced in the lesson plans designed by pre-service teachers.

In these results, participation in the construction of the rubric played a normative role. In this study, it was observed that the rubric offers guidance on what will be evaluated and how it will be evaluated; also, it seems to promote the appearance of other modeling knowledge not directly declared in the rubrics, but which can be valuable for pre-service teachers. This result recalls the criticism that Panadero and Jonsson [26] have called standardization and reduction of the curriculum. According to the authors, it is questionable the way rubrics standardize assessments by providing simple lists of criteria for complex skills and by creating a tendency on students and teachers to guide their actions exclusively towards those criteria.

Another characteristic of the pre-service teacher’s formative assessment was the continuous feedback achieved. During the course, in all class activities (workshops, homework, readings, discussions), there were reflections on: What was learned? Why was it important? And how could this be integrated into their future profession? Additionally, spaces for continuous advice were created in class and extra-class times. During the class, oral presentations were made about progress in the lesson plans; both teachers and pre-service teachers could comment and criticize each team. In extra-class spaces, pre-service teachers dialogued with teachers about their progress. Teachers permanently invited pre-service teachers to reflect on: why to do what is proposed? What does the literature say about it? etc. This allowed a reflection on the nature of modeling in mathematics school teaching. As an example, Josefina, a member of the lesson plan 2, indicated:

Josefina: We want to propose our class for third grade children, we liked the document we read about geometry and modeling in primary school, so we would like to do something similar with the children.

Teacher Educator: But, how is modeling conceived there (in the document)? What is the most relevant thing the authors talked about? What is different from other ways of modeling?

Josefina: Well, what most caught our attention is that the authors showed that modeling allows students to establish a relationship with space, in such a way that geometric notions become a means of decision.

Teacher Educator: And what does that mean? How did the authors propose it? Is it a matter of getting the students to move in space or is there something else that requires planning?

In response to these questions, in their lesson plan document, the team described in greater detail the arguments they extracted from that bibliographic reference to design the four stages of the plan and the transition between the real displacement and the map location activity. A similar situation happened while giving advice to the team of class 3.

Alexander: Teacher, we don’t know how to integrate the assessment part into our lesson plan, we don’t want the assessment to focus only on mathematical concepts; we don’t want the assessment to scare students either.

Teacher Educator: Alexander, but according to what we have experienced in the course, how do you think your processes have been assessed? What tools and forms of assessment have we used or studied? Ideally, everything we have developed in the course contributes to the construction of your lesson plans.

Alexander: Teacher, you have accompanied us with questions that guide us or questions that make us realize the errors or weaknesses we have.

Teacher Educator: Accordingly, how should assessment processes be included in your lesson plans?

Alexander: Teacher, then it would be like not even telling the students that they are being assessed, but teachers should be very attentive and assess what the students are doing and try to redirect what may not lead them in the right direction. But in that scenario, don’t we have to apply an exam or a rubric or a final assessment?

Teacher Educator: The idea is that you make the decision about how you will carry out the assessment process and, in general, how you will build your lesson plan. But what is clear is that you do not have to use the rubric as an evaluation instrument, you can use other resources or instruments. What is necessary is that you indicate how the evaluation process would be developed in your lesson plan.

The third team’s lesson plan showed that the elements discussed in advise sessions offered clarity to the students (pre-service teachers). In particular, this work team integrated, during the four stages of the lesson plan, feedback processes, and support to the students and made possible an assessment that facilitated orientation and success of the students.

Feedback can be considered a key strategy within formative assessment [10,11]. In the case of the present study, the feedback was conceived as a continuous dialogue and questioning about what pre-service teachers were proposing, thereby offering them opportunities to reflect on their proposals and helping them to improve their arguments and actions. Pres-service teacher’s arguments were based on the reviewed literature and also on the projection of other variables present in the institutional context. According to Romo-Vázquez et al. [5], teacher training should not only be based on the design of tasks and its implementation in class, but also on knowledge of the curriculum and other institutional considerations. Despite these reflections, no important evidence of the presence of such knowledge was included in the lesson plans. This can be justified by the fact that pre-service teachers had not yet had contact with school environments and, therefore, were unaware of the diversity of institutional conditions that may be present in daily school life.

5. Conclusions

In the first part of this article, conceptions about the notion of teachers’ knowledge assessment were presented. Those conceptions are aligned with the notion of measurement and certification of teachers’ knowledge and abilities. It also debated the need for this notion to transcend into a formative assessment of teacher’s knowledge in the context of training courses and professional programs.

This article offers evidence that, in the context of a course, the notion of formative assessment of pre-service teachers’ knowledge requires a conceptual delimitation of the knowledge that is expected to be achieved and the strategies to achieve it. The courses, by their nature, are delimited in space and time; therefore, their purposes, methodologies, and scope are also conditioned. In the case of this study, a conceptualization of two broad categories of modeling knowledge in teaching was offered: modeling as a tool and modeling as a learning object. In this framework, this study offers evidence of the knowledge showed by pre-service teachers in their lesson plans and on the contributions and limitations of rubrics and feedback in the strengthening of this knowledge. In this regard, this study highlights two important results.

The first result that stands out is the local character of the knowledge that is achieved in a course for pre-service teachers about teaching of (and through) modeling. The literature has shown the complexity involved in integrating modeling into everyday school life and the high demands that it implies for teachers. Faced with this panorama, the scope of a course is only part of that knowledge; the teaching practice will be conditioned by the opportunities and limitations that pre-service teachers have about school practice. It will also depend on the environment and strategies implemented during the course. In this sense, the second important result derived from this study is related to the opportunities and limitations offered by continuous advice and participation in the construction of rubrics. As argued in this study, some research supports the use of rubrics for student learning, academic performance, and self-regulation; however, rubric design requires care. In this study, participation in the rubrics contributed to the development of pre-service teacher’s knowledge about “teaching by and through modeling” and conditioned the appearance of other important knowledge in this category. Regarding advise sessions, its contributions to continuous feedback were important, but it also became clear that these contributions may be conditioned by the possible existence of other knowledge, for instance, the institutional context. These results can be used by mathematics teacher educators as an insight to the opportunities and limitations of the formative assessment for developing preservice teacher knowledge on mathematical modeling. Some formative assessment strategies would need to be reworked to afford a generation of other knowledges on mathematical modeling among pre-service teachers.

One limitation of the study is that pre-service teacher knowledge was analyzed through lesson plans. Other studies could analyze pre-service teacher knowledge in professional authentic situations (for instances, practicum) that might provide more differentiated descriptions of their prospective professional work; but as our interest was in the knowledge on modeling as both object and content we found lesson plans more appropriate. The variety of knowledge found in the participants informs about contributions of rubric and feedback, but we cannot generalize all our findings to other formative strategies uses or mathematics teacher education programs. In this sense, this study suggests the need for new research that accounts for the contributions of other strategies to the development of pre-service teachers’ knowledge. New studies on the design of rubrics are suggested, to address the participation of pre-service teachers and the formative/normative tension described in this article.