Supporting Preservice Teachers in Analyzing Curriculum Materials

Abstract

Developing evidence-based curricula is a common approach in science education research to improve the quality of teaching. Curriculum developers recurrently provide curriculum materials (CMs) to support teachers who design instruction in the classroom. However, these CMs are often designed for students, and the features of CMs that are supportive of student learning are not sufficiently explained. Extant studies indicate that teachers struggle to identify these features in CMs or reject certain features of CMs. Therefore, we developed a teaching and learning sequence (TLS) for teacher education programs to support preservice physics teachers in analyzing CMs. We designed a tool that provides a scheme for systematically analyzing CMs and investigated if this tool is suitable for supporting preservice physics teachers in analyzing CMs. We implemented the TLS in a bachelor seminar (N = 8) of our teacher education program, conducted short, guided interviews as well as problem-centered interviews, and collected several learning products. The tool helps preservice teachers to discover a broader range of features of CMs; however, they struggle to argue the role of these features in facilitating student learning. Further, we discuss the refinement of the tool and provide design conjectures for the development of a similar TLS.

1. Introduction

Physics education research aims at improving the quality of teaching physics, for example, by developing evidence-based curricula (e.g., [1,2,3]). In this article, an evidence-based curriculum describes a set of coherent strategies, such as structuring content—using representations, analogies, or models, etc.—to guide learners on a certain learning trajectory (cf. [4]). The term evidence-based indicates that the curriculum is based on research. Developers of such curricula usually intend to support teachers in school practice. Therefore, interaction between developers and teachers is necessary.

A common approach to mediate this interaction between developers and teachers in school practice is to provide curriculum materials (CMs), since such materials are easy to disseminate and can often be used directly in the classroom [5,6]. CMs are instructional resources, such as student handbooks or teacher guides, that support teachers in designing instruction in the classroom based on the corresponding curriculum (cf. [4,7]). Framed in the context of a communication process, these materials serve as tools for communication. Developers aim to improve the quality of teaching by influencing teachers’ instructional decisions. Therefore, they send a message—the evidence-based curriculum, that is, its underlying strategies—to teachers. Developers embed their messages in CMs. Teachers, in the role of receivers, perceive the CMs and need to decode and make sense of the embedded message.

However, certain studies indicate that teachers struggle to identify the messages of curricula in corresponding CMs or do not accept these messages [8,9,10]. A study that included Swedish mathematics teachers on the reform message of a new curriculum revealed that teachers often interact with CMs in a superficial manner and interpret CMs in a manner that is in alignment with their beliefs regarding teaching and learning; therefore, they do not decode the reform message [10]. Another study with German physics teachers indicates that teachers are more likely to look for interesting features—mostly tasks—of CMs to integrate into their lessons than engage with the innovative idea that the curriculum may be based on [8].

To understand why teachers often do not receive the message underlying CMs, it is important to understand how teachers use CMs [11]. Teachers use CMs very differently when preparing and enacting instructions [11]. They perceive and interpret CMs through the lens of their personal goals, beliefs, professional knowledge, professional identities, etc. [12]; they focus on different aspects of CMs [11]; they adopt different perspectives when evaluating CMs [13]; and they offload instructional decisions to a different degree to CMs [7]. When interacting with CM, teachers must evaluate “[…] the constraints of the classroom setting, balance tradeoffs, and devise strategies—all in the pursuit of their instructional goals” [7] (p. 18); decide whether they believe the curriculum is appropriate to support them in designing their instruction; and, finally, adapt the curriculum to meet their own instructional goals [14]. Thus, teachers play a vital role in transforming CMs into the enacted curriculum in the classroom—the teacher is a co-developer of the enacted curriculum (cf. [11,12]).

Another difficulty for teachers is that CMs are often designed for direct use in classrooms and address students rather than teachers [5]. Moreover, certain strategies underlying the curriculum are only indirectly evident through certain features of CM, such as the use of certain forms of representation in the materials. To enact the curriculum in the classroom as intended by the developers, teachers need to identify these features—for example, representations—as relevant to student learning.

We argue that for successful communication between developers and teachers, the role of the teacher as co-developer of the enacted curriculum must be taken into account (cf. [5,11,15]). Developers must consider how the content of their message—the embedded strategies and corresponding features—can be identified by teachers (cf. [5]) and how the curriculum will be accepted by teachers for designing their own specific instruction. On the other hand, it is also important that teachers and preservice teachers learn how to analyze CMs to identify embedded strategies (cf. [16]) in order to make instructional decisions in line with evidence-based curricula.

A general goal of teacher education is to support preservice teachers in developing professional competencies that are required for their future work in school practice [17]. Preparing and enacting instruction can be considered the core business of a teacher [18]. When preparing instructions, teachers need to make decisions. Therefore, we designed a teaching and learning sequence (TLS) as part of a design-based research (DBR) approach to support preservice teachers in secondary school in analyzing CMs as a basis for instructional decisions. When analyzing CMs, preservice teachers need to perceive or discover various features of CMs—for example, in which order central key ideas are introduced or which analogies for certain phenomena are chosen—and reflect upon the roles of these features for student learning.

A central element of the TLS is a tool to support preservice teachers in analyzing CM in a systematic manner: the representation of essential features (REF). In this article, we investigate whether the prototypical REF is a suitable tool within the TLS to support preservice teachers in analyzing CMs. Furthermore, we discuss how this prototypical REF and the TLS can be refined for further development cycles based on this investigation.

2. Theoretical Framework

As mentioned earlier, in this study, our aim is to investigate the suitability of a tool for a specific purpose. In general, a tool is intended to support a user in a certain activity. The REF is a tool designed to support preservice teachers in analyzing CMs. We consider the REF to be suitable if it supports preservice teachers in analyzing CMs and if preservice teachers accept the REF as supportive for analyzing CMs. In this section, we first conceptualize the analysis of CMs and provide a brief overview of the difficulties preservice teachers face in analyzing CMs based on previous findings. We then present a framework for the acceptance of tools.

2.1. Defining the Analysis of Curriculum Materials

Curriculum materials play a central role in instructional decisions [12]. Therefore, several studies have investigated how preservice teachers interact with CMs [16,19]. For example, Sherin and Drake [19] developed a framework for examining preservice teachers’ interactions with CMs before, during, and after the enactment of instruction. Based on literature, they identified three categories of the interpretive activities of preservice teachers when engaging with CMs: reading, evaluating, and adapting. In this context, reading refers to what information preservice teachers are looking for and what they focus their interest on when interacting with CMs. Reading also includes what features of CMs preservice teachers become aware of and what features or information they eventually reflect upon. Evaluating refers to how preservice teachers interpret or critique these identified features of CMs, or what strengths and weaknesses they identify in CMs (cf. [16]). Adapting refers to the changes that preservice teachers make to CMs when preparing or enacting instruction. This last activity represents the instructional decisions that preservice teachers make when transforming CMs into practice based on the evaluation of their interpretation of the CMs.

Lloyd and Behm [20] summarize the activities of reading and evaluating CMs as analyzing CMs. In our study, we focus only on how preservice teachers read and evaluate CMs as a basis for instructional decisions. Therefore, in this study, analyzing CMs implies the activities of reading CMs and evaluating CMs, as defined by Sherin and Drake [19], as the basis for instructional decisions.

Teachers analyze CMs very differently. Remillard [21] reports that two teachers in a case study read CMs selectively and focused on different parts or elements of CMs when designing instruction for a lesson. Teachers also appear to use different strategies when reading CMs for preparing instruction [19]: While some tend to obtain an overview of CMs and focus on student activities, others read more in detail and attempt to identify specific guidance. In particular, preservice teachers tend to focus more on the content of CMs than on student learning when evaluating CMs [13]. In general, (preservice) teachers tend to rely on their previous (teaching) experiences [13,20]. Sherin and Drake [19] also observed that teachers adopt different perspectives when evaluating CM: students, teachers, and others (such as administration or parents).

To analyze CMs, (preservice) teachers require certain professional competences. Blömeke et al. [22] conceptualize professional competence as a process that leads from a person’s resources, such as latent traits and skills, to performance as observable behavior in real-life situations. A teacher must have certain dispositions, like certain cognitive or affective resources (such as content knowledge or beliefs), to acquire certain skills to engage in a certain behavior in a specific situation. We consider analyzing CMs to be a professional action, and certain skills are required for the activities of reading and evaluating:

• When reading CMs, preservice teachers need to examine different aspects of CMs and perceive or discover features of these CMs. For example, in our case, a preservice teacher goes through a CM for an optics curriculum, identifies representations used in the CM, and notices that in each figure, light propagation is represented by diverging arrows. Becoming aware of different features or patterns in a CM is necessary to be able to reflect upon these features’ roles in student learning.
• When evaluating CMs, preservice teachers need to reflect upon the role of these features for student learning, consider the strengths and weaknesses of these features from the perspective of physics education, and decide whether the features they have identified are relevant for student learning. In our case, a preservice teacher needs to reflect on the possible consequences for student learning of diverging arrows as representations of light propagation. After weighing the strengths and weaknesses of this format, the preservice teacher decides that using diverging arrows in instruction might support students in their learning processes and, hence, is a relevant feature of the curriculum.

2.2. Analysis of CMs in Teacher Education

The idea of supporting preservice teachers’ analysis of CMs in teacher education is not new [13,16,23,24,25]. In this subsection, we describe several approaches and summarize their findings for guiding the design process of the REF.

For proposing strategies for teacher education, Ben-Peretz et al. [13], for example, investigated which criteria preservice teachers at different stages of their education used compared to in-service teachers when examining CMs. In this study, they observed that the criteria of first- to third-year preservice teachers shift from a focus on the content of CM to teaching strategies, student learning, and students’ cognitive demands. However, experienced teachers relied more on teaching strategies and learning tasks. Ben-Peretz et al. conclude that the criteria selected by preservice teachers strongly rely on their past (teaching) experiences. Therefore, they recommend introducing a scheme for analyzing CMs in teacher education. Such a scheme can introduce unfamiliar criteria to preservice teachers and may also enable them to identify the characteristics of CMs. Ben-Peretz et al. also state that “curriculum interpretation is a strategy for engaging teachers and preservice teachers reflectively in one of their professional activities, namely, making educational sense of curriculum materials.” (p. 9).

Lloyd and Behm [20] found similar results when they used specific lesson analysis assignments in teacher education, where 23 preservice teachers had to analyze and compare two mathematics textbooks. One textbook was reform-oriented, and the other was more traditional. The participants’ familiarity with traditional components of the textbooks appeared to be central to their evaluation. Most of the preservice teachers even explicitly indicated that they drew on their past teaching and learning experience with traditional textbooks when analyzing the two textbooks. Interestingly, familiarity appears to lead to a bias in interpreting the components of the textbooks: preservice teachers perceive the reform-oriented textbook through “traditional goggles.” Lloyd and Behm also state that preservice teachers struggled to identify the pedagogical value of the reform-oriented textbook. Similar to Ben-Peretz et al. [13], Lloyd and Behm conclude that preservice teachers might benefit from analyzing CMs in teacher education—particularly reflecting on and articulating what makes materials effective for student learning. Lloyd and Behm also suggest more classroom discussions of various teaching and learning theories that underlie CMs in order to create awareness among preservice teachers of their own assumptions regarding teaching and learning.

Further, in a study with 20 preservice elementary teachers, Davis [23] investigated which criteria these preservice teachers used to evaluate lesson plans (as a representation of a curriculum) to ascertain where they needed support in teacher education. Davis observed that the quality of the preservice teachers’ critiques depended on the presence or absence of a scaffold for analyzing lesson plans. Based on her findings, Davis emphasizes the role of evaluating CMs as an authentic element of teaching and the importance of scaffolding this evaluation in teacher education to support preservice teachers in focusing on other important aspects of CMs than they usually do.

The tenor of these studies is that preservice teachers need support in analyzing CMs. This support might be some kind of scaffold, like a set of criteria to evaluate CMs [23] or an analysis scheme [13] that guides their analysis of CMs. According to these studies, such a scaffold or scheme should guide the focus of analysis, support preservice teachers in becoming aware of different aspects of CMs that they usually do not consider, and provide guidance regarding what to examine more closely. Preservice teachers also need support in how to reflect upon features or aspects of CMs that they have gained awareness of.

2.3. Acceptance of Tools

Acceptance describes the behavioral intention of a user to use a tool for its intended purpose. Whether a tool—either physical or cognitive—is finally used depends on several factors. According to the theory of planned behavior (see [26]), a certain behavior of a person, like performing a task or using a tool, is predicted by the person’s behavioral intention for performing the task or using the tool. This behavioral intention is influenced by the person’s perception of how easy or difficult the behavior of interest is, the person’s attitudes toward the behavior, and subjective norms. Davis [27] developed the more specific technology acceptance model (TAM) based on the theory of planned behavior. The TAM describes how individual variables are interrelated to explain user behavior in cases where information technology systems (IT) are used as tools (see Figure 1). Central to the TAM are two basic determinants of user behavior: perceived usefulness and perceived ease of use [27]. Thus, whether a person rejects or adopts a new technology depends strongly on how useful the person perceives the technology to be for performing a task and how easy it is to use the technology. Several studies confirmed that the model is applicable to a broad spectrum of tools and contexts in various empirical studies (for a review, see [28]), even beyond IT. As mentioned earlier, the TAM is based on theories of general psychology, such as the theory of planned behavior; therefore, we consider the TAM appropriate to investigate the acceptance of the REF as a tool for analyzing CMs. In the context of our study, we particularly employ the idea of perceived usefulness and perceived ease of use of the REF for analyzing CMs.

3. The Design-Based Research Project

The study described in this article is part of a design-based research (DBR) project. To put the study in context, we describe the DBR project first.

3.1. Research Aims of the DBR Project

The idea of DBR in education is to address a real-world problem by designing a research-based solution considering the constraints and conditions of the naturalistic setting. Iteratively, it is investigated whether and how this solution works in this setting [29,30,31]. Therefore, on the one hand, we aim to provide a prototypical TLS that supports preservice teachers in analyzing CMs. We also aim to provide a set of practical strategies for teacher education on how to design such a TLS. On the other hand, we aim to contribute to a local theory on how to support preservice teachers in analyzing CM. Therefore, the overarching research question of our DBR project is “How can a TLS in a teacher education program be designed to support preservice teachers in analyzing CMs”?

Therefore, we designed a prototypical TLS for a bachelor’s seminar in our physics teacher education program to systematically investigate how to support preservice teachers in analyzing CMs. The design process of the TLS was guided by several research-based conjectures on how such a support can be realized—so-called design conjectures (cf. [32]). These conjectures are based on general theories of teaching and learning—particularly in teacher education—as well as on more specific local theories and empirical evidence from prior research on analyzing or interacting with CMs. The prototypical TLS consists of four seminar units of 135 min each.

In the first seminar unit, the preservice teachers analyze a chapter of CM for an introductory optics curriculum in an unguided manner. The second seminar unit is about curriculum development and evidence-based curricula, with the following central key idea: Developers make numerous instructional decisions based on empirical findings and teaching and learning theories. In several smaller activities, the preservice teachers are prompted to reflect upon the developers’ decisions. In the third seminar unit, the REF is introduced as a tool for analysis (see Section 3.2), and the preservice teachers analyze another chapter of the CM for introductory optics with the REF. In the last seminar unit, the preservice teachers presented the features of the CM they identified as being supportive of student learning and reflected upon their experience of analyzing the CM with the REF. At the end of the fourth seminar unit, the preservice teachers analyze the same chapter of the CM from the first seminar unit again. Figure 2 presents the main key ideas of the seminar units of the TLS and the central activities of these units.

3.2. The REF as a Tool for Analysis

A central design conjecture of the TLS is that preservice teachers require tools that support them in systematically analyzing CMs (cf. [25]). Therefore, we needed to identify which components such a tool might consist of and how to structure these components. Ben-Peretz et al. [13] suggest providing preservice teachers with a frame of reference that “may stem from the rationale of curriculum developers, and lead to an ‘internal’ scheme of analysis.” [13] (p. 53). Therefore, we designed such a tool based on the REF framework [33]. We developed this REF framework originally to support curriculum developers to organize the characteristic strategies of curricula in different dimensions (see

Table 1. REF framework—dimensions of essential features with their description.

	Dimensions of REF	Description of the Dimension
content structure	Content key ideas	Key ideas are the central ideas of concepts and principles that emerge in the course of an educational reconstruction [34].
	Representations	Representation can include both pictorial and verbal representations to develop ideas [35].
	Order of key ideas	The sequence of the key ideas.
	Models, analogies	Models represent relevant sections of a theory—for example, elements and their relationships. Analogies are comparisons between two models to describe similarities [36].
	Contexts	Contexts are the link between learning content and the “real world” [37].
	Strategies to support conceptual change	Strategies to guide students toward a physically adequate view of a term or concept.
	Student activities and tasks	Principles and schemes in tasks or activities for encouraging students to be cognitively activated [38].
	Subject-specific methods	Typical and characteristic working methods/action patterns for a discipline (such as measuring, observing, etc.).
organizational structure	Instructional media	Instructional media are non-personal teaching aids (e.g., worksheets, video projectors, audio media, etc.) used to transmit information.
	Class organization	Class organization includes free work, project teaching, direct instruction, etc. [39].
	Teaching methods	Teaching methods include genetic teaching, exemplary teaching, etc. [39].
	Structure of lessons	The structure of lessons defines the sequence of different phases—for example, the phases of motivation and elaboration [39].
	Group organization	Group organization defines the group size, constellation, etc. [39].
	Learning tools	Learning tools include ways of working, interaction structures, and communication elements [39].

)—such as the sequence of key ideas, use of models and analogies, representations, etc. [33]—in a compact and systematic manner. The strategies of a curriculum reflect evidence-based instructional decisions made by developers in the curriculum design process. The dimensions of the REF are derived from the literature (see Table 1). This systematic overview of strategies embedded in CMs is intended to help teachers identify which features of the CMs are essential for guiding students on the learning trajectory suggested by the curriculum. We call these features essential features (EF) of a curriculum.

The prototypical tool for analyzing CMs is a DIN A4 table named REF. The REF displays the different dimensions of the REF framework, their descriptions, one or two examples of EF of each dimension, and corresponding rationales on why these features are supportive of student learning (see Figure 3). The dimensions of the REF are intended to serve as a scheme for preservice teachers to systematically analyze CMs (cf. [13,23,25]) from different perspectives. The examples of EF are intended to support preservice teachers in becoming aware of what exactly to look at, and the exemplary rationales are intended to help preservice teachers articulate why the discovered features are supportive of student learning.

3.3. Introduction of the REF and Group Activity for the Analysis of CMs

The REF was introduced to the preservice teachers in the third seminar unit of the TLS. This seminar unit begins with an instruction of approximately 10 min, where the REF is introduced to the preservice teachers. Thereafter, the preservice teachers apply the REF in groups of two or three in a scaffolded manner. Figure 4 presents a schematic representation of the third unit, including a description of the key ideas of the instruction and the tasks of the group activity. The task of this group activity is to analyze a chapter of a student handbook corresponding with an evidence-based curriculum for introductory optics [2]. Each group is assigned a different chapter of the student handbook. The aim of this group activity is more related to the process of analyzing CMs with the REF as a central scheme of analysis than deciding which features of the student handbook are essential for student learning. The learning products of the group activity are the basis for group discussions in the fourth seminar unit. The preservice teachers present (1) which features they consider essential for student learning and why; and (2) reflect upon their experience with analysis using the REF compared to their analysis experience without the REF (which was in seminar unit 1) to become aware of their own internalized strategies for analyzing CM (cf. [13]).

4. Aim of the Study

The focus of our research is to investigate how to design a TLS for a teacher education program in order to support preservice teachers in analyzing CMs. The aim of this study is to investigate if the designed prototypical tool—the REF—is suitable for guiding the preservice teachers’ analysis of CM in the group activity within the TLS and how this tool and the TLS can be improved. We consider the REF suitable for analyzing CMs in the group activity if (1) the preservice teachers discover different features of the CM—particularly unfamiliar ones; (2) reflect upon their role in student learning; and (3) accept the tool for analyzing CMs. Therefore, the following is our first research question:

Q1.

Which features of CMs do preservice teachers identify as supportive of student learning when analyzing CMs with the REF in the group activity, and how do they argue in favor of why the identified features are supportive of student learning?

To understand how preservice teachers analyze CMs with the REF in the group activity and to derive hints for improvement of the TLS or the REF, it is important to address the following questions:

Q2.

How do preservice teachers comprehend the key ideas of the introduction of the REF and the tasks of the group throughout the TLS?

Q3.

How do preservice teachers use the REF for analyzing the student handbook in the group activity in the prototypical TLS?

According to the TAM (Davis, 1989), preservice teachers need to accept the REF (or at least the underlying analysis scheme) as a useful and easy-to-use tool for analyzing CMs, even if they understand the key aspects associated with its introduction, apply the REF as intended, and successfully identify features of CMs that are supportive of student learning. Therefore, we also aim to find the answer to the following question:

Q4.

How do preservice teachers accept the REF as a tool for analyzing CM?

These four research questions are intended to shed light on different perspectives regarding the suitability of the REF as a tool for analyzing CM in teacher education.

5. Methods

5.1. Context of Implementation and Participants

The TLS was implemented in 4 weekly live sessions of 135 min each as a part of a bachelor’s seminar in physics education for secondary school in the fourth semester (recommended) of our teacher education program in the winter semester of 2021/22. The TLS was attended by five male and three female preservice teachers at the bachelor’s level. To obtain access to this seminar, the preservice teachers had to attend another seminar on physics education in advance. They were preservice teachers who barely had any practical experience in physics classrooms. One preservice teacher was absent from the group activity since they were applying the REF. Further, the TLS was implemented online due to the COVID-19 pandemic, and the group activity for using the REF took place in three break-out rooms with two to three preservice teachers in each. The REF was provided digitally in the form of a PDF.

5.2. Data Collection

We selected a mixed methods approach to answer the research questions (see Figure 5), conducted both guided short interviews (n = 6) and problem-centered interviews (n = 4), and collected two types of learning products.

After the introduction of the REF, a scaffolded group activity was conducted. During this activity, we conducted two short, semi-structured interviews of 3–12 min each with each group—one at the beginning and one at the end. In the first short interview, we investigated how the preservice teachers comprehended and accepted the key ideas of the introduction and if the task and the REF appeared clear to them (Q2). In the second short interview, we asked how the preservice teachers experienced analyzing the student handbook with the REF; we prompted the preservice teachers to reflect upon their attitudes toward the REF and how they believed the REF supported them in analyzing CMs (Q2, Q3, Q4). Figure 6 presents the guiding questions for the short interviews.

We also collected the learning products that the preservice teachers had created during the group activity to obtain further insights into their performance when analyzing CMs (Q1). Each group had to complete a REF, which contained features of the student handbook they identified as supportive of student learning and the rationales for their decisions. Each group also had to create a presentation about the features they identified in the student handbook as supportive of student learning and how they perceived analyzing the student handbook.

Two and four weeks later, we conducted problem-centered interviews [40] of 45–75 min with 4 voluntary preservice teachers who attended the TLS. The interviews aimed at obtaining deeper insights into how the preservice teachers experienced analyzing the CM with the REF (Q2, Q3, Q4).

The problem-centered interviews were semi-structured and conducted online via video call due to the COVID-19 pandemic by one of the authors. The interview was structured in five parts. The first part dealt with the preservice teachers’ experience of analyzing the CM in the group activity—for example, how they applied the REF and how useful they perceived analyzing the CM with the REF. The second and third parts of the interview dealt with different learning opportunities within the TLS. In the fourth and fifth parts, we attempted to obtain further insights into the attitudes of the preservice teachers toward evidence-based CMs and how preservice teachers reflect upon the consequences of a single representation regarding the visual process for students’ learning processes.

5.3. Data Analysis

We analyzed the completed REF of each group created during the group activity to investigate which features preservice teachers believe to be supportive of student learning and how they argue in favor of their decisions (Q1). Each feature of the CM that preservice teachers identified as supportive of student learning was rated according to the questions presented in

Table 2. Questions and corresponding ratings for the analysis of REF.

Question	Rating	Explanation
Is this feature supposed to be essential for student learning based on what is generally accepted from the perspective of physics education research?	Yes
	Maybe	There is no clear consensus on whether or not the identified feature is essential for student learning.
	No
Do preservice teachers assign this feature to the appropriate dimension of the REF?	Yes
	Arguable	The identified feature is not assigned to the appropriate dimension, but it is understandable how the preservice teacher arrives at the assignment.
	No
Is the rationale the preservice teachers provide reasonable (from the perspective of student learning)?	Yes
	Yes, but too superficial	The rationale is too general or vague, like “The tasks support students to develop competences.”
	No
What perspective do preservice teachers adopt in the provided rationale?	Physics	The rationale only evaluates the physics underlying the identified features of the student handbook. Student learning is neglected.
	Learning physics	The rationale takes the learner into account and evaluates the role of the identified feature in the process of learning physics.
	Others (pedagogic, design)	The rationale does not relate to physics or learning physics, like “the colors and fonts of the headlines in the student handbook support students because they provide a clear structure.”

. Further, we rated the tables separately and compared the results. One of the raters was centrally involved in the development of the student handbook.

The short interviews and problem-centered interviews were transcribed and analyzed with a structured qualitative content analysis in accordance with Kuckartz [41]. We developed a category system (

Table 3. Deductive category system with category name, description, and a simplified example translated from German.

Main Category	Category Description	Simplified Examples (Translated from German)
Comprehension	This category includes all statements of preservice teachers in which they explicitly paraphrase the key ideas of the introduction and the intended purpose of these key ideas, or statements that allow conclusions to be drawn regarding how the preservice teachers understood the key ideas of the introduction.	“I think the table [REF] was introduced to critique instructional materials.”
Application	This category includes all statements of the preservice teachers regarding(a) how they used the REF (or the underlying scheme for analysis) in the group activity;	(a) “[…] we all first read our chapter of it [curriculum] separately, and then discussed together what its most important statements were for each of us.”
	(b) which aspects/features of the CM the preservice teachers noticed or discussed;	(b) “[…] aspects of content structure are usually always more present for me.”
	(c) how the preservice teachers decided, whether or not the noticed features were important for student learning.	(c) “[…] usually, you do not think so much about what didactic considerations are behind it [CM] or whether there are any at all, but you primarily just think: Ok, I could use that because it matches the topic or something.”
Acceptance	This category includes all statements of the preservice teachers regarding(a) the perceived usefulness of the REF (or the underlying scheme) for analyzing CMs;	(a) “I thought beforehand that it would be time-consuming to go through it all in detail. However, also that it will bring a lot. Additionally, I think that is still the same.”
	(b) the preservice teachers’ perceived ease of use of the REF (or the underlying scheme), when analyzing CMs;	(b) “I think it’s convenient mostly because it sharpens your focus and it is just a piece of paper you can easily handle and put it aside.”
	(c) the preservice teachers’ attitudes toward the REF (or the underlying scheme);	(c) “Yes, as I said, I find it quite practical that there is something that has a hand and foot and not just something.”
	(d) the preservice teachers’ intentions for using the REF (or the underlying scheme) for analyzing CM in school practice.	(d) “I do believe that I pick that [REF] up again, especially when I start my internships and use it even with a completely different topic.”

) for answering research questions Q2, Q3, and Q4. The main categories correspond with these three questions. These deductive categories serve as lenses to analyze the data from three perspectives: comprehension (of the introduction) (Q2), application (of the REF) (Q3), and acceptance (of the REF) (Q4). These categories are interdependent and therefore overlap at several points. How preservice teachers apply the REF for analyzing the student handbook depends, for example, on how they understood the introduction of the REF. Whether or not preservice teachers accept the REF as a tool for analyzing CMs depends on how they analyzed the student handbook with the REF in the group activity.

Following the procedure suggested by Kuckartz [41], we coded the transcriptions of the interviews with respect to the main categories. In the second phase, we analyzed the codes of the main categories for deriving inductive subcategories. To support the intersubjectivity of the coding process in the qualitative content analysis, we consensually coded the categories following a subjective assessment procedure [42]. Therefore, we iteratively coded selected sections of the transcription separately, discussed different interpretations, looked for consensus, and revised the category system.

Finally, we triangulated the results of the qualitative content analysis with the results of the analysis of the learning products.

6. Results

In this section, we structure the results according to the perspectives of questions Q1–Q4 as a basis for discussing the suitability of the REF as a tool and as a scheme of analysis for the preservice teachers within the TLS.

6.1. Performance: Which Features of CM Do Preservice Teachers Identify as Supportive of Student Learning When Analyzing CMs with the REF in the Group Activity and How Do They Argue Why the Identified Features Are Supportive? (Q1)

Our aim is to identify which features preservice teachers believe to be supportive of student learning and whether preservice teachers prefer certain dimensions of the REF. In the group activity, every group analyzed a different chapter of the student handbook for introductory optics with the task of identifying features of their chapter that they believe are supportive of student learning. This analysis was guided by the REF. One of the tasks was to complete an REF for the student handbook (see Figure 3) and to assign the identified features to the corresponding dimensions. We also evaluated whether the identified features could be considered essential for student learning from an education research perspective and how preservice teachers reasoned their decision.

Table 4. Ratings for the completed REF of the preservice teachers in Group 2. The dimensions without any identified features are not depicted (e.g., “models, analogies”).

Dimension of REF	Number of Identified Features	Supportive of Student Learning	Appropriate Dimension	Rationale
				Reasonable	Perspective
Key content ideas	4	Yes	Yes	Yes	Student Learning
		No	Yes	No rationale provided
		Yes	Yes	Superficial	Student Learning
		Maybe	Yes	No rationale provided
Representations	2	No	Arguable	Superficial	Other
		No	Arguable	No	Student learning
Order of key ideas	1	Yes	Yes	No	Student learning
Contexts	2	Yes	Yes	Superficial	Student learning
		Yes	Yes	Superficial	Student learning
Strategies to support conceptual change	1	Maybe	No	No	Student learning
Student activities and tasks	1	No	Arguable	Yes	Student learning
Subject-specific methods	1	No	No	No	Student learning
Instructional media	3	No	Yes	Superficial	Student learning
		No	No	Superficial	Student learning
		No	Yes	No	Student learning

presents the ratings of the completed REF of Group 2 and

Table 5. Two examples of features in the group activity identified as being supportive of student learning by preservice teachers, along with the rationale, and a summary of the researchers’ rating of these two examples.

Identified Feature by the Preservice Teachers	The Preservice Teachers’ Rationale for Being Supportive of Student Learning	Exemplary Summary of the Researchers’ Rating:
To see an object, light from the object must reach the eyes.	Tackles the student’s misconception that they can see in the dark and that seeing is an active process.	The identified feature is correctly assigned to the dimension of content key idea that can be considered essential for student learning. The rationale is reasonable from a physics education perspective.
The semi-structured experiments	They provide a red thread, but stimulate thinking.	The identified feature was wrongly assigned to the dimension of representations; it cannot be considered essential for student learning. The rationale is superficial from a physics education perspective and lacks a reason for why the experiments stimulate thinking or how they provide a red thread.

presents two exemplary features the preservice teachers identified supportive of student learning with the rationale they provided together with our ratings.

All three groups identified between 13 and 15 features as being supportive of student learning in the chapter of the student handbook they were assigned to analyze. The number of features that can be considered essential for student learning from an education research perspective differs among the groups. Only 2 of the 15 features identified as supportive by Group 1 can be considered essential for student learning. In Group 2, 5 of the 15 identified features can be considered essential, and in Group 3, 8 of 13 were considered essential.

Further, the features of certain dimensions appear to be easier to discover for the preservice teachers than others. For example, the dimensions of key content ideas, order of key ideas, and context appear to be easier to discover. All three groups are able to successfully identify at least certain features related to these dimensions as supportive of student learning. The preservice teachers also successfully noted that in their chapters, no analogies and models were used. Moreover, two groups correctly mentioned in the interviews that their chapters barely contained any features assigned to the organizational structure. However, Group 1 identified features in the dimensions of group organization, structure of lesson, and class organization. Based on the short interviews, we assume that a few preservice teachers felt that they needed to find appropriate features for all dimensions: “Since it was not mentioned before that you do not need some things at all and you can leave things free” (Translated from German; short, guided interview; Student 7, Group 3)

The preservice teachers appeared to struggle with the dimensions of activities and task schemes and representations. All groups identified one feature of the dimension activities and task schemes as supportive of student learning, but neither the mentioned features are to be considered essential for student learning nor are most of them assigned to the right dimension. Only group 3 successfully identifies features in the dimension representation as supportive of student learning. The features mentioned in this dimension by the other two groups suggest that the description of the dimension was not clear to these groups. While one group identifies a specific experiment as a representation of a phenomenon, the other group identifies several aspects of tasks as representations.

Further, certain groups also struggle with arguing why the features they discovered are supportive of student learning. All groups appeared to have at least a few difficulties explaining their decisions regarding why certain features are supportive of student learning. While Groups 1 and 2 only provide 2 out of 11 reasonable arguments and 4 or 6 superficial arguments, Group 3 provides 11 out of 13 reasonable arguments and 2 superficial arguments. Almost all the arguments mentioned by the preservice teachers to support their decisions are from an educational physics perspective and take the learner into account.

6.2. Comprehension: How Do the Preservice Teachers Comprehend the Key Ideas of the Introduction of the REF and the Tasks of the Group Activity before and after the Group Activity? (Q2)

Before the group activity, the idea of EF, the REF as a tool, and the different dimensions of the REF are introduced to the preservice teachers in approximately 10 min. We investigate how the preservice teachers comprehend the key ideas of the introduction and what those key ideas are intended for to better understand why the preservice teachers assigned certain features to certain dimensions; why they applied the REF in the manner they did in the group activity; or why they accept or reject the REF—that is the underlying analysis scheme.

In the following subsection, we first describe a key idea of the introduction and then how the preservice teachers comprehended these key ideas.

Essential features (EF) are the features of CMs that are essential for student learning. In the short interviews, all three groups stated that the idea of EF is comprehensible to them. Nonetheless, all preservice teachers describe EF differently—from the basic building blocks of a curriculum to a method for analyzing or reflecting upon curricula. Two preservice teachers defined EF as the core messages (German: “Kernbotschaften”) of a curriculum—that corresponds only with the dimension of key content ideas—and one preservice teacher as the structure plan of a teaching sequence. The members in Group 3 describe EF as the main ideas of a curriculum, what is important, and what is the rationale behind it.

Representation of essential features (REF) is a table that serves as a tool to guide the analysis of CMs with examples of EF and rationales for these EF. Although the preservice teachers disagree on what EF are and what the goal of the analysis is, they perceive the REF as a structured guideline or checklist to analyze CMs. After the group activity, all groups describe the REF as some kind of thread or checklist to guide what they examine in more detail. All groups perceived the REF coherently as something for analyzing CMs, and for “reflecting more” (translated from German) regarding different aspects of CMs, whereas they are usually unspecific in what exactly they reflect upon more.

However, the groups attribute different objectives to the activity of analyzing CMs with the REF. For example, Group 2 stated that the REF helps to identify the core messages of a curriculum and find them more quickly. The interviews with one member of Group 2 reveal that this member distinguishes EF with the core messages. Nevertheless, this preservice teacher conceptualizes the REF as a tool for considering how to teach these core messages to students in terms of which analogies or contexts are suitable for their intended learning processes. Another preservice teacher from Group 2 mentions that the REF provides the criteria for systematically critiquing and contrasting different curricula. A preservice teacher in Group 3 also describes the goal of analysis as being to critique CMs and eventually decide whether or not to use them in the classroom. Interestingly, this preservice teacher is the only one who explicitly mentions that a goal of the TLS is to learn how to reflect upon the didactic ideas of a curriculum in order to eventually use this curriculum effectively in the classroom.

The REF contains different dimensions of EF (see Table 1). The preservice teachers’ learning products indicate that certain dimensions of the REF are easier to understand than others (also see Section 6.1). The dimensions of activities and task schemes, representations, and subject-specific methods appear to be difficult to understand for the preservice teachers, because all groups appear to have very different perceptions of which aspects of the student handbook these dimensions are supposed to address. The preservice teachers also mention that they have difficulties with instructional dimensions. On the other hand, the dimensions of key content ideas, order of content, analogies and models, and contexts appear to be easier to understand. The preservice teachers interpret the dimension representations differently—from specific experiments to tasks.

6.3. Application: How Do the Preservice Teachers Analyze the Student Handbook with the REF in a Group Activity in the Prototypical TLS? (Q3)

In the group activity, the preservice teachers applied the REF to analyze a chapter of the student handbook. On the one hand, we want to know how they applied the REF for analyzing this chapter and what aspects of the CMs they examined; on the other hand, we want to know how they usually engage with CM.

According to the interviews, the preservice teachers analyzed the chapter of the student handbook with the REF differently in all groups. In Group 2, every group member reads the student handbook independently to obtain an overview, and thereafter, they discuss what is important, guided by the REF. In contrast, Group 1 uses the REF as a guiding framework when reading the student handbook for the first time. Instead of “merely reading” the student handbook, they focus on single aspects of the handbook and reflect upon what didactic idea is “behind” these aspects.

The preservice teachers mention that they became aware of certain new aspects of the student handbook that they may not have noticed without the REF. For example, Student 3 stated, “With the table, I looked more on these, or also because of the seminar, I became more aware of that these terms and strategies for conceptual change or which terms are used [in the student handbook]: Are these really the physical terms or are these elementarized terms—I just call them now this way, I do not know, what they are exactly called; that especially in the optics curriculum, there was often used [light] sender instead of [light] source, so that the pupils can imagine that [light is permanently emitted by an object] more easily” (translated from German; problem-centered interview; Student 3, Group 1). Without the REF, members of Group 2, for example, mentioned that they would not have looked at dimensions related to the organizational structure and would not have become aware of student conceptions while analyzing the student handbook. The preservice teachers in Group 1 were of the opinion that they focused more on strategies to support conceptual change because of the REF. Group 3 recognizes representations, instructional media and teaching methods when using the REF. Group 2 concludes from their analysis that there are barely any suggestions for the organizational structure in the CM. In contrast, Group 3 concludes that there are far more suggestions for the organizational structure in the CM than they expected, except for indications on how to organize a lesson as well as what students are supposed to do in different tasks. This indicates that the dimension activities and task schemes was not clear to them. This assumption is confirmed by the features of the CM that this group assigns to the dimension of activities and task schemes in the learning products.

In the interviews, the preservice teachers were also asked what they usually look at when engaging with a CM without the REF. All the preservice teachers stated that they usually examine more of what content a curriculum tackles. A few preservice teachers also tend to examine graphical representations, contexts, and the sequence of the content. The decision regarding whether the CMs are useful usually tends to be based on how difficult or comprehensible the preservice teachers perceive the content for their students and if the contexts may be interesting for their students. One preservice teacher even mentions that he usually does not think much about the ideas underlying different aspects of CM. In two interviews, a clear shift of focus was evident from the content of CM (what) to the strategies for teaching or learning the content used in CM (how) when analyzing CM with the REF. For example, Student 3 noticed: “When I myself first looked at curriculum or teaching materials, I first looked at what is explained, what topic it is about. Additionally, how profoundly does the material go into this topic. With the table, it’s a bit more about how? (…) So which approaches are used [in the CM]” (translated from German; Student 3, Group 2).

6.4. Acceptance: How Do the Preservice Teachers Accept the REF as a Tool for Analyzing CM? (Q4)

Based on the TAM, we investigate how useful and easy-to-use the preservice teachers perceive the REF to be for analyzing CM. We are also interested in their attitudes toward using the REF for analyzing CM and if they mention a behavioral intention to use the REF—that is the analysis scheme for analyzing CM in school practice.

All the preservice teachers perceive at least a few aspects of the REF as useful for analyzing CM after they have engaged in the group activity, according to the short interviews during the group activity. Before the group activity, the members of Group 1 were not sure if the REF would help them “reflect more [upon CM]” or “get better results [of the analysis]” (translated from German) when analyzing CM, mentioning the higher time requirement. After the group activity, the same group stated that the REF was helpful during the group activity, because they had a thread to follow and “look closer” at different aspects of the CM. Nevertheless, they cannot imagine using the REF in school practice, because of its time-consuming nature. Interestingly, they also mention that perhaps using the REF more often will help them develop some kind of intuition regarding how to analyze using the REF, and they may get faster at it. The other two groups perceive the REF as useful before and after applying it in the group activity because the REF provides a good orientation, helps structure the analysis, and reflects more upon different aspects of CMs. In the interview, Student 3 stated, “So it’s definitely more structured with the table,” and later, “It definitely makes sense to do it in a more structured way than just looking at it” (translated from German, Student 3, Group 2). The same preservice teacher also mentions the time-consuming nature of the REF and, therefore, believes that it is not useful for quickly analyzing all the CMs with the REF he finds in his future career as a teacher, but rather for deciding whether or not to use a particular curriculum for his class. Further, another preservice teacher perceived the REF as being generally useful for analyzing curriculum material that was new to him.

Overall, the preservice teachers perceive the REF as easy to use, but a few of the names and descriptions of the dimensions of the REF appear to be difficult to understand. They outline the checklist-like character of the REF and say that this helps focus on different aspects of CMs. A few of these aspects or dimensions of the REF are not clear to them. That may indicate that some of the names and descriptions of the dimensions are not self-explanatory or comprehensible. As mentioned earlier, the learning products also reveal that a few dimensions tend to be easier to understand than others. One preservice teacher mentions difficulties in distinguishing certain dimensions—particularly dimensions referring to instructional aspects of CM.

The preservice teachers’ attitudes toward using the REF for teacher education is very positive, except for the time-consuming nature of the REF. For example, Student 4 describes the experience of working with the REF in the following manner: “Basically, it [the experience] was good, and it forces you to take a closer look at it [CM] and you perhaps think about things [features] that you would not have thought about otherwise” (translated from German; problem-centered interview; Student 4, Group 2)

The intention to use the REF in their future work differs among the preservice teachers. Some of them can imagine using the REF in other seminars as well, but in a more concise version. One argument is that it is too time-consuming for school practice. Nevertheless, two preservice teachers mentioned in the interviews that they can imagine using the REF for analyzing curricula and that they may use it in school to obtain more insights into how to teach a certain topic.

7. Discussion

The aim of our overarching DBR research project was to develop a TLS for preservice teachers in teacher education programs. We derived several conjectures from teaching and learning theories and prior research to design such a TLS. The following is a central design conjecture in this study:

• Preservice teachers need tools that support them in analyzing CMs in a systematic manner (cf. [13,19,25]).

In this study, we aim to investigate whether the REF we developed is a suitable tool to support preservice teachers in analyzing CMs in the context of the TLS we developed. We consider the REF suitable for analyzing CMs within the TLS, if the REF supports the preservice teachers (1) in discovering different features of CM, (2) reflecting upon the role of these features for student learning, and (3) whether the REF—that is, the analysis scheme—is accepted by the preservice teachers as a tool for analyzing CMs, and (4) particularly for their future work. We also aim to refine the REF for the TLS and the TLS itself to improve support for preservice teachers. Therefore, in this section, we discuss the limitations of the study, the suitability of the REF, and the consequences we draw for refining or adding design conjectures. These design conjectures are epistemologically on the level of hypotheses or well-grounded assumptions (cf. [29,43]), which will guide future designs of the TLS and the research on these designs. We also suggest changes in the REF and the TLS for further designs.

The study has several limitations. The sample size is very small, and, therefore, the findings cannot be generalized. Nonetheless, the findings can be related to other research in similar contexts to obtain an initial idea of whether there are any specialties in the specific context of our study. Another limitation concerns the setting for the data collection. We collected the data from preservice teachers, who get graded, in a seminar of our teacher education program. This aspect is particularly important when interpreting the self-assessments regarding their experiences in the group activity. To address this aspect, the preservice teachers were explicitly told that their contributions would not affect their grading, and the teacher educator and the interviewer were not the same person in most of the cases. Nonetheless, the preservice teachers were aware that the interviewer and the researcher were from the same institute. Following the paradigm of DBR, we attempted to keep the seminar setting as realistic as possible. Therefore, the preservice teachers played the role of learners and we played the role of educators, which led to an imbalance of power.

7.1. Suitability of the REF to Discover Different Features of CMs

In the group activity, the preservice teachers discovered several features of CM from different dimensions of EF. Not all of the discovered features are considered supportive of student learning from a physics education perspective. In particular, features of the dimensions of key content ideas, order of key ideas, and contexts are predominantly successfully identified as supportive of student learning by the preservice teachers. According to the interviews, these content-related dimensions are the most familiar to them—as well as the most important ones for them.

The focus on content is also reported by Ben-Peretz et al. [13]: In their study, the preservice teachers at the beginning of their teacher education program relied more on criteria related to the content than on teaching and learning strategies. Ben-Peretz et al. explain their findings based on the past teaching experience of the preservice teachers. Lloyd and Behm [20] also report that familiarity with components of CMs plays a central role in preservice teachers’ analysis of CMs: preservice teachers tend to examine more closely the aspects of CMs that are familiar to them. The preservice teachers in our study barely had any teaching experience; therefore, we assume that the preservice teachers draw on their learning experience as students in school (cf. [20]).

Nonetheless, they also discover features that are new to them—according to their self-assessment in the interviews—and even identify some of them correctly as being supportive of student learning. In the interviews, they explicitly mention having considered the dimensions of strategies to support conceptual change and representation, as well as dimensions relating to the organization’s structure, when analyzing the student handbook. This supports the suggestion of Ben-Peretz et al. [13] to provide the preservice teachers with new criteria for analysis in order to facilitate a richer analysis. The preservice teachers in our study also mentioned a shift in their focus of analysis: from content to strategies used to support student learning of the content. Such a shift is also described by Ben-Peretz et al. [13] throughout teacher education. We are aware that, with the design of our study, we cannot claim that the preservice teachers discover more features or look at different aspects of the material with the REF than they do without the REF.

Further, the preservice teachers often struggle to assign unfamiliar features to the appropriate dimension. One possible reason for this is that the preservice teachers do not understand the meaning of this dimension. The dimensions of the REF should guide the analysis. For example, when a preservice teacher analyzes CM from the perspective of, for example, the dimension of representation, they should look at graphics and become aware of different elements of these graphics (e.g., the way propagation of light is represented). If preservice teachers do not understand what the dimensions mean, it will prevent them from adopting certain perspectives on CMs.

Based on these findings, we derive the following design conjectures:

• Preservice teachers require a scaffold (e.g., the REF) to guide the analysis of CM, thereby supporting them in examining a broader range of aspects—especially unfamiliar ones—to discover features of CM;
• Preservice teachers need additional support in discovering the unfamiliar features of CM.

Additionally, we suggest refining the descriptions of the dimensions of representation, activities and task schemes, subject-specific methods, and strategies of conceptual change based on the misinterpretations of the preservice teachers. Furthermore, we suggest providing further information regarding the dimensions and activities for understanding the dimensions in the TLS in advance of the group activity to support preservice teachers in discovering the features of these dimensions. Moreover, the short interviews also revealed that the preservice teachers struggle to distinguish dimensions related to the organizational structure. A few of the dimensions we derived from the literature are very theoretical and hardly ever play a role in curricula. Therefore, we reduce these dimensions to instructional media, learning tools, and methodological suggestions. The latter summarizes class organization, structure of lesson, teaching methods, and group organization.

7.2. Suitability of REF to Reflect upon Discovered Features

Two of the three groups appeared to struggle with arguing their decision regarding why certain features could be considered supportive of student learning. Nonetheless, in the interviews, the preservice teachers claimed to “reflect more” and analyze the student handbook “more intensively” when using the REF. Although we cannot support this self-assessment with any further data, we can at least claim that the preservice teachers reflected most of the time in the group activity by observing their discussions in the breakout rooms. We conclude that the prompt to reflect upon features of the CM leads at least to discussions regarding the role of several features for student learning.

Further, the preservice teachers report difficulties in articulating their decision in the presentation regarding their analysis process in the group activity. Assuming that students do indeed reflect more on the role of individual features in student learning, there are several possible reasons for these struggles. The preservice teachers, for example, may (1) lack the ability to reflect upon the role of features in student learning, (2) lack the ability to verbalize or articulate the results of their reflection, or (3) reflect upon a different interpretation of the discovered features than intended by the developers. Our study design is not able to distinguish between (1) and (2). We will first discuss (3) and then (1) and (2) by taking a closer look at the rationales provided.

Lloyd and Behm [20] state that the interpretation of discovered features and the reflection upon their role in student learning are often biased by the preservice teachers past learning experiences. They report that numerous preservice teachers interpreted a reform-oriented textbook in a traditional manner. One preservice teacher, for example, criticized the fact that there were no group tasks in the textbook, because there was no hint that students could work in groups. One group in our study came to a similar interpretation and identified “students work individually” as a feature of the dimension “social organization”.

Although the preservice teachers’ arguments are predominantly from a physics education perspective and take the learner into account, the arguments of the two groups rarely refer to pedagogical content knowledge [44]. The arguments are either superficial or unreasonable. In this study, we did not investigate how they came to their decision, but we aim to address this question in further research—this is another cycle of iterations of the DBR project. Therefore, we can only attempt to find possible reasons. The preservice teachers may lack, for example, pedagogical content knowledge to draw on. In a literature review, Remillard [11] emphasizes the role of different teacher characteristics, like pedagogical content knowledge or perceptions of student needs and capacities, in the interaction of teachers with CMs. In our current study, the preservice teachers did not attend many seminars on physics education and, therefore, may need more pedagogical content knowledge in advance. Additionally, the preservice teachers may not yet have developed reasonable perceptions of student needs and capacities. In their study on 20 mathematics preservice elementary teachers, Lloyd and Behm [20] describe that the majority of preservice teachers have naïve theories regarding teaching and learning mathematics. More research is required to investigate what the preservice teachers of our teacher education program draw on when reflecting upon the role of different features in student learning.

Although they struggle to articulate their decision, even the preservice teachers in Groups 1 and 2 identify a few EF of the CM as supportive of student learning, though the argumentation appears to be intuitive (e.g., “It supports student understanding”) or not reasonable to us.

In a case study with six elementary education students, Ball and Feiman-Neimser [24] conclude that preservice teachers need to learn how to justify their choices, particularly when developing standards for their justification. We conclude that preservice teachers need more learning opportunities in the TLS, in which their reflecting processes are scaffolded. Preservice teachers seem to need support in what they are reflecting on, what knowledge they can draw on, and how they can verbalize their reflection to justify their choices. Therefore, we derive the following design conjecture:

• Preservice teachers need support in reflecting upon features of CM and in verbalizing their reflection.

7.3. Preservice Teacher’s Perceived Suitability of REF for Analyzing CM

Even the most effective tool is worthless if the user does not utilize it for its intended purpose. According to the TAM [27], a person is more likely to use a tool when the person perceives the tool as useful and easy to use. Therefore, we investigated how suitable preservice teachers perceive the REF for analyzing CMs. To view the results in the correct light, we mention at this point again that the results may be overstated due to the relationship between the researcher/educator and the learner.

The preservice teachers perceive the REF, particularly after the group activity, as useful for analyzing CMs, despite the time-consuming nature of the analysis. Their attitudes towards the REF are mainly positive, and they agree on the usefulness of the REF for analyzing CM in group activity. According to the preservice teachers, the REF act as a guideline or checklist and contributes to a more systematic analysis than without the REF. Though the analysis takes more time than without the REF, they perceive the analysis of the student handbook as an important learning opportunity. Particularly after the analysis of the student handbook in the group activity, they emphasize how the REF has guided them and that they have found more features and analyzed CMs in more detail than without REF. According to the preservice teachers, the REF helps them to examine a broader range of aspects of the CM, and they perceive it as a useful guideline for analyzing CMs.

The REF appears to be broadly accepted by the preservice teachers as helpful for analyzing CM within the group activity. For the TLS, we assume that the learning opportunity in the group activity for analyzing CM with the REF plays a role in their perceived usefulness and their attitudes toward the REF for analyzing CM. Because this assumption is based only on the preservice teachers’ own reflection on the change of their attitudes before and after the group activity, a closer investigation of the change of these attitudes in future research is necessary to support this conclusion. With regard to the design of the TLS, we derive the following conjecture:

• Preservice teachers need opportunities to apply the REF to get used to the underlying scheme for analyzing CM and accept this scheme for analyzing CM.

7.4. Preservice Teacher’s Perceived Suitability of REF for Analyzing CM in Their Future Work

The rationale for the overarching DBR project is that, in school practice, teachers struggle with identifying and accepting important features of evidence-based curricula. Therefore, a central question is, “Do preservice teachers perceive this analysis scheme (or at least the idea of it) as useful for school practice?” The preservice teachers in this study barely have any experience in school practice. Nonetheless, we are interested in their perception of the usability of the REFs in school practice. If the preservice teachers already reject an in-depth analysis of CM for preparing and designing instruction before they are in school practice, it is important for us to address this issue in the TLS.

Most of the preservice teachers argue that the REF is not practical for everyday use in their future work because of the time-consuming nature of the analysis. Despite the positive attitudes toward the REF and the detailed analysis of the CM triggered by the REF in the group activity, the preservice teachers differ in their intention to use the REF for their future work. The main argument against using the REF is the time-consuming nature of the analysis. Some of the preservice teachers suggest using only a slimmed-down version of the REF. One group believes that analysis will go faster if they have more practice applying the REF.

The actual goal of the REF within the TLS is to support students by providing a strategy/scheme for analyzing CMs. Therefore, the REF represents this strategy/scheme for analyzing CMs and is more of a tool or scaffold to learn how to use this strategy for analyzing CMs than a tool for analyzing CMs in school practice. Because the preservice teachers mentioned that a slimmed-down version of the REF would be more suitable for school practice, we assume emphasizing this idea of the REF as a scaffold supports the acceptance amongst preservice teachers to use this strategy or parts of it to guide the analysis of CMs in the future. We assume that the preservice teachers need one or more additional opportunities in the TLS to analyze other CMs with the REF to familiarize themselves with the strategy underlying the REF—in particular, to support the idea of the REF as a scaffold to guide the analysis and to experience that they analyze faster when they do it more often. Therefore, we refine the design conjecture derived in the previous subsection:

• Preservice teachers need several opportunities to apply the scaffold to get used to the underlying strategy for analyzing CM and accept this strategy for analyzing CM;
• Emphasizing that the REF is not a tool for analyzing CM but a scaffold for learning to internalize a strategy of analysis supports preservice teachers’ perceived usefulness of the analysis strategy.

The provided design conjectures inform another cycle of iterations of our DBR project and guide the refinement of the REF and the TLS. These conjectures may also inform other researchers in designing a similar tool or scaffold to support (preservice) teachers in analyzing CM or a TLS with similar goals.