The Effects of a Modeling and Computational Thinking Professional Development Program on STEM Educators’ Perceptions toward Teaching Science and Engineering Practices

Abstract

Teachers’ integration of the Next Generation Science Standards and corresponding Science and Engineering Practices (SEPs) illustrate current science education reform in the United States. Effective teacher professional development (PD) on SEPs is essential for reform success. In this study, we evaluated the Nebraska STEM Education Conference, a PD program for middle school, high school, and first- and second-year post-secondary STEM teachers. This SEP-oriented PD program focused predominantly on the SEPs ‘developing and using models’ and ‘using mathematics and computational thinking.’ An electronic survey was used to measure participants’ (n = 45) prior integration of SEPs, influential factors and barriers to using SEPs, and changes to interest and confidence in using SEPs as a result of attending the PD program. Our results showed that teachers had limited prior use of SEPs in their teaching. Student interest and learning outcomes were the factors found to be most influential to teachers’ use of SEPs, while limited knowledge, confidence, and resources were the most commonly identified barriers. As a result of attending the PD program, participants significantly improved their confidence and interest to incorporate SEPs. We recommend continued SEP-oriented PD to foster successful NGSS integration and to advance reforms in science education.

1. Introduction

Science education in the United States has experienced considerable reform during the last several decades due to consistent reports of students’ underwhelming achievement in science [1]. Reforms in science instruction reflected the emergent knowledge of how students learn the best. Instead of direct instruction which often places students as passive learners, instructional shifts in science education emphasized the role of active learning. Guided by the theoretical underpinnings of constructivism, inquiry-based instruction became a leading approach to foster active learning in science education [2,3,4].

Although varying perspectives of what constitutes inquiry-based instruction exist [5,6], there is agreement that inquiry-based approaches are student-centered and engage students in investigations that require them to reason, obtain information, and derive solutions [7]. Elements of inquiry-based instruction are similar to the process of inquiry used in the science community to make discoveries [8]. Models to implement inquiry-based instruction, such as the Biological Sciences Curriculum Study (BSCS) 5E Instructional Model [9], align well with calls for science education to “cultivate students’ scientific habits of mind, develop their capability to engage in scientific inquiry, and teach them how to reason in a scientific context” [4] (p. 41).

The highly renowned publication, A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas [4], simplified as The Framework, expands the call for science learning to be discovery-based, while stressing the need for students to progressively understand and apply core ideas in science and engineering [10]. The Framework accentuates the development of students’ understanding of the process of science and engineering, not only to expand their problem solving and critical thinking skills but also to aid their ability to become better consumers of scientific information [11].

The Framework paved the way for a collaborative state-led process between the National Research Council (NRC), the National Science Teachers Association, the American Association for the Advancement of Science (AAAS), and Achieve to create the Next Generation Science Standards (NGSS) [12].

1.1. The Next Generation Science Standards

The NGSS have been adopted by 20 states, and 24 additional states have developed similar standards based upon The Framework [13]. In line with The Framework, the NGSS were designed to expose students to situations where they mimic the processes used by scientists and engineers to create knowledge and solve problems. Furthermore, the NGSS were created to better position students to discover scientific principles through explaining processes connected to larger questions as opposed to investigating isolated processes [14].

Both the NGSS and The Framework stress the importance of the integration of engineering aspects in science education in order to extend the relevance of STEM to students’ everyday life [15]. Incorporating engineering principles into science curricula is a critical reform to widespread science education and the inclusion of engineering principles within the NGSS expands the need for science teachers to implement teaching methods beyond traditional inquiry [16]. According to the National Science Teaching Association (2014) [15], “scientific inquiry involves the formation of a question that can be answered through investigation, while engineering design involves the formation of a problem that can be solved through design” (para. 1). The NGSS highlight the need for students to understand the importance and relationship between scientific investigations and engineering design to solve complicated problems through innovations [17,18,19]. The primary goal of the NGSS is to provide all students with the scientific and technological knowledge and skills to make informed decisions and to provide interest and career avenues in STEM disciplines [4].

1.2. Science and Engineering Practices (SEPs)

The NGSS include three dimensions embedded in instruction at all levels: core ideas, crosscutting concepts, and science and engineering practices (SEPs). Core ideas illustrate specific content and subject matter, while crosscutting concepts illustrate the connectedness between foundational areas of science. The SEPs are the practices identified by The Framework as critical for students to understand, and mirror the practices used by scientists and engineers. A key component of the NGSS is the integration of SEPs within teaching content at all levels.

The eight SEPs are:

• Asking questions and defining problems.
• Developing and using models.
• Planning and carrying out investigations.
• Analyzing and interpreting data.
• Using mathematics and computational thinking.
• Constructing explanations and designing solutions.
• Engaging in argument from evidence.
• Obtaining, evaluating, and communicating information.

1.3. Modeling and Computational Thinking SEPs

While the formation of some SEPs (e.g., asking questions and defining problems; planning and carrying out investigations; analyzing and interpreting data) aligned with the historic process of scientific discovery and with the development of traditional inquiry-based instruction [9], other SEPs emerged in response to rapid changes seen in science and engineering industries and the corresponding need for a skilled 21st-century workforce. For example, the inclusion of modeling and computational thinking was influenced by technological advancements in science and engineering leading to the collection and analyses of large data sets [20,21], the promotion of systems thinking to combat complex problems [22,23,24], and the global development of information and communication technology (ICT) in industry [25,26,27] and in broader education [28,29,30].

The Framework provides specific goals that students should be able to do for each SEP by grade 12. Example goals of the second SEP, developing and using models, state the students should be able to: “construct drawings or diagrams as representations of events or systems”, “represent and explain phenomena with multiple types of models”, and “use [provided] computer simulations or simulations developed with simple simulation tools as a tool for understanding the investigation aspects of a system” [13]. Students’ understanding and use of basic modeling should start at the earliest of grade levels and can include pictures and scale modes [13]. As students’ progress in age and ability, modeling activities should become more advanced, such as having students use models to illustrate their own ideas, findings from investigation, and systematic relationships [13]. Whereas modeling focuses on more visual representations, the fifth SEP, using mathematical and computational thinking, focuses on the role of numerical representations to examine relationships and outcomes. The use of mathematics is fundamental in science and engineering and begins with simple measurements and data comparisons. As students advance, they should be exposed to equations and the use of computer programs (e.g., spreadsheets, graphs) to analyze large data sets and illustrate findings [13]. In general, computational thinking applies the use of mathematical operations to provide solutions to complex problems using concepts of computer science [31].

The Framework describes students’ use of computers as an integral component to accomplish the goals described in both SEP 4 and 5. Advanced work may span across SEPs, such as using mathematical relationships to build computer models. Computational modeling provides relevance to modern scientific discovery and engineering design [13] and has been shown to foster systems thinking and equitable learning opportunities among diverse student populations [32,33,34].

1.4. Professional Development and SEPs

The NRC, the National Science Teaching Association, the AAAS, and many other national and state organizations and governing bodies call for the successful integration of the NGSS and corresponding SEPs into science classrooms across the United States. In states that have adopted the NGSS or similar standards, school leaders and science teachers are tasked with the responsibility of incorporating them in the curriculum. The appropriate integration of SEPs in instruction may be particularly challenging as recent literature indicates teachers’ minimal use of active learning strategies, including the use of inquiry-based approaches in science-oriented classrooms [35,36,37,38]. Reasons for teachers’ limited use of such strategies include a lack of confidence incorporating these approaches [39], minimal preparation and opportunities for development [40,41], insufficient time needed to effectively implement them [42], and low access to instructional resources [43].

Teacher professional development (PD) on SEPs and their integration with the NGSS has been described as an essential component for teacher and student success [18,44,45]. Following a PD in engineering education with secondary STEM teachers, researchers found that teachers significantly improved their confidence in teaching engineering [18]. Furthermore, after the PD, participants held positive perceptions toward integrating SEPs in their disciplinary content, especially to develop students’ critical thinking and problem-solving skills. In another study [10], researchers qualitatively assessed a four-day teacher PD on project-based learning and alignment to NGSS. The researchers found the PD improved teachers’ understanding of the NGSS and implementation of project-based learning. Participants also described the PD as valuable to their development as science teachers. Prior research [17] has found that collaborative PD focused on the integration of SEPs expanded teachers’ motivation to reform their instructional practices. In a study that evaluated the impact of an inquiry-based PD experience for Nebraska teachers, researchers found the program improved teachers’ confidence using inquiry-based approaches and helped participants meet state standards [46]. The present study examined the impact of a SEP-oriented PD program held in-person during the July of 2021 for middle school, high school, and first- and second-year post-secondary STEM educators in Nebraska, USA.

2. Purpose and Research Questions

The purpose of this research was to evaluate the effectiveness of the Nebraska STEM Education Conference and to utilize this PD as a case study that can be modified and replicated to promote teachers’ integration of SEPs in science classrooms. It has been illustrated that successful PD increases teacher knowledge and skills and promotes changes in teachers’ attitudes and beliefs, which lead to positive changes in instruction and ultimately improved student learning [47]. Improving teacher perceptions regarding a teaching concept influence their confidence and teaching effectiveness toward the concept [48]. The goal of the Nebraska STEM Education Conference was to improve teachers’ confidence and interest toward implementing SEPs in their teaching, with a particular focus on modeling and computational thinking. Our vision was to ultimately increase teachers’ use of active learning strategies and SEPs in instruction, thereby improving student performance, motivation, and interest in STEM. The research questions that guided this study were:

• How often did participants use SEPs in their teaching prior to attending the PD?
• How did PD attendance impact participants’ interest and confidence using SEPs?
• What factors are associated with participants’ post-PD confidence using SEPs?
• What factors are the most influential to participants’ use of SEPs?
• What barriers exist that limit participants’ use of SEPs?
• What benefits and recommendations do participants identify regarding the PD?

3. A Framework for Teacher Professional Development and the Nebraska STEM Education Conference

Desimone’s (2009) [47] framework for analyzing teacher PD was used to guide the development of our study and has been described as an ideal framework for PD evaluation studies [49]. According to the framework, the core features of PD should be described in PD evaluation. The core features of Desimone’s (2009) [47] framework include: content focus; active learning; coherence; duration; and collective participation.

In the following sections, we describe characteristics of the core features and how the PD event under our investigation, the Nebraska STEM Education Conference, utilized these features.

3.1. Content Focus

Content focus describes the subject matter of the PD. Appropriate content is the most important component for successful PD events [47]. Researchers [49] have summarized that “content-focused professional development enhances teachers’ knowledge, reforms teaching practice, or improves student learning” (p. 13). Prior studies evaluated PD events with content related to scientific inquiry [46], project-based learning [10], integrating SEPs [17], and engineering practices [18]. In addition to the subject matter of the PD, best practices in delivering PD should also include how students best learn the content [50].

The Nebraska STEM Education Conference was designed to increase teachers’ awareness, knowledge, confidence, and ability to integrate SEPs and core scientific competencies into STEM-related curriculum. The conference focused on two of the most seemingly difficult SEPs to implement related to modeling and computational thinking. Throughout the PD, content focused on the use of inquiry and problem-based learning strategies to promote students’ skill development. Sessions introduced participants to NGSS (crosscutting concepts, core ideas, and SEPs) and strategies to enhance student inquiry and design. While the conference provided participants with some experience in most of the SEPs, this framework was applied specifically toward helping educators gain experience with the SEPs of developing and using models and using mathematics and computational thinking. Several sessions exposed participants to concrete examples of these practices in use (e.g., Cell Collective Workshop, Brome Inquiry Workshop).

3.2. Active Learning

Active learning describes how teachers will be engaged and interact with the content of the PD. Best practices in delivering teacher PD are those that engage teachers in active learning, including providing opportunities for them to observe, receive feedback, and make presentations, over passive learning, such as solely listening to speakers [50].

The Nebraska STEM Education Conference was designed to mimic the active learning styles the conference promoted. For example, teachers participated as ‘students’ in several sessions that required them to use inquiry and problem-solving skills. Furthermore, participants worked in teams to design conceptual models and scientific experiments and developed code maps to investigate an algorithmic solution to a biological problem. Participants also were tasked to determine how they would integrate active learning strategies and SEPs into their respective curricula, as well as utilizing reflection strategies after being exposed to content areas. Participants were given time to reflect on their understanding of these particular SEPs and how they relate to others that they were more familiar with. Lastly, throughout the conference, participants learned from each other through teacher-led panels, small group discussions, and formal and informal peer presentations (e.g., think-pair-share).

3.3. Coherence

Coherence refers to the alignment of the PD to existing knowledge, beliefs, and academic standards. Appropriate PD should meet the needs of students, teachers, and administrators. When describing best practices in teacher PD, researchers have posited that appropriate coherence means that the PD is guided by “school, district, and state reforms and policies” [50] (p. 253). The promotion of the NGSS by national science education organizations and states’ mass adoption of NGSS or NGSS-like standards demonstrate a clear agenda for science education reform. Teacher PD on SEPs and their integration with the NGSS have been described as an important factor for reform success [18,44,45].

Appropriate coherence was a foundational component of the Nebraska STEM Education Conference. In 2017, Nebraska adopted the Nebraska’s College and Career Ready Standards for Science (NCCRS-S). These standards closely resemble the NGSS and are distinguished as NGSS-like standards that are guided by The Framework. The NCCRS-S include SEPs, disciplinary core ideas, and crosscutting concepts. The PD itself was offered with the purpose of improving Nebraska STEM teachers’ implementation of NCCRS-S, with a specific focus on SEPs, and additional content on disciplinary core ideas and crosscutting concepts. The PD also intended to bridge the gap between secondary and post-secondary education by illustrating connections between NGSS and Vision and Change reform in undergraduate science education [51]. The PD was offered in collaboration and support between the University of Nebraska-Lincoln, Doane University, Nebraska’s Established Program to Stimulate Competitive Research (EPSCoR), and the Nebraska Association of Teachers of Science (NATS), which demonstrated additional coherence.

3.4. Duration

Duration includes the length and amount of contact time offered by the PD. Prior research has found that the number of hours of PD attended by teachers is positively associated with their use of inquiry-based teaching practices and an investigative classroom culture [52]. Many states and teaching districts have required teachers to obtain a certain number of continuing education hours that PD events can count toward [53]. Although the duration of PD programs can vary from a single-hour session to recurring, multi-hour sessions throughout the year, PD activities that include 20 or more contact hours and have some sustained duration throughout the school year follow best practices [50].

The Nebraska STEM Education Conference was held in-person for two consecutive weekdays in July of 2021 and included approximately 20 hours of participant contact time. Lastly, all participants received complimentary registration to the yearly NATS state science teacher conference that was held after the PD and received a complimentary year membership to the NATS organization. We believe the duration of the PD aligned well to best practices and offered participants opportunities for sustained discourse in PD-related activities.

3.5. Collective Participation

Collective participation refers to the ability of the PD to provide discussion and collaboration among teachers to build an interactive learning community. Collective participation is enhanced when participants share similar characteristics such as teaching at the same school, teaching the same grade level, or teaching the same subject [50].

Collective participation was a foundational component of the Nebraska STEM Education Conference. All participants held shared experiences teaching STEM. To foster collective participation, lead teachers who were exemplary in executing SEPs were identified. The lead teachers participated in the PD and panel discussions, presented material, and led small group discussions with participants. Throughout the PD, collective participation was stressed with the goal of all participants learning from each other. The inclusion of middle school, high school, and first- and second-year post-secondary STEM educators was intentional to promote sustained collaborations after the PD; however, small-group opportunities for participants to work within the same grade level and subject were provided. Additional information on the Nebraska STEM Education conference can be found within the supplemental materials.

4. Methods

4.1. Conference Attendees and Study Participants

A total of 55 attendees participated in the Nebraska STEM Education Conference. Immediately following the conclusion of the PD program, all attendees were invited to complete an electronic survey administered through Qualtrics by a third-party provider. A total of 47 of 55 attendees responded to the survey indicating an 85.5% response rate. The survey completion rate was 95.7% for a total of 45 usable responses for our sample.

Table 1. Participant demographics (n = 45).

Participant Characteristics	Frequency (n)	Percentage (%)
Gender
Male	12	26.7%
Female	33	73.3%
Age
19–25	3	6.7%
26–35	9	20.0%
36–45	13	28.9%
46–55	16	35.6%
56–65	4	8.9%
Education Level
Bachelors	6	13.3%
Bachelors + 18 or more credits	8	17.8%
Masters	9	20.0%
Masters + 18 or more credits	11	24.4%
Doctorate	11	24.4%
Years of Teaching Experience
0–2 Years	4	9.1%
3–5 Years	9	20.5%
6–10 Years	5	11.4%
11–15 Years	6	13.6%
More than 15 Years	20	45.5%
Teaching Position
Middle School Teacher	5	11.1%
High School Teacher	24	53.3%
College or University Teacher	8	17.8%
Other	8	17.8%

illustrates demographic information for the participants in this study.

4.2. Instrumentation and Data Collection

An electronic survey was used to assess the effectiveness of the Nebraska STEM Education Conference. Informed consent was collected from all participants at the start of the survey. The survey contained a total of 26 questions and included several quantitative scales and open-ended questions related to participants’ implementation of SEPs, several questions related to assessment of the conference, and nine questions pertaining to participant demographics and characteristics.

Five scales were used to measure participants’ perceptions toward using SEPs. Each of these scales included five items, with each item describing an SEP that was central to the PD. The SEPs under our investigation were: (1) asking questions and defining problems; (2) planning and carrying out investigation; (3) analyzing and interpreting data; (4) developing and using models; and (5) using mathematics and computational thinking.

The first scale on our survey asked respondents to consider a STEM-related course they teach and to report their frequency of use, prior to attending the conference, for each of the five SEPs under our investigation. Frequency options were rarely or never, a few times during the course, once every two to three weeks, about once a week, and more than once per week.

Interest toward using SEPs was measured by a scale containing five items, with each item corresponding to one of the five SEPs under our investigation. Respondents were asked to indicate their interest toward using each SEP through a 5-point, Likert-type scale (1 = very disinterested, 2 = somewhat disinterested, 3 = neither interested nor disinterested, 4 = somewhat interested, 5 = very interested). The same scale was used twice, once to measure participants’ interest toward using SEPs prior to attending the PD, retrospectively [54,55,56], and again to measure interest after completion of the PD.

Two scales were used to measure participants’ confidence toward using SEPs. The same five SEPs under our investigation were measured through a 4-point Likert-type scale (1 = not at all confident, 2 = slightly confident, 3 = moderately confident, 4 = extremely confident). Participants’ confidence using SEPs prior to the PD was also measured retrospectively [54,55,56].

Factors having the most influence on participants’ decision to use SEPs were determined by a ranking question. Participants were asked to rank a list of nine items from one, most influential, to nine, least influential, based upon the degree of relative influence items had on their decision to use SEPs in their teaching. In addition, several open-ended questions were included on the survey to provide qualitative data to strengthen our evaluation. Open-ended questions asked respondents to report their top three barriers to implementing SEPs, how their perceptions of implementing modeling or computational thinking have changed as a result of the PD, and what teaching adjustments they plan on making in the future. All participants were asked to report demographic information pertaining to their age, gender, race/ethnicity, education level, years of teaching experience, and level of teaching position.

4.3. Data Analysis

All data were analyzed using SPSS version 26. Descriptive statistics, including means and frequencies, were used to address research questions 1 and 4. Paired-samples t-tests were used to answer research question 2, and multiple regression was used to assess research question 3. Qualitative coding and frequency reports were used to address questions 5 and 6. Effect size statistics were analyzed by Cohen’s d for paired-samples t-test and by R² for multiple regression [57]. Significance of inferential statistics were established at a p value of less than 0.05.

4.4. Validity and Reliability

The validity and reliability of the survey instrument developed for this study was assessed in several ways. The retrospective design used to measure change has been shown to improve respondent precision and awareness over standard pretest and posttest designs [54]. Additionally, the full survey instrument was reviewed by a panel of experts to improve validity [58]. Post hoc reliability analysis of the two interest scales yielded Cronbach’s alphas of 0.87 and 0.94, and therefore, were determined to be reliable [59]. Lastly, post hoc reliability of the two confidence scales achieved Cronbach’s alphas of 0.87 and 0.83 and also were determined to be reliable [59].

5. Results

5.1. Participants’ Prior Use of SEPs in Teaching

Respondents were asked to consider STEM-related courses they teach and to estimate the frequency they implemented five SEPs into their courses during the last year.

Table 2. Frequencies of participants’ prior use of SEPs during teaching (n = 43).

SEP	Rarely or Never	A Few Times during the Course	Once Every 2 to 3 Weeks	About Once a Week	More Than Once per Week
Asking questions and defining problems	0 (0.0%)	6 (14.0%)	15 (34.9%)	10 (23.3%)	12 (27.9%)
Planning and carrying out investigations	1 (2.3%)	11 (25.6%)	18 (41.9%)	10 (23.3%)	3 (7%)
Analyzing and interpreting data	0 (0.0%)	4 (9.3%)	12 (27.9%)	19 (44.2%)	8 (18.6%)
Developing and using models	1 (2.3%)	7 (16.3%)	12 (27.9%)	17 (39.5%)	6 (14.0%)
Using mathematics and computational thinking	1 (2.3%)	7 (14.0%)	17 (39.5%)	10 (23.3%)	9 (20.9%)

illustrates respondents’ frequencies of use for the SEPs under our investigation.

As illustrated in Table 2, the practice of having students analyzing and interpreting data was most frequently used. Nineteen (44.2%) respondents implemented this practice about once a week, followed by 27.9% (n = 12) of teachers using it once every two to three weeks. The next most frequently implemented SEP was having students asking questions and defining problems. Approximately 28% (n = 12) of teachers used this SEP more than once a week, while a little over one-third (n = 15) of teachers used it once every two to three weeks. Teachers’ use of having students developing and using models and using mathematics and computation thinking was similar. The majority of teachers (39.5%, n = 17) had students developing and using models about once a week and had students using mathematics and computational thinking once every two to three weeks. Interestingly, the least commonly used SEP was having students planning and carrying out investigations, with most (41.9%, n = 18) using the practice once every two to three weeks and over a quarter implementing this practice a few times during the course or less.

5.2. Changes to Participants’ Interest and Confidence in Using SEPs as a Result of the PD

Changes to participants’ interest in using SEPs were assessed by analyzing pre- and post-test responses to a 5-point Likert scale. Paired-sample tests were used to determine if changes were significant.

Table 3. Participants’ interest using SEPs before and after attending the PD (n = 43).

Item		Very Disinterested	Somewhat Disinterested	Neither Interested nor Disinterested	Somewhat Interested	Very Interested	M (SD)	Paired t-Test
Asking questions and defining problems	Before	0 (0.0%)	2 (4.7%)	5 (11.6%)	12 (27.9%)	24 (55.8%)	4.35 (0.87)	t = −2.60 p = 0.013 *
	After	0 (0.0%)	0 (0.0%)	2 (4.7%)	12 (27.9%)	29 (67.4%)	4.63 (0.58)
Planning and carrying out investigations	Before	0 (0.0%)	0 (0.0%)	7 (16.3%)	13 (30.2%)	23 (53.5%)	4.37 (0.76)	t = −2.35 p = 0.024 *
	After	0 (0.0%)	0 (0.0%)	3 (7.0%)	11 (25.6%)	29 (67.4%)	4.60 (0.62)
Analyzing and interpreting data	Before	1 (2.3%)	0 (0.0%)	6 (14.0%)	15 (34.9%)	21 (48.8%)	4.28 (0.88)	t = −3.18 p = 0.003 *
	After	0 (0.0%)	0 (0.0%)	3 (7.0%)	10 (23.3%)	30 (69.8%)	4.63 (0.62)
Developing and using models	Before	0 (0.0%)	4 (9.3%)	4 (9.3%)	16 (37.2%)	19 (44.2%)	4.16 (0.95)	t = −4.06 p < 0.001 *
	After	0 (0.0%)	0 (0.0%)	2 (4.7%)	10 (23.3%)	31 (72.1%)	4.67 (0.57)
Using math and computational thinking	Before	0 (0.0%)	3 (7.0%)	11 (25.6%)	14 (32.6%)	15 (34.9%)	3.95 (0.95)	t = −4.39 p < 0.001 *
	After	0 (0.0%)	0 (0.0%)	3 (7.0%)	13 (30.2%)	27 (62.8%)	4.56 (0.63)
SEP Interest Scale	Before						4.22 (0.72)	t = −4.34 p < 0.001 *
	After						4.62 (0.54)

* significance at p < 0.05.

illustrates participants’ interest using SEPs before and after attending the PD.

As reported in Table 3, prior to attending the PD, participants largely indicated being more interested than uninterested in using SEPs in their teaching. In fact, the average interest using the five SEPs prior to attending the PD session was 4.22 (SD = 0.72) on a five-point Likert scale, categorized as being somewhat interested when considering real limits. The SEPs of most interest were planning and carrying out investigations (M = 4.37, SD = 0.76) and asking questions and defining problems (M = 4.35, SD = 0.87). Respondents were least interested, yet still being somewhat interested, in using math and computation thinking (M = 3.95, SD = 0.95) prior to the PD.

Interest in using SEPs increased after participants attended the PD as noted by the scale average increasing to 4.62 (SD = 0.52), categorized as being very interested when considering real limits. In fact, interest toward using each of the five SEPs increased. The SEP item with the highest post-PD interest was developing and using models (M = 4.67, SD = 0.57), followed by analyzing and interpreting data (M = 4.63, SD = 0.62) and asking questions and defining problems (M = 4.63, SD = 0.58). Paired-samples t-tests were conducted to determine if changes were significant. Results demonstrate that participants held a significant increase in interest toward using SEPs after attending the PD (t = 4.34, p < 0.001). The effect size statistic using Cohen’s d was 0.66, which can be interpreted as a medium effect size [57]. Significant increases were also observed for each of the individual SEPs, respectively. The SEP item with the largest change in interest pre- to post-PD was using mathematics and computational thinking.

Changes to participants’ confidence in using SEPs were measured before (retrospectively) and after attending the PD using a four-point Likert-scale. Paired-sample tests were used to determine if changes were significant.

Table 4. Participants’ confidence using SEPs before and after attending the PD (n = 43).

SEP		Not at All Confident	Slightly Confident	Moderately Confident	Extremely Confident	M (SD)	Paired t-Test
Asking questions and defining problems	Before	1 (2.3%)	14 (32.6%)	19 (44.2%)	9 (20.9%)	2.84 (0.79)	t = −5.1 p = < 0.001 *
	After	0 (0.0%)	2 (4.7%)	25 (58.1%)	16 (37.2%)	3.33 (0.57)
Planning and carrying out investigations	Before	2 (4.7%)	14 (32.6%)	17 (39.5%)	10 (23.3%)	2.81 (0.85)	t = −4.9 p = < 0.001 *
	After	0 (0.0%)	4 (9.3%)	24 (55.8%)	15 (34.9%)	3.26 (0.62)
Analyzing and interpreting data	Before	2 (4.7%)	14 (32.6%)	20 (46.5%)	7 (16.3%)	2.74 (0.79)	t = −5.2 p = < 0.001 *
	After	0 (0.0%)	3 (7.0%)	28 (65.1%)	12 (27.9%)	3.21 (0.56)
Developing and using models	Before	6 (14.0%)	15 (34.9%)	12 (27.9%)	10 (23.3%)	2.60 (1.00)	t = −6.5 p < 0.001 *
	After	1 (2.3%)	3 (7.0%)	12 (51.2%)	17 (39.5%)	3.28 (0.70)
Using math and computational thinking	Before	7 (16.3%)	17 (39.5%)	13 (30.2%)	6 (14.0%)	2.42 (0.93)	t = −5.9 p <.001 *
	After	3 (7.0%)	4 (9.3%)	23 (53.5%)	13 (30.2%)	3.07 (0.83)
SEP Confidence Scale	Before					2.68 (0.71)	t = −7.11 p < 0.001 *
	After					3.23 (0.51)

* significance at p < 0.05.

illustrates participants’ confidence toward using SEPs before and after attending the PD.

As can be seen from Table 4, respondents reported being between slightly and moderately confident (M = 2.68, SD = 0.71) toward using SEPs prior to attending the PD. Of the five SEPs, respondents were least confident using mathematics and computational thinking (M = 2.42, SD = 0.93). Most respondents (39.5%, n = 17) indicated being only slightly confident using this practice. Participants initially had the most confidence asking questions and defining problems (M = 2.84, SD = 0.70), followed by planning and carrying out investigations (M = 2.81, SD = 0.85).

Confidence toward using SEPs increased for all practices after the PD. Participants still reported having the most confidence asking questions and defining problems (M = 3.33, SD = 0.57). A significant mean increase of 0.49 was observed for this practice. Respondents still indicated being the least confident using mathematics and computational thinking (M = 3.07, SD = 0.83). However, confidence using this SEP improved by 0.55, and over 80% of respondents identified being either moderately or extremely confident at the end of the PD. Paired-samples t-tests were conducted to determine if gains in confidence toward SEPs were significant. The mean increase in confidence using SEPs increased from 2.68 (SD = 0.71) to 3.23 (SD = 0.51), and this change was significant (t = 7.11, p < 0.001). The effect size statistic using Cohen’s d was greater than 0.8, and therefore considered a large effect size [57]. Significant increases were also observed for each SEP practice at p < 0.001.

5.3. Factors Associated with Participants’ Post-PD Confidence in Using SEPs

To answer research question 3, we used a multiple linear regression with participants’ mean SEP confidence score as the dependent variable. Due to a sample size of 45 participants, four predictor variables were appropriate for the model. The four predator variables selected to have potential impact were respondents’: (a) level of education; (b) years of teaching experience; (c) prior frequency of SEP implementation; and (d) interest using SEPs. Results of the multiple regression are illustrated in

Table 5. Multiple Linear Regression Model Showing Predictors of Post-PD Confidence Using SEPs.

Predictor Variable	B (Coefficient)	SEB	β (Standardized Coefficient)	t	p-Value
Constant	1.488	0.715		2.081	0.044 *
Years of Teaching Experience	−0.083	0.061	−2.41	−1.355	0.184
Educational Attainment	0.096	0.046	0.388	2.094	0.043 *
Post-PD SEP Interest	0.237	0.137	0.251	1.724	0.093
Prior SEP Frequency	0.153	0.106	0.221	1.443	0.157

* significance at p < 0.05.

.

Results indicated a significant model (p < 0.05) that explained 24.9% of the variance, which is between a medium and large effect size [57]. However, the only significant predictor variable in the model was respondents’ level of education. In our model, respondents’ level of education was classified by seven linear categories (1 = bachelors, 2 = bachelors + 18 credits, … 7 = doctoral degree). For every unit of increase in educational level, teachers improved their confidence by 0.39 on the 4-point SEP confidence scale. Interestingly, years of teaching experience, prior frequency of SEP implementation, and interest using SEPs were not significant predictors of confidence using SEPs. A follow-up test showed no significant difference between grade level taught and confidence incorporating SEPs. Table 5 illustrates the results of the model.

5.4. Factors Most Influential to Using SEPs

To identify the perceived factors most influential to participants’ use of SEPs, respondents were asked to rank a list of nine factors from 1, most influential, to 9, least influential.

Table 6. Participants’ rankings of factors most influential to using SEPs during teaching (n = 45).

Rank	Factor	Mean 1	SD
1	Student learning outcomes	3.20	2.50
2	Student interest	3.42	2.12
3	The current topic	4.44	2.67
4	Time to plan lessons to incorporate SEPs	4.53	2.12
5	Student ability	4.96	2.29
6	Time in class relative to other activities	5.07	2.02
7	Previous experience using SEPs	5.51	2.46
8	The encouragement or experience of peers	6.29	1.89
9	Administrative mandates or recommendations	7.58	2.22

¹ Mean ratings are based on a scale of 1 = most important to 9 = least important.

illustrates response rankings from most influential to least influential.

As outlined by Table 6, respondents ranked the most influential factor as student learning outcomes, with a mean ranking of 3.20 (SD = 2.50). Other factors determined most influential were student interest (M = 3.42, SD = 2.12), the current course topic (M = 4.44, SD = 2.67), and the time needed to plan lessons incorporating SEPs (M = 4.53, SD = 2.12). The least influential factors, as determined by respondents, were administrative mandates or recommendations (M = 7.58, SD = 2.22), encouragement or experiences of peers (M = 6.29, SD = 1.89), and instructors’ previous experience using SEPs (M = 5.51, SD = 2.46).

5.5. Barriers to Incorporate SEPs into Teaching

Research question 5 was addressed by an open-ended response question on our survey. We asked participants to focus on the SEPs of modeling and computational thinking and to identify their top three to five barriers to incorporating these SEPs into their classroom. Responses were coded and frequencies were used to identify the most commonly mentioned barriers. The most mentioned barriers are presented in

Table 7. Participants’ most described barriers to implementing modeling and computational thinking SEPs.

Rank	Barrier	n
1	Lack of knowledge and low perceived ability to effectively implement the SEPs	22
2	Limited access to resources and materials on the SEPs	17
3	Lack of time to adequately plan lessons that integrate the SEPs	16
4	Student limitations (e.g., low ability of students)	11
5	The social environment (e.g., social norms, reluctance of peers, etc.)	7

below.

5.6. Participants’ Perceived Benefits and Recommendations Regarding the PD

Research question 6 was addressed by several open-ended response questions on the survey instrument. Responses were coded and frequencies were identified. After the PD, participants reported positive changes to their perceptions of implementing SEPs. Most notably, teachers identified dramatic changes related to the use of modeling and computational thinking in the classroom. Some teachers reported a better fundamental understanding of how students create or develop models (n = 7), and how computational thinking can be implemented in the classroom (n = 10). Other teachers (n = 7) expressed that the PD helped them to better understand that modeling can happen in a variety of ways in the classroom. For example, one participant stated,

“It is important to think of other ways to implement and purposefully embed modeling and computational thinking into the learning environment… there is no one set way to teach modeling and computational thinking.”

For others, the PD reinforced the importance of modeling (n = 5) and computational thinking (n = 3). Many participants described that they were able to learn about new resources that will better allow them to integrate SEPs in the classroom, as described by one participant,

“I am more aware of some resources that are out there. I will definitely try to use these things now that I am aware of them.”

In fact, most participants (n = 37) described specific tools identified in the conference that they intend to use in the future, including the Cell Collective (n = 7), 3D tools (n = 5), and coding software (n = 4).

Overall, qualitative findings demonstrated that teachers had a positive experience attending the Nebraska STEM Education conference. Teachers (n = 11) appreciated the conference accommodations (e.g., venue, hotel, meals), and many shared general positive comments about the conference. Seven teachers specifically mentioned enjoying the interaction and networking opportunities with teachers of other grade levels. For example, one teacher stated,

“The strategies [used in the PD] to help get to know lots of people were great! I really appreciated making connections to other science teachers,”

and another teacher added,

“…this conference was very well done and allowed connections between secondary and higher education that were desperately need.”

Participants’ recommendations for improving the conference were largely centered around conference logistics. Several teachers suggested that additional breaks were needed and that the first day of the conference was too long. A couple of teachers suggested extending the conference to three days as opposed to two days to spread out the content more.

6. Discussion

The Framework describes SEPs as a necessary component to science education reform in the United States to meet the needs of the 21st century. The SEPs distinguished in The Framework are a fundamental component of the NGSS, and many NGSS-like standards, including Nebraska’s College and Career Ready Standards for Science. There has been limited research documenting the extent that teachers are incorporating SEPs in science classrooms. However, there have been reports that the actual use of inquiry-based instruction remains minimal in a majority of science classrooms despite numerous calls for its implementation [35,36,37,38]. Our findings indicated that the teachers participating in our study were incorporating the five SEP practices under our investigation prior to the PD, however, their extent of use varied by each practice and by the teacher. For example, teachers implemented the SEP of analyzing and interpreting data at the highest rate. Over half of teachers had students utilize this SEP about once a week or more. Other SEPs were used less often by teachers, for example, over half of teachers had students planning and carrying out investigations once every two to three weeks or less. On average, we found that teachers’ existing use of SEPs was less frequent than ideal, however it was favorable to see that a majority of teachers were using these practices and therefore had some familiarity with their use prior to the PD.

Improving teachers’ perceptions toward a teaching concept can improve teachers’ confidence and teaching effectiveness toward the concept [48]. Although there has been limited research that has analyzed teachers’ confidence to incorporate SEPs, teachers’ lack of confidence to incorporate inquiry-based approaches has been identified as one reason for teachers’ limited use of instructional strategies involving student inquiry [39]. In addition, and due to prior reports of science teachers’ limited knowledge and training with engineering [17,18,44], we expected teachers to exhibit low confidence incorporating SEPs that extended beyond traditional science inquiry. Our findings indicated that teachers were on average, moderately confident incorporating SEPs prior to the conference. Participants were more confident incorporating SEPs that aligned to science practices related to traditional science inquiry (e.g., having students ask questions and define problems, and having student plan and carry out investigations) compared to other SEPs. For example, prior to the PD approximately half of teachers were only slightly confident or not at all confident when it came to having students developing and using models, and using mathematics and computational thinking. This finding supports prior literature that suggests science teachers are less familiar incorporating these practices [18,60,61,62,63]. Despite having moderate confidence incorporating SEPs, our participants indicated, on average, being somewhat interested in all SEPs prior to attending our PD.

As prior researchers have noted [18,44,45], PD on SEPs and their integration with NGSS and NGSS-like standards are essential for successful reform in science education. Although our study was limited to teachers’ perceptions on SEPs, we were able to conclude that teachers significantly improved their confidence in incorporating SEPs into the classroom and became significantly more interested in doing so. Our findings were comparable to other evaluation studies that examined similar PD [17,18,46]. Interestingly, after the PD, participants described being most interested in having students developing and using models, and using math and computation thinking—The two practices that teachers were least confident in prior to the PD. This suggests that PD may be particularly useful for SEPs that extend beyond traditional scientific inquiry.

The last several decades of reform in science education clearly emphasize student-centered learning over teacher-centered learning. These reforms, including the NGSS, have been led by findings in science education research that show active teaching methods improve student learning outcomes and increase students’ interest in science [6,64,65]. Results from our study indicated that our participants, on average, believed student learning outcomes and student interest were the most influential reason for them to use SEPs during their teaching. This important finding demonstrated that teachers believe the integration of SEPs are valuable for students. Our results also indicated the current topic of instruction influenced teachers’ use of SEPs, thereby, expanding the need to demonstrate the linkage between core ideas, cross-cutting concepts, and SEPs illustrated in the NGSS [13]. Prior findings [42] described insufficient time needed to effectively implement science teaching practices as being influential to their limited use. In our study, teachers recognized time to plan lessons to incorporate SEPs as being fourth most influential out of a given list of nine factors, supporting prior findings [42].

In our study, teachers’ identified barriers to implementing SEPs on modeling and computational thinking. Similar to previous reports (e.g., [39]), our findings indicated that many teachers believe they have a low level of knowledge and a lack of confidence when it comes to implementing specific SEPs. Prior findings [43] suggested that teachers’ lack of instructional resources can be a barrier for teachers to appropriately integrate inquiry-oriented curriculum. Our findings supported this notion, as our participants identified limited access to resources and materials as the second largest barrier.

Despite the many barriers that teachers face that limit their physical or perceived ability to implement SEPs in their teaching, our results showed that after a SEP-oriented PD, teachers are more confident, motivated, and have a better understanding of how to integrate SEPs. Furthermore, our results showed that many of the identified barriers that our participants held toward SEP implementation (e.g., low self-efficacy, lack of resources, lack of knowledge) were broken down as a result of attending the PD. Another important finding was that teachers did not experience institutional barriers, which is a helpful sign to enable teaching reform. Overall, the PD reduced barriers, as summarized by one participant,

“I feel like I can now incorporate [SEPs] in my classroom so some of the barriers are lifted a bit.”

Our findings were similar to other research that evaluated the impacts of SEP-oriented PD [10,17,18,46], offering further evidence on the value of PD for science education reform.

7. Conclusions and Recommendations

Our study evaluated the Nebraska STEM Education Conference and provided baseline data on science teachers’ SEP integration prior to attending this PD. Our findings suggested that SEPs are still under-utilized in science classrooms. However, our evaluation of the PD demonstrated successful outcomes that encourage teachers’ future use of SEPs. After attending the PD, teachers significantly improved their interest and confidence in incorporating SEPs into their teaching. Teacher’s level of education was a significant predictor of their post-PD confidence to use SEPs. Our findings also indicated that the needs of students (e.g., student learning outcomes, student interest) have the most influence on why teachers incorporate SEPs. Based upon this finding, we recommend future PD programs highlight how using SEPs improves student learning and fosters students’ motivation and interest in science.

Lastly, our findings shed light on the perceived barriers teachers face to integrate SEPs in their teaching. We found the most common barriers to be teachers’ self-identified lack of knowledge and low perceived ability. Teachers also mentioned limited access to resources and materials as major barriers. However, after attending the PD, teachers described these barriers as being reduced (e.g., increase in knowledge and self-efficacy, having access to resources) and proclaimed that they intend to incorporate SEPs more in the next school year.

8. Study Limitations

Although we believe our findings to be of significant value, our study was limited by the relatively small number of participants in our teacher PD and corresponding study sample. The results of this study should be compared with future, similar work, that utilizes larger sample sizes if plausible. Furthermore, our study was limited by only focusing on post-PD teacher perceptions. Follow-up data on teachers’ actual SEP integration after attending SEP-oriented PD would be helpful to evaluate true implementation.