Correlation between High School Students’ Computational Thinking and Their Performance in STEM and Language Courses

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Introduction 

The introduction starts with a broad overview of CT research and later discusses the connection between CT and STEM education.

The manuscript needs to be reorganized to improve the readability. 

There is some repetition in discussing the challenges of defining CT. 

The statement of the research problem or the significance of CT in education is subtle.

 The article introduces multiple definitions and perspectives of CT, making it somewhat confusing for the reader to grasp a clear understanding of the concept.

The manuscript lacks concrete examples or case studies to illustrate how CT is integrated into STEM courses or real-world problem-solving scenarios.

The assessment of CT is challenging due to the absence of a clear definition and consensus on its features.

Various methods have been used, including programming environments, surveys, interviews, and multiple-choice tests.

Methodology 

The actual methodology used for the research is not thoroughly explained.

The study is described as longitudinal, which is a suitable approach for tracking changes over time.

However, it's not clear from this section how data was collected longitudinally or if there were any control groups or comparisons.

The study involved 80 students but it's important to consider the generalizability of findings to a broader population. 

The manuscript could be improved for clarity and organization.

Some sentences are long and complex, which can make it challenging for readers to follow the content. 

Results

The article examines the correlation between students' latent CT trait and their grades in STEM subjects, finding no statistically significant correlation with information technology.

Further validation and testing are necessary to ensure the instrument's reliability and validity in different contexts.

Discussions

The discussion lacks a clear introduction to set the context and purpose of the research, which could help readers better understand the significance of the findings.

 The research aims to investigate the latent CT trait of students, but the discussion doesn't explicitly state why this is important or relevant.

 The discussion briefly mentions the correlations between CT and STEM/language subjects, highlighting strong correlations in Physics and Biology but not in Computer Science. 

There is limited interpretation of why these correlations exist or their practical implications. The discussion could benefit from more in-text citations to provide direct support for the arguments being made. 

 

Manuscript needs to improve language and grammar to enhance overall clarity.

Comments on the Quality of English Language

Extensive editing of English language required. Clarity is a problem through the article. Manuscript needs to improve language and grammar to enhance overall clarity. The manuscript could be improved for clarity and organization. Some sentences are long and complex, which can make it challenging for readers to follow the content.

Author Response

Response to Reviewer 1

1. There is some repetition in discussing the challenges of defining CT.

Response: Thank you very much for your comment. This repetition is intentional. The first reference to the definitions of CT is in the introduction of our text, where the definitions are not given in detail but are given in broad strokes. Then, in Theoretical Perspectives, under the title Computational Thinking and STEM courses, we give an extensive reference to the definitions of CT also with the help of tables. This repetition and extensive reference highlight the confusion regarding the definition of CT and its consequent confusion in its assessment methods. Given that we were trying to construct a questionnaire to assess this controversial concept, we needed to repeat and extensively report on the topic.

2. The statement of the research problem or the significance of CT in education is subtle.

Response: We appreciated your comment and made the necessary changes by introducing our research questions formulated as clearly as possible in the Research Objectives section.

«Given all the above, we came to the formation of the following research questions:

1. Is our research tool adequate for estimating students' CT levels?
2. Is there a detectable correlation between students' CT levels and their performance in STEM and language courses?
3. Is there a detectable correlation between students' CT levels and their choice of field of study? » (Research Objectives, paragraph 7, lines 311-316)
4. The article introduces multiple definitions and perspectives of CT, making it somewhat confusing for the reader.

Response: Although we wanted to ensure the potential reader understands, this cannot be avoided. This is precisely the problem with computational thinking, and we wanted to make the potential readers aware. Depending on the field of research and occupation of each researcher, we have a different aspect of the definition of CT. It would not be appropriate to isolate the definition that suits us and present it exclusively to the potential reader. It is a matter of honesty towards potential readers. It should be made clear that the confusion in the definition of CT also leads to a difference of opinion on how to assess it. We present and evaluate a possible way of assessing CT based on one of the definitions that identifies CT as a set of skills required for the problem-solving process. CT is a relatively new field of research where almost all researchers agree that there is no "de facto" definition and way of assessment.

Method

4. The actual methodology used for the research needs to be thoroughly explained.

The study is described as longitudinal, a suitable approach for tracking changes over time.

However, this section needs to clarify how data was collected longitudinally or whether there were any control groups or comparisons.

Response: Your comment has proven particularly useful and to the point. We are grateful for pointing out this shortcoming.

Following your suggestion, we have included a short paragraph on the Methodology under Settings and Participants. This paragraph is reproduced below.

«These written tests are preceded by teaching with worksheets, laboratory activities and problem-solving activities during the first semester of study. As such, they review the work done in class during the first semester. These data were collected in collaboration with the physics, biology, mathematics, informatics, and language teachers and with the student's consent. » (Method, lines 322-326)

Discussion

5. The Discussion needs a clear introduction to set the context and purpose of the research, which could help readers better understand the significance of the findings.

 The research aims to investigate the latent CT trait of students, but the Discussion does not explicitly state why this is important or relevant.

Response: Regarding your comments on the Discussion, we have tried to keep our corrections short and targeted so as not to overload the text and be as straightforward as possible.

After introducing the research questions (comment 2), we transformed the Discussion section by listing our findings per research question and introducing additional references to support our findings. To follow your suggestions to improve the text's flow and make its content more transparent, we have included small changes in almost every paragraph.

Paragraph 2: «In particular, we concluded that items q2, q3, q4, q5 and q6 should be removed since, on the one hand, other items cover their difficulty level, and on the other hand, they have a lower discriminative capacity. » (Discussion, lines 479-481)

Paragraph 3: «These findings, in general, agree with the conclusions of the studies of Polat et al. [81] of Sun et al. [83] for primary education and with the research of Chongo et al. [84] and Hava & Koyunlu Ünlü [85] for secondary education. In support of our conclusions, we should mention that according to Orban and Teeling-Smith [57], regarding Physics, this is to be expected, as most of the CT skills are taught in Physics. » (Discussion, lines 491-496)

Paragraph 4: «We have not identified any research that tests the correlation between CT levels and the field of study. However, we felt it was appropriate to investigate the existence of this correlation since, as explained above, in science and economics studies, students take STEM subjects (Maths et al.) to a greater extent than in humanities. At this point, we should mention that the research of Chongo et al. [84], which investigated the correlation between mathematics achievement and CT levels, involved students of science based on the same criterion. » (Discussion, lines 511-518)

6. There needs to be more interpretation of why these correlations exist or their practical implications.

Response: We appreciate your feedback and do our best to accommodate it. Regarding the practical implications of the existence of a correlation between CT and performance in language and STEM subjects, there are our suggestions in the Limitations and Implications section as follows:

Modern education aims to introduce CT elements in as many courses as possible without necessarily using computers, as stated in the reports of the National Research Council US (2010, 2011) [8, 9, 10 in our text references].

These reports clearly state that «CT is a set of skills transferred between different disciplines» and that «At its core, CT is independent of technology....to be a competent user of CT is not necessarily related to one's ability to use modern information technology» (Theoretical perspectives, lines 128-131).

Therefore, the research aims to investigate whether there are correlations between CT and subjects such as language and STEM. If this correlation is confirmed, then «the next stage is to recognize those practices of STEM and language courses that are essentially CT practices and strengthen and enrich them with the ultimate goal of developing and empowering future citizens' CT. It is, therefore, imperative to enrich the curriculum with appropriate activities in the context of STEM and language courses, which promote and enhance student's acquisition of the characteristic of CT.» (Limitations and Ιmplications, lines 542-547)

Comments on the Quality of English Language

Response: The article underwent a linguistic proofreading process before it was resubmitted.

 

Reviewer 2 Report

Comments and Suggestions for Authors

Overall, this is a highly readable text on an interesting topic. I have a few suggestions that could enhance your contribution:

You point out that "few studies have focused on this correlation" (between CT and STEM subjects). I would recommend doing some more research on this - for example, the conference proceedings of the International Conferences on Computational Thinking and STEM Education (CTE-STEM 2021, 2022 and 2023) could be a starting point. Also, your paper would benefit from including more literature from this year (2023).

In the theoretical perspectives, there is a section on computational thinking and STEM courses, but not on CT and language courses, although the latter is also examined later. This should be added. In addition, the section 'Research objectives' gives a good overview of why implementing computational thinking is important. It should be considered whether this section might not be more useful at the beginning of the contribution.

The complete current state of research on correlations between CT and STEM subjects and CT and language courses is missing. Are there already correlations (possibly for another age group or in other countries)? Where is your research desideratum? What research questions do you have and what research hypotheses should you formulate based on the current state of research? Unfortunately, neither concrete research questions nor hypotheses were stated and then tested. These would have to be included in any case.

In principle, the results are presented clearly, but since no research questions and hypotheses have been posed beforehand, they cannot be answered clearly either. The table references in the results section are not correct. Table 1 is mentioned here, but what is meant is Table 3. This definitely needs to be revised. Overall, comparatively few results (lines 403-410 and Table 3) are reported on the actual topic of the paper - the relationship between CT and STEM and language courses. This raises the question of the focus of the paper.

In the discussion you state that "another focus of the study was to explore whether there was a correlation between the latent trait of CT and the orientation choice made by the  students who participated in the survey", but this topic is only mentioned once before (line 67) - this topic is not theoretically substantiated and does not appear as an objective of the paper. Since no current state of research was established, your results cannot be discussed in the discussion section against the background of other studies or at least shown whether hypotheses (which were not established) are to be accepted or rejected. Accordingly, the discussion section is more a summary of the results section than a more in-depth discussion.

Comments on the Quality of English Language

quite good but some minor issues detected (e.g. in the caption of table 3 it should be fit indices instead of fit indicis)

Author Response

Response to Reviewer 2

1. You point out that "few studies have focused on this correlation" (between CT and STEM subjects). I recommend doing more research on this. For example, the conference proceedings of the International Conferences on Computational Thinking and STEM Education (CTE-STEM 2021, 2022 and 2023) could be a starting point. Also, your paper would benefit from including more literature from this year (2023).

Response: Thank you very much for your recommendation. After a careful search of the recent literature, we were able to track down some attempts to investigate the correlation between students' computational thinking levels and their academic performance in STEM and language courses. The paragraph referring to the above was introduced in the section of the study under the title Research Objectives and is cited below:

«CT and all aspects of it, especially about language and STEM courses, is an emerging field of research. One of the main reasons is that many countries are choosing to introduce CT into their curriculum through its integration into existing subjects. Thus, there has been an effort to investigate the correlation between CT levels and students' academic performance in language and STEM courses in recent years. However, the number of relevant studies is still limited. In particular, we identified three relevant studies related to primary education carried out in the last three years. The research by Polat et al. [81] concerns school students in Turkey and suggests a positive effect of good performance in mathematics and CS on students' CT levels. The study states that the effect of mathematics on students' CT levels is stronger than the effect of CS. Sun et al. [82], in their research involving school students in China, investigated the correlation between students' attitudes towards STEM subjects and their CT skills. They found that the type of learning attitude predicted students' CT levels. Also, Sun et al. [83], in their research, which again involved Chinese students, found significant bidirectional correlations between students' academic performance in STEM and language courses and CT skills. Finally, we identified two studies on secondary education conducted in the last three years. The research by Chongo et al. [84] involved 128 students of science field of study from Malaysia. The study found a statistically significant correlation between students' CT skills and mathematics achievement. Hava & Koyunlu Ünlü [85] found a significant correlation between Turkish high school students' CT skills, interest in STEM careers, and attitudes towards inquiry learning. In addition, the above researchers' observations confirm Lei et al.'s meta-analysis [86], which concerned earlier similar studies and reported positive correlations between CT levels and academic achievements for students. » (Research Objectives, paragraph 4, lines 264-287)

2. In the theoretical perspectives, there is a section on computational thinking and STEM courses but not on CT and language courses, although the latter is also examined later. This should be added.

Response: Your comment has proven particularly useful and to the point. We are grateful for pointing out this shortcoming.

Following your suggestion, we have included an additional paragraph to the Theoretical Perspectives under “Computational Thinking and Language courses”. This paragraph is reproduced below.

«Computational Thinking and Language courses

Unfortunately, there is not enough material in the literature that directly links computational thinking to language learning. This is because computational thinking skills are directly related to computer science. Nevertheless, the role of language is crucial in cultivating and promoting computational thinking skills since language is the tool through which articulation and reflection occur in all processes that require the activation of these skills. Reinforcing this belief is the presentation by Lu and Fletcher in 2009 at the 40th ACM Technical Symposium on Computer Science Education [62]. In addition, we have identified a small number of cases where attempts have been made to integrate CT processes and skills into Language courses [63, 64]. Some of these have involved modelling [65, 66] or even the production of computer games [67].

However, we have not detected any research correlating the skills students acquired in language classes with CT skills. » (Theoretical Perspectives, Heading 2: Computational Thinking and Language courses, lines 152-165)

3. The current state of research on correlations between CT and STEM subjects and CT and language courses needs to be included. Are there already correlations (possibly for another age group or other countries)?

Response: The response to this comment is included in our response to comment 1. However, it is appropriate to include it here.

«CT and all aspects of it, especially about language and STEM courses, is an emerging field of research. One of the main reasons is that many countries are choosing to introduce CT into their curriculum through its integration into existing subjects. Thus, there has been an effort to investigate the correlation between CT levels and students' academic performance in language and STEM courses in recent years. However, the number of relevant studies is still limited. In particular, we identified three relevant studies related to primary education carried out in the last three years. The research by Polat et al. [81] concerns school students in Turkey and suggests a positive effect of good performance in mathematics and CS on students' CT levels. The study states that the effect of mathematics on students' CT levels is stronger than the effect of CS. Sun et al. [82], in their research involving school students in China, investigated the correlation between students' attitudes towards STEM subjects and their CT skills. They found that the type of learning attitude predicted students' CT levels. Also, Sun et al. [83], in their research, which again involved Chinese students, found significant bidirectional correlations between students' academic performance in STEM and language courses and CT skills. Finally, we identified two studies on secondary education conducted in the last three years. The research by Chongo et al. [84] involved 128 students of science field of study from Malaysia. The study found a statistically significant correlation between students' CT skills and mathematics achievement. Hava & Koyunlu Ünlü [85] found a significant correlation between Turkish high school students' CT skills, their interest in STEM careers and their attitudes towards inquiry learning. In addition, the above researchers' observations confirm Lei et al.'s meta-analysis [86], which concerned earlier similar studies and reported positive correlations between CT levels and academic achievements for students. » (Research Objectives, paragraph 4, lines 264-287)

4. Where is your research desideratum? What research questions do you have?

Response: We appreciated your comment and made the necessary changes by introducing our research questions formulated as clearly as possible in the Research Objectives section.

«Given all the above, we came to the formation of the following research questions:

1. Is our research tool adequate for estimating students' CT levels?
2. Is there a detectable correlation between students' CT levels and their performance in STEM and language courses?
3. Is there a detectable correlation between students' CT levels and their choice of field of study? » (Research Objectives, paragraph 7, lines 311-316)
4. The table references in the results section are not correct. Table 1 is mentioned here, but what is meant is Table 3. This needs to be revised.

Response: Thank you very much for pointing this out. All incorrect references in the existing tables have been corrected.

6. In the Discussion, you state that "another focus of the study was to explore whether there was a correlation between the latent trait of CT and the orientation choice made by the students who participated in the survey", but this topic is only mentioned once before (line 67). This topic is not theoretically substantiated and does not appear as an objective of the paper. Since no current state of research was established, your results cannot be discussed in the discussion section against the background of other studies or at least show whether hypotheses (which were not established) are to be accepted or rejected. Accordingly, the discussion section is more a summary of the results section than a more in-depth discussion.

Response: We appreciated your comment and made the necessary changes. In order to support this choice, we need to set out our reasoning as follows.

Previous research, included in our survey and cited in our response to comment 1, finds a positive correlation between students' levels of computational thinking and their performance in STEM courses [81-86 in our text references].

In addition, the Greek educational system requires students to choose a course of study in the third grade of high school between science, economics, computer science and humanities. In science com, computer science, and economics, the teaching of STEM subjects is more extensive than in the humanities. This leads to greater engagement of these students with STEM course practices.

Combining the above data, we were led to the thought that the choice of course of study will be reflected in students' computational thinking levels. In order to support this hypothesis and test the existence of this correlation, we introduced the following section in our text. The following is the corresponding paragraph from our text.

«It should be noted that in Greece, students choose a field of study from the second grade of high school. Thus, in science and economics studies, they attend STEM courses (Mathematics, Physics, Chemistry, Biology and Language) to a greater extent, which is not the case in humanities. Based on this fact and given the direct relationship between STEM courses and CT, we decided to investigate the existence of a correlation between the field of study and students' CT.» (Research Objectives, lines 295-300)

Comments on the Quality of English Language

Response: The article underwent a linguistic proofreading process before it was resubmitted.