A Practical Model for the Evaluation of High School Student Performance Based on Machine Learning

Round 1

Reviewer 1 Report

The article provides important information for debate and action in the area of its focus.
It is well structured.

We suggest that in the summary, and also in the materials and methods section, an overview/synthesis of the method used should be included, in terms of the scientific approach and type of scientific study developed. This will further help the scientific clarity of the work performed.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

A very well designed and implemented research project tested with empirical data.  The study should be of great interest to secondary educators as a means of reducing the administration time required of teachers in assessing the academic performance of their students.

Please check the wording in line 77; "and students are only assess with grades in each lessons", should this read "and students are only assessed with grades in each lesson".

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

The paper presents an interesting topic which fits within the scope of the Sustainability Journal.

The abstract should be more rigorous and precise in the information presented. The authors should delete the topics “Background”, “Methods”, “Results”, “Conclusions” and present an integrated and complete version of the content of the paper. Please see the authors guidelines for more details.

The theoretical background and empirical research on the topic should be improved, including studies and literature review from the Education field, focused on the evaluation of student performance from an educational perspective. Also, some background information and description of the educational system of Iran would also be important to understand the results and implications of the study.

The research questions or objectives are not clearly stated in the Methods section. The objective of the study is presented at the end of the Introduction section: “The main object of this study is to design and develop an ML system for classifying the performance of high school students in four classes very well, good, medium and bad.” (line 59). Please correct – objective and not object. This objective is not very clear. I suggest the authors provide a clear relationship between the objectives and the results of the study. In the discussion and conclusions, this relationship should be made clear.

In the discussion section, authors must go deeper in their conclusions and identify implications for practice and future work to be developed in this field. The limitations of the study should also be referred.

In general, the paper needs some editing as there are several sentences and paragraphs that do not start with capital letter (see lines 154 or 158, just to show an example). The English language should also be reviewed, as there are a few errors throughout the manuscript and the paper could be improved with professional English writing.

Author Response

Dear Prof. Dr. Nikola Lučić,

Editor-in-Chief

Dear Prof. Hong-Ren Chen, dearProf. Wen-Shan Lin, dear Prof. Hao-Chiang Koong Lin

Guest Editors

Sustainability

September, 2021

Re: sustainability-1354752, entitled: “A Practical Model for evaluation of high school student performance based on Machine learning”

We thank both reviewers for providing us highly constructive and insightful comments to improve our manuscript. We have carefully revised the manuscript, following the reviewers’ suggestions, and have responded in detail to each comment. The next section contains our point-by-point responses (in blue) and changes and referencing to the manuscript (in green), based on the reviewers’ comments (in italic). We believe that our manuscript has substantially improved and is more readable for broader audiences. The authors decided to all one more author (Dr. Ali Esmaeily) in the process of responding the reviewers as well as his previous contribution in this paper.

We look forward to hearing from you. Also, we would be glad to respond to any further questions and comments that you may have.

 

Yours Sincerely,

 

Abolghasem Sadeghi-Niaraki,

 

 

                                                                                                          

 

 

Reviewer 1

 

The Comment:

The abstract should be more rigorous and precise in the information presented. The authors should delete the topics “Background”, “Methods”, “Results”, “Conclusions” and present an integrated and complete version of the content of the paper. Please see the authors guidelines for more details.

Response:

Thank you for pointing out this misunderstanding to us. Mentioned changes were applied and the abstract was revised in manuscript

“The objective of this research is to develop an ML-based system that evaluates the performance of high school students during the semester, and identify the most significant factors affecting the student performance. It also specifies how the performance of models affects when models run on data that only includes the most important features. Classifiers employed for the system include Random Forest, Support vector machines, Logistic regression and Artificial neural network Techniques. Moreover, the Boruta algorithm was used to calculate the importance of features. The dataset includes behavioral information, individual information and scores of students that were collected from teachers and a one-by-one survey through an online questionnaire. As a result, the effective features of the database were identified and the least important features were eliminated from the dataset. The ANN accuracy, which was the best accuracy on the original dataset, was reduced on the decreased dataset. On the contrary, SVM performance was improved which was the highest accuracy among of other models with 0.78. Moreover, the LR and RF model could provide the same performance on the decreased dataset. The results showed that ML models are influential for evaluating students and stakeholders can use identified effective factors to improve education.”

 

The Comment:

The theoretical background and empirical research on the topic should be improved, including studies and literature review from the Education field, focused on the evaluation of student performance from an educational perspective.

Response:

the theoretical background and empirical research are listed in the references and to make it clear, the references with similar issues that evaluate student performance with ML, were placed together and identified with a headline. all references are categorized in a thematic order. also, more references were added to emphasize studies conducted on student performance evaluation based on ML models from an educational perspective. It reads (Page 2, Paragraph 4):

“Many research works have been focused on the integration of AI and ML in different parts of education and various methods and tools have been used to carry out such tasks. One of these parts is an assessment of student performance. The performance of students can be evaluated from various aspects. Several studies evaluated students in general as student performance [16-20] and some other studies are evaluated students for a specific purpose such as academic achievement [21,22], Reading ability [23,24], Grading [25-27], Dropout prediction [28-31], etc. Below, some state-of-the-art research studies have been discussed for each of mentioned tasks that evaluated the student performance from different aspects.”

It reads (Page 3, line 109): “For monitoring student performance of Brazilian technical high school, authors in ref [33] 6807 developed an ML system based on NB, SVM, Tree-based method (SimpleCart), Rule-based method (OneR) algorithms. Their dataset contained information about: course name, age, sex, birthplace, course duration, identification of each discipline studied, number of faults in the 1st – 4th bimester, and student status at end of the course. In this research, the usefulness of ML was highlighted for monitoring student performance. In ref [16], student performance was predicted using a Multiclass Support Vector Classification. Real data from 395 Portugal secondary students was employed which was collected by questionnaire method and school reports. The data contained grades assigned and student info, and each instance is categorized in five levels as A to F. This research indicates that by predicting the performance of the students, they can provide appropriate extra tasks to improve students' education. In ref [34], answers of 702 students in the 10th grade was analyzed in chemistry lab classes. The researchers used the K-Mean Clustering Method for segment answers then a DL algorithm was employed for the development of explanatory models of the segmentation. Their results highlight that one key factor for chemistry learning is attention and involvement of student in classes.”

16. Athani, S.S.; Kodli, S.A.; Banavasi, M.N.; Hiremath, P.G.S. Student performance predictor using multiclass support vector classification algorithm C3 - Proceedings of IEEE International Conference on Signal Processing and Communication, ICSPC 2017. 2018; pp. 341-346.
17. De Melo, G.; Vasconcelos Filho, E.P.; Oliveira, S.M.; Calixto, W.P.; Ferreira, C.C.; Furriel, G.P. Evaluation techniques of machine learning in task of reprovation prediction of technical high school students C3 - 2017 CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies, CHILECON 2017 - Proceedings. 2017; pp. 1-7.
18. Figueiredo, M.; Lurdes Esteves, M.; Neves, J.; Vicente, H. Lab classes in chemistry learning an artificial intelligence view C3 - Advances in Intelligent Systems and Computing. 2014, 299, 565-575, doi:10.1007/978-3-319-07995-0_56.

 

 

The Comment:

Also, some background information and description of the educational system of Iran would also be important to understand the results and implications of the study.

Response:

Thanks for pointing out this comment. The following information has been applied in the revised version of the manuscript. It reads (Page 2, Paragraph 2):   

“The educational system in Iran is divided into two main levels: K-12 education and higher education consisting of 6 years of primary education and 6 years of high school education. Students spend 24 hours in class each week. The curriculum contains mathematics, science, foreign language and so on. High school students aged 12 to 18 years old are divided into four fields (streams): humanism, science, technical and vocational. Choosing their stream is based on his/her grades and the results of his/her examination, not based on their interest. Furthermore, grades are determined on a scale between 0 and 20 in all levels of education and students assess during the semester and end of the semester as well. The lowest score to pass a lesson is 10. Some of the most highlighted features of the Iranian educational system include: teaching strategy is Teacher-centered type in all schools and students are only assess with grades in each lessons [15].”

15. Clark, N. Education in Iran. Available online: world education news reviews (accessed on https://wenr.wes.org/2017/02/education-in-iran).

The Comment:

The research questions or objectives are not clearly stated in the Methods section. The objective of the study is presented at the end of the Introduction section: “The main object of this study is to design and develop an ML system for classifying the performance of high school students in four classes very well, good, medium and bad.” (line 59). Please correct – objective and not object. This objective is not very clear.

Response:

We appreciate the reviewer’s comment. The research questions were specified in the list in the revised version of the manuscript. It reads (Page 4, Paragraph 4):

“The main objective of this study is to design and develop an ML system for classifying the performance of high school students in four classes very good, good, medium and bad. In particular, the study aims to seek answers to the following research questions:

• To make the right decisions and appropriate policy, what are the main factors and most significant features for evaluating student performance? Also, which of the three types of data including demographic, behavior and grades, have the most influence?
• What is the best and most effective ML model for evaluating and classifying student performance that can determine a good boundary in data?
• In order to make a more efficient and convenient model for predicting new instances, when we run models on data that only includes the most important features, what impact on their performance is occurs?”

 

The Comment:

I suggest the authors provide a clear relationship between the objectives and the results of the study. In the discussion and conclusions, this relationship should be made clear.

Response:

Thank you for the reviewer’s suggestions. We have applied this valuable suggestion to the revised version of our manuscript. It reads (Page 13, Paragraph 2):

“The developed models in this research evaluate the overall performance of students promptly, and this evaluation is without bias and intention that makes the system impressive. For more accurate assessment of the student, all the features of the original dataset can be used in analysis, otherwise, this can be done faster way with 17 of the features mentioned earlier. moreover, our results allow decision-makers, schools and teachers to better understand the factors affecting students' performance at the individual level. This valuable information can be used by politicians to adopt correct decisions and choosing appropriate strategies to improve education. The information obtained can be used to understand the improvement of student academic achievement by teachers. Efficient models and explainable AI could allow the Iran Ministry of Education to make an informed decision for better student achievement and the policies of employing AI in schools.”

 

The Comment:

In the discussion section, authors must go deeper in their conclusions and identify implications for practice and future work to be developed in this field.

Response:

Concepts for practice are referred to in the previous comment revise. It reads (Page 13, Paragraph 2). Furthermore, for more discipline and a better summary, the conclusion section is separated from the discussion section, and the Future section is added in this part of the manuscript. It reads (Page 13, Paragraph 3)

“For further researches, the collected data can be used for other work such as dropout pre-diction and academic achievement by add related labels. also, the performance of models can be improved by adding more new data. As mentioned above, the student's attention is an influential factor, so, with the help of Deep Learning, a system can be developed to track students' attention and warn the teacher or students if there is lack of attention from students. such a system helps student maintain their attention in class.”

 

The Comment:

The limitations of the study should also be referred.

Response:

We thanks for the reviewer’s comment. the limitation of the research was referred to in the discussion section of the revised manuscript. It reads (Page 12, Paragraph 3):

“Due to the lack of an appropriate educational database and insufficiency of connection between the available databases, the process of data collection in this way was difficult. The questionnaires were filled by students online, and there was no supervision on filling the questionnaire process. Consequently, some information was unreliable due to lack of student's proper understanding of questions so these data were removed. Further-more, some further features because the absence of resources was not included in data such as teacher quality and parent salary. their existence could provide a more accurate assessment.”

 

The Comment:

In general, the paper needs some editing as there are several sentences and paragraphs that do not start with capital letter (see lines 154 or 158, just to show an example). The English language should also be reviewed, as there are a few errors throughout the manuscript and the paper could be improved with professional English writing.

Response:

Thank you for pointing out these mistakes. We have submitted our manuscript to an English proofreading service. The probable editorial mistakes have been corrected in the revised version of the manuscript.

 

 

 

Reviewer 2

 

The Comment:

The paper analyzes school performance in four classes classification (very well, good, medium, and bad) based on Machine Learning. The methodology points to a sample of students of different levels and in fields such as humanities, science, and technology. In particular, only in the results (Table 1) can it be seen that these are subjects such as mathematics, science, language, and sports. Are these subjects some of the variables? If they are part of the 35 variables. How are they related to school performance? Why there are not any analysis of the results of these variables?

Response:

Yes, the grade of these classes is part of the variables that teachers have provided us with the evaluation of students during the semester. Iran's educational assessment system is completely based on the score. This will make grades play a decisive role in assessing students, so for a comprehensive evaluation, in addition to demographic and behavioral information of students, we added two grade assessments of mentioned classes into the dataset which specified in the database with "first class name midterm" and "second class name midterm". the description about these features was added in the "3.1. dataset" section and the finding related to these features was analyzed in the discussion. also, the name of these two features was modified in the revised manuscript.

It reads (Page 5, Paragraph 3): “Due to the fact that grades play a major role in student assessment in the educational system of Iran, the scores of student exams in different classes were added to the data as a variable for evaluating his/her performance. The wide range of class subjects includes math, science, foreign language, sport and art, was used two times in the semester for a comprehensive evaluation. These grades were obtained from the assessment of students during the semester by teachers of each class which specified in the dataset with "first class name midterm" and "second class name midterm".”

It reads (Page 12, Paragraph 2, line 444): “Another important factor is the student grades in the lessons which have much more weight compared to the rest of the variables. To evaluate student performance, only these criteria should not be considered, also other influential criteria such as problem-solving, critical thinking, creativity and Etc should be somehow included in calculations.”

 

The Comment:

There is a lack of a definition of what the authors consider to be school performance, specifying the variables to be taken into account and how they are interrelated to give; as a result, school performance.

Response:

We appreciate the reviewer’s comment. The variables used in this study are described in the "3.1. dataset" section. General explanations of variable type are on (Page 5, Paragraph 3, line 214), descriptions for the demographic data is on (Page 5, Paragraph 3, line 217), descriptions for behavioral data is on (Page 5, Paragraph 3, line 225), and descriptions for grades data is on (Page 5, Paragraph 3, line 231) that was revised for another comment. However, additional explanations were added about the definition of student performance and related variables.

 It reads (Page 2, Paragraph 1, line 56): “Evaluation of student performance refers to how much a student approaches the educational goals and specifies how well a student learned, how much motivated to learn and how good the teaching method was [14]. The information obtained from the evaluation gives teachers this insight so they can make the right decisions to improve the learning of the student and gives appropriate feedback. Individual differences of each student, such as personality, motivation, self-efficacy, intelligence and self-control have a close relationship to his/her performance, so in this research, all these differences were covered by choosing the proper features.”

14. Cassano, R., V. Costa, and T. Fornasari, An Effective National Evaluation System of Schools for Sustainable Development: A Comparative European Analysis. Sustainability, 2019. 11(1): p. 195.

It reads (Page 5, Paragraph 3, line 231): “Due to the fact that grades play a major role in student assessment in the educational system of Iran, the scores of student exams in different classes were added to the data as a variable for evaluating his/her performance. The wide range of class subjects includes math, science, foreign language, sport and art, was used two times in the semester for a comprehensive evaluation. These grades were obtained from the assessment of students during the semester by teachers of each class which specified in the dataset with "first class name midterm" and "second class name midterm".”

 

The Comment:

In the discussion and conclusion section, the paper shows that some more important characteristics to the school performance like absence, math, attention, science, sport, art, science, language, internet use, sport, and study time. Also, it shows the characteristics less important like age, free time, father and mother education, extra class, work, go-out, friendship, and TV use. These results are exciting, but it is necessary for a better discussion centered on school performance. All this information is exciting but required a better discussion and conclusion.

Response:

The authors completely agree with the reviewer’s comment that this sentence is not clear. We have applied this and improved discussion to the revised version of our manuscript. It reads (Page 12, Paragraph 2, and Line 435):

“Indeed, attention in the class is very important, and whatever the attention of the student is more in class, they will have more focus and more understanding of the class content [34]. So, to improve the student's performance, attractions in the classroom must be created in order for students have more enthusiasm to pay attention. One of the disadvantages in the education system of Iran is the teacher-centered classroom. by changing this method to student-centered class, more attraction will be created and the attention of students will be attracted. moreover, in cases where the student becomes absent, he/she should not be left behind from the curriculum and it should be compensated by arrangements such as recording class for them, additional classes, additional support and so on. Another important factor is the student grades in the lessons which have much more weight compared to the rest of the variables. To evaluate student performance, only these criteria should not be considered, also other influential criteria such as problem-solving, critical thinking, creativity and Etc should be somehow included in calculations. Excessive usage of the internet and social media reduce the attention of youth and distract them [56], so less use of the internet increases the consideration and performance as well.”

56. Abbas, J., et al., The impact of social media on learning behavior for sustainable education: Evidence of students from selected universities in Pakistan. Sustainability, 2019. 11(6): p. 1683.

 

 

The Comment:

How could these results help to understand the school performance?

Response:

According to the reviewer’s comment, the discussion section was improved and the required explanation information has been applied to the revised version of the manuscript. It reads (Page 13, Paragraph 2, and Line 459)

“This valuable information can be used by politicians to adopt correct decisions and choosing appropriate strategies to improve education. The information obtained can be used to understand the improvement of student academic achievement by teachers. Efficient models and explainable AI could allow the Iran Ministry of Education to make an informed decision for better student achievement and the policies of employing AI in schools.”

 

The Comment:

Regarding the format of the document, it requires a detailed revision of the punctuation.

Response:

We thanks for the reviewer’ comment. We have submitted our manuscript to an English proofreading service. The probable editorial mistakes have been corrected in the revised version of the manuscript.

 

The Comment:

As for the references, it is necessary to review references number 25, 33, 35, 36, and 46, which lack information or are out of the required format.

Response:

Thanks for pointing out this comment. The references style converted to the ACS style mentioned in Instructions for authors.

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper analyzes school performance in four classes classification (very well, good, medium, and bad) based on Machine Learning. The methodology points to a sample of students of different levels and in fields such as humanities, science, and technology. In particular, only in the results (Table 1) can it be seen that these are subjects such as mathematics, science, language, and sports. Are these subjects some of the variables? If they are part of the 35 variables. How are they related to school performance? Why there are not any analysis of the results of these variables?

There is a lack of a definition of what the authors consider to be school performance, specifying the variables to be taken into account and how they are interrelated to give; as a result, school performance.

In the discussion and conclusion section, the paper shows that some more important characteristics to the school performance like absence, math, attention, science, sport, art, science, language, internet use, sport, and study time. Also, it shows the characteristics less important like age, free time, father and mother education, extra class, work, go-out, friendship, and TV use. These results are exciting, but it is necessary for a better discussion centered on school performance. All this information is exciting but required a better discussion and conclusion.

How could these results help to understand the school performance?

Regarding the format of the document, it requires a detailed revision of the punctuation.

 

As for the references, it is necessary to review references number 25, 33, 35, 36, and 46, which lack information or are out of the required format.

Author Response

Dear Prof. Dr. Nikola Lučić,

Editor-in-Chief

Dear Prof. Hong-Ren Chen, dearProf. Wen-Shan Lin, dear Prof. Hao-Chiang Koong Lin

Guest Editors

Sustainability

September, 2021

Re: sustainability-1354752, entitled: “A Practical Model for evaluation of high school student performance based on Machine learning”

We thank both reviewers for providing us highly constructive and insightful comments to improve our manuscript. We have carefully revised the manuscript, following the reviewers’ suggestions, and have responded in detail to each comment. The next section contains our point-by-point responses (in blue) and changes and referencing to the manuscript (in green), based on the reviewers’ comments (in italic). We believe that our manuscript has substantially improved and is more readable for broader audiences. The authors decided to all one more author (Dr. Ali Esmaeily) in the process of responding the reviewers as well as his previous contribution in this paper.

We look forward to hearing from you. Also, we would be glad to respond to any further questions and comments that you may have.

 

Yours Sincerely,

 

Abolghasem Sadeghi-Niaraki,

 

 

                                                                                                          

 

 

Reviewer 1

 

The Comment:

The abstract should be more rigorous and precise in the information presented. The authors should delete the topics “Background”, “Methods”, “Results”, “Conclusions” and present an integrated and complete version of the content of the paper. Please see the authors guidelines for more details.

Response:

Thank you for pointing out this misunderstanding to us. Mentioned changes were applied and the abstract was revised in manuscript

“The objective of this research is to develop an ML-based system that evaluates the performance of high school students during the semester, and identify the most significant factors affecting the student performance. It also specifies how the performance of models affects when models run on data that only includes the most important features. Classifiers employed for the system include Random Forest, Support vector machines, Logistic regression and Artificial neural network Techniques. Moreover, the Boruta algorithm was used to calculate the importance of features. The dataset includes behavioral information, individual information and scores of students that were collected from teachers and a one-by-one survey through an online questionnaire. As a result, the effective features of the database were identified and the least important features were eliminated from the dataset. The ANN accuracy, which was the best accuracy on the original dataset, was reduced on the decreased dataset. On the contrary, SVM performance was improved which was the highest accuracy among of other models with 0.78. Moreover, the LR and RF model could provide the same performance on the decreased dataset. The results showed that ML models are influential for evaluating students and stakeholders can use identified effective factors to improve education.”

 

The Comment:

The theoretical background and empirical research on the topic should be improved, including studies and literature review from the Education field, focused on the evaluation of student performance from an educational perspective.

Response:

the theoretical background and empirical research are listed in the references and to make it clear, the references with similar issues that evaluate student performance with ML, were placed together and identified with a headline. all references are categorized in a thematic order. also, more references were added to emphasize studies conducted on student performance evaluation based on ML models from an educational perspective. It reads (Page 2, Paragraph 4):

“Many research works have been focused on the integration of AI and ML in different parts of education and various methods and tools have been used to carry out such tasks. One of these parts is an assessment of student performance. The performance of students can be evaluated from various aspects. Several studies evaluated students in general as student performance [16-20] and some other studies are evaluated students for a specific purpose such as academic achievement [21,22], Reading ability [23,24], Grading [25-27], Dropout prediction [28-31], etc. Below, some state-of-the-art research studies have been discussed for each of mentioned tasks that evaluated the student performance from different aspects.”

It reads (Page 3, line 109): “For monitoring student performance of Brazilian technical high school, authors in ref [33] 6807 developed an ML system based on NB, SVM, Tree-based method (SimpleCart), Rule-based method (OneR) algorithms. Their dataset contained information about: course name, age, sex, birthplace, course duration, identification of each discipline studied, number of faults in the 1st – 4th bimester, and student status at end of the course. In this research, the usefulness of ML was highlighted for monitoring student performance. In ref [16], student performance was predicted using a Multiclass Support Vector Classification. Real data from 395 Portugal secondary students was employed which was collected by questionnaire method and school reports. The data contained grades assigned and student info, and each instance is categorized in five levels as A to F. This research indicates that by predicting the performance of the students, they can provide appropriate extra tasks to improve students' education. In ref [34], answers of 702 students in the 10th grade was analyzed in chemistry lab classes. The researchers used the K-Mean Clustering Method for segment answers then a DL algorithm was employed for the development of explanatory models of the segmentation. Their results highlight that one key factor for chemistry learning is attention and involvement of student in classes.”

16. Athani, S.S.; Kodli, S.A.; Banavasi, M.N.; Hiremath, P.G.S. Student performance predictor using multiclass support vector classification algorithm C3 - Proceedings of IEEE International Conference on Signal Processing and Communication, ICSPC 2017. 2018; pp. 341-346.
17. De Melo, G.; Vasconcelos Filho, E.P.; Oliveira, S.M.; Calixto, W.P.; Ferreira, C.C.; Furriel, G.P. Evaluation techniques of machine learning in task of reprovation prediction of technical high school students C3 - 2017 CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies, CHILECON 2017 - Proceedings. 2017; pp. 1-7.
18. Figueiredo, M.; Lurdes Esteves, M.; Neves, J.; Vicente, H. Lab classes in chemistry learning an artificial intelligence view C3 - Advances in Intelligent Systems and Computing. 2014, 299, 565-575, doi:10.1007/978-3-319-07995-0_56.

 

 

The Comment:

Also, some background information and description of the educational system of Iran would also be important to understand the results and implications of the study.

Response:

Thanks for pointing out this comment. The following information has been applied in the revised version of the manuscript. It reads (Page 2, Paragraph 2):   

“The educational system in Iran is divided into two main levels: K-12 education and higher education consisting of 6 years of primary education and 6 years of high school education. Students spend 24 hours in class each week. The curriculum contains mathematics, science, foreign language and so on. High school students aged 12 to 18 years old are divided into four fields (streams): humanism, science, technical and vocational. Choosing their stream is based on his/her grades and the results of his/her examination, not based on their interest. Furthermore, grades are determined on a scale between 0 and 20 in all levels of education and students assess during the semester and end of the semester as well. The lowest score to pass a lesson is 10. Some of the most highlighted features of the Iranian educational system include: teaching strategy is Teacher-centered type in all schools and students are only assess with grades in each lessons [15].”

15. Clark, N. Education in Iran. Available online: world education news reviews (accessed on https://wenr.wes.org/2017/02/education-in-iran).

The Comment:

The research questions or objectives are not clearly stated in the Methods section. The objective of the study is presented at the end of the Introduction section: “The main object of this study is to design and develop an ML system for classifying the performance of high school students in four classes very well, good, medium and bad.” (line 59). Please correct – objective and not object. This objective is not very clear.

Response:

We appreciate the reviewer’s comment. The research questions were specified in the list in the revised version of the manuscript. It reads (Page 4, Paragraph 4):

“The main objective of this study is to design and develop an ML system for classifying the performance of high school students in four classes very good, good, medium and bad. In particular, the study aims to seek answers to the following research questions:

• To make the right decisions and appropriate policy, what are the main factors and most significant features for evaluating student performance? Also, which of the three types of data including demographic, behavior and grades, have the most influence?
• What is the best and most effective ML model for evaluating and classifying student performance that can determine a good boundary in data?
• In order to make a more efficient and convenient model for predicting new instances, when we run models on data that only includes the most important features, what impact on their performance is occurs?”

 

The Comment:

I suggest the authors provide a clear relationship between the objectives and the results of the study. In the discussion and conclusions, this relationship should be made clear.

Response:

Thank you for the reviewer’s suggestions. We have applied this valuable suggestion to the revised version of our manuscript. It reads (Page 13, Paragraph 2):

“The developed models in this research evaluate the overall performance of students promptly, and this evaluation is without bias and intention that makes the system impressive. For more accurate assessment of the student, all the features of the original dataset can be used in analysis, otherwise, this can be done faster way with 17 of the features mentioned earlier. moreover, our results allow decision-makers, schools and teachers to better understand the factors affecting students' performance at the individual level. This valuable information can be used by politicians to adopt correct decisions and choosing appropriate strategies to improve education. The information obtained can be used to understand the improvement of student academic achievement by teachers. Efficient models and explainable AI could allow the Iran Ministry of Education to make an informed decision for better student achievement and the policies of employing AI in schools.”

 

The Comment:

In the discussion section, authors must go deeper in their conclusions and identify implications for practice and future work to be developed in this field.

Response:

Concepts for practice are referred to in the previous comment revise. It reads (Page 13, Paragraph 2). Furthermore, for more discipline and a better summary, the conclusion section is separated from the discussion section, and the Future section is added in this part of the manuscript. It reads (Page 13, Paragraph 3)

“For further researches, the collected data can be used for other work such as dropout pre-diction and academic achievement by add related labels. also, the performance of models can be improved by adding more new data. As mentioned above, the student's attention is an influential factor, so, with the help of Deep Learning, a system can be developed to track students' attention and warn the teacher or students if there is lack of attention from students. such a system helps student maintain their attention in class.”

 

The Comment:

The limitations of the study should also be referred.

Response:

We thanks for the reviewer’s comment. the limitation of the research was referred to in the discussion section of the revised manuscript. It reads (Page 12, Paragraph 3):

“Due to the lack of an appropriate educational database and insufficiency of connection between the available databases, the process of data collection in this way was difficult. The questionnaires were filled by students online, and there was no supervision on filling the questionnaire process. Consequently, some information was unreliable due to lack of student's proper understanding of questions so these data were removed. Further-more, some further features because the absence of resources was not included in data such as teacher quality and parent salary. their existence could provide a more accurate assessment.”

 

The Comment:

In general, the paper needs some editing as there are several sentences and paragraphs that do not start with capital letter (see lines 154 or 158, just to show an example). The English language should also be reviewed, as there are a few errors throughout the manuscript and the paper could be improved with professional English writing.

Response:

Thank you for pointing out these mistakes. We have submitted our manuscript to an English proofreading service. The probable editorial mistakes have been corrected in the revised version of the manuscript.

 

 

 

Reviewer 2

 

The Comment:

The paper analyzes school performance in four classes classification (very well, good, medium, and bad) based on Machine Learning. The methodology points to a sample of students of different levels and in fields such as humanities, science, and technology. In particular, only in the results (Table 1) can it be seen that these are subjects such as mathematics, science, language, and sports. Are these subjects some of the variables? If they are part of the 35 variables. How are they related to school performance? Why there are not any analysis of the results of these variables?

Response:

Yes, the grade of these classes is part of the variables that teachers have provided us with the evaluation of students during the semester. Iran's educational assessment system is completely based on the score. This will make grades play a decisive role in assessing students, so for a comprehensive evaluation, in addition to demographic and behavioral information of students, we added two grade assessments of mentioned classes into the dataset which specified in the database with "first class name midterm" and "second class name midterm". the description about these features was added in the "3.1. dataset" section and the finding related to these features was analyzed in the discussion. also, the name of these two features was modified in the revised manuscript.

It reads (Page 5, Paragraph 3): “Due to the fact that grades play a major role in student assessment in the educational system of Iran, the scores of student exams in different classes were added to the data as a variable for evaluating his/her performance. The wide range of class subjects includes math, science, foreign language, sport and art, was used two times in the semester for a comprehensive evaluation. These grades were obtained from the assessment of students during the semester by teachers of each class which specified in the dataset with "first class name midterm" and "second class name midterm".”

It reads (Page 12, Paragraph 2, line 444): “Another important factor is the student grades in the lessons which have much more weight compared to the rest of the variables. To evaluate student performance, only these criteria should not be considered, also other influential criteria such as problem-solving, critical thinking, creativity and Etc should be somehow included in calculations.”

 

The Comment:

There is a lack of a definition of what the authors consider to be school performance, specifying the variables to be taken into account and how they are interrelated to give; as a result, school performance.

Response:

We appreciate the reviewer’s comment. The variables used in this study are described in the "3.1. dataset" section. General explanations of variable type are on (Page 5, Paragraph 3, line 214), descriptions for the demographic data is on (Page 5, Paragraph 3, line 217), descriptions for behavioral data is on (Page 5, Paragraph 3, line 225), and descriptions for grades data is on (Page 5, Paragraph 3, line 231) that was revised for another comment. However, additional explanations were added about the definition of student performance and related variables.

 It reads (Page 2, Paragraph 1, line 56): “Evaluation of student performance refers to how much a student approaches the educational goals and specifies how well a student learned, how much motivated to learn and how good the teaching method was [14]. The information obtained from the evaluation gives teachers this insight so they can make the right decisions to improve the learning of the student and gives appropriate feedback. Individual differences of each student, such as personality, motivation, self-efficacy, intelligence and self-control have a close relationship to his/her performance, so in this research, all these differences were covered by choosing the proper features.”

14. Cassano, R., V. Costa, and T. Fornasari, An Effective National Evaluation System of Schools for Sustainable Development: A Comparative European Analysis. Sustainability, 2019. 11(1): p. 195.

It reads (Page 5, Paragraph 3, line 231): “Due to the fact that grades play a major role in student assessment in the educational system of Iran, the scores of student exams in different classes were added to the data as a variable for evaluating his/her performance. The wide range of class subjects includes math, science, foreign language, sport and art, was used two times in the semester for a comprehensive evaluation. These grades were obtained from the assessment of students during the semester by teachers of each class which specified in the dataset with "first class name midterm" and "second class name midterm".”

 

The Comment:

In the discussion and conclusion section, the paper shows that some more important characteristics to the school performance like absence, math, attention, science, sport, art, science, language, internet use, sport, and study time. Also, it shows the characteristics less important like age, free time, father and mother education, extra class, work, go-out, friendship, and TV use. These results are exciting, but it is necessary for a better discussion centered on school performance. All this information is exciting but required a better discussion and conclusion.

Response:

The authors completely agree with the reviewer’s comment that this sentence is not clear. We have applied this and improved discussion to the revised version of our manuscript. It reads (Page 12, Paragraph 2, and Line 435):

“Indeed, attention in the class is very important, and whatever the attention of the student is more in class, they will have more focus and more understanding of the class content [34]. So, to improve the student's performance, attractions in the classroom must be created in order for students have more enthusiasm to pay attention. One of the disadvantages in the education system of Iran is the teacher-centered classroom. by changing this method to student-centered class, more attraction will be created and the attention of students will be attracted. moreover, in cases where the student becomes absent, he/she should not be left behind from the curriculum and it should be compensated by arrangements such as recording class for them, additional classes, additional support and so on. Another important factor is the student grades in the lessons which have much more weight compared to the rest of the variables. To evaluate student performance, only these criteria should not be considered, also other influential criteria such as problem-solving, critical thinking, creativity and Etc should be somehow included in calculations. Excessive usage of the internet and social media reduce the attention of youth and distract them [56], so less use of the internet increases the consideration and performance as well.”

56. Abbas, J., et al., The impact of social media on learning behavior for sustainable education: Evidence of students from selected universities in Pakistan. Sustainability, 2019. 11(6): p. 1683.

 

 

The Comment:

How could these results help to understand the school performance?

Response:

According to the reviewer’s comment, the discussion section was improved and the required explanation information has been applied to the revised version of the manuscript. It reads (Page 13, Paragraph 2, and Line 459)

“This valuable information can be used by politicians to adopt correct decisions and choosing appropriate strategies to improve education. The information obtained can be used to understand the improvement of student academic achievement by teachers. Efficient models and explainable AI could allow the Iran Ministry of Education to make an informed decision for better student achievement and the policies of employing AI in schools.”

 

The Comment:

Regarding the format of the document, it requires a detailed revision of the punctuation.

Response:

We thanks for the reviewer’ comment. We have submitted our manuscript to an English proofreading service. The probable editorial mistakes have been corrected in the revised version of the manuscript.

 

The Comment:

As for the references, it is necessary to review references number 25, 33, 35, 36, and 46, which lack information or are out of the required format.

Response:

Thanks for pointing out this comment. The references style converted to the ACS style mentioned in Instructions for authors.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors have replied to the reviewers comments and included several changes in the paper. The overall quality of the paper has improved. It can be accepted in its current form.

Author Response

Please kindly see the attached pdf file including the responses to the reviewers as well as the revised manuscript.

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors of this paper included new references in the introduction that allow a better understanding of the Iranian educational context. They also better described the way in which is proposed characteristics of the research for understanding and analyzing the results.

However, questions related to the methodology were not fully answered. For example, it was asked whether the subjects listed in Table 1 correspond to the aspects that would be analyzed to evaluate student performance. The answer is ambiguous since that they tell something, but they do not explain in what way. How do the 35 characteristics indicated by the authors reflect the students' performance in the indicated subjects? What does it mean that the 35 characteristics were selected for further research? Are the characteristics the same as the variables mentioned in the discussion? How was it determined the 35 selected characteristics from a total of 65? Why will these characteristics help to evaluate the students' performance?

The research questions are not fully answered. For example, "which of the three types of data, including demographic, behavior, and grades, have the most influence?" It was never discussed. Another question: "When we run models on data that only includes the most important features. What impact on their performance is occurs?" They were not shown in the paper.

The discussion section does not clarify why the characteristics considered best reflect student performance. From the results, the authors infer that attention is an essential factor. Based on this, they concluded that this is the problem with Iran's educational model. Later, the authors write that the results with the most negligible value are the use of the internet and social media. However, these implications are not deduced from the results. In addition, the authors point out that the study presents a series of problems, such as not having an adequate educational database. Also, they had problems in the data collection because there was no control over the students. As a result, it is unknown if the students understood all the questions, among other aspects. How can one be sure of this? With these methodological problems, is it possible to make inferences about student achievement?

In the conclusions, the activities carried out are summarized, and the attention finding is emphasized, but without further support or articulation with other information about the population. In the discussion, the authors write that this is the problem and that what needs to be done is to make sure that students do not lose attention to improve the educational system.

 

Author Response

Please kindly see the attached pdf file including the responses to the reviewers as well as the revised manuscript.

Author Response File: Author Response.pdf