Studying Learner’s Perception of Attaining Graduate Attributes in Capstone Project Units Using Online Flipped Classroom
Round 1
Reviewer 1 Report
Assessing Learner's Level of Attainment of Graduate Attributes in Capstone Project Units Trialing Online Flipped Classroom
General note: The study focuses on a key subject in higher education research that concerns the ways to develop graduates’ employability. It uses original data and contributes to the available literature by showing the application of knowledge and skills acquired at school, and the benefits of flipped learning in the context of online classes. However, there are aspects that can be clarified to improve the paper. Some details are presented in the next paragraphs.
Introduction: It includes the major elements of the study. I think it should not include a list of graduates’ attributes and its definition. This should be moved to theoretical framework.
Literature review: The Authors include relevant literature related with the subject of the study but should say something about other ways of increasing employability. The study is about pedagogical methods, and the authors should provide a summary of studies about the how pedagogical methods influence employability skills. The LR may be organised in the different way: the employability skills and graduates’ attributes; ways to improve both; different pedagogical methods that attempt to such outcomes; and finally, the flipped learning.
As regards the end of literature review, the Authors should include some hypotheses or propositions that would guide the empirical analysis. Instead, they repeat the goals and add information on the sample, which should be placed in the method section.
Methodology and data: It lack information on the population of students and how much the sample is representative of that population. At the end of this section, the authors should provide summary statistics on sociodemographic characteristics of students, their grades, perception of the usefulness of learned knowledge and skills, among others. This is relevant to understand whether some unobserved characteristics affect the outcomes and identify the true benefits of flipped learning. A comparison among men/women, young/adult would help to reinforce or relativise the achieved results.
Empirical evidence: Care should be taken in the writing of the findings. Expressions such as “strong evidence” should be avoided. The sample is small, information of its representativeness is lacking. Some additional estimation should be performed with data of socioeconomic features and other characteristics reported earlier.
Conclusions: The study has limitations, which are not reported in the paper. What are implications of this study for higher education policy and pedagogy?
Final note: The study has potential for publication but there some writing planning and methodological issues that deserve due attention.
Author Response
Response to Reviewer 1 Comments
Point 1: Introduction: It includes the major elements of the study. I think it should not include a list of graduates’ attributes and its definition. This should be moved to the theoretical framework.
Response 1: We agreed that the definition of graduate attributes did not contribute significantly to the introduction part of the manuscript - therefore, it has been moved to another part (section 2.1 methodological framework) of the paper, as recommended.
Point 2: Literature review: The Authors include relevant literature related to the subject of the study but should say something about other ways of increasing employability. The study is about pedagogical methods, and the authors should provide a summary of studies about how pedagogical methods influence employability skills. The LR may be organised in a different way: the employability skills and graduates’ attributes; ways to improve both; different pedagogical methods that attempt to such outcomes; and finally, the flipped learning.
Response 2: We agreed that the literature review could provide a more systematic discussion on how different teaching pedagogy influences the students' graduate attributes (intimately linked to the student employability, as well). We have included more literature to provide a more systematic view of the current educational pedagogies. The literature demonstrates how teaching pedagogy influences the students' experiences and learning outcomes. The literature then focuses on the role(s) of flipped learning within the realm of teaching pedagogies. As the current manuscript is of small scale, it could not carry a comprehensive review of the wider teaching pedagogies. Still, the manuscript now includes an improved list providing a better-structured view. The intention was to refer the audience to further reading and information, as will be needed.
- Strelan, P., Osborn, A., & Palmer, E. (2020). The flipped classroom: A meta-analysis of effects on student performance across disciplines and education levels. Educational Research Review, 30, 1-22.
- Brewer, R., & Movahedazarhouligh, S. (2016). Successful stories and conflicts: A literature review on the effectiveness of flipped learning in higher education. Journal of Computer Assisted Learning, 34, 409 - 416.
- Fadol, Y. Aldamen, H., & Saadullah, S. (2018). A comparative analysis of flipped, online and traditional teaching: A case of female Middle Eastern management students. The International Journal of Management Education, 16, 266 – 280.
Point 3: As regards the end of the literature review, the Authors should include some hypotheses or propositions that would guide the empirical analysis. Instead, they repeat the goals and add information on the sample, which should be placed in the method section.
Response 3: We agreed and improved the latter part of the literature review – to highlight the proposition of the paper and its intended contributions – which now provides a better motivation to the methodological and experimental sections. Furthermore, we have clarified in the Introduction (Section 1) our primary research question and the associated hypothesis to guide the empirical analysis based on the collected data.
Point 4: Methodology and data: It lacks information on the population of students and how much the sample is representative of that population. At the end of this section, the authors should provide summary statistics on sociodemographic characteristics of students, their grades, perception of the usefulness of learned knowledge and skills, among others. This is relevant to understand whether some unobserved characteristics affect the outcomes and identify the true benefits of flipped learning. A comparison among men/women, young/adult would help to reinforce or relativise the achieved results.
Response 4: We agreed that more information on the student demographics would communicate the experimental outcomes in a more meaningful manner. Therefore subsection 2.2 now carries more information including, survey participation rates, sex of participants, their level of studies (postgraduate vs. undergraduate), and their discipline of studies (e.g., Business vs. IT). More comprehensive information on the student demographic is available on the published website of the institute. However, as we are sampling a fraction of the students from the institute (we are sampling only the students in their final year), it may be inappropriate to state extra information that could be erroneous. Nonetheless, the current demographic information provides a broad view of the student discipline, sex, age group (e.g., postgraduate vs. undergraduate), and coupled with the survey participation rates from each cohort – representing various important information in relation to this study – which is discussed in full within the manuscript.
Point 5: Empirical evidence: Care should be taken in the writing of the findings. Expressions such as “strong evidence” should be avoided. The sample is small, information of its representativeness is lacking. Some additional estimation should be performed with data of socioeconomic features and other characteristics reported earlier.
Response 5: We agree that we could take more care when reporting some outcomes. We acknowledge that such a tone of voice may be inappropriate for a scientific paper. As such, the manuscript was revised to remove such tones/remarks.
Point 6: Conclusions: The study has limitations, which are not reported in the paper. What are implications of this study for higher education policy and pedagogy?
Response 6: The COVID19 is having a profound effect on the higher education industry overall, and the aftermath of such the pandemic is expected to have prolonged lingering effects. In such an uncertain time, we believe any worthwhile study will be of value and of contributions. As we study the perception of the students on flipped learning – on online platforms (with significant uptake due to the unexpected pandemic), the survey study has the potential to inform future improvements. This study is expected to make a significant contribution to inform educational practitioners of these outcomes – in better preparation of their own teaching methodologies and inform the relevant authority in future consideration based on the study of students’ perception.
Point 7: Final note: The study has potential for publication but there some writing planning and methodological issues that deserve due attention.
Response 7: The manuscript has been revised significantly in its structure to align the project objectives, methodologies, evaluations, and contributions to the community in educational science.
The authors would like to express sincere gratitude for the review comments, which helped to shape the manuscript to be more readable, structured, and overall better focused. Thanks.
Reviewer 2 Report
This paper provides a good analysis of the impact of the flipped classroom approach on learner’s attainment of graduate attributes in five capstone project units. The paper is well organized and presented. However, it would be better if the research questions could be put more notably such as Research question (RQ)1 - has the flipped classroom approach affected our students’ level of attainment of graduate attributes in capstone project units? (p. 3, line 102) Then, to answer the RQ1 and other RQs accordingly in the Result Section. Furthermore, it would be useful to give more background information about the capstone projects such as to delineate more about the flipped classroom approach. For example, materials to be learned online and activities that learners have to interact with one another so that readers could have a clearer idea of what has been done. It would also be useful if future research direction is included at the end of the paper. Congratulations for submitting such an insightful paper.
Author Response
Response to Reviewer 2 Comments
This paper provides a good analysis of the impact of the flipped classroom approach on learners' attainment of graduate attributes in five capstone project units. The paper is well organized and presented.
Point 1: However, it would be better if the research questions could be put more notably, such as Research question (RQ)1 - has the flipped classroom approach affected our students’ level of attainment of graduate attributes in capstone project units? (p. 3, line 102) Then, to answer the RQ1and other RQs accordingly in the Result Section.
Response 1: The authors appreciate the constructive feedback and encouragements from the reviewer.
The authors have accepted the recommendation – and included a clear ‘research question’ in the introduction (highlighted in Italic). The whole manuscript has since addressed one research question – its purpose/objectives, literature review, experimental testing including surveys, and conclusion based on the analysis of the survey data. The entire manuscript is now better structured and focused on answering the research question on "the student perception of learning improvement based on flipped learning."
Point 2: Furthermore, it would be useful to give more background information about the capstone projects, such as to delineate more about the flipped classroom approach. For example, materials to be learned online and activities that learners have to interact with one another so that readers could have a clearer idea of what has been done. It would also be useful if future research direction is included at the end of the paper. Congratulations for submitting such an insightful paper.
Response 2: The authors have accepted and updated the manuscript literature review section that addresses how flipped learning framework has been carried out in the institute. This addresses how the pre-recording was made available, how the classroom was managed, and how the instructor interacted and engaged with the students. The updates should provide a better insight into the study and strengthen the discussion within the paper.
The authors would like to express gratitude for the feedback which helped the manuscript to be more insightful and relevant. Thank you.
Reviewer 3 Report
The paper presents a current topic but not new, on which there are already many studies. Therefore, the literature on this topic is huge, hence the literature review could be broader and also include more recent literature. I suggest authors should go through the following papers (all of them about flipped classroom and its results):
- Brewer and Movahedazarhouligh (2018). Journal of Computer Assisted Learning.
- Fadol, Aldamen and Saadullah (2018). International Journal of Management Education.
- Strelan, Osborn and Palmer (2020). Educational Research Review
- Zheng, Bhagat, Zhen and Zhang (2020). Educational Technology & Society.
I think the real objective of the paper is not clear. What are the research questions? Why don’t you stablish some hypothesis? I miss the research model.
It does not explain in detail how the flipped classroom was carried out, in these five courses. Since it is the central axis on which the article is developed, it should have been worked more in depth. It is not enough just to say that this methodology was used in these groups, but also to explain in detail how these sessions were carried out, how many sessions were carried out, what contents were worked on, what was the work done before, during and after the class; as well as the evaluation method and the relationship between pupil and teacher.
As for the item A5, it is not clear to me why it is included in the “Affective” dimension. It seems to me that it would fit better, from its description, in the “Professional” dimension (it is more similar to a professional competence: written and oral communication).
The authors present some descriptive analysis and ANOVA to find differences between courses. Again, as it is not clear the purpose of the study, why do you do this analysis? Why not a comparison between these groups and others without flipped methodology?
In the discussion section, I consider there are some paragraphs that fit better in the results section, as they only show percentages of responses (for example: page 13, from line 392 to line 407 and from line 422 to 427 and from line 433 to 436; page 14, from line 445 to 448 and from line 450 to 458). The discussion section should show the interpretation that authors make from the results and the coherence or not with previous studies from other authors. Therefore, I consider that this section is the poorest one of the paper and needs to be enriched.
In the conclusion section, authors stablished that “this article has examined… their academic performance” (of students) but It is not true. The study does not analyse any type of students’ performance (marks, results, grades…)
It is not clear which is the final conclusion of the article. What this work brings to the knowledge of flipped classroom? It is also interesting to present both the theoretical and practical implications of this research.
Finally, the study has some limitations, but authors doesn’t include them at the end of the paper. For example, there is no control group and the number of students is not large enough to consider that results could be generalized. I recommend to include some limitations and future research lines at the end of the conclusion section.
Author Response
Please see the attachment.
Author Response File: 
Author Response.docx
Round 2
Reviewer 3 Report
The authors have included many of the recommendations made in my previous review and, as a result, I believe the article is much improved.
However, I still have concerns with the Discussion and Conclusion sections, which I still find a little weak.
I think that more comparison with previous results from other studies (even if they do not refer to the same type of subject) and a more in-depth analysis of the results obtained are needed.
Also, the limitations of the study and future lines of research are still missing in the paper.
Author Response
Response to Reviewer 3 Comments (2nd Round)
- The authors have included many of the recommendations made in my previous review and, as a result, I believe the article is much improved.
Response: We express deep appreciation for the constructive comments from the reviewer that have enabled us to make considerable improvements on the article.
- I still have concerns with the Discussion and Conclusion sections, which I still find a little weak. I think that more comparison with previous results from other studies (even if they do not refer to the same type of subject) and a more in-depth analysis of the results obtained are needed.
Response: In response to the reviewer’s concerns, we have included additional content in the Discussion section to compare our study with the results from other related studies. These related studies have also been mentioned in Section 1.1 on Literature Review.
For ease of reference, the additional contents are highlighted in the revised manuscript. Their locations in the revision are summarised below:
- Lines 422-436 (pages 13-14)
- Lines 451-456 (page 14)
- Lines 460-466 (page 14)
- Lines 488-499 (page 15)
- Lines 514-522 (page 15)
- The limitations of the study and future lines of research are still missing in the paper.
Response: In response to this comment, we have expanded the parts on limitations of the study and the future lines of research in the Conclusions.
For ease of reference, the additional contents are highlighted in the revised manuscript. Their locations in the revision are summarised below:
- Lines 552-573 (page 16)
