Next Article in Journal
Impact of the COVID-19 Pandemic on CSR Activities of Healthcare Providers
Previous Article in Journal
The Impact of Retired Immigrants on Quality of Life for the Local Aging Population: Results from the Southeast Spanish Coast
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 20
Issue 1
10.3390/ijerph20010367
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Subjective Outcome Evaluation of Instructional Videos in Leadership Education

by
Xiang Li
,
Daniel T. L. Shek
*,
Tingyin Wong
and
Lu Yu
Department of Applied Social Sciences, The Hong Kong Polytechnic University, Hung Hom, Hong Kong, China
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2023, 20(1), 367; https://doi.org/10.3390/ijerph20010367
Submission received: 28 November 2022 / Revised: 23 December 2022 / Accepted: 23 December 2022 / Published: 26 December 2022
(This article belongs to the Section Mental Health)
Versions Notes

Abstract

:
The aims of this study were to examine the psychometric properties of two subjective outcome evaluation tools measuring students’ perceptions of 24 instructional videos and to understand the profiles of students’ perceptions of the videos. Online teaching and learning played an important role when school lockdown measures were imposed during the COVID-19 pandemic. To facilitate online teaching in a college-level leadership education subject, we developed and piloted 24 instructional videos, including 15 animated videos and 9 case-based videos, in the 2021/22 academic year. To understand students’ perceptions of the videos, we developed two subjective outcome evaluation scales (one for the animated videos and another for the case-based videos) to assess the subjective perceptions of 1308 students. Results showed that the developed tools possessed good psychometric properties, including factorial, convergent and discriminant validity. The findings of this study also revealed the students had positive attitudes towards the developed videos, including positive perceptions of the videos’ design and the benefits gained from watching them. The present study suggests teachers can meaningfully use the 24 instructional videos in the context of leadership education in higher education.

1. Introduction

Hong Kong university students are facing various challenges and encountering different developmental issues, such as personal problems, unhealthy lifestyles, mental health issues and social connection problems [1,2]. Moreover, mental health problems are prevalent among Hong Kong youths, including depression, anxiety and excessive stress [3,4,5], internet addiction [6,7], materialistic beliefs and egocentric behaviors [8]. Unfortunately, efforts to promote life skills and social-emotional learning skills amongst young people are weak in different education sectors, including higher education [9].
With globalization, the changing economic ecology requires students to acquire skills specific to the 21st century [10]. Cornali [11] argued that both hard skills (e.g., job- and discipline-specific skills, technical and cognitive abilities) and soft skills (e.g., creativity, flexibility, resilience, communication and problem-solving skills) are essential to one’s employability. In Hong Kong, the service economy requires employees to have strong intrapersonal and interpersonal competences [12]. However, during the COVID-19 pandemic, the cultivation of soft skills such as interpersonal competence, effective communication and problem-solving skills was particularly difficult because of social distancing and school lockdown measures [13].
At one university in Hong Kong, we have developed a leadership subject entitled “Tomorrow’s Leaders” based on the positive youth development (PYD) approach, which focuses on young people’s strengths and potentials. The PYD approach was conceptualized by Lerner et al. [14] as a developmental process resulting from an individual’s engagement with his/her ecology in a positive manner. According to this view, positive development in young people is shown by five qualities—character, competence, confidence, connection and caring. Moreover, Catalano et al. [15] proposed 15 PYD constructs (e.g., competence in emotional management and social relationships, morality, spirituality and prosocial norms) as basic indicators of PYD. Based on these PYD theories, we developed and launched the 3-credit “Tomorrow’s Leaders” subject in the 2012/13 academic year. This subject consists of 13 lectures covering ten intrapersonal and interpersonal qualities, such as social competence, cognitive competence, resilience, morality, spirituality, self-leadership and law-abidance leadership [16,17]. In each lecture, we use experiential learning pedagogies, including class and group discussion, self-reflection, role-playing activities and case studies, to enrich students’ learning experience. Since the launch of the “Tomorrow’s Leaders” subject in the 2012/13 academic year, we have conducted regular evaluations. For quantitative evaluation, objective outcome evaluations [17,18], subjective outcome evaluations [19] and process evaluations [20] have been implemented. For qualitative evaluation, we have collected data based on personal reflections [21] and focus group interviews [22]. These evaluation studies revealed the effectiveness of this subject in promoting PYD attributes in university students and strengthening their mental well-being.

1.1. The Need for Virtual Teaching and Learning Materials

Electronic learning (e-learning) is conceptualized as the use of technology (e.g., media and digital electronic tools) to facilitate learning [23]. Empirical studies show that various technologies (e.g., mobile phone, online platform and 3-D virtual world) have brought benefits to students’ learning [24,25,26]. During the COVID-19 pandemic, school lockdowns negatively affected the learning of nearly 1.6 billion students, or 94% of learners, globally [27]. Therefore, online teaching and learning plays an important role. Online teaching offers synchronous (e.g., the delivery of lectures through video conferencing platforms) and asynchronous (e.g., pre-recorded lecture videos) learning environments [28]. In Beijing Normal University, out of the 4036 courses offered in the semester starting in February 2020, 3238 were delivered online [29]. There are studies suggesting that many applications of online courses during the pandemic were crisis-response migration methods adopted by universities [30,31]. Many teachers and students were not well-prepared for online learning. This situation was particularly severe in developing countries where students faced various technical and monetary issues in online learning [32]. Obviously, there is a need to reflect on the issues and policies that surrounded teaching and learning in an online setting during the COVID-19 pandemic [33,34].
There is a growing trend of video-based learning, where educational videos are combined with different teaching and learning activities [35], such as hands-on skills exercises and virtual case discussions in dental schools [36] and PowerPoint slides, readings and face-to-face practical tutorials in data science labs [37]. Before the COVID-19 pandemic, Kay [38] found that learners had positive affective and cognitive attitudes towards video podcasts (including audiographs, video streaming and webcasting). Scagnoli et al. [39] reported that video lectures enabled students to have control over the media and feel the presence of the instructors, thus increasing their sense of engagement with the online course content. Other studies also supported the effectiveness of video lectures in promoting students’ learning outcomes [40]. During the pandemic, Wikandari et al. [41] found that using animated videos in flipped learning significantly enhanced students’ ability to comprehend and apply biochemistry knowledge. Wu et al. [42] also showed that theoretical knowledge and student satisfaction in the experimental group (video-based program plus flipped classroom) were higher than that of students in the control group. In addition, other studies demonstrated the effective use of instructional videos in reinforcing teamwork activities amid the pandemic [43]. To conclude, although virtual teaching and learning had already shown to offer promising learning outcomes before the pandemic, its teaching and learning effectiveness during the pandemic has not been clearly established. Therefore, the development and assessment of educational videos in a college-level leadership subject during the pandemic could fill this knowledge gap in the literature.

1.2. Research Gaps in the Literature

There are three research gaps regarding the use of videos in leadership education, particularly during the pandemic. First, there are very few attempts to develop video-based teaching and learning materials. Jenkins [44] suggested that class discussion, individual projects, group presentations, self-reflections and personal assessments were the dominant pedagogies in U.S. undergraduate leadership education. Within the classroom setting, videos are especially useful for experiential learning [45]. Evaluating the use of video simulation of clinical scenarios (e.g., moral dilemmas, patient management) in nursing education, Sharpnack et al. [46] found that video simulation was able to enhance students’ critical thinking skills and develop other competencies related to leadership and safety. These findings highlight the significant benefits of video-based teaching for university leadership education.
Second, there are relatively few attempts to develop video-based teaching and learning materials for undergraduate programs in Hong Kong. Searching the Web of Science database with the keywords “video-based teaching and learning” and “Hong Kong” on 15 November 2022, there were only 28 citations. Furthermore, most of the papers were on teacher professional training and secondary school students, with only three studies investigating video-based teaching in undergraduate courses [47,48,49]. Moreover, although traditional Chinese students expected didactic teaching and learning, the younger generation increasingly favor interactive pedagogies in recent years. With the use of smart phones and exposure to social media, young people are more receptive to audio-visual teaching materials.
Third, there are very few existing evaluation studies on video-based materials in virtual teaching and learning. Evaluation is needed to ensure the instructional effectiveness of newly developed teaching materials. Howe [50] used a 12-item student survey to assess students’ perceived educational benefits of a CD-ROM with images and instructional videos on basic veterinary surgical skills. However, the psychometric properties of the assessment tool are not clear. Schooley et al. [51] highlighted the lack of standardized scales to measure educational videos’ quality and developed the instructional video quality checklist (IVQC) to assess how multimedia principles have improved students’ learning efficiency. However, the psychometric properties of the scale, particularly its dimensionality, are not clear.
The development of virtual teaching and learning (VTL) materials such as videos was urgently needed during the pandemic due to the heavy reliance on online teaching and learning. With the support of the University Grants Committee, our project team started developing animated videos and case-based videos to facilitate the teaching of “Tomorrow’s Leaders” in 2021. In Semester 2 of the 2021/22 academic year, 15 animated videos and 9 case-based videos were developed and piloted. We have developed more animated videos than case-based videos (15 versus 9) for two reasons. First, since we already had some case videos but no animated video, we developed more animated videos to fill this gap in our teaching materials. Second, studies found that animated videos are appealing to young people, and they can produce significant learning outcomes [52,53,54]. As such, we believed that developing more animated videos could bring significant benefits to students’ learning.
Moreover, because of social distancing measures, this subject was conducted using an online synchronous teaching mode. At the same time, we applied the flipped classroom approach to deliver the 24 videos. Students were required to watch these videos to learn the course content and finish the reflection activities before attending the online lectures. The intention in developing this e-learning module was to enrich students’ understanding of related topics before joining the 2-h face-to-face or online synchronous lectures. A brief description of these videos and reflection activities are listed in Table 1 and Table 2. Each lecture contains at least one animated video and one case-based video (except for the lectures on moral competence and law-abiding leadership which contain no case-based video).

1.3. The Present Study

As mentioned above, we have conducted evaluations to measure the teaching effectiveness of the “Tomorrow’s Leaders” subject since the 2012/13 academic year. One evaluation strategy is the application of a subjective outcome evaluation to measure students’ satisfaction with and perceptions of the instructional effectiveness of the subject via the client satisfaction approach [19]. Researchers in many fields (including education and social work) have applied the client satisfaction approach to assess the degree to which consumers are satisfied with the process and outcomes of an intervention program. It is also used to measure service recipients’ perceptions of how their needs are met after an intervention [55]. In higher education, studies have emphasized the importance of assessing education quality from the students’ perspective. Wang, Sun, & Jiang [56] argued that students’ perceptions of education quality could be divided into three dimensions—students’ engagement in learning, students’ satisfaction and students’ evaluation of teaching. A previous study revealed that subjective outcome evaluation was significantly associated with objective outcome evaluation [57]. Zhu and Shek [58] also highlighted several advantages of subjective outcome evaluation, such as efficiency, cost-effectiveness and significance through data accumulation [58].
In this study, based on an integration of the previous studies and existing measures, we developed two subjective outcome evaluation scales to assess students’ perceptions of the 24 videos: one for the animated videos (12 items) and one for the case-based videos (14 items). Each scale measures the content, implementation quality and effectiveness of the videos. Since the 24 videos were part of the lectures, two items were added to evaluate students’ perceived effectiveness of each lecture. There are two research objectives of this study:
  • To examine the psychometric properties of two subjective outcome evaluation scales for evaluating animated videos and case-based videos.
  • To evaluate students’ perceived quality of 24 newly developed videos in terms of their content, implementation and effectiveness using the two subjective outcome evaluation scales.

2. Materials and Methods

To facilitate online teaching of the “Tomorrow’s Leaders” subject, our project team developed and piloted 15 animated videos and 9 case-based videos. These videos were uploaded to a digital teaching support system, “Blackboard Learn”, along with other e-learning modules of the subject. Moreover, two evaluation scales, including one for animated videos and another for case-based videos, were designed and uploaded to the e-learning platform. Students were invited to complete the evaluation forms after watching the videos every week in a voluntary and anonymous manner. The psychometric properties of these scales and students’ perceptions of the videos were analyzed at the end of the semester.

2.1. Participants

In the second semester of the 2021/22 academic year, 1308 Year 1 undergraduate students took “Tomorrow’s Leaders.” These students studied at the following seven schools or faculties: School of Fashion and Textiles, School of Design, School of Hotel and Tourism Management, Faculty of Science, Faculty of Health and Social Sciences, Faculty of Engineering and Faculty of Construction and Environment. To maintain anonymity of the participants and to encourage their participation in the evaluation, we did not collect demographic information from the students. Generally speaking, Year 1 students of the university have a mean age of 18 and a gender ratio of roughly one female to one male. For each of the 24 videos, 598 to 1114 students evaluated the video. For 20 of the videos, at least 900 students provided their evaluation. Table 1 and Table 2 show the content of the videos.

2.2. Procedures

We obtained ethical approval from the institutional review board of the university and consent from each participant before data collection. “Tomorrow’s Leaders” consists of 13 lectures, including one lecture for introduction, 10 lectures for different topics related to leadership and two lectures for student group presentations. From Lecture 2 to Lecture 11, all students were required to watch two to three animated or case-based videos in the e-learning module before attending the lecture each week to fulfil the learning requirement. Meanwhile, we invited all students to complete the two evaluation scales plus two items on the lecture’s effectiveness in a self-administration manner.

2.3. Instrument

We created two evaluation scales, including one for animated videos and another for case-based videos. We designed these measures with reference to the Subjective Outcome Evaluation Scale (SOES) specifically developed for participants to evaluate the quality of a project in terms of its content, implementation and effectiveness [19,57,58]. While we used 12 items for the animated videos (e.g., “the animated video has helped me to reflect on the topic”), we used 14 items to evaluate the case-based videos (e.g., “the video-based activity has enabled me to apply the theories in real life cases”). There are two more items for case-based videos because case-based videos contain classroom discussion during the lectures. As such, items assessing the interaction between the teacher and the students and the interaction between the students were included in the scale for case-based videos. Moreover, two items were added to measure the effectiveness of each lecture. For each item, the students rated it on a six-point Likert scale, with 1 representing ‘strongly disagree’ and 6 representing ‘strongly agree’. Higher ratings indicate a better appraisal of the quality of the videos by students. The reliabilities of the two scales were excellent, with scores ranging from 0.96 to 0.99 for the 15 animated videos and 0.97 to 0.99 for the 9 case-based videos.

2.4. Data Analytic Plan

In this study, Mplus Version 8.8 was used to conduct confirmatory factor analyses (CFAs) to assess the two rating scales’ factorial validity. To evaluate the one-factor model for each model, we examined a series of model fit indices, including the comparative fit index (CFI), the non-normed fit index (NNFI), the standardized root mean square residual (SRMR) and the root mean square error of approximation (RMSEA). For CFI and NNFI, equal or above 0.90 values indicate a good fit; for SRMR and RMSEA, equal or below 0.08 values imply a close fit [59].
Furthermore, convergent and discriminant validity were assessed by calculating the average variance extracted (AVE). AVE ≥ 0.50 represents a favorable convergent validity. Reliability of the scale was evaluated by calculating the composite reliability (CR). CR ≥ 0.70 indicates a good reliability [60,61]. We performed separate factor analyses for the responses to each of the 24 videos.
After confirming the validity and reliability of the two rating scales, we used the two scales and the two items on lecture effectiveness to evaluate students’ perceptions of the content, implementation and educational effectiveness of the 24 videos. Descriptive statistics, particularly percentages of positive responses to each item of the scales, were conducted for each video. Our previous studies on subjective outcome evaluation adopted a similar approach to measure students’ perceptions of the quality and effectiveness of the “Tomorrow’s Leaders” subject [17,19].

3. Results

3.1. Psychometric Properties of the Two Evaluation Tools

The findings of the CFAs showed that the one-factor model fits the data very well for both scales of the animated videos and case-based videos separately (see Table 3 and Table 4). For the 15 animated videos, the one-factor model of the 12-item measure showed a good model fit: CFIs ≥ 0.94, NNFIs ≥ 0.92, SRMRs ≤ 0.04 and RMSEAs ≤ 0.08. Good AVEs ≥ 0.65 and CRs ≥ 0.96 were found. For the nine case-based videos, the one-factor model of the 14-item measure showed a good model fit as well: CFIs ≥ 0.93, NNFIs ≥ 0.92, SRMRs ≤ 0.04 and RMSEAs ≤ 0.07. Good AVEs ≥ 0.66 and CRs ≥ 0.96 were also found. All model fit indices supported the one-factor model of the scales for both animated videos and case-based videos.

3.2. Client Satisfaction Responses

For the animated videos, the findings revealed most of the students had positive views towards the content of the videos and their implementation and educational effectiveness (Table 5). For example, for the item “the animated video is interesting”, the mean scores of the 15 animated videos ranged from 4.53 to 4.9, and the positive response rates varied between 90.81% and 96.65%. In addition, for the item “the animated video can benefit my development”, the 15 videos received mean scores ranging from 4.68 to 4.95, with positive response rates varying between 93.58% and 97.94%. These findings suggested that the majority of students had positive perceptions of the animated videos’ content, the delivery of the videos in the online modules and the contribution of these videos to their learning in this subject.
Similarly, for the case-based videos, students had a good appraisal of the video content, implementation and their educational effectiveness (Table 6). For example, for the item “the content of the video is stimulating”, the nine case-based videos received mean scores from 4.70 to 5.01, with positive response rates varying between 96.05% and 98.02%. In addition, the item “the video-based activity has enabled me to apply the theories in real life cases” had mean scores ranging from 4.68 to 4.99, and its positive response rates varied between 93.60% and 97.67% across the nine videos. These figures revealed the majority of students had positive views towards the content of the case-based videos, their utilization as well as educational effectiveness.
Finally, the last two items measured students’ perceptions of the general effectiveness of the lectures. These two items received highly positive responses (Table 7). For example, the topic of Lecture 3 is “Cognitive Competence”, and it contains two videos, “Cognitive Competence” and “Myths of MMR Vaccine”. The findings showed that 97.44% of students perceived the lecture had enriched their development, with a mean score of 4.79. The evaluation results showed that students were very satisfied with the integration of the 24 videos in “Tomorrow’s Leaders” lectures.

4. Discussion

To enrich the teaching materials for a leadership subject, we developed 24 instructional videos (15 animated videos and 9 case-based videos) and conducted an evaluation of the developed videos. There are a number of special features of this study. First, as there are few instructional videos for leadership education in Hong Kong universities and colleges, we filled this gap by developing videos with reference to different aspects of leadership qualities based on the positive youth development perspective. Second, to promote the learning motivation of students, we developed both animated and case-based videos. Third, we conducted a subjective outcome evaluation of the videos based on a large-scale student survey. Fourth, we conducted confirmatory factor analyses to support the psychometric properties of the two measures for the animated videos and case-based videos. Fifth, the evaluation findings provide support for the implementation of instructional videos in teaching leadership in the higher education sector.
Conceptually, the evaluation results from this study support the proposition that instructional videos are useful in teaching and learning. In the literature on instructional videos, there are views suggesting the positive value of instructional videos in promoting student motivation and teaching effectiveness [36,37,38,39,40,41,42,43]. During the pandemic, Doudesis and Manataki [37] found that undergraduate students preferred video-based online labs over instructor-led online labs in data science. Students enjoyed video-based online labs more and found them to be more useful. Wu et al. [42] revealed that relative to traditional teaching methods, college students in a video-based plus flipped classroom program showed significantly higher scores of theoretical knowledge, overall satisfaction, learning efficiency and enthusiasm, as well as problem-solving and independent learning abilities. The present study supports the existing literature on the benefits of instructional videos on students’ learning. Furthermore, the present findings expand the scope of the value of instructional videos to university leadership education in a Chinese cultural context. Of course, there is a need to understand the mechanisms involved in generating the positive impact of instructional videos, such as increasing students’ control over learning and improving their study habits [39]. Moreover, this study further supports the importance of enriching online teaching and learning for students during the pandemic. The study by Leung, Shek and Dou [62] illustrated the delivery of service-learning in an online setting during the pandemic, and the subjective and objective outcome evaluation in the study found that both college students and service recipients benefited from the online service-learning. The current study also demonstrated that instructional videos conducted in an online setting could benefit college students during this unprecedented time.
Methodologically, the present study generated two measures of subjective outcome evaluation—one for animated videos and one for case-based videos. As there are few validated tools for the evaluation of instructional videos, the findings of this study are innovative additions to the existing literature. Particularly, studies that examine related evaluation tools are almost non-existent in the current scientific literature. For example, Howe [50] and Schooley et al. [51] developed scales to measure students’ perceptions of instructional videos’ quality and their learning efficiency using these videos. However, the psychometric properties and the dimensionality of their assessment tools are unclear. There are three reasons why we should have validated assessment tools for instructional videos. First, we need scientific tools to measure students’ perceptions. By listening to student responses, teachers can have more dialogues with students on the subject matter that would further promote the teacher–student relationship. Second, it is common for tertiary education institutions to conduct subjective outcome evaluations, which is also regarded as a routine mechanism for quality assurance. Although studies have argued that the concept of quality in higher education is very complicated, it is still apparent that students are the most important stakeholders [63]. Moreover, similar subjective outcome evaluation tools have been widely used [17,19,64]. Third, evaluation findings provide cues on how teachers could improve the instructional materials as well as their teaching. It is noteworthy that some teachers used subjective outcome evaluation to evaluate the hybrid mode of teaching [65] and service-learning [66] in connection with leadership education.
Practically, the present study demonstrates the possibility of developing animated and case-based videos for leadership education built from the positive youth development theories. Because positive youth development attributes significantly contribute to the healthy development of young people and form the foundation of effective leadership, nurturing students’ positive youth development attributes is important [16]. The present study suggests the value of using instructional videos to impart knowledge on PYD. In addition, as young people welcome images (e.g., Instagram), using educational videos can facilitate college students’ learning. In particular, research has suggested the significant educational effectiveness of using instructional videos during the COVID-19 pandemic, which is still affecting many people around the world [41,42,43].
Despite the pioneer nature of the instructional videos development project, as well as the evaluation study, several limitations should be acknowledged. First, besides subjective outcome evaluation, the use of other evaluation strategies, such as objective outcome evaluation and qualitative evaluation, would produce useful findings [17]. For example, we can conduct a single group pre- and post-test study to evaluate students’ positive changes after watching the videos [18]. Moreover, collecting qualitative data is useful for us to further understand students’ subjective experiences of watching the videos. Various studies have used focus group or individual semi-structured interviews to examine students’ perceptions of newly developed teaching and learning activities and generated results that are conceptually and practically significant [67,68]. Second, as this is the first attempt to validate the instructional videos, replications of the findings at a different time using different samples are needed. As pointed out by Schmidt [69], replication is much needed in social science research to control for errors or to generalize research findings to a different or a larger population. Third, besides university students, teachers’ perceptions of the 24 videos’ value and effectiveness could offer us a new perspective to understand the value of videos in college-level leadership education. A recent study by Toto and Limone [70] also demonstrated the importance of teachers’ perceptions to the implementation of distance learning in college during the pandemic.

5. Conclusions

The present study demonstrates the use of instructional videos to educate college students on leadership knowledge based on the positive youth development approach. Strengthening young people’s positive youth development attributes could enhance their healthy development and cultivate their leadership qualities. A subjective outcome evaluation of these videos using a large-sample student survey was conducted. Two scales were developed to measure students’ perceptions of the content, implementation and effectiveness of the animated videos and case-based videos, respectively. Confirmatory factor analyses showed that the two scales possessed good statistical validity and reliability. The evaluation results suggested that the majority of students had highly positive perceptions of the videos’ content, delivery and the benefits gained from watching them. This study supports the use of instructional videos in university leadership education and offers insights on how colleges and universities could strengthen young people’s healthy development from a positive youth development perspective. As there is a need to conduct more studies on the quality of life (such as academic satisfaction) in Chinese people [71], the present study is a constructive response.

Author Contributions

Conceptualization, D.T.L.S., X.L. and L.Y.; methodology, D.T.L.S. and X.L.; formal analysis, X.L. and D.T.L.S.; resources, D.T.L.S., X.L. and L.Y.; writing—original draft preparation, D.T.L.S., X.L. and T.W.; writing—review and editing, D.T.L.S. and X.L.; supervision, D.T.L.S. and X.L.; project administration, D.T.L.S. and X.L.; funding acquisition, D.T.L.S. and L.Y. All authors have read and agreed to the published version of the manuscript.

Funding

The University Grants Committee financially support the development and evaluation of virtual teaching and learning materials in this project (89PZ). The drafting of this paper is also supported by Wofoo Foundation and the RGC Research Matching Fund (R.54.CC.83Y7).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and the related studies were approved by the Ethics Committee of The Hong Kong Polytechnic University (protocol code HSEARS20210113005; April 2021).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Shek, D.T.L.; Ma, H.K.; Sun, R.C.F. A brief overview of adolescent developmental problems in Hong Kong. Sci. World J. 2011, 11, 2243–2256. [Google Scholar] [CrossRef] [PubMed]
  2. Shek, D.T.L. Protests in Hong Kong (2019–2020): A perspective based on quality of life and well-being. Appl. Res. Qual. Life 2020, 15, 619–635. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Li, X.; Shek, D.T.L.; Shek, E.Y.W. Psychological morbidity among university students in Hong Kong (2014–2018): Psychometric properties of the Depression Anxiety Stress Scales (DASS) and related correlates. Int. J. Environ. Res. Public Health 2021, 18, 8305. [Google Scholar] [CrossRef] [PubMed]
  4. Shek, D.T.L.; Dou, D.; Zhu, X.; Wong, T.; Tan, L. Need satisfaction and depressive symptoms among university students in Hong Kong during the COVID-19 pandemic: Moderating effects of positive youth development attributes. Front. Psychiatry 2022, 13, 931404. [Google Scholar] [CrossRef]
  5. Shek, D.T.L.; Dou, D.; Zhu, X. Prevalence and correlates of mental health of university students in Hong Kong: What happened one year after the occurrence of COVID-19? Front. Public Health 2022, 10, 857147. [Google Scholar] [CrossRef]
  6. Shek, D.T.L.; Zhu, X.; Dou, D. Influence of family processes on internet addiction among late adolescents in Hong Kong. Front. Psychiatry 2019, 10, 113. [Google Scholar] [CrossRef] [Green Version]
  7. Shek, D.T.L.; Zhu, X.; Ma, C.M.S. The influence of parental control and parent-child relational qualities on adolescent internet addiction: A 3-year longitudinal study in Hong Kong. Front. Psychol. 2018, 9, 642. [Google Scholar] [CrossRef]
  8. Shek, D.T.L.; Ma, C.; Siu, A.M.H.; Lee, B.M. Materialism and egocentrism in Chinese adolescents in Hong Kong: Perceptions of teachers. Int. J. Child Health Hum. Dev. 2019, 12, 269–279. [Google Scholar]
  9. Shek, D.T.L.; Lin, L.; Ma, C.M.S.; Yu, L.; Leung, J.T.Y.; Wu, F.K.Y.; Leung, H.; Dou, D. Perceptions of adolescents, teachers and parents of life skills education and life skills in high school students in Hong Kong. Appl. Res. Qual. Life 2021, 16, 1847–1860. [Google Scholar] [CrossRef]
  10. Kennedy, T.J.; Sundberg, C.W. 21st century skills. In Science Education in Theory and Practice; Akpan, B., Kennedy, T.J., Eds.; Springer: Cham, Switzerland, 2020; pp. 479–496. [Google Scholar]
  11. Cornali, F. Training and Developing Soft Skills in Higher Education. In Proceedings of the 4th International Conference on Higher Education Advances (HEAd’18), Universitat Politècnica València, Valencia, Spain, 20–22 June 2018. [Google Scholar]
  12. Shek, D.T.L.; Chung, P.P.Y.; Leung, H. Manufacturing economy vs. service economy: Implications for service leadership. Int. J. Disabil. Hum. Dev. 2015, 14, 205–215. [Google Scholar] [CrossRef]
  13. Volkov, A.; Rishko, Y.; Kostyukhin, Y.; Sidorova, E.; Boboshko, D.; Savinova, D.; Ershova, V. Using digital tools to teach soft skill-oriented subjects to university students during the COVID-19 pandemic. Educ. Sci. 2022, 12, 335. [Google Scholar] [CrossRef]
  14. Lerner, R.M.; Lerner, J.V.; Lewin-Bizan, S.; Bowers, E.P.; Boyd, M.J.; Mueller, M.K.; Schmid, K.L.; Napolitano, C.M. Positive youth development: Processes, programs, and problematics. J. Youth Dev. 2011, 6, 38–62. [Google Scholar] [CrossRef] [Green Version]
  15. Catalano, R.F.; Berglund, M.L.; Ryan, J.A.M.; Lonczak, H.S.; Hawkins, J.D. Positive youth development in the United States: Research findings on evaluations of positive youth development programs. Ann. Am. Acad. Political Soc. Sci. 2004, 591, 98–124. [Google Scholar] [CrossRef]
  16. Shek, D.T.L. Development of a positive youth development subject in a university context in Hong Kong. Int. J. Disabil. Hum. Dev. 2012, 11, 173–179. [Google Scholar] [CrossRef] [Green Version]
  17. Chai, W.; Li, X.; Shek, D.T.L. The effectiveness of a leadership subject using a hybrid teaching mode during the pandemic: Objective outcome and subjective outcome evaluation. Int. J. Environ. Res. Public Health 2022, 19, 9809. [Google Scholar] [CrossRef]
  18. Li, X.; Shek, D.T.L. Objective outcome evaluation of a leadership course utilising the positive youth development approach in Hong Kong. Assess. Eval. High. Educ. 2020, 45, 741–757. [Google Scholar] [CrossRef] [Green Version]
  19. Yu, L.; Lin, L.; Shek, T.L.D.; Chai, W. Students’ perceived attributes and benefits of a leadership course: Subjective outcome evaluation. Res. Soc. Work Pract. 2022, 32, 199–214. [Google Scholar] [CrossRef]
  20. Shek, D.T.L.; Sun, R.C.F. Process evaluation of a leadership and intrapersonal development subject for university students. Int. J. Disabil. Hum. Dev. 2013, 12, 203–211. [Google Scholar] [CrossRef]
  21. Shek, D.T.L.; Leung, J.T.Y. Perceived benefits of a university subject on leadership and intrapersonal development. Int. J. Disabil. Hum. Dev. 2014, 13, 481–488. [Google Scholar] [CrossRef]
  22. Shek, D.T.L.; Sun, R.C.F. Focus group evaluation of a positive youth development course in a university in Hong Kong. Int. J. Disabil. Hum. Dev. 2012, 11, 249–254. [Google Scholar] [CrossRef] [Green Version]
  23. Kumar Basak, S.; Wotto, M.; Bélanger, P. E-learning, M-learning and D-learning: Conceptual definition and comparative analysis. E-Learn. Digit. Media 2018, 15, 191–216. [Google Scholar] [CrossRef]
  24. Crompton, H.; Burke, D. The use of mobile learning in higher education: A systematic review. Comput. Educ. 2018, 123, 53–64. [Google Scholar] [CrossRef]
  25. Wang, C.; Lan, Y.; Tseng, W.; Lin, Y.R.; Gupta, K.C. On the effects of 3D virtual worlds in language learning–a meta-analysis. Comput. Assist. Lang. Learn. 2020, 33, 891–915. [Google Scholar] [CrossRef]
  26. Jarmon, L.; Traphagan, T.; Mayrath, M.; Trivedi, A. Virtual world teaching, experiential learning, and assessment: An interdisciplinary communication course in second life. Comput. Educ. 2009, 53, 169–182. [Google Scholar] [CrossRef]
  27. United Nations. Policy Brief: Education during COVID-19 and Beyond. Available online: https://www.un.org/development/desa/dspd/wp-content/uploads/sites/22/2020/08/sg_policy_brief_covid-19_and_education_august_2020.pdf (accessed on 25 October 2022).
  28. Lapitan, L.D.S.; Tiangco, C.E.; Sumalinog, D.A.G.; Sabarillo, N.S.; Diaz, J.M. An effective blended online teaching and learning strategy during the COVID-19 pandemic. Educ. Chem. Eng. 2021, 35, 116–131. [Google Scholar] [CrossRef]
  29. Zhu, X.; Liu, J. Education in and after COVID-19: Immediate responses and long-term visions. Postdigit. Sci. Educ. 2020, 2, 695–699. [Google Scholar] [CrossRef] [Green Version]
  30. Ferri, F.; Grifoni, P.; Guzzo, T. Online learning and emergency remote teaching: Opportunities and challenges in emergency situations. Societies 2020, 10, 86. [Google Scholar] [CrossRef]
  31. Adedoyin, O.B.; Soykan, E. COVID-19 pandemic and online learning: The challenges and opportunities. Interact. Learn. Environ. 2020, 1–13. [Google Scholar] [CrossRef]
  32. Adnan, M.; Anwar, K. Online learning amid the COVID-19 pandemic: Students’ perspectives. J. Pedagog. Sociol. Psychol. 2020, 2, 45–51. [Google Scholar] [CrossRef]
  33. Shek, D.T.L. COVID-19 and quality of life: Twelve reflections. Appl. Res. Qual. Life 2021, 16, 1–11. [Google Scholar] [CrossRef]
  34. Shek, D.T.L.; Leung, J.T.Y.; Tan, L.D. Social policies and theories on quality of life under COVID-19: In search of the missing gaps. Appl. Res. Qual. Life 2022, in press.
  35. Sablić, M.; Mirosavljević, A.; Škugor, A. Video-based learning (VBL)—Past, present and future: An overview of the research published from 2008 to 2019. Technol. Knowl. Learn. 2021, 26, 1061–1077. [Google Scholar] [CrossRef]
  36. Van Doren, E.J.; Lee, J.E.; Breitman, L.S.; Chutinan, S.; Ohyama, H. Students’ perceptions on dental education in the wake of the COVID-19 pandemic. J. Dent. Educ. 2021, 85, 1187–1189. [Google Scholar] [CrossRef] [PubMed]
  37. Doudesis, D.; Manataki, A. Data science in undergraduate medicine: Course overview and student perspectives. Int. J. Med. Inform. 2022, 159, 104668. [Google Scholar] [CrossRef]
  38. Kay, R.H. Exploring the use of video podcasts in education: A comprehensive review of the literature. Comput. Human Behav. 2012, 28, 820–831. [Google Scholar] [CrossRef]
  39. Scagnoli, N.I.; Choo, J.; Tian, J. Students’ insights on the use of video lectures in online classes. Br. J. Educ. Technol. 2019, 50, 399–414. [Google Scholar] [CrossRef] [Green Version]
  40. Vaganova, O.I.; Rudenko, I.V.; Markova, S.M.; Smirnova, Z.V.; Kutepov, M.M. The use of educational video materials in educational process of a higher educational institution. Amazon Investig. 2019, 8, 216–222. [Google Scholar]
  41. Wikandari, R.; Putro, A.W.; Suroto, D.A.; Purwandari, F.A.; Setyaningsih, W. Combining a flipped learning approach and an animated video to improve first-year undergraduate students’ understanding of electron transport chains in a biochemistry course. J. Chem. Educ. 2021, 98, 2236–2242. [Google Scholar] [CrossRef]
  42. Wu, Y.-Y.; Liu, S.; Man, Q.; Luo, F.-L.; Zheng, Y.-X.; Yang, S.; Ming, X.; Zhang, F.-Y. Application and evaluation of the flipped classroom based on micro-video class in pharmacology teaching. Front. Public Health 2022, 10, 838900. [Google Scholar] [CrossRef]
  43. Martí Calatayud, M.C.; Cifuentes-Cabezas, M.; Rodríguez-López, A.D.; Hernández-Pérez, L.; Carrillo-Abad, J. Development of hybrid practice sessions in the chemistry lab reinforced by teamwork activities and video-enhanced learning. Rev. De Docencia Univ. (REDU) 2022, 20, 51–67. [Google Scholar] [CrossRef]
  44. Jenkins, D.M. Exploring signature pedagogies in undergraduate leadership education. J. Leadersh. Educ. 2012, 11, 1–27. [Google Scholar] [CrossRef] [Green Version]
  45. Zipp, G.P.Z.; Maher, C. Use of video-based cases as a medium to develop critical thinking skills in health science students. J. Coll. Teach. Learn. 2010, 7, 1–4. [Google Scholar] [CrossRef]
  46. Sharpnack, P.A.; Goliat, L.; Baker, J.R.; Rogers, K.; Shockey, P. Thinking like a nurse: Using video simulation to rehearse for professional practice. Clin. Simul. Nurs. 2013, 9, e571–e577. [Google Scholar] [CrossRef]
  47. Wong, M.S.; Lemaire, E.D.; Leung, A.K.L.; Chan, M.F. Enhancement of prosthetics and orthotics learning and teaching through e-learning technology and methodology. Prosthet. Orthot. Int. 2004, 28, 55–59. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  48. Wong, J.K.W.; Oladinrin, O.T.; Ho, C.M.F.; Guilbert, E.; Kam, R. Assessment of video-based e-learning in a construction measurement course. Int. J. Constr. Manag. 2022, 22, 1–7. [Google Scholar] [CrossRef]
  49. Kor, P.P.K.; Liu, J.Y.W.; Kwan, R.Y.C. Exploring nursing students’ learning experiences and attitudes toward older persons in a gerontological nursing course using self-regulated online enquiry-based learning during the COVID-19 pandemic: A mixed-methods study. Nurse Educ. Today 2022, 111, 105301. [Google Scholar] [CrossRef]
  50. Howe, L.M.; Boothe, H.W.; Hartsfield, S.M. Student assessment of the educational benefits of using a CD-ROM for instruction of basic surgical skills. J. Vet. Med. Educ. 2005, 32, 138–143. [Google Scholar] [CrossRef]
  51. Schooley, S.P.; Tackett, S.; Peraza, L.R.; Shehadeh, L.A. Development and piloting of an Instructional Video Quality Checklist (IVQC). Med. Teach. 2022, 44, 287–293. [Google Scholar] [CrossRef]
  52. Adri, H.T.; Yudianto, S.A.; Mawardini, A.; Sesrita, A. Using animated video based on scientific approach to improve students higher order thinking skill. Indones. J. Soc. Res. 2020, 2, 9–17. [Google Scholar] [CrossRef]
  53. Upson-Taboas, C.F.; Montoya, R.; O’Loughlin, V.D. Impact of cardiovascular embryology animations on short-term learning. Adv. Physiol. Educ. 2019, 43, 55–65. [Google Scholar] [CrossRef]
  54. Alam, K.; Suryatna, U.; Kusumadinata, A.A. Improving student literacy skills through visual communication media. Indones. J. Soc. Res. 2020, 2, 126–133. [Google Scholar] [CrossRef]
  55. Brestan, E.V.; Jacobs, J.R.; Rayfield, A.D.; Eyberg, S.M. A consumer satisfaction measure for parent-child treatments and its relation to measures of child behavior change. Behav. Ther. 1999, 30, 17–30. [Google Scholar] [CrossRef]
  56. Wang, D.; Sun, Y.; Jiang, T. The assessment of higher education quality from the perspective of students through a case study analysis. Front. Educ. China 2018, 13, 267–287. [Google Scholar] [CrossRef]
  57. Shek, D.T.L. Subjective outcome and objective outcome evaluation findings: Insights from a Chinese context. Res. Soc. Work Pract. 2010, 20, 293–301. [Google Scholar] [CrossRef] [Green Version]
  58. Zhu, X.; Shek, D.T.L. Subjective outcome evaluation of a positive youth development program in mainland China. Res. Soc. Work Pract. 2021, 31, 285–297. [Google Scholar] [CrossRef]
  59. Kline, R.B. Principles and Practices of Structural Equation Modeling, 4th ed.; Guilford Publications: New York, NY, USA, 2015. [Google Scholar]
  60. Fornell, C.; Larcker, D.F. Evaluating structural equation models with unobservable variables and measurement error. J. Mark. Res. 1981, 18, 39–50. [Google Scholar] [CrossRef]
  61. Hair, J.F. Multivariate Data Analysis, 6th ed.; Pearson Prentice Hall: Upper Saddle River, NJ, USA, 2006; ISBN 0130329290. [Google Scholar]
  62. Leung, H.; Shek, D.T.L.; Dou, D. Evaluation of service-learning in project Wecan under COVID-19 in a Chinese context. Int. J. Environ. Res. Public Health 2021, 18, 3596. [Google Scholar] [CrossRef]
  63. Degtjarjova, I.; Lapina, I.; Freidenfelds, D. Student as stakeholder: “Voice of customer” in higher education quality development. Mark. Manag. Innov. 2018, 2, 388–398. [Google Scholar] [CrossRef] [Green Version]
  64. Coman, C.; Țîru, L.G.; Meseșan-Schmitz, L.; Stanciu, C.; Bularca, M.C. Online teaching and learning in higher education during the coronavirus pandemic: Students’ perspective. Sustainability 2020, 12, 10367. [Google Scholar] [CrossRef]
  65. Shek, D.T.L.; Zhu, X.; Li, X.; Dou, D. Satisfaction with HyFlex teaching and law-abiding leadership education in Hong Kong university students under COVID-19. Appl. Res. Qual. Life 2022, 17, 2833–2858. [Google Scholar] [CrossRef]
  66. Shek, D.T.L.; Li, X.; Yu, L.; Lin, L.; Chen, Y. Evaluation of electronic service-learning (e-service-learning) projects in mainland China under COVID-19. Appl. Res. Qual. Life 2022, 17, 3175–3198. [Google Scholar] [CrossRef] [PubMed]
  67. Licorish, S.A.; Owen, H.E.; Daniel, B.; George, J.L. Students’ perception of Kahoot!’s influence on teaching and learning. Res. Pract. Technol. Enhanc. Learn. 2018, 13, 9. [Google Scholar] [CrossRef] [Green Version]
  68. Shek, D.T.L. The use of focus groups in programme evaluation: Experience based on the Project, P.A.T.H.S. in a Chinese context. In A New Era in Focus Group Research: Challenges, Innovation and Practice; Barbour, R.S., Morgan, D.L., Eds.; Palgrave Macmillan: London, UK, 2017; pp. 129–153. [Google Scholar]
  69. Schmidt, S. Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Rev. Gen. Psychol. 2009, 13, 90–100. [Google Scholar] [CrossRef]
  70. Toto, G.A.; Limone, P. From resistance to digital technologies in the context of the reaction to distance learning in the school context during COVID-19. Educ. Sci. 2021, 11, 163. [Google Scholar] [CrossRef]
  71. Shek, D.T.L.; Peng, H.; Zhou, Z. Editorial: Children and adolescent quality of life under socialism with Chinese characteristics. Appl. Res. Qual. Life 2022, 17, 2447–2453. [Google Scholar] [CrossRef] [PubMed]
Table
Table 1. Description of the 15 Animated Videos.
Table 1. Description of the 15 Animated Videos.
Video TopicVideo NumberDescription
Self-LeadershipA1
(N = 924)		Self-Understanding and Self-Management:
The video introduces concepts and theories of self-leadership, including self-understanding and self-management. It also covers emotional management skills and steps to cope with negative emotions.
Cognitive Competence A2
(N = 901)		Cognitive Competence:
The video introduces concepts and examples related to misinformation, disinformation and propaganda. Students work with their peers to find daily examples of misinformation, disinformation or propaganda.
Social-Emotional Competence A3
(N = 1039)		Social-Emotional Competence:
This video explains the definition of social-emotional competence and the four-dimensional framework of social-emotional competence.
A4
(N = 1012)		Egocentrism:
This video presents theories related to egocentrism and illustrates the concept by analyzing the performance of an egocentric person.
ResilienceA5
(N = 1092)		Resilience and Resilient Qualities:
This video introduces definitions of resilience and the concepts of internal and external resilient qualities that form protective factors of a person.
A6
(N = 1076)		Richardson’s Model of Resilience:
This video introduces Richardson’s resiliency model that highlights the process of resilience development.
Moral CompetenceA7
(N = 1114)		Moral Development:
This video illustrates Kohlberg’s Theory of Moral Development (including Heinz’s dilemma) and explains the three levels and six stages of moral development with real-life examples.
A8
(N = 1105)		Is “Ghostwriting” Acceptable?
This video describes the phenomenon of hiring ghostwriters among university students. It also introduces definitions and examples of ghostwriting and contract cheating.
SpiritualityA9
(N = 1069)		Spiritual Leadership Model:
The video introduces the Spiritual Leadership Model and explains each component with the support of daily examples. This model explains why spiritual development is the fundamental need of both leaders and followers.
Law-Abiding LeadershipA10
(N = 971)		Concepts of Law-abiding Leadership:
This video introduces the basic concepts and theories related to law-abiding leadership and socially responsible leadership.
A11
(N = 968)		The Importance of Law-abiding Leadership:
In this video, the importance of law-abiding leadership from the individual and organizational levels is discussed.
Cultural CompetenceA12
(N = 1083)		Definitions and Theories of Cultural Competence:
The video introduces definitions and models of cultural competence and encourages students to reflect on cross-cultural differences.
Effective CommunicationA13
(N = 1025)		REPAIR Strategies:
This video introduces the six steps in the REPAIR strategies for resolving interpersonal conflicts and invites student to share their opinions about the REPAIR strategies.
Team Building A14
(N = 1046)		Five Conflict Resolution Strategies:
This video introduces five conflict-handling modes in the Thomas-Kilmann Model of Conflict Management and discusses the effectiveness of these conflict-handling modes in different situations.
A15
(N = 1040)		Win-Win Strategies:
This video introduces the four steps to reach a win-win situation in the Conflict Resolution Model. In addition, the four typical win-win solutions are explained.
Table
Table 2. Description of the Nine Case-based Videos.
Table 2. Description of the Nine Case-based Videos.
Lecture TopicVideo Number Description
Self-Leadership C1
(N = 898)		Anger/Hatred Management Arising from Differences in Viewpoint:
This role-play video presents a conflict between two good friends over their lifestyle choices. Students analyze emotional management in the case and share similar experiences of their own.
Cognitive Competence C2
(N = 875)		Myths of MMR Vaccine:
The video presents two distinct views towards the MMR vaccine (a vaccine against measles, mumps and rubella). Students analyze the arguments in the case and propose ways to distinguish true information from misinformation.
Social-Emotional Competence C3
(N = 1007)		Story of a Family Issue:
This role-play video uses storytelling technique to present a conflict between a mother and a daughter over the choice of television program at home. Students reflect on the concepts of social awareness and relationship management and discuss ways to handle the situation in the case.
ResilienceC4
(N = 1073)		A Story of Resilience (Eric Ma, a life-fighter):
The video uses storytelling technique to describe the challenges faced by Eric Ma, who suffers from a rare disease that affects his normal functioning and appearance. Students identify the resilient qualities shown in the case and reflect on their own experiences of using coping strategies to handle changes and stressful events.
SpiritualityC5
(N = 1065)		Cases of Forgiveness:
In this role-play video, three students with different experiences of betrayal and forgiveness are interviewed. Students reflect on their own experiences of forgiveness.
Cultural CompetenceC6
(N = 1080)		How to Live Harmoniously with People from Different Cultural Background?
This case-based video is an authentic interview with an ethnic minority family in Hong Kong on their difficulties. Students discuss issues encountered by people from different cultural backgrounds in Hong Kong.
Effective CommunicationC7
(N = 1022)		Relationship Repair with Peers:
The video presents a case about resolving conflict with peers with reference to face-enhancing strategies and face-detracting strategies. Students share their own or others’ experiences of using face-enhancing strategies to resolve conflict.
Team Building C8
(N = 598)		Absence of Trust:
This video presents problems in a dysfunctional team where team members are hesitant to discuss with each another. Students identify problems in the team and propose solutions based on related concepts and theories.
C9
(N = 611)		Avoidance of Accountability:
This video presents problems in a dysfunctional team with a free rider. Students identify problems in the team and propose solutions based on related concepts and theories.
Table
Table 3. CFA findings of the 12-item SOES for the 15 Animated Videos.
Table 3. CFA findings of the 12-item SOES for the 15 Animated Videos.
ModelDescriptionχ2dfCFINNFISRMRRMSEA [90% CI]AVECRα
M1L2 Self-understanding (N = 924)352.10540.940.920.040.08 [0.07, 0.09]0.650.960.96
M2L3 Cognitive competence (N = 901)290.51540.940.930.030.07 [0.06, 0.08]0.690.960.96
M3L4 Social-emotional (N = 1039)301.66540.950.940.030.07 [0.06, 0.07]0.730.970.97
M4L4 Introduction to egocentrism (N = 1012)186.24540.980.970.020.05 [0.04, 0.06]0.760.970.98
M5L5 Resilience (N = 1092)245.38540.960.950.030.06 [0.05, 0.06]0.710.970.97
M6L5 Richardson’s model (N = 1076)170.82540.980.970.020.05 [0.04, 0.05]0.730.970.97
M7L6 Moral development (N = 1114)271.57540.970.960.020.06 [0.05, 0.07]0.770.980.98
M8L6 Is ghostwriting acceptable (N = 1105)118.20540.990.990.010.03 [0.03, 0.04]0.770.980.98
M9L7 Spiritual leadership model (N = 1069)155.61540.980.980.010.04 [0.03, 0.05]0.790.980.98
M10L8 Concept of law-abiding (N = 971)240.76540.970.960.020.06 [0.05, 0.07]0.860.990.99
M11L8 Importance of law-abiding (N = 968)198.27540.980.970.010.05 [0.05, 0.06]0.860.990.99
M12L9 Cultural competence (N = 1083)203.75540.970.970.020.05 [0.04, 0.06]0.810.980.98
M13L10 REPAIR strategies (N = 1025)156.60540.980.980.010.04 [0.04, 0.05]0.820.980.98
M14L11 Conflict resolution (N = 1046)229.47540.970.970.020.06 [0.05, 0.06]0.820.980.98
M15L11 Win-win strategies (N = 1040)162.94540.980.980.010.04 [0.04, 0.05]0.820.980.98
Note. α = Cronbach’s α (reliability). L = Lecture number.
Table
Table 4. CFA findings of the 14-item SOES for the Nine Case-Based Videos.
Table 4. CFA findings of the 14-item SOES for the Nine Case-Based Videos.
ModelDescriptionχ2dfCFINNFISRMRRMSEA [90% CI]AVECRα
M16L2 Anger management (N = 898)418.16770.930.920.040.07 [0.06, 0.08]0.660.960.97
M17L3 Myths of MMR vaccine (N = 875)403.79770.940.930.040.07 [0.06, 0.08]0.680.970.97
M18L4 Story of a family issue (N = 1007)313.69770.960.950.030.06 [0.05, 0.06]0.740.980.98
M19L5 Story of resilience (N = 1073)520.15770.930.920.040.07 [0.07, 0.08]0.700.970.97
M20L7 Case of forgiveness (N = 1065)318.86770.970.960.020.05 [0.05, 0.06]0.780.980.98
M21L9 Diverse cultural background (N = 1080)342.83770.960.950.020.06 [0.05, 0.06]0.800.980.98
M22L10 Relationship with peers (N = 1022)232.78770.980.970.020.04 [0.04, 0.05]0.810.980.98
M23L11 Absence of trust (N = 598)264.36770.950.940.020.06 [0.06, 0.07]0.820.980.99
M24L11 Avoidance of accountability (N = 611)258.28770.960.950.020.06 [0.05, 0.07]0.810.980.98
Notes. α = Cronbach’s α (reliability). As there are two case-based videos for Lecture 11, half of the students were required to watch and evaluate “Absence of trust”, while the other half were required to watch and evaluate “Avoidance of accountability”. L = lecture number.
Table
Table 5. Positive Responses Rates of Post-Lecture Video Evaluation for the 15 Animated Videos on Theories/Concepts.
Table 5. Positive Responses Rates of Post-Lecture Video Evaluation for the 15 Animated Videos on Theories/Concepts.
Questionnaire ItemA1
N = 924		A2
N = 901		A3
N = 1039		A4
N = 1012		A5
N = 1092		A6
N = 1076		A7
N = 1114		A8
N = 1105		A9
N = 1069		A10
N = 971		A11
N = 968		A12
N = 1083		A13
N = 1025		A14
N = 1046		A15
N = 1040
Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %
1The animated video is well designed. 95.9598.2096.6998.0297.9697.3998.1997.7396.8694.7894.6996.8297.1497.5897.87
2The animated video is relevant to the lecture topic.98.5898.5598.4598.9198.6298.8898.2098.0197.8394.6295.7497.9797.9597.8897.78
3The animated video is easy to understand.97.6098.4398.1598.5198.2597.1197.7498.0996.0695.4395.1296.9497.7598.1898.26
4The animated video is interesting.90.8194.1993.0295.4395.4895.2695.7896.6594.0092.1391.4194.8996.0995.5995.75
5The teacher used the animated video effectively. 96.6197.6597.8698.0198.2698.6097.7498.2897.4695.0395.1297.5997.2597.7097.87
6The animated video has enabled me to understand related concepts.97.2898.5598.4597.7298.7198.2398.5697.8296.4495.0394.9198.0597.5698.1798.08
7The animated video has deepened my understanding of the topic.97.7198.2097.6897.8298.4398.4298.0098.2797.0994.4294.8297.5997.4697.1298.27
8The animated video has enhanced my interests in the topic.90.6294.2094.1095.3394.8695.2495.7596.6494.2890.8891.3994.9995.7994.2696.62
9The animated video has helped me to reflect on the topic.95.4196.3196.9996.9296.7797.7597.1197.4596.7092.4693.3496.2997.4597.4097.69
10I prefer more such animated video in the teaching and learning of this subject.95.3995.7696.2396.8297.1396.8297.5797.4696.9094.1194.7296.4895.9996.8297.49
11The animated video can benefit my development.95.7196.8596.5297.8097.5097.7697.8496.9996.6193.7093.5897.1397.9497.4197.40
12Overall speaking, I am satisfied with the animated video. 97.4898.1998.0698.3198.5298.2298.2998.0996.9194.4294.4997.6797.6597.6997.68
Notes. (items 1 to 12) All items were rated on a 6-point Likert scale with 1 = Strongly Disagree, 2 = Disagree, 3 = Slightly Disagree, 4 = Slightly Agree, 5 = Agree, 6 = Strongly Agree.
Table
Table 6. Positive Responses Rates of Post-Lecture Video Evaluation for the Nine Case-based Videos (Application and Discussion).
Table 6. Positive Responses Rates of Post-Lecture Video Evaluation for the Nine Case-based Videos (Application and Discussion).
Questionnaire ItemC1 (N = 898)C2 (N = 875)C3 (N = 1007)C4 (N = 1073)C5 (N = 1065)C6 (N = 1080)C7 (N = 1022)C8 (N = 598)C9 (N = 611)
Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %
1The video-based activity is well designed.95.0997.8296.4297.7596.2397.6897.1497.3198.04
2The video-based activity is relevant to the topic.98.5598.9798.3198.6097.7498.2498.0398.3298.69
3The video is easy to understand.97.3298.6297.4198.3197.1797.5998.2398.4998.19
4The content of the video is interesting.92.4896.2194.2996.4495.3196.4695.2895.6497.02
5The content of the video is stimulating.96.2097.9396.4197.4896.0597.0296.7697.4898.02
6The video-based activity has increased my interaction with the teacher.86.3290.3991.6291.5093.5093.7694.2894.4796.39
7The video-based activity has increased my interaction with classmates.82.6089.4392.1289.7991.8993.1193.7193.6394.74
8The teacher has used the video effectively.96.9898.0597.9998.0397.5598.0497.4497.9998.52
9 I have been very engaged in the video-based activity.92.6194.1094.4995.9795.1996.5695.3996.6597.87
10The video-based activity has deepened my understanding of the topic.96.8798.2897.6098.3297.2697.4997.4498.3298.03
11The video-based activity has enabled me to apply the theories in real life cases.93.6096.8996.4996.0696.4197.6796.3796.9897.54
12The video-based activity has enhanced my interests in the topic.92.8495.5295.3096.2695.5695.7396.1895.6497.21
13The video has helped me reflect on the subject matters.96.0797.1197.2098.0496.6197.5997.3597.9997.54
14I prefer more such video-based activities in the teaching and learning of this subject.94.0897.1396.9097.7596.8996.8396.7696.9697.21
Notes. (items 1 to 14) All items were rated on a 6-point Likert scale with 1 = Strongly Disagree, 2 = Disagree, 3 = Slightly Disagree, 4 = Slightly Agree, 5 = Agree, 6 = Strongly Agree.
Table
Table 7. Positive Responses Rates of Post-Lecture Video Evaluation for Lectures 2 to 11 (Two Items Evaluating the Effectiveness of the Lecture).
Table 7. Positive Responses Rates of Post-Lecture Video Evaluation for Lectures 2 to 11 (Two Items Evaluating the Effectiveness of the Lecture).
Lecture TopicSelf-LeadershipCognitive CompetenceSocio-Emotional CompetenceResilienceMoral CompetenceSpiritualityLaw-Abiding LeadershipCultural CompetenceEffective CommunicationTeam Building
Video NumberA1 and C1 (N = 892)A2 and C2 (N = 867)A3, A4 and C3 (N = 999)A5, A6 and C4 (N = 1072)A7 and A8 (N = 1105)A9 and C5 (N = 1062)A10 and A11 (N = 964)A12 and C6 (N = 1076)A13 and C7 (N = 1019)A14, A15, C8 and C9 (N = 1040)
Questionnaire ItemPositive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %Positive Response Rate %
1Overall speaking, the lecture has enriched my development.96.6197.4498.1898.7897.6397.6292.7797.5698.1197.39
2On the whole, I am very satisfied with this lecture.96.2698.1597.8897.6597.8297.2692.7197.5898.0497.68
Notes. (items 1 to 2) All items were rated on a 6-point Likert scale with 1 = Strongly Disagree, 2 = Disagree, 3 = Slightly Disagree, 4 = Slightly Agree, 5 = Agree, 6 = Strongly Agree.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Li, X.; Shek, D.T.L.; Wong, T.; Yu, L. Subjective Outcome Evaluation of Instructional Videos in Leadership Education. Int. J. Environ. Res. Public Health 2023, 20, 367. https://doi.org/10.3390/ijerph20010367

AMA Style

Li X, Shek DTL, Wong T, Yu L. Subjective Outcome Evaluation of Instructional Videos in Leadership Education. International Journal of Environmental Research and Public Health. 2023; 20(1):367. https://doi.org/10.3390/ijerph20010367

Chicago/Turabian Style

Li, Xiang, Daniel T. L. Shek, Tingyin Wong, and Lu Yu. 2023. "Subjective Outcome Evaluation of Instructional Videos in Leadership Education" International Journal of Environmental Research and Public Health 20, no. 1: 367. https://doi.org/10.3390/ijerph20010367

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop