Web-Based Technology: Trials and Tribulations of Using Online Quizzes in an EFL Course ^†

Abstract

One of the tools to use in the ever-increasing implementation of the flipped classroom is the online quiz to monitor students’ formative learning. However, in useful and convenient administering online quizzes, how students engage with them creates a paradox. This case study discusses four main components: the instructor’s reasoning for using online quizzes, two different implementation approaches, how the students behaved whilst completing them, and reflecting on how closely the learning results compared to the original intentions. As a result, ways to keep the students on the intended learning path are proposed to limit divergence.

1. Introduction

The flipped classroom has become one of the emerging instructional pedological trends since Bergmann and Sams [1] wrote about how they tried to help their students. Although the initial premise was to provide the information taught in class to students who could not attend, Bergmann and Sams discovered that this information was also consumed by students who were attending class. As students were previewing the class material before starting class, the time spent in class could be spent on more useful activities, such as experiments and projects. Furthermore, teachers can provide more personalized one-on-one instruction.

Whilst other researchers examined the flipped classroom approach, comparisons were drawn to Bloom’s Taxonomy, in particular to his cognitive thinking [2,3,4]. Bloom’s Taxonomy, in the revised version, refers to six different levels of cognitive learning, comprising remembering, understanding, applying, analyzing, evaluating, and creating [5,6]. Administering online quizzes typically requires the student to use lower-order thinking, which applies to learning new information before class. In higher-order thinking skills, analyzing, evaluating, and creating are used with critical thinking activities during class. The benefits of this teaching approach are twofold. The first is that while students are learning the basic information, it is more fruitful for them to do so at their own pace and using their own preferred learning style rather than being tied to the teacher’s style and pace [7,8]. The second is that during class time, the activities assigned allow students to explore the learning content more deeply and autonomously [9]. Thus, teachers are allowed to focus on those students who need more personalized instruction.

2. Literature Review

One of the key points to successfully implementing a flipped classroom is ensuring that all students have a similar understanding of the preliminary information at the start of class. One way to achieve this is to administer online quizzes, and there have generally been many benefits from this approach [10,11], including the convenience and flexibility when compared to paper quizzes, as well as the ability to provides prompts and immediate feedback to the students. Theoretically, implementing a standard quiz for the topic at hand to all students aims to ensure that a consistent minimum level of knowledge is understood before moving on to the more cognitively challenging tasks to be assigned in class.

Having established that online quizzes are generally an important checkpoint for students to move from lower-order thinking tasks to higher-order, the type of questions posted in online quizzes become essential to ensure that the correct understanding is measured. Clay [12] discussed the many considerations of writing and using online quizzes, including the types of questions and the considerations needed for them. The most common categories of questions used are single and multiple choice, true or false, matching items, fill-in-the-blank, and essay style questions that include short and long answers. With all types of online questions, a great deal of thought is needed for the language used to ensure that questions are not ambiguous and they measure the intended level of knowledge. With the majority of question types, except for long essay answers, each of the question types generally requires the students to use their lower-order thinking skills of remembering, understanding, and applying.

3. Implementation and Reflection of Online Quizzes

For this case study, the researcher’s role as a lecturer in the Department of Applied English reflects on the same group of freshmen students taking an oral training course over two semesters. A set course textbook is used within eight units, each having a broad unit question, two listening tracks that discuss issues surrounding the unit question, as well numerous practice activities probing deeper meanings. These practice activities from the book are then used as online quiz questions intended to show that students have the prerequisite understanding of the topic before proceeding with critical thinking and expression activities. To address some of the issues that Clay discussed [12], the reliability and content must be of a high standard. The other attributes of online quizzes that need consideration are rules, including time limits, the ability to retake the quiz, showing incorrectly answered questions, as well as revealing correct answers.

This case study reflects on the freshmen’s engagement and observed actions over two semesters for an oral training course. Each semester, the researcher used a different set of quiz-taking rules. For the autumn semester, students were immediately shown a score as well as the incorrect answers. Then, the students could go back into the quiz, revise the incorrect answers and resubmit. In the spring semester, this condition was changed, and students would only be allowed one attempt. Then, after the class finished, the score was revealed along with all answers.

4. Observations of Students’ Behavior and Engagement

The following section discusses several issues surrounding each approach for online quizzes. For both semesters, it was observed that very few, if any students prepared by previewing the instructed content before class, despite instructions being provided numerous times including at the start of the semester, as well as closing the previous week’s class.

4.1. Autumn Semester

The strategy for implementing this set of online quiz rules was to allow students to see the areas from the day’s learning activities from the course textbook that they did not fully understand, allow them to search for and then revise their answers, then resubmit and hopefully see their improvement. The number of retries was unlimited and the quiz would close at the end of the class day. However, after a few weeks, the students quickly adopted their quiz-taking strategies. Rather than reviewing the book’s information, the students would immediately attempt the quiz by guessing all of the answers. For the single-answer multiple-choice questions, the students would see which answers were incorrect. Then, they went back into the quiz and randomly guessed another answer before submitting again. For the more-than-one-answer multiple-choice questions, it was observed that several students selected all choices. Thus, the online automatic marking function was deceived because all the correct answers were selected whilst not checking that all incorrect answers were not selected. Another approach that students adopted was for one of the smarter students to correctly guess all answers and then share the answers via an instant message app with the class. By the end of the semester, the students, unfortunately, appeared to have completed the quizzes by brute force rather than engaging and learning from the course textbook’s content.

4.2. Spring Semester

The strategy for implementing this set of online quizzes was aimed to address the issues from the autumn semester while hoping to achieve the learning objectives. In the autumn semester, the first problem mentioned was that students were not engaging with the course textbook’s content before attempting the online quiz. The class instruction changed to allocate time for the students to listen to the audio, write their answers in their book, and randomly call students to discuss their answers in front of the class, then the quiz was opened up with only one attempt allowed. The consequences of this approach were that students had to follow the instructor’s pace and those students who were not called to speak could passively wait for the answers. From the autumn semester, the second problem mentioned regarding more-than-one answer multiple choice questions was manually reviewed, and if students selected all answers, then their respective scores would be altered to zero and feedback was provided not to do this again. The consequences of this were the loss of the auto-marking benefits, as well as more review work needed. From the autumn semester, the third problem mentioned about students sharing answers appeared. This was addressed by creating a class discussion about the content and online quiz questions before students were instructed to take them. Thus, it was hoped that it would negate the need for students to share answers via instant messaging apps.

4.3. Reflection on Setting the Rules around Online Quizzes

Over these two semesters, two different approaches were taken to administering online quizzes, and as a result, it affected the underlying teaching approach. It is worth noting that very few students, if any, did not preview before class. Hence, strictly adhering to the flipped classroom approach had to be adapted to be more like a blended learning approach. Furthermore, the intended benefits of using online quizzes to foster student-centered learning seemed to be lost, as well as the potential for deeper learning achievements.

One of the main goals of using online quizzes was to allow timely and specific feedback to each student about their understanding of the day’s topic. This served two goals. Firstly, an opportunity was provided for students to delve deeper into the course textbook’s content and seek the answers to the points where they need improvement. The successful completion of the online quiz was intended to show that the student understood the prerequisite information and would be able to build on this to develop his/her own well-thought-out opinions. Secondly, the successful completion of the online quizzes was to encourage to move onto deeper meaning activities that use higher-order cognitive thinking, thus building up a conversation in a more meaningful and intelligent manner, using relevant points and critical thinking.

However, the students’ alternative approach to taking online quizzes resulted that the intended path to the learning objectives was not followed. The original instructional approach intended to be beneficial became a faint idealism. The first concern with the alternative approach was that more time was needed at the start of class to spend on discussing the book’s activities before the online quizzes could be completed. This resulted in less time that could be spent on critical thinking activities, which was the original intended use of class time. The second concern with the alternative approach was the benefits of student-centered learning, which included that students could use their preferred learning style, go at their own pace, and choose the order they complete the book’s activities that have been lost. Therefore, the best-laid plans fell apart by underestimating the students’ desire not to follow them.

5. Strategies to Setup Online Quizzes

Upon reflection on the students’ online quiz-taking strategies, a more comprehensive philosophy needs to be thought out. One strategy to make online quizzes more effective is to impose time limits [13,14]. Clay [12] offered some suggestions based on the type of questions being asked. It is recommended that for question types that are true or false questions, students must be allowed 30 s for each question. For multiple-choice questions, 60 s were allowed for each. For short answer questions, students must be allowed 120 s for each. Finally, students must be provided between five to ten minutes to review the work. Ideally, these recommendations need to be adjusted based on the complexity and knowledge level of the students.

The second strategy suggested is to randomly shuffle the answers within each question [12,14]. Many online quiz-setting applications including Google Forms and Microsoft Teams have this option to select. To take this one step further, several LMS applications including TronClass and Moodle can create a unique quiz for each student by randomly drawing questions from an established question bank [14,15]. Ideally with no two online quizzes the same, it hopefully makes it more difficult for students to share answers and forces them to seek answers from their knowledge and learning.

The third strategy is to foster a more positive attitude in students about taking the quizzes. Faculty Focus [16] is a collection of short articles on strategies to make online quizzes more enjoyable, engaging, and beneficial. Some of the strategies include allowing collaboration between students while completing the quiz, using different formats of questions, renaming quizzes to extra-credit exercises, and changing the times when students take the online quiz, just to name a few. Therefore, possibly, a more important issue than the pedagogical reasoning to implement online quizzes is the packaging and presentation of them to encourage students to use them positively.

6. Conclusions

Although the literature has found many positive results of implementing a flipped classroom pedagogy [9,17,18] and using online quizzes as one of its tools [19], this is the tip of the iceberg for an instructor’s implementation plan. Managing students’ attitudes towards online quizzes, creating a set of rules to minimize students’ divergence from the learning objectives, and ensuring that the online quizzes are beneficial, must to be considered to achieve effective results. The overall outcome requires instructors to be watchful of students’ behavior and be vigilant with a set of strategies to keep students on the desired learning path.