Students and Teachers Using Mentimeter: Technological Innovation to Face the Challenges of the COVID-19 Pandemic and Post-Pandemic in Higher Education

Abstract

The COVID-19 pandemic has prompted higher university lecturers to develop their digital skills in order to adapt to online teaching. A group of university teachers decided to evaluate the educational uses of Mentimeter to promote student participation and active learning. A questionnaire was answered by 400 students and 12 participating academics. These 12 academic respondents also participated in a focus group after experiencing this software during an academic course. Qualitative and quantitative data was collected and analyzed to conclude that this software not only facilitated student participation during the pandemic (both face-to-face and online) in synchronous and asynchronous ways but also improved attention, engagement, collaborative learning and interaction. Immediate feedback made it possible for teachers to monitor the students’ learning processes and to adjust the content and pace accordingly. Students and educators highlighted the inclusive potential of this tool, as it allows participation from a diverse audience with different backgrounds and capacities, ensuring inclusive and equitable education for all. Some opportunities for improvement were also identified, namely more functions to make the software more attractive and adapt it to different educational objectives.

1. Introduction

Technological innovation has been a key element to promote methodologies that facilitate the active involvement of students [1]. The use of information and communication technologies (ICT) has been shown to “contribute to improving the understanding of concepts related to a specific subject, amplify the possible teaching strategies and contribute to students acquiring a dimension of greater responsibility towards their own teaching and learning process” [2].

Research has shown that strategies that promote student participation increase both class attendance and the acquisition of competences [3]. The COVID-19 pandemic has implied radical changes in teaching, including in higher education. Two of them are highlighted here [4]: online learning, sometimes in combination with face-to-face teaching, has assumed a leading role in its different modalities and, consequently, teachers have had to quickly incorporate and develop their digital skills.

In this context, a group of university lecturers decided to innovate through the use of online software (Software as a Service, or SaaS) during the 2020–2021 academic year, to promote and facilitate participation, collaborative learning and interaction between all the individuals involved in the learning process. This approach focuses on the learning process, since it allows joint work between educators and students. To do this, we initially explored a variety of tools, including Kahoot, Mentimeter, Socrative, Wooclap and Quizizz, that enable students to be actively involved during classes. This software also facilitates gamification strategies, offering significant motivation for the students and a great opportunity for innovation to enhance teaching and learning processes [5,6].

After conducting a bibliographic review [7,8,9,10,11,12,13,14,15], we concluded that the use of Mentimeter in educational processes resulted in students significantly increasing their attention and participation, while promoting inclusion and commitment to the learning process. Gokbulut [16] investigated the use of two software tools in education: Kahoot and Mentimeter. He observed that both of them had a strongly positive effect on online teaching: students enjoyed learning and, specifically with Mentimeter, they were observed to participate more actively in classroom activities. Gokbulut concluded that Mentimeter helped teachers to develop digital skills and become aware of the importance of using technologies in the classroom.

We, hence, decided to use and evaluate Mentimeter in our courses in order to identify its advantages and weaknesses in teaching and learning. Mentimeter is online software that allows educators to create interactive presentations and engage their students in real time. We describe how it is used in the Materials and Methods section. After testing other previously mentioned possibilities (Kahoot, Mentimeter, Socrative, Wooclap and Quizizz), the educators participating in this study agreed that Mentimeter was more dynamic.

It offers a greater degree of flexibility and has a wider range of functions, allowing teachers to explore more types of questions and interactions. Mentimeter also presents an attractive and easy-to-use interface, both for students and teachers, through any mobile device with access to the internet (mobile, tablet or computer) and without the need to install any type of software or specific program [17].

Previous research [7,8,9,10,11,12,13,14,15] has predominantly focused on students’ opinions of using Mentimeter. While keeping students at the center of the teaching and learning processes, we also wanted to know the uses, possibilities and limitations that teachers report when applying this tool. Through the use of quantitative and qualitative methods, as well as investigating the use of Mentimeter in face-to-face and synchronous contexts, we have examined its suitability for online teaching situations and its asynchronous use.

Due to the COVID-19 pandemic, the implementation of blended and online modes of teaching in higher education has made it increasingly urgent to integrate ICTs as a resource that, given the context, is no longer complementary, but fundamental for teaching [4]. The challenges posed by the pandemic have stimulated new knowledge and competences in terms of the use of audience response systems (ARS) and gamification tools, which will also be useful in a post-pandemic process and reshape the way we teach and learn at universities. In this paper, we show that the use of Mentimeter not only increases the participation, engagement, gamification, concentration and attention of students in face-to-face classes but also enables their participation and improves inclusivity in online settings.

2. Materials and Methods

The exploratory research evaluating how the use of Mentimeter can improve teaching and learning experiences took place during the academic year of 2020–2021 (September 2020 to July 2021, according to the academic calendar in Spain). Eleven higher education teachers from Complutense University and one from Rey Juan Carlos University in Madrid (Spain) (eight female and four male) took part in the project and used the software in other universities and settings where they lectured. The group had an interdisciplinary background covering various fields: psychology, pedagogy, linguistics, social anthropology, social work, medicine, computer engineering, mathematics and biology. In the following sections, we explain how these educators used Mentimeter and the data collection methods.

2.1. Experiencing Mentimeter

Four of the twelve participating teachers had been previously unaware of the software (33%), three had known of it but never used it (25%) and the remaining five had used it before: one (9%) with a paid license and four (33%) using its free version. All in all, the proportion of teachers who had never used the application (58%) was higher than those who had (42%). Most of them had previously used software during classes, mainly for purposes of presentation (Powerpoint), assessment (Google Forms) and review and games (Kahoot), while not taking full advantage of other more interactive platforms, such as Quizizz or Socrative.

The teacher participants did not start from the same position in terms of knowledge of theories on gamification and/or educational use of online digital tools in general and Mentimeter in particular. This diversity of situations permitted an active exchange of ideas and advice on the use of Mentimeter, shaping the project team as a learning community. The first step of the study was holding a seminar where all teacher participants learned how to use Mentimeter, and those who had already used it reported their experiences.

Each teacher then prepared the questions or content slides that they wanted to use for any specific session. During the classes, they shared the presentation using a projector in the case of face-to-face teaching, on screen for synchronous online teaching and embedded in a virtual learning management system (LMS), such as Moodle in the case of asynchronous online teaching.

To use the Mentimeter instant online audience response system, the audience must go to the page www.menti.com (accessed on 30 August 2021) and insert a unique numeric code provided by the application to access the presentation or scan a QR code (quick response code) provided by the presenter. Both software and data are hosted in the cloud, and thus there is no need to download or install anything. The audience responds to the questions, and the results are displayed on the screen in real time, while the number of people who have responded appears in the lower right corner. The program allows educators to download the results in pdf or Excel format, which makes it possible to review and work on them at the end of the lecture. This immediate feedback offers valuable input to measure comprehension and adapt contents in the current session or in subsequent classes or to improve teaching in future academic years.

The group of twelve teacher participants produced 160 Mentimeter presentations in different teaching activities, with an average of 13 presentations per professional, ranging from a minimum of 2 to a maximum of 35. Considering that each term is approximately 15 weeks long, each teacher used the tool every two or three weeks. However, and for various reasons, some teachers only used the application during one term, increasing the real frequency of use. Bearing in mind that some participants had never previously used Mentimeter, these numbers indicate their willingness to use it in their teaching.

The teacher participants used Mentimeter in both regular higher education courses (in undergraduate, master’s and doctorate studies) and other learning settings, such as the “university for the elderly” (a series of courses specially designed for seniors), cross-curricular subjects and free-choice credits, presentations at workshops and conferences, teacher training in university contexts and many other types of training beyond the scope of higher education, as well as in secondary education.

Each presentation records the number of times that each audience member participates. The twelve teacher participants collected 4033 “voices” (the term Mentimeter uses to describe contributions to a presentation). At least 1350 students had contact with Mentimeter due to this experience. We estimate an average of three contributions per student, since, although some teachers used Mentimeter several times with the same group, others used it only once.

2.2. Data Collection

At the end of the final term (from the end of May to the beginning of July 2021), after all the participants had had the chance to try the application, three tools were used to collect data about the use of Mentimeter in higher education:

• All twelve teachers were invited to a focus group with the objective of evaluating their experience and describing the potential of Mentimeter for teaching and learning, the strategies they developed, the difficulties they encountered and how they had overcome them, together with tentative suggestions to optimize its use during online and face-to-face classes. The discussion was guided by a script created by the team, recorded on video and subsequently analyzed using a qualitative approach that investigates the understanding perceptions and experiences of educators using Mentimeter during their courses.
• A questionnaire for educators was collectively designed by the entire team and uploaded to Mentimeter. It consisted of thirteen questions: eleven were closed-ended and were statistically analyzed, while the last two were open-ended and were analyzed more descriptively. The questionnaire was open for four weeks, and all 12 participants answered it. All participants had access to the results as soon as the questionnaire was closed. The first open-ended question received 23 answers commenting on Mentimeter’s benefits for teaching. Similarly, the last question asked for free ideas and received 13 replies, amounting to more than one per person. This fact speaks for itself, reflecting the teachers’ high levels of engagement in the project and their satisfaction with Mentimeter after their experience.
• A questionnaire was also designed for students and once again adapted and uploaded to Mentimeter. It consisted of six quantitative questions and three open-ended questions. The research team opted for a very short and easy questionnaire for students to allow them to provide quick answers. There were no questions about gender or field of study, and cross-tabulation was not possible owing to the use of Mentimeter. Therefore, no statistical analysis will be presented here, but rather the descriptive outcome provided by this software (not intended for research). The total number of respondents comprised 400 students. More than half of the sample were undergraduates (65%), while 11% were graduate students (MA, MSc and PhD), 7% belonged to training courses for university staff, 5% were attending cross-curricular subjects or free-choice credits, and 5% were secondary school students (in their fourth year of Spanish secondary education). The remaining 7% came from other training environments. Most of the students were members of Complutense University (59%), Rey Juan Carlos University (10%), Polytechnic University of Madrid (9%) or Autonomous University of Madrid (6%), with 5% studying at other higher education institutions (International University of La Rioja, Nebrija University and Valencia International University), 5.5% at José Saramago Secondary School (Arganda del Rey, Madrid) and the remaining 5.5% being participants in staff training. A total of 629 comments were collected in the last three open-ended questions for students about their positive impressions of using Mentimeter in class, opportunities for improvement and other open comments.

Both questionnaires were designed in Mentimeter, and a link for each one was sent to all the teachers participating in the project and distributed among all the audiences that had attended presentations where the software was used within the framework of this study. Both questionnaires were open for four weeks. This asynchronous use of Mentimeter facilitates data collection without the presence of the researcher [18] and proves its capacity to collect both qualitative and quantitative descriptive data not only with a teaching objective, but also for research, albeit subject to significant limitations in terms of statistical analysis. For the research team, using Mentimeter made sense in order to also test its potential use in research.

3. Results and Discussion

As we analyzed both quantitative and qualitative data, the results are presented together with the discussion, taking into account the input from both teachers and students. We grouped the information on Mentimeter and impressions of its use, possibilities and limitations for teaching into categories and subcategories to facilitate an in-depth exploration of the data. The results are presented accordingly.

Teachers highly valued the use of Mentimeter for pedagogical objectives, giving it an average score of 9.5/10 on a Likert scale, strongly agreeing with the sentence “Mentimeter is a good tool for teaching and learning” (N = 12). They were also asked to rank what they considered to be the main reasons for using Mentimeter in class. As shown in Figure 1, teachers coincided on this application enhancing student participation, especially in the case of students with social interaction difficulties. Second and third were didactic purposes: keeping the students’ attention by changing the lesson’s pace and making the class enjoyable.

The teachers then referred to a variety of feedback goals, such as getting to know the students’ previous knowledge or how well they understood explanations in class. This set of answers was followed by another set referring to the educator’s desire to know their audience better, asking for opinions or background with the intention of creating a warm, welcoming environment. The final set related to evaluation and testing questions for final exams. An extra use of evaluating the interpretation of graphs and data was mentioned and is meaningful for specific disciplines, such as mathematics.

The same question was posed to students (Figure 2). Two aspects received the highest rankings: students felt that Mentimeter allowed shy audience members to participate and that it enabled everyone to participate. This indicates that students also predominantly value the participatory nature of this tool, facilitating the involvement of audience members who do not usually participate due to reasons that will be described later. Engaging and paying attention were ranked in third position, together with the fun side of Mentimeter relating to gamification targets (ranked fifth). Students also identified Mentimeter as a tool for self-assessment, explaining that Mentimeter let them check if they were understanding lessons well. Finally, student participants also highlighted how inclusive Mentimeter can be, since it is an easy-to-use way to interact with classmates, even if there are linguistic or physical barriers.

Students were asked to rank their opinion of their teachers’ use of Mentimeter during a 2020–2021 course on a Likert scale (from 0 to 10). As we can see in Figure 3 and Figure 4, students were very positive in terms of their educators using this software with an educational purpose (8.7). They especially valued its simplicity and easiness of use (9.1) and, again, its encouragement of active participation, both for themselves and others (8.5 and 8.9). They highlighted the opportunity to hear classmates’ opinions (8.1) and to have the feeling that their opinion also counted (8.2). Mentimeter was considered quite fun (8.3) and a good way of practicing questions for the final exam (7.7). All in all, most students reported a very positive feeling that using Mentimeter improved learning (7.9).

All these elements will be examined in more detail below, integrating the answers to the open-ended questions and the impressions teachers shared during focus groups. In the following section, we will examine how the different participating educators used Mentimeter in their courses. We will then consider different items that have proven relevant for the use of this software for educational purposes.

3.1. The Use of Mentimeter

This project looked into the development of relevant teaching–learning processes, such as participation, engagement, maintaining students’ attention and improving inclusion. This is truly necessary in the pandemic context, which has forced students to attend all kinds of lectures in formats other than the traditional face-to-face approach. For instance, students attended online lectures or were even provided with asynchronous e-learning materials only in some cases.

The educators involved in this study implicitly assume constructivist approaches to learning based on the theories of Piaget and Vygotsky, who stated that people construct new knowledge from experiences. These approaches require students and teachers to overcome the traditional passive roles assigned to learners, committing to active and immersive student participation. In this framework, educators assume a mediating role in an interactive learning scenario to propose new ways of learning and teaching, in line with Alexander’s [19] “dialogic teaching” or “dialogic conversation” between students and teachers.

Mentimeter is highly suitable for a teaching approach based on the idea of multiple intelligences. The range of questioning environments makes it possible to address a variety of student learning styles, stimulating concentration and interest and making classes more enjoyable. Participation with Mentimeter is more sophisticated, since it is not limited to a simple question “thrown out” in the classroom, but the result of a reflection on the most appropriate type of question and format for each learning objective: quantitative (multiple choice, ranking, scales and quizzes) or qualitative (word cloud and text open-ended questions).

Teacher participants reported a preference for question types that facilitate further and easier participation (see Figure 5): word clouds, multiple choice, rankings, open-ended questions, scales and quiz competitions, in that order. It is noteworthy that teachers chose participatory questions that allowed them to get the necessary feedback from their audience, thus, better meeting their needs. However, teachers do not seem to have favored the ludic aspect by resorting more frequently to quiz competitions, which could enhance gamification for teaching.

Content slides, which do not foster participation, ranked seventh out of nine. As stated, this group of educators preferred other software for expository presentations, which they used simultaneously with Mentimeter. The last two items were questions and answers, which were commonly used as a way of reviewing and advanced questions (i.e., 100 points or 2 × 2 grids), which likely require a higher degree of expertise for more frequent use by teachers.

Mentimeter also allowed lecturers to prepare and carry out entire classes through the platform, since, in addition to question slides, it offers other content presentation slides for the posting of photos or images, videos, documents, citations, paragraphs, bullet points or numbers. With each of these types of content presented through Mentimeter, students can interact if they wish through their mobile device, with an Instagram-style heart, a thumbs-up or thumbs-down in the style of Facebook’s “like” or “dislike” function, or a question mark if they find the presentation unclear.

These are all icons, language, logics and environments that are especially familiar to younger students. The presenter can also enable an option to ask questions anonymously through the mobile device during the presentation, using the Q&A tool. The presenter controls how, when and what questions they want to integrate into their discourse thread and they always retain the power to manage the pace of the class. However, only a few patterns of content slides are available, making it very difficult to adapt the entire content of a class to this software. This proved frustrating for the teachers that attempted to do so. Moreover, it is not easy (and sometimes not possible) to upload slides or presentations from other platforms.

The teacher participants mostly made short presentations consisting of an average of 1–5 slides (seven participants) or 6–10 slides (five participants) in which they asked questions. The content of the class itself was normally presented using other instruments, such as the now traditional PowerPoint, which was minimized on the screen to insert the Mentimeter questions at the right time, whether at the start, middle, or end of each class or tutoring session.

In fact, one teacher participant used Mentimeter to obtain feedback from students during online group tutorials, allowing anonymous questions or asking questions to ascertain the knowledge acquired by the small group. In any case, transferring all presentation content to the Mentimeter format implies an unacceptable level of effort, time and work for most teacher participants. However, participants who did transfer all of their content noted that it represents a new and different way of receiving constant, real-time and non-disruptive feedback from the students on each slide that is shown.

This is one of the most significant innovations that this technology brings to the teaching process, since it requires time and reflection to systematize participation, while providing the advantage of its inclusion in a planned and regular way. For students, it requires an active attitude and continuous attention in the learning process to be able to become involved. It also promotes the idea that technologies (mobile, tablet or computer, as well as SaaS) can be allies in both face-to-face and online teaching.

3.2. Student Participation

In Figure 1, Figure 2, Figure 3 and Figure 4, we observed that allowing and encouraging the participation of all students was the main reason for taking advantage of the different features of Mentimeter for both teachers and students. Teachers were aware of the need to use a participatory methodology for the acquisition and evaluation of knowledge, the creation of knowledge in a participatory way and the generation of connections and links in learning communities.

Using Mentimeter promotes participation by all students, whether teaching is face-to-face or online. This is due to its mass nature (it can support a large number of responses at the same time) and because its anonymity facilitates the participation of students who (for various reasons) do not usually raise their hands or interact with their voices. Students have also highly valued this innovation, since it promoted a different way of being able to participate actively, in many cases for the first time, during classes.

Students emphasized the democratizing potential of Mentimeter insofar as it allows all students to participate anonymously on equal terms, without excluding those who usually feel more insecure about participating. In their own words, “all students are encouraged to participate anonymously and all opinions are valued equally without value judgements”. Mentimeter is therefore “more accessible to shy students” who can participate and “generate debate on issues that may not be easily expressed orally”, promoting diversity in the class.

Students pointed out a fundamental and innovative aspect of using Mentimeter rather than other traditional, non-technological methods of student participation: it facilitates and improves feedback processes that have been shown to be fundamental for learning [20]. Feedback through Mentimeter is quick and immediate.

First, everyone receives opinions and responses from other students, and then the presenter reacts based on the responses of the attendees as a whole and not merely the small percentage of students who usually participate. In particularly large groups, as it is often the case in the first years of university degrees in Spain, it is especially difficult for teachers to take into account the concerns of the entire class and to adapt explanations and clarifications to the real needs of all students in the absence of ICTs, such as Mentimeter.

A variety of studies have demonstrated the positive impact of participation [3], gamification [6,21,22], feedback processes [20,23] and student motivation [12] in teaching–learning processes. The comments from students highlighted the effect that greater participation has on improving their learning: “Being anonymous allows you to put in what you think and you don’t have that pressure of failing, which means you can learn more”; “It greatly increases activity in class, students who normally don’t express themselves are given a voice, and we often come across interesting new ideas that we might not have thought of in a ‘standard’ class without these tools”; “I find it very useful and empowering to see that the teacher reacts to the answers and so, although it is sometimes anonymous, you do feel that you have somehow contributed in class. This is essential: to take a second to look at the answers and evaluate”; “It helps you understand the content better and internalize it in the class itself, so it is easier to study afterwards”; “It facilitates learning and concentration”.

Educators highly valued Mentimeter generating a relaxed atmosphere in the classroom, which cut through routines and encouraged participation: the fact that the answers were sent anonymously fostered a sense of freedom to express opinions without fear of censorship by the other participants, as well as providing a friendly framework for participation by students that might experience inhibition and anxiety in anticipation of judgment from their teacher and classmates. As answers are not associated with a particular individual, it is easier to ask questions or share opinions that might be controversial or betray a lack of understanding about previously taught content. This facilitates greater group interaction without judgment. The anonymity provided by Mentimeter, thus, creates a friendly and stimulating learning environment for both students and teachers.

The teacher participants considered anonymity to be good for promoting participation and assessing student needs and progress: “With anonymity, they lose the fear of giving a wrong answer, which allows you to better know the students’ knowledge”. But while anonymity is an advantage for shy students, it can be a disadvantage for teachers, since they do not know which students are participating or contributing with their comments or answers. In any case, some teachers expressed the view that Mentimeter “allows you to bring out things that would not have come out in a conventional class”. If students feel comfortable that they can say what worries them and ask questions without fear, this is positive for teaching and learning in and of itself.

3.3. Attention and Engagement

Keeping students’ attention during a class is one of the fundamental challenges identified by teachers at institutions including universities. Lack of attention is commonly cited as one of the key learning problems in both online and face-to-face teaching [2]. In addition to encouraging participation, a large majority of the students that took part in this study reported an improvement in their own attention and concentration during class thanks to Mentimeter.

Several participants commented on its originality and playful potential, including through quizzes that facilitated real-time gameplay and anonymous individual or collective competition or collaboration to solve a specific activity or complete an exercise online. The variety of stimuli increased student motivation and made it possible to maintain their attention, meaning learning became more meaningful and enjoyable: “It helps to maintain attention, plus it’s an incentive that forces you to process what the teacher is explaining in order to respond, and it is fun. It is usually the information we remember most at the end of the term”; “It is a break in the teacher’s presentation that allows us to take a second, think and remember the class better afterwards”; “It is a different and entertaining method. Either for the ‘competition’ or to get out of the routine/basic class (textbook and listening to lectures)”.

With regards to increasing student engagement and involvement in their own learning process, the students surveyed often referred to the way in which the use of Mentimeter demonstrated the teacher’s interest in students’ opinions, perceptions and experiences. The application was often used to gain immediate and continuous feedback from students in terms of usefulness, relevance and interest in relation to the pedagogical materials, activities and means of training assessment used in each course. Reactions to each question can be displayed immediately, so that student answers are visible even before the teacher’s feedback.

This enhancement of teacher–student dialogue in relation to the teaching–learning process makes students feel included, taken into account and, hence, co-responsible for the dynamics and results of the class and their own learning process: “It gives the feeling that teachers are interested in you, in your opinion and in your own education”; “The teacher can adapt the contents to the students’ opinions and adapt them to her classes”; “Above all [I value] the fact that we all talk together about what we thought of certain subject contents”.

As seen through the analysis of the responses from students who participated in the study, the use of Mentimeter was proven to have great potential to improve attention, including groups of students who would otherwise have been left out of parts of the teaching–learning processes. It also fosters understanding and critical and reflective capacity, improves involvement and encourages students to engage in self-assessment: “It changes the dynamics of the class in a positive way; we go from theory to being able to apply it in a reflective task and at the same time, you learn from and about the rest of your classmates’ opinions”; “It makes you keep up with the explanation, it even makes you pay more attention to it, it stimulates interest”; “It makes it easier to learn even things that are important in the final assessment by taking away the pressure of other study methods”.

Compared to traditional learning and teaching, in which technology is used in a unidirectional way (from instructors to students), Mentimeter permits bidirectionality through student interaction with teachers and the content that is being taught. Teachers have to plan in advance the points in their courses and sessions when they are open to student participation with this tool. There are some naturally appropriate windows: the start of the course (as an ice-breaker, to identify audience expectations and get to know the group) and the end (for evaluation). However, Mentimeter facilitates this interaction throughout the entire term, during any teaching or tutoring session: before starting, in the middle or at the conclusion of each lecture, seminar or meeting.

However, the introduction and use of any tool based on an audience response system does not guarantee a better learning experience by itself without prior practical and pedagogical planning. According to Mayhew [11], teachers should explain in the classroom why Mentimeter is going to be used, the intended objectives and the expected benefits. Teachers must, hence, bear in mind the need to prepare questions with a clear learning purpose, encouraging students to participate and discuss issues and enhancing meaningful learning. Questions can be answered individually, in pairs or groups, with previous discussion of the chosen answer.

The response time can be controlled and, depending on the type of question or planned objective, time can be allowed for analysis and discussion of each result, creating a fun and motivating learning environment. The dynamics offered by the use of this tool can contribute to transforming lectures from passive to active learning. Certain educators may feel that they are losing some power and control, but they will hopefully consider that this is offset by the benefit of educational innovation. Mentimeter was also used by students to get to know other classmates and their opinions and to generate discussions both within and beyond the classroom. Teachers also referred to the potential of Mentimeter for student use in this regard, since it can facilitate teamwork and peer-to-peer learning.

The importance that teachers placed on the opportunity for immediate feedback on their work in terms of knowledge acquisition by the students stood out in their responses: “In a very short time you collect a lot of information, in a minute you can have the opinion of 60 people”. They also highlighted the benefit for their own performance, in terms of knowing whether they were playing an adequate role and involving all students, those present in the classroom and those who are online (if they are actively “in” the class, especially when not connecting their cameras during online sessions): “Feedback, especially feedback: where the students are during class (mentally speaking), if they are really there (when you teach online synchronously). If students are keeping pace with the content presented”. Playing the role of mediator in learning processes and the development of competences is also very important for teachers. As one such participant stated, Mentimeter allowed “group reflection by knowing the opinions of others and the participation of all the students”.

3.4. Assessment and Continuous Improvement

Teacher participants in this study used Mentimeter to promote analysis, reflection, argumentation, demonstration and comprehension of concepts and contents. It also enabled the diagnosis of previous student knowledge, facilitating the construction of a class on sound foundations. Teachers followed Ausubel’s Theory of Meaningful Reception Learning [24], which states, among other things, that learning occurs when it is related to the students’ previous knowledge.

With this approach in mind, the most used type of question were word clouds (Figure 5). This type of question practices competences among students including the capacity for synthesis, facilitates diagnosis of knowledge and helps to clarify concepts, since observing the words on screen can trigger a discussion (“making them talk”) and the teacher’s intervention. All of this can be presented in a highly attractive and esthetic manner, as one educator stated: “The gorgeous and colorful word-cloud questions, I think they are great! I do not know of anything like it. They are great to start any type of class and get to know the audience and their preconceptions”.

According to Walss [25], for assessment to fulfill its training function, it must provide students with information that allows them to make decisions and adjustments in their learning process. The use of ICTs, such as Mentimeter, means that assessment can be integrated into the teaching process by facilitating feedback and automating otherwise time-consuming activities. Questions can be asked to check whether students are adequately incorporating knowledge. In this sense, it is important to stress that teachers can detect student needs that are not perceived or expressed either by themselves or by others: distinction between concepts, their applicability, or other fundamental relationships for optimal learning management. As a result, the learning process can be continuously evaluated by both teacher and student. This view was shared by both teachers (“An evaluation that the student makes of himself/herself at the moment, by answering the questions anonymously”) and students (“It helps me to see if I have understood the theory”).

An evaluation of the didactic resources used by teachers (texts, audiovisual materials, dynamics, assessment system, etc.) facilitates instant and face-to-face decisions. This can help in the design of future editions of the course, as well as reshaping a course that is ongoing. Mentimeter was used here to involve students in making certain course-related decisions (including choosing materials to be discussed or analyzed in class or types of work to be carried out as part of continuous assessment). This made students co-responsible for the development of the course, including their preferences and interests whenever possible.

3.5. Diversity and Inclusion

The use of Mentimeter promotes a more democratic and inclusive method of participating in class, as it facilitates a simultaneous response from a large number of people. This is not feasible when participation is oral, since the order of contribution has to be regulated and hierarchies are, hence, created. When there is no time limit for answering through Mentimeter, such as when it is used in asynchronous mode, participation is also made possible for those who require more time to prepare their answers and for those who are not synchronously present in the class.

As mentioned, certain personal characteristics, such as a lack of confidence, shyness or diversity factors, can limit participation. The leadership of some students can also be a barrier to other students speaking up, as they may not wish to challenge points of view or even be able to find the time to express their thoughts. However, the right to remain silent is also secure with Mentimeter. Nobody can know who has or has not answered, and students are not subjected to pressure from teachers or peers to disclose their answers (or non-answers): participation is anonymous and remains so even when educators subsequently download the data.

Students notably emphasized the inclusive potential of Mentimeter not only in relation to the increased participation of students who would not normally share their opinions with the group due to shyness or embarrassment, but also because the method of including the opinions of all class members is more egalitarian, eliminating potential biases or barriers, such as those related to gender, shyness, embarrassment, self-consciousness, fear of mistakes, lack of command of the language, speaking skills, stuttering, speech disorders, personality characteristics or any form of diversity: “Interacting in this way can be fun and easier for people who do not feel comfortable participating”; “As well as making lessons fun and different, it seeks to engage students with equal opportunities and on an ongoing basis”; “I feel more included”; “It avoids a gender bias whereby boys participate more than girls in class, equalizing all participation”.

Mentimeter also allows information, activities and even forms of assessment to be presented in different ways, making learning environments more flexible in order to better adapt to the diversity of ways of learning, managing and communicating information present in the classroom. For years, the need to vary the didactic approaches used in classrooms has been emphasized as a way of making teaching–learning processes truly inclusive, including higher education. Universal Design for Learning (UDL), a theoretical-practical model focused on offering learning opportunities to all students based on the diversity intrinsic to any class group, proposes to vary the options for presenting information: offering diverse options for expression and communication; providing multiple forms of involvement to capture interest; minimizing feelings of insecurity and offering effective feedback as well as providing options for self-regulation by developing self-assessment and reflection [26].

Visually impaired people have challenged the use of Mentimeter in class as they cannot read the questions and answer on the device, or directly view the results in real time. During this study, students with this disability in the classroom were sent the questions in advance so that they could answer them at the appropriate time using adapted devices. The teacher also has to orally narrate the results that appear in the presentation so that all students are aware of them.

One student referred to the poor adaptability of the tool in the case of people with functional diversity. It is not enough to have good online connectivity and electronic devices that permit students to access the software and participate in the activities proposed during the class; adapted devices must be available when required. Moreover, the tool itself does not allow questions to be voiced, certain fonts to be used or the font size to be modified, which could make accessibility difficult for certain groups of students and limit its inclusive potential in the future.

3.6. COVID-19 and the Post-Pandemic Context: Improving Digital Skills

Mentimeter was initially designed for face-to-face and synchronous presentations, but the circumstances arising from the COVID-19 pandemic have shown that it can also be used online and asynchronously. As mentioned, the pandemic has forced many universities worldwide to abruptly transition to online teaching. This has sometimes taken the form of blended learning, with both online and face-to-face sessions, but academics have frequently had to transform all their courses to provide exclusively online learning, with synchronic and asynchronous activities.

In higher education institutions in Spain, a majority of students have negatively evaluated this transition to online learning and the performance of their educators during the transition, largely due to a lack of innovation associated with the strategic management of ICTs [27]. Prior to the pandemic, several studies had already warned about the need to incorporate ICTs to a greater extent in university teaching; however, the pandemic and its enormous challenge in terms of teaching and assessment have rendered this an imperative need to maintain teaching quality [28].

The proper use of ICTs in class improves positive evaluations of courses and attitudes towards learning: students have a better opinion of teachers who innovate by introducing these tools. There is widespread demand from primary and secondary-level students, who frequently ask their teachers to do a Kahoot on Fridays or at the end of class. García-Valcárcel [29] identifies group work (55%), doing practical activities (39%), working with friends (35%) and using the computer (31%) as students’ preferred learning methods.

In this study, we asked students about their previous experience with different online software tools intended to encourage participation in class. A significant 71.5% of students claimed to have previously used Kahoot, compared to 13.5% who stated they had had some kind of experience with Mentimeter before this project. Other software applications (student participants were allowed to select more than one option) experienced at least once by students for participation in the classroom were: 47% Google Forms; 17.75% Doodle with an educational use; 17.25% surveys in Blackboard Collaborate; and 19.5% who had used any other software, including Quizizz, Wooclap, Socrative and Ahaslides. Finally, 11.75% of participants reported never having used any of the aforementioned tools, meaning they had never been exposed to the most common forms of ICT participation software in education.

Most online SaaS are easily adaptable to both face-to-face and synchronous teaching and blended or online teaching (also synchronous or with a degree of asynchrony). They proved to be a highly useful resource for adjusting teaching during the 2020–2021 academic year due to face-to-face restrictions, physical distancing in the classroom, the use of masks and mobility limitations imposed by lockdowns and other restrictions to contain the effects of COVID-19. In this new scenario, with online classes and video calls, student and audience participation is one of the main challenges for teaching.

Lecturers have been forced to innovatively incorporate or adapt educational resources. According to our research, Mentimeter has helped not only to increase student participation but also to make it possible in situations where everyone was online, as well as in other situations with asynchronous teaching through Moodle and in situations with some students in the classroom and others following online at the same time, in settings as diverse as their home, library, study room or any other public space where speaking is not permitted and microphone use is, hence, impossible.

In fact, between 50% and 75% of all teaching done during 2020–2021 by the twelve professionals who participated in this study was online. Nine of them used Mentimeter in face-to-face settings and eight in synchronous online lessons. Although teachers used Mentimeter mainly to obtain instant synchronous feedback from their students, it is significant that two of them also used it for asynchronous feedback by embedding the questionnaires and their results in the Moodle site of their courses.

Teachers have integrated Mentimeter into their courses in many different ways. During a course marked by restrictions and online teaching, it offered a useful way to break the ice and create a sense of belonging: in some cases, students were not meeting in person in a classroom at all, or were doing so only once or twice a month. Using this software helped to reduce the feeling of intimidation that students can feel, especially when in the first year and attending university for the first time. Fostering group belonging and mutual knowledge has been strategically key to promoting collaborative and fruitful work during the course, especially in semi-face-to-face contexts [30]. For many of the students who participated in this research, using Mentimeter represented one of the few ways, if not the only one, to participate during online classes: “It allowed me to participate during online classes for the first time”.

Students also highlighted Mentimeter’s ability to help them maintain attention and concentration during online sessions. It was especially difficult for students at home to follow class effectively and attentively via the computer: “Very useful for maintaining attention, especially now with the online system. For me personally, it has helped me to be at my best in class”; “It helps to keep my attention in class, which is appreciated, as online classes are very tiring. So it makes the class more interactive and dynamic. I really like it”.

Mentimeter was not only useful for online activities. When face-to-face classes were possible, students and lecturers had to wear a facemask during the entire class. This was also a barrier to participation, as it made it difficult for the speaker to express themselves and fully communicate, and, especially in very large classrooms with high attendance and social distancing, it was often hard to properly hear what anyone without a microphone was saying. The application prevented people from having to raise their voices (a risky practice for the transmission of the virus) and made it possible, for example in open-ended questions, for the whole audience to read what classmates wanted to share.

3.7. Limitations and Opportunities for Improvement

In this section, we will discuss some difficulties, challenges and opportunities for improvement that have arisen for educators and students after using Mentimeter.

The main limitation of Mentimeter is its cost. Although the company has an educational pricing scheme, most universities do not purchase licenses for their staff. This means that educators would have to pay for the product themselves if they want to have access to all its features. Teachers can trial Mentimeter with a free license, but the trial version is limited to only two question slides and five quiz slides.

Some teachers who had never previously used Mentimeter found it difficult to get started. Issues included how to create slides, how to present results and how to embed slides in Moodle. However, the collaborative, team-based approach implied peer support in this respect. These initial difficulties generated resistance among some teachers, but this was overcome through use of the software in class, with teachers ultimately considering it worthwhile in light of the results. Other educators reported that the interface was less intuitive than Kahoot, which they considered better suited for quizzes and competitions due to certain features, such as music and graphics. There was also an impression that the novelty effect and excitement that Mentimeter initially created among students when it was used in class faded slightly over time.

Certain educators only used certain kind of questions, focusing on two or three types and not exploring or benefiting from the full potential of the application. On the other hand, some teachers took full advantage of Mentimeter, with one even being recognized by the application team staff as one of the most frequent Mentimeter users (top 1%).

Several opportunities for improvement for the program were mentioned. These included facilitating the integration of Mentimeter into Power Point presentations, expanding the option of editing formulas using LaTeX not only in questions but also when answering them, improving the availability of different fonts and size and the possibility of open-ended questions with more characters in the answer.

When lecturers received synchronous feedback, unexpected questions, doubts or comments sometimes emerged, which they had to make time to address. This can slow the pace of the class or endanger the schedule and planned timing, but most teachers considered this a worthwhile risk, as it favored the solidity of learning of each student. However, some teachers also felt that they were unable to incorporate as much participation as they would have liked because of time constraints. With many topics and issues to be covered during the course, these teachers reported that they did not have time to include many slides for feedback. The COVID-19 pandemic also played an important role in this regard, in many cases causing reductions in available teaching time.

The students who participated in the study identified certain aspects that could be improved in relation to Mentimeter and its use in learning. Their main suggestions and contributions were as follows:

• Formalizing and increasing the use of this software in lessons as a whole, by integrating it throughout the class and not exclusively at the start and end. Some students wanted to analyze the results of the exercises, questions and activities in more detail. Others warned about the risk of abusing Mentimeter, arguing that it could lose its capacity to surprise, break routine dynamics and thus capture student attention: “Use it more often in between different questions, as it is normally used at the beginning or end of the class and there are ideas that can be missed”; “It has been used too little in my opinion, the tool has more potential”; “...But without using it too much, otherwise you lose interest”; “Make more use of the results, interpret their implications, for example”.
• Improving the design of the interface and the variety of questions used. Some students suggested incorporating a greater number of images and sounds: “The design of the application is a bit monotonous and terse”; “Maybe add sound”; “Make it more visually attractive, so that it attracts more attention”; “You can use more audiovisual stuff and graphics”.
• Addressing the problems arising from the digital gap, which has been especially significant during the pandemic [31]. Although most students had their own Internet connections, others depended on the Wi-Fi connection offered by the university and there were some complaints that it was not stable enough, making it difficult to participate. The use of Mentimeter requires adequate, reliable devices and good connectivity for students who are at home and following the presentations online as well as those in university classrooms: “Make sure that everyone has devices to participate”; “Improve classroom connectivity”; “The time for answering should be extended as there are people who may experience connection problems”. Despite the efforts made by universities to improve classroom connectivity and the availability of both connection and devices for students, a small group of classrooms and students have found their use of the tool hindered for these reasons.

4. Conclusions

As previous research has shown [7,8,9,10,11,12,13,14,15,16], Mentimeter has been found to be a useful digital software in higher education for promoting participation. The opportunity for students to participate anonymously and enjoyably enhances democratic contribution to each session and the whole course. As a result, students experience higher engagement with their learning process, as they pay more attention and feel inclined to collaborate with educators and peers in a more relaxed atmosphere than is found in traditional passive learning. However, further research is needed to test whether engagement and attention are retained over time, as the novelty effect of this software fades.

The cost of accessing the full range of functionalities that this application offers can be seen as the main barrier to its implementation. There are also other limitations when using Mentimeter in higher education: some educators find it difficult to start working with this software, and most gave up on the idea of using it to present all their course content owing to the limited range of available templates. Difficulties uploading or transferring content from other programs and platforms certainly point to an area where there is room for improvement. Most teacher participants only used certain kinds of questions and interactions, meaning they could have achieved the same goals of student participation and engagement using simpler software. Having access to a proper device and online connection can be critical to implement the use of Mentimeter and similar ICTs in education.

However, developing these student-centered methodologies does shed light on the learning process, reinforcing joint work between educators and students. In the context of COVID-19, with many activities shifted online, Mentimeter has shown its motivational effect and its potential to enable student participation and improve attention and engagement.

The versatility of Mentimeter facilitates planning participation in addition to selection and sequencing of content. Real-time feedback promotes continuous improvement by making it possible to identify gaps in knowledge, which makes it easier for educators to readjust the content and pace of classes. Most teachers expressed the effectiveness of the program for identifying students’ learning needs and assessing students’ comprehension and performance, as well as their own teaching practice and materials.

By paying attention to different types of learning and enabling Universal Design for Learning, this kind of software has the potential to contribute to achieving Sustainable Development Goal (SDG4) of the United Nations 2030 Agenda: to ensure inclusive and equitable quality education that promotes learning opportunities for all [32]. In terms of inclusion, however, Mentimeter requires improvements to facilitate the participation of visually impaired students and other students with functional diversity.

The satisfaction reported by participating teachers and students with the educational use of Mentimeter invites us to consider exploring other functionalities and find ways of overcoming the challenges and taking advantage of the opportunities that ICTs pose in education. Future research could include a comparison between this and other free or paid-for software that is available for educators.