Comparison of the Effectiveness and Performance of Student Workgroups in Online Wiki Activities with and without AI ^†

Abstract

Collaborative learning has been widely acknowledged as a successful teaching method within the education field, with research indicating its positive impact on student outcomes. During the COVID-19 pandemic, when all courses transitioned online due to lockdown measures, many universities employed learning management systems to facilitate continued group work among students. However, forming effective student groups remained challenging, particularly given the large number of enrolled students. To address this issue, this study proposes the application of an artificial intelligence (machine learning) solution to automatically group students based on their behaviours and interactions within an e-learning environment. This paper explores the potential of machine learning (ML) algorithms in assisting educators to create heterogeneous groups, considering various student attributes, such as behaviour and performance, to optimise collaborative learning outcomes. Students’ performance within a module was compared using a wiki activity that employed group work over the course of two academic years. In the first experiment, groups were formed randomly, while in the second experiment, students with similar behaviours were firstly identified using a clustering algorithm and then organised by an additional algorithm into heterogeneous groups. The results demonstrate the efficacy of the machine learning solution compared to the random approach in assisting educators with group formation for a collaborative activity such as the wiki, confirmed by a comparative analysis showing an improvement in student performance and satisfaction. This research contributes to the advancement of online education through the creation of more effective group dynamics using machine learning algorithms, thereby improving overall student learning.

1. Introduction

Successful collaborative activities, which often require considering two essential aspects when creating student groups such as heterogeneity and the number of group members, can influence learning, helping students to increase their skills and to achieve their learning outcomes [1,2]. During the COVID-19 pandemic emergency, educators struggled to plan collaborative activities due to lockdown measures. A solution adopted by universities to continue using this educational approach remotely consisted of the use of learning management systems, such as Moodle, thanks to their specific tools for collaborative activities [3].

A successful collaborative activity should consider at least the following two important factors when the workgroups are created:

• Groups composed of four or five learners each [4];
• The heterogeneity of students involved in each group in terms of cognitive resources, characteristics, and behaviours [5].

While Moodle supports the creation of five-member groups, it lacks an intelligent mechanism to generate adequately heterogeneous groups of students. Educators struggle to manually identify different categories of students to guarantee heterogeneity in each group, especially in the pandemic period that forced educators to use only the learning analytics extracted by Moodle. Consequently, this difficulty led several educators to use randomised grouping for online collaborative activities.

A plausible remedy to mitigate the shortcomings of e-learning and education lies in the utilisation of software underpinned by artificial intelligence (AI) [6] and machine learning (ML) [7,8]. These technologies, armed with precise models trained on learning analytics, have the capacity to make accurate predictions and offer valuable assistance to educators in their teaching activities. This paper explores the potential of machine learning algorithms in assisting educators to create heterogeneous groups, considering various student attributes, to optimise students’ collaborative learning outcomes.

The primary objective is to assess the impact of machine learning software, part of a Moodle Plugin (already tested) [8] on an online collaborative wiki activity conducted within the “User Experience (UX) Design” module. This assessment involves comparing responses from module perception questionnaires and the performance of the educational activity across two distinct academic years. In one academic year, groups were formed randomly, while in the other, a well-tested machine learning software was employed to create heterogeneous groups. This evaluation aims to gauge the effectiveness of the ML approach in terms of student satisfaction and performance. The primary inquiries to be addressed are as follows:

• Does online wiki based on heterogenous groups with the ML approach contribute to greater improvements in the students’ performance compared to the same activity based on the random approach?
• Does the use of the ML approach in creating heterogeneous groups influence the improvement in students’ perception, compared with the random approach?

2. Methods

2.1. Description of the Activity

The “User Experience (UX) Design” module is a blended learning course included in the last year of the undergraduate degree in Computer Science at Middlesex University in the UK. It drives students towards grasping the foundational theoretical principles and applied methodologies that hold significance in the context of human involvement, not only in the structuring of design and its procedural aspects but also in integrating a user-oriented outlook into the creation of products and services.

Students are assessed through an individual task characterised by a multiple-choice quiz and a collaborative task to be performed on the Moodle platform consisting of an online wiki activity based on workgroups, where they can also share experiences, negotiate understandings, build shared knowledge, and support each other in the learning process.

This study examines two iterations of the online wiki activities conducted during the academic years 2020/2021 and 2021/2022. During the academic year 2020/2021, the online wiki engagement encompassed 141 international students (107 males and 34 females) representing diverse regions such as Africa, Europe, Arabia, India, and Asia. Within the same timeframe, a total of 26 groups (each comprising 5–6 members) were chosen through a random selection process. In the subsequent academic year, 2021/2022, the participation extended to 111 international students (90 males and 21 females) who shared the same nationalities as the prior year. Notably, this edition featured the creation of 20 groups, maintaining an equivalent membership count as the previous iteration. These groups were formed utilising machine learning (ML) software specifically designed to foster heterogeneity among the groups.

2.2. Machine Learning Approach

The educators encountered challenges in profiling student behaviour based on learning analytics monitored through Moodle (Moodle log data). This hindered the successful formation of diverse workgroups, leading to the frequent use of a random selection approach. This approach, however, did not consistently ensure diversity within the collaborative activities.

For this purpose, one of the two editions of the “User Experience (UX) Design” course adopted a machine learning approach. This approach aimed to create diverse and useful groups of students for the online wiki activity. The goal was to enhance the learning outcomes and performance of the students by fostering collaboration within these groups.

The machine learning software included in the tested Moodle plugin used in this study performs the following functions:

• Initially, it employs the K-means clustering algorithm to establish clusters of students based on their behaviour and interaction within the “User Experience (UX) Design” course. This process results in the creation of distinct categories of students.
• Once the clusters are formed, the plugin assembles groups of five members each. These groups are carefully curated to include students from various clusters, thereby maximising diversity within each group [8].

The dataset employed for this study comprised a selection of Moodle log data. These logs were utilised to identify various features related to the students’ learning process, such as activity interaction, study patterns, and presence coefficients. These features aided in effectively distinguishing between different student profiles [9].

2.3. Data Collection and Methodology

The research inquiries necessitated an analysis involving a comparison between students’ scores in online wiki activities and their responses in anonymous questionnaires. This comparison was based on prior similar investigations focusing on students’ perceptions of collaborative activities [10]. Two specific academic years (2020/2021 and 2021/2022) were selected for examination due to their inclusion of the same activity and a roughly equivalent number of international students. The analysis of scores was executed using MS Excel, encompassing mean and standard deviation comparisons for each academic year to measure the dispersion of the data [11]. The evaluation of questionnaires, consisting of 14 questions, was also conducted using MS Excel ver. 16.80. This entailed the selection of questions concerning perceptions of group composition within the online wiki activity, specifically targeting those rated on a five-point Likert scale (comprising the values strongly disagree (SD), disagree (D), neutral (N), agree (A), and strongly agree (SA)). The objective was to contrast perceptions between the two editions.

3. Results and Discussion

The analysis of students’ scores and satisfaction regarding the online wiki activity required an examination based on workgroups to address the research questions. For the academic year 2020/2021, students were randomly assigned to groups, while those from the academic year 2021/2022 were grouped heterogeneously using machine learning software, which included clustering and sorting algorithms.

After creating the dataset using Moodle logs, which encompassed login frequency, last login, total time spent online, number and frequency of video views, and number and frequency of files opened, the K-means clustering algorithm identified four clusters representing students with similar behavioural and interactional aspects within the online module. The sorting algorithm then automatically distributed students into 20 groups, ensuring the inclusion of at least one student from a different cluster in each group, thereby maximising heterogeneity.

Following group formation, students engaged in the online wiki activity through the Moodle platform. At the activity’s conclusion, each student was assigned a grade between 0 and 100 by the teacher. A comparative analysis of the scores achieved by students in workgroups formed through random assignment and those formed through machine learning was conducted to examine the disparities between the two approaches.

3.1. Does Online Wiki Based on Heterogenous Groups with the ML Approach Contribute to Greater Improvements in the Students’ Performance Compared to the Same Activity Based on the Random Approach?

Figure 1 depicts the performance outcomes after the online wiki activity in both editions. Students engaged in the wiki activity within heterogeneous groups exhibited a noticeable enhancement in their grades in contrast to the alternative approach.

This pattern is further substantiated by the data presented in

Table 1. Comparison between grades obtained by students in different online wikies.

	Wiki with Random Groups	Wiki with Heterogeneous Groups
Score mean	60	65
Scores more than 70	23%	48%
Scores between 60 and 69	28%	25%
Scores between 40 and 59	37%	13%
Scores less than 40	12%	14%
Standard deviation	26	9

. In the heterogeneous group approach, 48% of students attained a grade of 70 or higher, and 73% of them successfully completed the activity with a grade exceeding 60. In contrast, the wiki activity based on random groups resulted in only 23% of students achieving a grade surpassing 70, and 51% obtaining a score of 60 or more.

The outcomes derived from the standard deviation analysis of the grades across both academic years further validate the efficacy of the machine learning software in forming successful heterogeneous student groups. The minimal standard deviation observed in the online wiki activity utilising the machine learning approach signifies a remarkably limited spread of data around the mean scores. This outcome is a result of the advantages offered by heterogeneous groups, which guarantee the inclusion of highly skilled students within each group, foster peer tutoring and support, alleviate anxiety—particularly among students struggling with the subject—and ultimately prompt learners to enhance both their understanding and their final academic achievements.

3.2. Does the Use of the ML Approach in Creating Heterogeneous Groups Influence the Improvement in Students’ Perception, Compared with the Random Approach?

The analysis conducted by comparing the two sets of questionnaires completed by students across the two academic years validates the efficacy of the machine learning approach, particularly in terms of perceptions. The questionnaire corresponding to the 2021/2022 academic year yielded more favourable outcomes in comparison to the 2020/2021 academic year. This improvement is evidenced by the responses provided by students engaged in heterogeneous groups through the machine learning approach, as shown in

Table 2. Results of the 5 questions selected by the questionnaire related to User Experience (UX) Design 2020/2021 and 2021/2022 using random and machine learning approaches. SD = strongly disagree; D = disagree; N = neutral; A = agree; SA = strongly agree.

Questions	Random Approach					Machine Learning Approach
	SA	A	N	D	SD	SA	A	N	D	SD
I believe that collaborative activities based on groupwork increase students’ engagement.	46%	18%	27%	0%	9%	38%	35%	18%	3%	6%
I believe that each member of my group gave the maximum contribution to the collaborative task based on their preparation.	36%	0%	18%	18%	28%	41%	29%	18%	9%	3%
It was motivating to complete the collaborative task with the members of my group.	45%	19%	19%	0%	17%	44%	29%	15%	9%	3%
I think my group was well balanced and offered benefits, especially for students that found studying this topic difficult.	27%	27%	18%	10%	18%	38%	32%	15%	9%	6%
I was surprised by the high quality of the contributions from the members of my group.	29%	18%	26%	11%	16%	24%	38%	26%	9%	3%

. These responses not only highlight advancements in performance but also indicate a heightened sense of satisfaction.

4. Conclusions

This research delves into the potential and effectiveness of employing a machine learning solution to aid educators in forming student workgroups, facilitating successful collaborative activities such as the online wiki within the Moodle platform. The same activity was implemented in the “User Experience (UX) Design” module for the final year of the undergraduate Computer Science program, executed in two distinct academic years: 2020/2021, where groups were formed entirely at random, and 2021/2022, characterised by the utilisation of heterogeneous student groups automatically generated by the machine learning solution.

This solution creates groups with diverse learner “profiles,” ensuring heterogeneity within each group. A comparison of these two approaches highlighted the efficacy of the machine learning approach, which exhibited superior enhancements in terms of performance. This was attributed to the advantages of heterogeneous groups which foster peer tutoring, reduce anxiety, and enhance productivity through the diverse skill sets of students within each group.

The effectiveness of creating heterogeneous groups was also affirmed by students’ responses in questionnaires. In the edition featuring heterogeneous groups, 73% (strongly agree + agree) believed that collaborative activities based on group work increased student engagement, compared to 64% in the random group edition. Similarly, responses regarding motivation indicated that 70% (strongly agree + agree) of students in heterogeneous groups considered their groups well balanced and advantageous, especially for students facing challenges. This contrasted with only 54% (strongly agree + agree) of students involved in the random approach.

A potential future challenge lies in extending this machine learning solution beyond collaborative activities to various teaching methodologies in different online courses to assess its effectiveness in diverse contexts.