Next Article in Journal
Research on Mechanical Behavior and Energy Evolution of Coal and Rocks with Different Thin Spray-On Liners Thickness under Uniaxial Compressive Loading
Next Article in Special Issue
Simulation-Assisted Learning about a Complex Economic System: Impact on Low- and High-Achieving Students
Previous Article in Journal
Strategies for Enhancing Construction Waste Recycling: A Usability Analysis
Previous Article in Special Issue
Flipped Classroom and Gamification Approach: Its Impact on Performance and Academic Commitment on Sustainable Learning in Education
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Examining the Effectiveness of Computer-Supported Collaborative Learning for Language Proficiency Purposes

Institute of Humanities, Peter the Great Saint-Petersburg Polytechnic University, 195251 St. Petersburg, Russia
*
Author to whom correspondence should be addressed.
Sustainability 2022, 14(10), 5908; https://doi.org/10.3390/su14105908
Submission received: 4 March 2022 / Revised: 20 April 2022 / Accepted: 6 May 2022 / Published: 13 May 2022
(This article belongs to the Collection Technology-Enhanced Learning and Teaching: Sustainable Education)

Abstract

:
(1) The main goal of this research was to assess the effectiveness of the computer-supported collaborative learning for language learning purposes using the indicators of students’ learning outcomes and the level of their engagement, as well as to determine the most effective benchmarks for teams’ forming. (2) Methods: A total of 81 undergraduate students studying at the Humanity Institute of Peter the Great Polytechnic University voluntarily participated in the study. For our research, we used the results on final English testing and survey results on students’ engagement. Each year, three groups of students were formed into teams according to three criteria: leadership skills, academic performance and personal choices. Microsoft Excel 2016 tools were used for data interpretation: testing samples for normality, a one-way analysis of variance (ANOVA) and comparison of means. Neural network dependencies of test results were built by means of Mathematica Wolfram Software. (3) Results: According to the results of this study, the underlying principles of forming teams highly influenced the indicators of students’ English proficiency; in particular, the experiment proved the effectiveness of selecting students according to their academic performance. In addition, the correlation analysis revealed that the engagement of students played an important role and influenced the results of their learning. This was especially seen in a group where teams were distributed due to the differences in academic performance. (4) Conclusions: As the COVID-19 pandemic is an ever-changing situation, it is important to implement effective learning models that promote higher learning outcomes and students’ engagement. This study contributes to such knowledge and provides insightful implications to academia.

1. Introduction

The devastating nature of the COVID-19 pandemic has affected almost every sector of society in the world, and higher education is no exception [1]. In terms of teaching, the COVID-19 pandemic has disrupted most existing practices, with the exception of specific online providers such as the Open University and forced the teaching and learning process to change unpredictably and quickly [2]. A variety of learning technologies and approaches were introduced to maintain the quality of higher education. Some of them proved to be effective for learning professional disciplines, others for language learning purposes. Many researchers have analyzed the transformation brought about by such a pedagogical approach as computer-assisted language learning (CALL), which appeared in the theory and practice of teaching in the 1990s. This methodology is evolving and will make it possible to apply digital education on a wide scale. This type of learning proved instrumental in the integration of individual and personality-oriented education. However, in view of the situation which the whole world faced during the COVID-19 pandemic, scientists and educators started to explore the ways of collaborative work among students isolated from one another.
The goal of the current research is to investigate the concrete technologies of collaborative learning for foreign language proficiency purposes with the help of computer-supported learning in the system of teaching foreign languages in technical universities, and to assess the effectiveness of the proposed teaching approaches using the indicators of students’ learning outcomes and the level of students’ engagement. Thus, the paper is focused on three major research questions:
  • Do the criteria of forming the teams influence the indicators of students’ learning outcomes (English proficiency)?
  • Do the criteria of forming the teams influence the level of students’ engagement?
  • Does the level of students’ engagement influence the students’ learning outcomes?
The rest of this paper is organized as follows. Section 1 describes the theoretical background on computer-assisted language learning (CALL) and computer-supported collaborative learning (CSCL). In Section 2, the methodological basis of the study is described. In Section 3, the results on English proficiency and engagement level in three groups of students are presented. Furthermore, a correlation analysis of students’ learning outcomes and engagement levels in CSCL are analyzed and discussed. Lastly, the conclusion, limitations of the study and future research directions are given in Section 4.

1.1. Literature Review

1.1.1. Computer-Assisted Language Learning (CALL)

In the last decade, computers have inevitably become an integral part of education. CALL is of particular relevance for language learning. In a broad sense, CALL is defined as “the search for and learning of applications on a computer for teaching and learning languages” [3]. Such a definition reveals the term as the use of computers for the presentation of linguistic material. However, Schofield [4] defines CALL as any form of language learning or language teaching performed using computers. Another definition of CALL is based on its variability characteristic, which is “any process in which a student uses a computer and improves his language as a result” [5]. This term is widely used to refer to “the field of technology and teaching and learning a second language” [6]. CALL has evolved to include material design, technology, pedagogical theories and teaching methods [5]. It is now used regularly in a variety of learning situations. Thus, the possession of digital devices for language learning; general computer literacy, i.e., higher digital and computer literacy to use a wide range of skills for general or “technical purposes”, as well as for educational purposes, being “tech savvy” [7]; and the frequency of use of these devices for language learning purposes, weighted by their ratio, can be considered as potential determinants of language achievement.
CALL helps learners not only improve their language skills, but also allows them to learn at their own pace and get fast feedback, bug and correction information and error analysis. In other words, it is the learner who gets to control the educational track of their own learning, construct meaning and evaluate and control their own work [8]. Jaber [9] also mentions that with the help of a computer, students can collaborate, use their critical thinking skills and find alternatives to problem solving in student-centered classrooms.
A literature analysis shows that foreign language learners usually have a positive attitude towards CALL. In a study by Al-Juhani [10], which examined the attitudes of EFL students towards CALL, it was noted that participants had a positive attitude towards CALL. In a study by Askar, Yavuz and Köksal [11], it was found that all perception scores for both computer and traditional environments were in favor of CALL. Students also reported positive changes, including focus and self-esteem while studying in the computer lab. Another study [12], which focused on student and teacher attitudes towards CALL use, noted that both students and teachers had a positive attitude towards CALL. CALL proved to be especially effective in teaching writing. Hirvela [13] explains that college writing is becoming increasingly computer-based, and the computer is becoming a popular tool of developing writing skills.
However, a study by Min [14], examining the attitudes of adult Korean students towards the use of computers in learning English as a foreign language, found that a significant proportion of adult learners of Korean did not express positive attitudes towards the use of computers in teaching English. In addition, Tang [15] sought to determine the attitudes of ESL learners towards CALL and found no significant differences in participants’ attitudes towards computer-assisted learning. In summary, although most studies show that students hailed CALL as a novel and fresh approach to teaching and learning foreign languages, it is important to note that in some studies students were less enthusiastic about CALL. However, it is now recognized that negative attitudes towards the use of technology for language education hinder language achievement [16], while positive attitudes promote the integration of technology into language teaching and learning and improve language skills [17].

1.1.2. Computer-Supported Collaborative Learning (CSCL)

Computer-supported collaborative learning (CSCL) is an area related to how computers can support learning in groups (co-located and distributed). This also applies to understanding computer-mediated actions in collaborative learning [18]. The research questions addressed in the CSCL include how people learn with domain-specific tools, how small groups interact and develop shared meanings over time and how online learning in communities (e.g., MOOCs) creates new conditions for teaching and learning at scale. In this rapidly developing field, Ludvigsen et al. [19] argue that CSCL is characterized by a more or less stable basis of two epistemological positions: individualism and relationism. Individualism in CSCL means that researchers use a cognitive perspective on group learning (e.g., collaborative cognition, predefined analytic categories, individualized knowledge), while relationism in CSCL is consistent with a sociocultural perspective (emergent collaboration, mediation, learning as a process).
CSCL promotes the creation of learning communities that are consistent with the educational paradigm emerging in today’s social and technological environment, in which participants can alternate between the roles of student, designer and active participant [20,21,22]. CSCL involves creative problem solving from an interdisciplinary perspective during a period of intense collaboration when students need to find a co-created solution, develop a project, create a prototype or produce a product. The collaborative process leads to multiple opportunities for constructing shared meaning, which entails both convergence and divergence of cognition or knowledge [23,24]. Convergence and divergence occur in a process of intense negotiation aimed at achieving a common vision and outcome, overriding previous individual constructs and therefore encouraging a restructuring of previous knowledge [25].
Learning processes within CSCL require detailed planning, in which all pedagogical, educational, social and technological aspects must be clearly defined [26,27,28,29]. Engagement in the search for a common goal is a key element through which the actions are carried out and the structure of cooperation is shaped [30].
At the stage of implementation of the educational process, organizing effective interaction is a problem at the organizational, social and cognitive levels [25,31]. Through planned exchanges of information, each student must restructure their previous knowledge and be ready to learn with others. Motivation, task engagement, in-group support and a sense of community are key factors for students’ success [32,33,34]. Computer-assisted language learning allows educators to adjust the learning input to the specific requirements of individual students. Moreover, learners can get instant feedback on their answers, which enables them to assess their own performance [35]. Teacher feedback is also very important, due to its influence on the management and adjustment of knowledge convergence and divergence processes [25,31,34,36].
When evaluating CSCL processes, one should take into account not only the achievement of goals or the final result in the form of a product, but also the dynamics of the interactions that took place, as well as cognitive and social elements [37,38]. In this respect, participatory appraisal is a very appropriate system for reviewing individual and group achievements, whether through summative or formative appraisal processes [37,38,39].
Collaboration and negotiation are essential to learning at CSCL as they influence the satisfaction levels of both students and teachers who participate in the experience [31,40]. Therefore, collaboration must be carefully defined during the design phase of the CSCL so that the implementation includes consideration of social relationship aspects and ensures the articulation of cognitive aspects related to task performance and intended knowledge construction [23,24,25,41]. In this regard, intragroup emotional support is especially important [33].
The technology tools used in CSCL have a significant positive impact on learning processes and the underlying dynamics of collaboration [42,43,44]. Technological resources should be chosen in accordance with the learning objectives pursued and should be consistent with the planned pedagogical, cognitive and social activities [45,46]. In this regard, a technology that enables collaborative learning must be able to structure complex tasks and facilitate group analysis and negotiations that will lead to task solution [47].
Computer-facilitated communication between students will go smoothly only if collaborative learning activities are in place [48]. To facilitate a networked learning environment conducive to the sharing of knowledge, it is essential to form cohesive groups of learners taking into consideration their individual preferences and characteristics.
CSCL gives all learners a number of advantages, even if they are backward in their studies. All team members are more motivated since teamwork reduces stress levels and saves time. CSCL produces a synergy effect as teams of students have higher levels of thinking and keep information longer than individual learners [49]. Students appreciate being given a lot of responsibility for their performance, which results in better learning outcomes [50].
CSCL coupled with structured, online intervention is especially beneficial for students with low levels of foreign language proficiency since educators can use the class time more effectively. In such an environment, peer support and technology become scaffolds in the learning process [51].
At the moment, CSCL is facilitated by special groupware developed for educational purposes, but these computer applications do not embrace social, motivational and emotional demands [52]. In-depth research needs to be conducted to meet these demands, and CSCL will produce the desired results only if teams of learners are formed not randomly, but in evidence-based ways.

1.1.3. Teams’ Forming

Student teams face many challenges, especially if they have little teamwork experience [53]. The following are some of the main factors that influence successful team building [53,54,55,56,57,58,59]: group composition, different motivations, expectations or obligations within teammates, personality conflicts, task ambiguity, role ambiguity, nervousness in knowledge levels, lack of interpersonal communication skills, etc. Among these, interpersonal conflicts related to unequal efforts and contributions of team members and poor time management stand out [60].
Several strategies have been considered in the work of some researchers to promote the correct formation of a team [53,56,57,61,62]: small group, processing group and individual contributions, coaching sessions, reflection sessions on teamwork, formal and informal communication, providing feedback on individual and collective work in a team, self-assessment and peer assessment, incentives, etc.
One of the ways is forming a team on the basis of psychometric data. In one study, the authors used Myers–Briggs Type Indicator metrics to evaluate the team members’ personality traits so as to strike the right balance between them and select the team members whose psychological characteristics are complementary [63]. Psychological compatibility can be measured in a number of ways: by getting students to complete pen-and-paper tests, watching their behavior in the course of teamwork, distributing questionnaires to determine the students’ social preferences, organizing psychological trainings or by simply allowing them to choose their learning partners.
Another way is to unite students who have high outcomes with those whose outcomes are weak.
Higher-level learners should act as motivators involving their peers in teamwork since they tend to initiate communication with the teacher and other students. Each team member should be given a clearly defined role in the team collaboration and assume collective responsibility for the final result [64].
A cohesive atmosphere (including psychological safety, positive emotions and a structured team organization) is directly related to the students’ performance [65], so it makes sense to do research into different ways of organizing collaborative learning. After conducting a number of experiments with different techniques, it will be possible to compare the learning outcomes, and the technique with the best outcome should then be integrated into the educational process. Whereas in the course of face-to-face learning a teacher could form teams on the basis of personal observations, it is virtually impossible in the distance learning environment.

2. Materials and Methods

On the basis of the Humanity Institute of Peter the Great St. Petersburg Polytechnic University we elaborated an educational model based on the computer-supported collaborative learning approach. During the whole semester, students were working in teams (special assignments were created in the Moodle, separate rooms in MS Teams were developed for collective discussions, the additional material was uploaded to Google platform). To distribute students into teams we used several criteria. The experiment, which aimed at identifying the best possible criterion according to which teams of learners can be formed, was conducted in two spring semesters (2020–2021) when the pandemic caused abrupt transition to distance learning. It was carried out at Peter the Great Polytechnic University in the groups of students majoring in linguistics in listening comprehension classes. Each year, groups were formed in accordance with three criteria, the first of which was personality traits. The students who possessed leadership skills worked in collaboration with the learners who were reluctant to assume responsibility for the completion of the assignment and willing to accept a subordinate role. The students’ leadership characteristics were identified on the basis of the test “Diagnosing leadership skills” [66]. The second principle of forming teams for collaborative work was the students’ personal preferences: they were guided by their personal affinities when they chose the partners to work with. In accordance with the third criterion, the students with a high level of foreign language proficiency worked in collaboration with the ones whose level of foreign language training was inadequate.
The experiment was conducted in the fall semesters of 2020–2021 in 6 groups of second-year students (75 females and 6 males, 81 total). In 2020 and 2021, we had different groups of students, since listening comprehension is studied through only one semester in accordance with the curriculum of the bachelor course for students specializing in linguistics. In 2021, we had a sample of 3 equal groups, 13 students in each of them, whereas in 2022 the number of students in each group equaled 14 (since more students had enrolled in the course). Each semester, we had 3 groups of students that were studying according to the proposed learning model, but the principles according to which the students were selected to the working teams were different. The group of students in which the teams were formed on the basis of psychological factors (leadership skills) is referred to as Group A (N = 27); the group in which the teams were formed depending on the level of academic attainments is Group B (N = 27); and the group in which teams were created according to the students’ individual choices is Group C (N = 27).
For our research, we used the scores of the initial and final testing on English proficiency. Both tests were structured in a similar way: they included one lexical multiple-choice assignment (maximum 20 points); one listening comprehension assignment in which it was necessary to identify whether the statement was true or false on the basis of the first part of an audio record (maximum 15 points); and one multiple-choice listening comprehension assignment on the basis of the second part of the same audio record (maximum 15 points). The final test was based on the lexical material and the topics which were studied in the course of the experiment. To evaluate students’ level of engagement we used the Academic Engagement Scale based on the three most common dimensions identified by researchers (behavioral, emotional and cognitive engagement). To collect the data, we conducted an online survey that consisted of three questionnaires. The first questionnaire measuring the students’ cognitive engagement included six statements:
(a)
I am willing to acquire new knowledge.
(b)
I feel confident in the course of acquiring new knowledge.
(c)
I take many efforts when I do my home assignment.
(d)
I am always involved in classroom discussions and interested in the topics under discussion.
(e)
I pay attention in class and do my home assignments very carefully; therefore, I understand the learning material.
(f)
If I do not understand the learning material, I ask questions.
The students were asked to give from 1 to 5 points to each statement (1—totally disagree; 5—absolutely agree).
In the second questionnaire, which was aimed at measuring emotional engagement, the students also rated their attitudes on a 5-point scale, as in the first questionnaire. The second questionnaire included four statements:
(a)
I enjoy attending foreign language classes.
(b)
I am willing to continue studying the foreign language after graduation.
(c)
I enjoy doing home assignments given by the foreign language teacher.
(d)
I feel tension and inner uneasiness when I have to study a foreign language.
Behavioral engagement was measured in percentages on the basis of the following three criteria:
(a)
The completed assignments were regularly uploaded in Moodle;
(b)
The students attended the classes in MS Teams;
(c)
The learners studied the material available on the Google platform.
The results of the questionnaires were then checked for validity using SPSS. We tested the validity of Pearson’s product-moment correlation using the principle of correlating, or connecting, each item’s score with the overall score derived from the respondents’ answers. The results of the questionnaire validity test (Pearson product-moment correlation) showed that all items are valid. After checking and testing the questionnaire, the Cronbach reliability coefficient alpha was 0.871.
Microsoft Excel 2016 tools were used for data interpretation [67]: testing samples for normality, a one-way analysis of variance (ANOVA) and comparison of means. Neural network dependencies of test results were built by means of Mathematica Wolfram Software.

3. Results

3.1. English Proficiency Results

At first, we checked all formed groups’ results distribution. The diagnostic test data of each team were distributed according to the normal distribution (based on Pearson’s chi-squared test). The means were equal at the 5 percent level based on the three sample Student’s t-test. Smooth histograms are presented in Figure 1. Thus, we can reliably state that there were no differences in the initial results between groups.
In all the groups participating in the experiment, the students worked with the course book by Dashkina, A.I, Popova, N.V. called Listening comprehension tasks for linguistic majors [68].
All the students completed some listening comprehension assignments individually in class, but the tasks that required listening for details and listening aimed at full understanding were completed in collaboration. The examples of such tasks included finding synonyms and words corresponding to the definitions as well as integrative assignments, in which the students were asked to find differences between the video they watched and the text they read. Sometimes, they had to prepare a monologue related to a listening comprehension topic using open electronic resources, produce a transcript of a video or watch another video dedicated to the same topic.
Teamwork was employed through discussion and correspondence in MS Teams. The learners had regular meetings in a distant format on the MS Teams platform; they added the teacher as a member of the team, and each meeting was recorded. The teacher was informed about the time of online meetings, which enabled him to join the discussion at some point and check how much the team members contributed to the teamwork. When the students did a lexical assignment, they recorded their answers in the MS Teams chat. Even though the teacher did not have time to join all the meetings of the small groups, he had access to their record.
Here is an example to illustrate how a small group of students prepared a monologue “Dressing for a job interview”. Before the meeting in MS Teams, they worked individually and found some information on the Internet about what clothes candidates should wear for an interview. At the MS Teams meeting, they took turns to read the recommendations they had found and discussed in English which of them could be included in the presentation. One of the team members recorded the plan of the monologue in the chat. Such a discussion on the MS Teams platform gave the learners a chance to speak English outside the classroom and also enabled the teacher to check whether the meeting actually took place.
All the groups that participated in the experiment were given exactly the same assignments; the only difference was the criteria, according to which teams were formed.
At the end of the semester, the students sat a final test based on the topics studied in the course of the experiment. It was a 50-point multiple-choice test in the same format as the diagnostic one. Smooth histograms of the final test data are presented in Figure 2.
Note that the final test data of the first two teams were still distributed according to the normal distribution and the null hypothesis was not rejected at the 0.05 level based on the Student’s t-test. The third team’s final test data had a different distribution. Figure 3 clearly shows the difference in the distribution of the increase in results for the two tests.
Further, for all the teams, we approximated the dependency of the final test result on the diagnostic test data by means of the neural network model. Neural network models for data allowed us to see an overall pattern and revealed hidden ones [69,70]. Figure 4 illustrates that these approximations were quite accurate.
Thus, we also have the neural network model of the increase in the students’ test results. The correspondent line graph is presented in Figure 5. It allows the interpretation of the difference between the final test data distribution of the first two, and third, teams.
The progress made by the students by the end of the experiment is clearly visible on the line graph built by a neural network. The results of the diagnostic test are on the horizonal axis, whereas the difference between the results of the final and the diagnostic tests is given on the vertical axis.
The most considerable progress was made in the groups where the teams consisted of the students with different levels of foreign language training (the green graph). The learners whose foreign language proficiency was just at a rudimentary level benefited from the collaboration with the students with high levels of language proficiency. For this reason, the starting point of the graph is the highest on the vertical axis. All the graphs bulge in the middle, which means that the students with the average results of the diagnostic text made more impressive progress than the learners with the lowest level of language training, since it was easier for them to internalize the new learning material than for the ones who had hardly any initial foreign language base.
The groups in which the teams were formed in accordance with the results of the psychological test, and in which learners with leadership qualities collaborated with the students who accepted a subordinate role (the blue graph in the middle), made less substantial progress than the previous group. The students with leadership skills did not necessarily have high levels of foreign language proficiency. They facilitated teamwork so that the whole process of completing assignments ran smoothly, but they were unable to help their partners when the latter had queries. Nevertheless, the progress achieved by these groups was more substantial than the progress in the groups in which the teams were formed on the basis of the students’ personal preferences. Very often, the team members did not produce the desired synergy effect because they could not share knowledge or facilitate teamwork. Indeed, students with low levels of academic performance often formed partnerships and completed assignments together, but they did not benefit from such collaboration. Anecdotal evidence suggests that top performers also tended to form partnerships, and they could certainly gain considerable advantage from working together. However, students with high levels of academic performance did not make much progress because the difference between their results in the final test and the diagnostic one was not substantial.
In all the groups, the students whose initial level of foreign language proficiency was very high did not have much room for improvement; therefore, the line graphs illustrate that they made less progress than the ones with the average results of the diagnostic test. Thus, the results of the tests illustrate that when small groups were formed on the basis of difference in academic performance, the progress made by students was more significant than when the other criteria were applied. The students who worked in small teams formed in accordance with this principle could clearly identify their own foreign language deficiencies by comparing themselves with the other team members [71]. When the students collaborated with the partners of their own choice as well as in the teams formed on the basis of the psychological test, it was more difficult for them to draw comparisons with their partners because in many cases they were all at the same rudimentary level of foreign language proficiency.

3.2. Results on Engagement

The highest levels of cognitive and emotional engagement were registered in the groups in which the teams were formed on the basis of the students’ individual preferences, whereas the lowest level of emotional engagement was observed in the groups in which the criterion of forming the teams was the difference in academic performance, and the lowest level of cognitive engagement was observed in the groups in which the criterion of forming the teams was leadership qualities. Figure 6 illustrates this observation.
The students felt more positive when they were free to choose the partners who they felt comfortable with. On the contrary, when they had to work with the partners that they sometimes had difficulty establishing rapport with, they perceived it as a restriction of their personal freedom, which resulted in the backlash and affected the levels of the cognitive and emotional engagement (see Figure 7).
However, the analysis of the behavioral engagement showed different results (see Figure 8).
The behavioral engagement in the groups formed on the basis of different levels of individual preferences was the lowest, while in the groups formed on the basis of different levels of academic performance, students showed the highest level of this indicator.

3.3. Correlation Analysis

Since one of the goals of our study was to identify the influence of students’ engagement on their academic outcomes, a Pearson correlation analysis was conducted to calculate the significance of the indicators’ impact in three groups. The results are shown in Table 1.
The correlation analysis revealed a positive correlation between all the indicators considered. It was a strong relationship between behavioral engagement and learning outcomes, especially in Group B (p < 0.001). Thus, most students who attended live online classes, completed assignments in Moodle and learned materials using the Google platform had higher results on final English testing. Emotional engagement significantly predicted results on English testing (p < 0.05). At the same time, the relationship between emotional engagement and behavioral engagement was weak in three groups (0.17 < R < 0.26). Furthermore, cognitive engagement had a slight impact on students’ outcomes. Thus, it can be concluded that the level of engagement significantly influences the students’ learning outcomes, and this should be taken into account by teachers and academic managers.

4. Conclusions

According to the results on academic outcomes, students improved their English proficiency in all groups, which confirms the adequacy of computer-supported collaborative learning approach usage for English learning purposes. At the same time, the group in which the students were distributed according to academic performance results (students with a high level of foreign language proficiency worked in collaboration with the ones whose level of foreign language training was quite low) showed the highest results, while the results of the group in which teams were formed on the basis of the students’ personal choices reflected an insignificant improvement. These conclusions are confirmed by the results of statistical processing. Thus, we can conclude that the criteria underlying the principles of forming teams highly influence the indicators of students’ English proficiency; in particular, the experiment proved the effectiveness of selecting students according to their academic performance.
Besides the improvements in academic performance, it is also essential that students should be highly motivated and involved in the collaborative activities. To this end, at the end of the experiment, the students were asked to complete two questionnaires measuring their cognitive and emotional engagement in learning the English language. The behavioral engagement level was also analyzed. Generally, the level of students’ engagement was high in all the groups, which corroborates the positive impact of computer-supported collaborative learning for linguists. Students who were grouped in accordance with their individual preferences showed the highest level of emotional and cognitive engagement, while students who were split into teams on the basis of their academic performance showed the highest level of behavioral engagement. Despite the fact that indicators did not vary significantly, we can confirm that the underlying principles of forming teams influence the students’ level of engagement.
According to the obtained results on academic performance and students’ engagement, we can state that the learning process based on this model is effective, as indicators showed positive values. After conducting a correlation analysis of all the indicators under consideration, we revealed that the engagement of students played an important role and influenced the outcomes of their learning, and this was especially seen in Group B. We can conclude that due to the implemented computer-supported collaborative learning and appropriate criteria of teams’ forming, students were highly engaged in the learning process and, therefore, accomplished high results.
The theoretical work of other scholars in this field has been a useful resource for planning and designing, and we expect that our study will provide something of value for future researchers, too.
The study was based on the work of scientists involved in the analysis of two different models used: CALL [11,12,13,14,15,16] and CSCL [43,44,45,46,47,48]. After studying the results of research on the application of such educational models in various settings, a proprietary model was developed, adapted to the local university environment. The main difference from previous works is the combination of models. In addition, an important part of the study is the formation of teams and the impact of teamwork on academic performance, English language development and student engagement.
The results of our study contribute to previous studies [10,11,27,34,46] in terms of the developed new educational model, which has a positive effect not only on learning a foreign language, but can also be applied in other areas. The main advantage is the successful combination of the two previously described models, which contributes to effective teamwork, stress reduction, increased motivation and, as a result, increased engagement in the educational process.
Our study has its limitations. Since the research was conducted only in Russia, cultural differences may influence the results of students in other countries. The study included only the students specializing in humanities, particularly the ones majoring in linguistics. Undergraduates studying in technical areas have their own peculiarities, and the results may differ. In addition, the sample size was quite small as the number of students studying linguistics at Peter the Great St. Petersburg Polytechnic University is small, but the results are supposed to be checked every year.

Author Contributions

Conceptualization, A.K. and A.D.; methodology, A.K.; software, T.L.; validation, T.L., I.G. and D.T.; formal analysis, D.T.; investigation, E.T.; resources, E.T.; data curation, A.K.; writing—original draft preparation, A.D.; writing—review and editing, A.D.; visualization, A.K.; supervision, E.T.; project administration, A.K.; funding acquisition, A.D. All authors have read and agreed to the published version of the manuscript.

Funding

The research is partially funded by the Ministry of Science and Higher Education of the Russian Federation under the strategic academic leadership program ‘Priority 2030′ (Agreement 075-15-2021-1333 dated 30 September 2021).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board of INSTITUTE OF HUMANITIES (protocol code 715, 1 September 2021).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Baranova, T.; Kobicheva, A.; Tokareva, E. Web-based Environment in the Integrated Learning Model for CLIL-Learners: Examination of Students’ and Teacher’s Satisfaction. In Advances in Intelligent Systems and Computing, Proceedings of the Digital Science 2019, DSIC 2019, Limassol, Cyprus, 11–13 October 2019; Antipova, T., Rocha, A., Eds.; Springer: Cham, Switzerland, 2020; Volume 1114, pp. 263–274. [Google Scholar]
  2. Baranova, T.; Kobicheva, A.; Tokareva, E. The Impact of an Online Intercultural Project on Students’ Cultural Intelligence Development. In Knowledge in the Information Society. PCSF 2020, CSIS 2020. Lecture Notes in Networks and Systems; Bylieva, D., Nordmann, A., Shipunova, O., Volkova, V., Eds.; Springer: Cham, Switzerland, 2021; Volume 184. [Google Scholar] [CrossRef]
  3. Levy, M. Computer-Assisted Language Learning: Context and Conceptualization; Oxford University Press: Oxford, UK, 1997. [Google Scholar]
  4. Schofield, J.W. Computers and Classroom Culture; Cambridge University Press: Cambridge, UK, 1995. [Google Scholar]
  5. Beatty, K. Teaching & Researching: Computer-Assisted Language Learning; Routledge: Oxfordshire, UK, 2013. [Google Scholar]
  6. Chapelle, C.A. Computer Applications in Second Language Acquisition; Cambridge University Press: Cambridge, UK, 2001. [Google Scholar]
  7. Dudeney, G.; Hockly, N.; Pegrum, M. Digital Literacies: Research and Resources in Language Teaching; Pearson Education Limited: London, UK, 2013. [Google Scholar]
  8. Hanson-Smith, E. Technology in the Classroom: Practice and Promise in the 21st Century. TESOL Professional Papers#2; Teachers of English to Speakers of Other Languages, Inc.: Alexandria, VA, USA, 1997. [Google Scholar]
  9. Jaber, W.E. A Survey of Factors Which Influence Teachers’ Use of Computer-Based Technology. Ph.D. Thesis, Virginia Polytechnic Institute and State University, Blacksburg, VA, USA, 1997. [Google Scholar]
  10. Al-Juhani, S.O. The Effectiveness of Computer-Assisted Instruction in Teaching English as a Foreign Language in Saudi Secondary School. Ph.D. Thesis, University of Denver, Denver, CO, USA, 1991. [Google Scholar]
  11. Aşkar, P.; Yavuz, H.; Köksal, M. Students’ perceptions of computer assisted instruction environment and their attitudes towards computer assisted learning. Educ. Res. 1992, 34, 133–139. [Google Scholar] [CrossRef]
  12. Önsoy, S. Students’ and Teachers’ Attitudes towards the Use of Computer-Assisted Language Learning at the Preparatory School of Celal Bayar University. Ph.D. Thesis, Bilkent University, Ankara, Turkey, 2004. [Google Scholar]
  13. Hirvela, A. Computer-based reading and writing across the curriculum: Two case studies of L2 writers. Comput. Compos. 2005, 22, 337–356. [Google Scholar] [CrossRef]
  14. Min, B.C. A Study of the Attitudes of Korean Adults toward Technology-Assisted Instruction in English-Language Programs. Ph.D. Thesis, Northern Illinois University, DeKalb, IL, USA, 1998. [Google Scholar]
  15. Tang, L.H. Whole Language Instruction in the Computer-Assisted Instruction Setting: ESL Learners’ Attitudes. Ph.D. Thesis, Southern Illinois University, Carbondale, IL, USA, 1995. [Google Scholar]
  16. Yang, S.C.; Huang, Y.F. A study of high school English teachers’ behavior, concerns and beliefs in integrating information technology into English instruction. Comput. Hum. Behav. 2008, 24, 1085–1103. [Google Scholar] [CrossRef]
  17. Chen, Y.L. Modeling the determinants of Internet use. Comput. Educ. 2008, 51, 545–558. [Google Scholar] [CrossRef]
  18. Ludvigsen, S.; Mørch, A. Computer-supported collaborative learning: Basic concepts, multiple perspectives, and emerging trends. Int. Encycl. Educ. 2010, 5, 290–296. [Google Scholar]
  19. Ludvigsen, S.; Lund, K.; Oshima, J. International handbook of computer-supported collaborative learning. In Computer-Supported Collaborative Learning Series; Springer: Cham, Switzerland, 2021; Volume 19. [Google Scholar] [CrossRef]
  20. Fischer, G.; Rohde, M.; Wulf, V. Community-based learning: The core competency of residential, research-based universities. Int. J. Comput.-Support. Collab. Learn. 2007, 2, 9–40. [Google Scholar] [CrossRef]
  21. Stahl, G.; Koschmann, T. Computer-supported collaborative learning: An historical perspective. In Cambridge Handbook of the Learning Sciences; Cambridge University Press: Cambridge, UK, 2006; pp. 409–426. [Google Scholar]
  22. Al-Emran, M.; Mezhuyev, V.; Kamaludin, A.; ALSinani, M. Development of M-learning application based on knowledge management processes. In Proceedings of the 2018 7th International Conference on Software and Computer Applications, Kuantan, Malaysia, 8–10 February 2018; pp. 248–253. [Google Scholar]
  23. Puntambekar, S. Analyzing collaborative interactions: Divergence, shared understanding and construction of knowledge. Comput. Educ. 2006, 47, 332–351. [Google Scholar] [CrossRef]
  24. Weinberger, A.; Stegmann, K.; Fischer, F. Knowledge convergence in collaborative learning: Concepts and assessment. Learn. Instr. 2007, 17, 416–426. [Google Scholar] [CrossRef] [Green Version]
  25. Borge, M.; Ong, Y.S.; Rosé, C.P. Learning to monitor and regulate collective thinking processes. Int. J. Comput.-Support. Collab. Learn. 2018, 13, 61–92. [Google Scholar] [CrossRef]
  26. Darling-Hammond, L.; Hyler, M.E. Preparing educators for the time of COVID… and beyond. Eur. J. Teach. Educ. 2020, 43, 457–465. [Google Scholar] [CrossRef]
  27. Haake, J.M.; Pfister, H.R. Scripting a distance-learning university course: Do students benefit from net-based scripted collaboration? Int. J. Comput.-Support. Collab. Learn. 2010, 5, 191–210. [Google Scholar] [CrossRef]
  28. Sobreira, P.; Tchounikine, P. A model for flexibly editing CSCL scripts. Int. J. Comput.-Support. Collab. Learn. 2012, 7, 567–592. [Google Scholar] [CrossRef]
  29. Sellés, N.H.; Sanmamed, M.G.; Carril, P.C.M. Planning collaborative learning in virtual environments. Comun. Media Educ. Res. J. 2014, 22. [Google Scholar]
  30. Näykki, P.; Isohätälä, J.; Järvelä, S.; Pöysä-Tarhonen, J.; Häkkinen, P. Facilitating socio-cognitive and socio-emotional monitoring in collaborative learning with a regulation macro script–an exploratory study. Int. J. Comput.-Support. Collab. Learn. 2017, 12, 251–279. [Google Scholar] [CrossRef]
  31. King, A. Scripting collaborative learning processes: A cognitive perspective. In Scripting Computer-Supported Collaborative Learning; Springer: Boston, MA, USA, 2007; pp. 13–37. [Google Scholar]
  32. Capdeferro, N.; Romero, M. Are online learners frustrated with collaborative learning experiences? Int. Rev. Res. Open Distrib. Learn. 2012, 13, 26–44. [Google Scholar] [CrossRef] [Green Version]
  33. Garrison, D.R.; Cleveland-Innes, M.; Fung, T.S. Exploring causal relationships among teaching, cognitive and social presence: Student perceptions of the community of inquiry framework. Internet High. Educ. 2010, 13, 31–36. [Google Scholar] [CrossRef]
  34. Vuopala, E.; Hyvönen, P.; Järvelä, S. Interaction forms in successful collaborative learning in virtual learning environments. Act. Learn. High. Educ. 2016, 17, 25–38. [Google Scholar] [CrossRef] [Green Version]
  35. Dimulescu, C.; Nechifor, A. Accommodating English for Specific Purposes to Computer Assisted Language Learning. Bull. Transylvania Univ. Brasov 2021, 14, 5–18. [Google Scholar] [CrossRef]
  36. Yilmaz, F.G.K.; Yilmaz, R. Impact of pedagogic agent-mediated metacognitive support towards increasing task and group awareness in CSCL. Comput. Educ. 2019, 134, 1–14. [Google Scholar] [CrossRef]
  37. Gikandi, J.W.; Morrow, D.; Davis, N.E. Online formative assessment in higher education: A review of the literature. Comput. Educ. 2011, 57, 2333–2351. [Google Scholar] [CrossRef]
  38. Evans, C. Making sense of assessment feedback in higher education. Rev. Educ. Res. 2013, 83, 70–120. [Google Scholar] [CrossRef] [Green Version]
  39. Sellés, N.H.; Carril, P.C.M.; Sanmamed, M.G. La e-evaluación en el trabajo colaborativo en entornos virtuales: Análisis de la percepción de los estudiantes. Edutec. Rev. Electrónica Tecnol. Educ. 2018, 65, 16–28. [Google Scholar]
  40. Kwon, K.; Liu, Y.H.; Johnson, L.P. Group regulation and social-emotional interactions observed in computer supported collaborative learning: Comparison between good vs. poor collaborators. Comput. Educ. 2014, 78, 185–200. [Google Scholar] [CrossRef]
  41. Borge, M.; Mercier, E. Towards a micro-ecological approach to CSCL. Int. J. Comput.-Support. Collab. Learn. 2019, 14, 219–235. [Google Scholar] [CrossRef]
  42. Bowman, N.D.; Akcaoglu, M. “I see smart people!”: Using Facebook to supplement cognitive and affective learning in the university mass lecture. Internet High. Educ. 2014, 23, 1–8. [Google Scholar] [CrossRef]
  43. Hamid, S.; Waycott, J.; Kurnia, S.; Chang, S. Understanding students’ perceptions of the benefits of online social networking use for teaching and learning. Internet High. Educ. 2015, 26, 1–9. [Google Scholar] [CrossRef]
  44. Molinillo, S.; Aguilar-Illescas, R.; Anaya-Sánchez, R.; Vallespín-Arán, M. Exploring the impacts of interactions, social presence and emotional engagement on active collaborative learning in a social web-based environment. Comput. Educ. 2018, 123, 41–52. [Google Scholar] [CrossRef]
  45. Tarun, I.M. The Effectiveness of a Customized Online Collaboration Tool for Teaching and Learning. J. Inf. Technol. Educ. 2019, 18, 275–292. [Google Scholar] [CrossRef] [Green Version]
  46. Lyons, K.M.; Lobczowski, N.G.; Greene, J.A.; Whitley, J.; McLaughlin, J.E. Using a design-based research approach to develop and study a web-based tool to support collaborative learning. Comput. Educ. 2021, 161, 104064. [Google Scholar] [CrossRef]
  47. Strijbos, J.W.; Martens, R.L.; Jochems, W.M. Designing for interaction: Six steps to designing computer-supported group-based learning. Comput. Educ. 2004, 42, 403–424. [Google Scholar] [CrossRef] [Green Version]
  48. Riga, A.; Ioannidi, V.; Papayiannis, N. Study on Computer Supported Collaborative Learning in Greek Inclusive Secondary Education. Curr. Res. Lang. Lit. Educ. 2021, 2, 45–56. [Google Scholar]
  49. Rao, P.S. Collaborative learning in English language learning environment. Res. J. Engl. Lang. Lit. RJELAL 2019, 7, 330–339. [Google Scholar]
  50. Laal, M.; Laal, M. Collaborative learning: What is it? Procedia—Soc. Behav. Sci. 2012, 31, 491–495. [Google Scholar] [CrossRef] [Green Version]
  51. Li, J.; Mak, L. The effects of using an online collaboration tool on college students’ learning of academic writing skills. System 2022, 105, 102712. [Google Scholar] [CrossRef]
  52. Hakkarainen, K. Emergence of Progressive-Inquiry Culture in Computer-Supported Collaborative Learning. Learn. Environ. Res. 2003, 6, 199–220. [Google Scholar] [CrossRef]
  53. Oakley, B.; Felder, R.M.; Brent, R.; Elhajj, I. Turning student groups into effective teams. J. Stud. Cent. Learn. 2004, 2, 9–34. [Google Scholar]
  54. Aritzeta, A.; Senior, B.; Swailes, S. Team role preference and cognitive styles: A convergent validity study. Small Group Res. 2005, 36, 404–436. [Google Scholar] [CrossRef]
  55. Del Barco, B.L.; Mendo-Lázaro, S.; Felipe-Castaño, E.; del Río, M.I.P.; Fajardo-Bullón, F. Potencia de equipo y aprendizaje cooperativo en el ámbito universitario. Rev. Psicodidáctica 2017, 22, 9–15. [Google Scholar] [CrossRef]
  56. Leung, W.S. Bad blood: Managing toxic relationships through belbin roles for first year software engineering students. In Proceedings of the 3rd International Conference on Communication and Information Processing, Tokyo, Japan, 24–26 November 2017; pp. 82–86. [Google Scholar]
  57. Loughry, M.L.; Ohland, M.W.; Woehr, D.J. Assessing teamwork skills for assurance of learning using CATME team tools. J. Mark. Educ. 2014, 36, 5–19. [Google Scholar] [CrossRef]
  58. Pieterse, V.; Thompson, L. Academic alignment to reduce the presence of ‘social loafers’ and ‘diligent isolates’ in student teams. Teach. High. Educ. 2010, 15, 355–367. [Google Scholar] [CrossRef] [Green Version]
  59. Prichard, J.S.; Stanton, N.A. Testing Belbin’s team role theory of effective groups. J. Manag. Dev. 1999, 18, 652–665. [Google Scholar] [CrossRef]
  60. Aranzabal, A.; Epelde, E.; Artetxe, M. Monitoring questionnaires to ensure positive interdependence and individual accountability in a chemical process synthesis following collaborative PBL approach. Educ. Chem. Eng. 2019, 26, 58–66. [Google Scholar] [CrossRef]
  61. Vasquez, E.S.; Dewitt, M.J.; West, Z.J.; Elsass, M.J. Impact of team formation approach on teamwork effectiveness and performance in an upper-level undergraduate chemical engineering laboratory course. Int. J. Eng. Educ. 2020, 36, 491–501. [Google Scholar]
  62. Batenburg, R.; van Walbeek, W. Belbin role diversity and team performance: Is there a relationship? J. Manag. Dev. 2013, 32, 901–913. [Google Scholar] [CrossRef] [Green Version]
  63. Ciubuc, C.; Dascalu, M.; Trausan-Matu, S.; Marhan, A.-M. Forming Teams by Psychological Traits—An Effective Method of Developing Groups in an Educational Environment. In Proceedings of the 2013 19th International Conference on Control Systems and Computer Science, Bucharest, Romania, 29–31 May 2013; pp. 597–602. [Google Scholar] [CrossRef]
  64. Wu, X.; He, Z.; Li, M.; Han, Z.; Huang, C. Identifying Learners’ Interaction Patterns in an Online Learning Community. Int. J. Environ. Res. Public Health 2022, 19, 2245. [Google Scholar] [CrossRef] [PubMed]
  65. Hühn, C.; Rausch, A. Collaboration and emotions during simulation-based learning in general management courses. Stud. Educ. Eval. 2022, 73, 101130. [Google Scholar] [CrossRef]
  66. Fetiskin, N.P.; Kozlov, V.V.; Manuylov, G.M. Social and Psychological Diagnostics of Development of the Personality and Small Groups; Izd-vo Instituta Psikhotherapii: Moscow, Russia, 2002; p. 490. (In Russian) [Google Scholar]
  67. Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; Erlbaum: Hillsdale, NJ, USA, 1988. [Google Scholar]
  68. Dashkina, A.I.; Popova, N.V. Anglijskij Yazyk. Uchebnoe Posobie po Audirovaniyu Dlya Lingvistov. [English. Listening Comprehension Tasks for Linguistic Majors]; Course Book; Saint Petersburg Peter the Great Polytechnic University: Saint Petersburg, Russia, 2016. [Google Scholar]
  69. Lazovskaya, T.V.; Tarkhov, D.A.; Berezovskaya, G.A.; Petrischev, N.N.; Zulkarnay, I.U. Possibilities of neural networks for personalization approaches for prevention of complications after endovascular interventions. In Advances in Neural Networks, Proceedings of the ISNN 2017, Hokkaido, Japan, 21–26 June 2017; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2017; Volume 10261, pp. 379–385. [Google Scholar] [CrossRef]
  70. Zulkarnay, I.U.; Dashkina, A.I.; Tarkhov, D.A. Measuring the students’ level of knowledge on the basis of other students’ subjective opinions. J. Phys. Conf. Ser. 2018, 1044, 012043. [Google Scholar] [CrossRef]
  71. Jones, R.W. Learning and Teaching in Small Groups: Characteristics, Benefits, Problems and Approaches. Anaesth. Intensive Care 2007, 35, 587–592. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Smooth histogram of diagnostic test data distribution of three teams.
Figure 1. Smooth histogram of diagnostic test data distribution of three teams.
Sustainability 14 05908 g001
Figure 2. Smooth histogram of final test data distribution of three teams.
Figure 2. Smooth histogram of final test data distribution of three teams.
Sustainability 14 05908 g002
Figure 3. Smooth histogram of the difference between the results of the final and the diagnostic tests of three teams.
Figure 3. Smooth histogram of the difference between the results of the final and the diagnostic tests of three teams.
Sustainability 14 05908 g003
Figure 4. Neural network approximation of the dependency of the final test result on the diagnostic test data and personality traits criteria of forming the team.
Figure 4. Neural network approximation of the dependency of the final test result on the diagnostic test data and personality traits criteria of forming the team.
Sustainability 14 05908 g004
Figure 5. Neural network approximation of the dependency of the difference between the results of the final and the diagnostic tests of three teams on the diagnostic test data.
Figure 5. Neural network approximation of the dependency of the difference between the results of the final and the diagnostic tests of three teams on the diagnostic test data.
Sustainability 14 05908 g005
Figure 6. Smooth histogram of emotional engagement test data distribution of three teams.
Figure 6. Smooth histogram of emotional engagement test data distribution of three teams.
Sustainability 14 05908 g006
Figure 7. Smooth histogram of cognitive engagement test data distribution of three teams.
Figure 7. Smooth histogram of cognitive engagement test data distribution of three teams.
Sustainability 14 05908 g007
Figure 8. Smooth histogram of behavioral engagement test data distribution of three teams.
Figure 8. Smooth histogram of behavioral engagement test data distribution of three teams.
Sustainability 14 05908 g008
Table 1. Correlation analysis.
Table 1. Correlation analysis.
Group AEmotional EngagementBehavioral EngagementCognitive EngagementLearning Outcomes
Emotional engagement1
Behavioral engagement0.241
Cognitive engagement0.28 *0.33 *1
Learning outcomes0.36 *0.44 **0.221
Group BEmotional EngagementBehavioral EngagementCognitive EngagementLearning Outcomes
Emotional engagement1
Behavioral engagement0.171
Cognitive engagement0.210.28 *1
Learning outcomes0.31 *0.67 ***0.37 *1
Group CEmotional EngagementBehavioral EngagementCognitive EngagementLearning Outcomes
Emotional engagement1
Behavioral engagement0.261
Cognitive engagement0.38 *0.151
Learning outcomes0.35 *0.29 *0.171
Note: * p < 0.05; ** p < 0.01; *** p < 0.001.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Dashkina, A.; Kobicheva, A.; Lazovskaya, T.; Tokareva, E.; Tarkhov, D.; Guselnikova, I. Examining the Effectiveness of Computer-Supported Collaborative Learning for Language Proficiency Purposes. Sustainability 2022, 14, 5908. https://doi.org/10.3390/su14105908

AMA Style

Dashkina A, Kobicheva A, Lazovskaya T, Tokareva E, Tarkhov D, Guselnikova I. Examining the Effectiveness of Computer-Supported Collaborative Learning for Language Proficiency Purposes. Sustainability. 2022; 14(10):5908. https://doi.org/10.3390/su14105908

Chicago/Turabian Style

Dashkina, Alexandra, Aleksandra Kobicheva, Tatiana Lazovskaya, Elena Tokareva, Dmitriy Tarkhov, and Irina Guselnikova. 2022. "Examining the Effectiveness of Computer-Supported Collaborative Learning for Language Proficiency Purposes" Sustainability 14, no. 10: 5908. https://doi.org/10.3390/su14105908

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop