Sustaining E-Learning Studies in Higher Education: An Examination of Scientific Productions in Scopus between 2019 and 2021

Abstract

The study aimed to gain a comprehensive picture of current e-learning development in higher education. In order to do this, research questions including distribution patterns, contributors in terms of journals, authors and countries, as well as theme development and evolution pattens of e-learning were proposed. To address the research questions, a bibliometric analysis was carried out by reviewing 877 peer-reviewed journal publications published from 2019 to 2021 in the Scopus database, with the help of Biblioshiny and VOSviewer software. The results indicated a fast annual growth in the number of publications from 2019 to 2021, with social science as the core subject area for e-learning development. Sustainability (Switzerland) and Education and Information Technologies were the most contributing journals, due to their high citation numbers and productivity. China, Malaysia, and Saudi Arabia made the highest contributions to the e-learning field in terms of the publication numbers and citation indicators. “Acceptance factors”, “perceptions”, “mental health”, “teachers”, “university students”, and “e-learning management and organization” were interest topics discussed between 2019 and 2021. More empirical research was featured to address the issues of e-learning in 2019, and the research interests focused on motivation, administration, and pedagogical aspects. Meanwhile, from 2020–2021, it was observed that themes such as machine learning, artificial intelligence, and information and communication technology in the e-learning field were booming. The study provided extensive insight into the existing publications of the e-learning landscape; accordingly, the results could be useful to researchers, policymakers, and practitioners in this field.

1. Introduction

The idea of a virtual instructional mode in a digital environment originated in the 1990s, with the development and advancement of the World Wide Web and information technology [1]. Since then, this learning mode has become increasingly important as a support tool, lifelong learning modality, and supportive training in various educational sectors [2]. Recently, e-learning, one of the remote learning modes, has been further stimulated by growth in its adoption during the current COVID-19 pandemic [3], where e-learning was not an option or a supplementary tool for improving educational quality, but a necessity for ensuring the continuity of normal educational activities in the institutional systems [4].

Taken as a multidisciplinary nature, e-learning is contributed to from different areas, including mathematics, for instance [5]. Research shows that e-learning studies in education have grown continuously in the last decades [6]. As stated by Sobral [7], the rates of e-learning publication have shown a stable and fast increase from 2000 to 2019. Consequently, there is a vast body of literature accessible, detailing advancements in this sector in the last two decades. Furthermore, in the current pandemic condition, studies on e-learning in the COVID -19 context have been increasingly published [8]. By studying these existing studies, a basic picture of e-learning research will be presented, which offers a better opportunity to understand what has happened in the e-learning field.

Higher education plays an important role in e-learning throughout the educational system. Higher education, as the most dynamic sector of internal and external change, needs to redesign learning processes regularly to meet newly created pedagogical, socio-cultural, and socio-economic needs [9]. More undergraduate and graduate students enrolled on online courses as e-learning became more technologically and operationally available [10]. As a result, some large institutions began to offer courses and degrees via distance learning, which was a booming development [11]. On the other hand, financial constraints, reduced infrastructure building, and an increase in non-traditional student enrollment emerged as motivations for institutions to provide online programs [12]. The unanticipated breakout of COVID-19 also spurred higher education to adopt e-learning to sustain its teaching activities. Higher education institutions are increasingly adopting e-learning as a long-term strategy with technological developments [13]. Accordingly, universities share best practices for delivering course materials over the internet, engaging students, and administering evaluation [1]. With the increasing popularity of e-learning in higher education, research into this academic setting has been conducted in the last decades to shed light on the issues from conceptual and empirical perspectives. These perspectives included, but were not limited to, barriers to adopting an e-learning system [14]; factors influencing teachers’ e-learning acceptance [15]; e-learning readiness [16]; continuance satisfaction with e-learning [17]; and quality assurance of e-learning [18].

However, after careful examination of the e-learning literature, it was found that there is a scarcity of review research on e-learning productions focusing on higher education, especially with regard to the new dynamics of the literature landscape of e-learning development. Previous review studies mainly discussed the scientific productions of e-learning up to 2018, such as the reviews carried out by Bai et al. [19], Fatima and Abu [20], and Rodrigues et al. [6]. Meanwhile, few review studies were carried out to take a comprehensive look at e-learning development in higher education after 2018. Since it is essential to obtain and understand the latest information in a field by adding ongoing knowledge by assessing newly published productions [1], the study was designed to respond to the demand by providing the important performances and trend development of presently published scholarly productions within the e-learning academic landscape. Hence, as follow-up research, the study chose to look at the publications published between 2019 and 2021. To examine the properties and recorded information of previous literature on e-learning with bibliometric indicators, the research questions listed below will be addressed:

RQ1: What were the main distribution patterns of e-learning documents in higher education between 2019 and 2021?

RQ2: Who or what were the most contributing in terms of journals, articles, authors, and countries of the e-learning field in higher education between 2019 and 2021?

RQ3: What were the main topics of interest and topic evolution patterns of the e-learning field in higher education between 2019 and 2021?

2. Literature Review of Sustainable E-Learning Studies

With the rising interest in e-learning, scholars from different fields have carried out reviewing work from various perspectives over the past decades, which promotes the sustainable development of e-learning studies.

Some review work focusing on specific research domains of e-learning have contributed to the existing knowledge of e-learning, such as AI in online learning [21]; training work corporations with e-learning [22]; and e-learning within social virtual reality environments [9]. In the review research of artificial intelligence-supported e-learning [21], Hwang et al. visualized and interpreted 224 journal publications published between 1997 and 2019. The findings revealed the research changes in themes focusing on intelligent tutoring system integration 1997–2009, learner-oriented learning 2010–2014, and the interactions between learners and systems 2015–2019. In the study by Kaizer et al. [22], the authors reviewed the literature on instructional training planning by e-learning modality. The study showed that some important factors such as feedback, self-regulation and cultural profiles should be paid a lot of attention in e-learning training. For the social virtual reality environments, Mystakidis et al. [9] conducted a systematic literature review to explore e-learning effectiveness in social virtual reality environments, by looking at 33 empirical studies published between 2004 and 2019. The findings highlighted the positive roles of e-learning in social virtual reality environments.

Moreover, the researchers employed different research methods to carry out e-learning studies. The systematic literature review studies addressing the general theme studies of e-learning have been conducted continually. Berge and Mrozowski [23] explored the themes from the publications of the earlier stage in distance education between 1990 and 1999 by analyzing journal articles and theses abstracts. Martin et al. [24] analyzed publication trends and patterns, research themes, research techniques, and research locations between 2009 and 2018, and compared research results with those from earlier decades. Valverde-Berrocoso et al. [25] discovered that the three primary nodes of e-learning were curriculum-interactive learning environments, online teachers, and online students, by examining 248 publications published between 2009 and 2018. These systematic review studies mainly identified study areas, important research approaches, and theme development based on their specific research objectives and research questions, by text analysis. Though a significant amount of time is spent on these systematic reviews, which are generally considered to be confined to a small number of articles [2], this kind of review does provide detailed content descriptions of the published publications.

Bibliometric analyses also have been adopted, to seek a more comprehensive picture of e-learning studies with a more significant number of studies. For example, Tibaná-Herrera et al. [26] carried out a bibliometric analysis of 39,244 papers published between 2003 and 2016. They reported the studies’ main contributors in terms of authors, countries and affiliations, and university collaborative networks. Similarly, GÜRCAN & ÖZYURT [27] examined 27,735 journal articles in the e-leaning field published between 2008 and 2018 from the Elsevier Scopus database. In their study, five main dimensions were found, including teaching tools, teaching areas, teaching models, learning environments, and assessment, all of which made a significant contribution to e-learning research. Sweileh [28] conducted another bibliometric analysis with 4576 records published between 1966 and 2020, to examine the recorded information of e-learning studies related to health sciences education. The results of Sweileh indicated the increasing growth trend in publication number, and popular research topics such as blended learning and the flipped classroom; the highest contribution countries in the field were in European and American regions, as well as the journal contributors. More recently, with a total of 602 documents from 2020–2021 indexed in the database of Web of Science, a bibliometric review was conducted by Brika et al. [1], to comprehend in full scientific productions focusing on the relationship of “e-learning” “pandemic time” and “higher education”. Based on the findings, deep learning, machine learning, artificial intelligence, and pedagogical methodologies were emerging research hotspots during COVID-19. These bibliometric analyses generally dealt with data properties such as publication trends, productivities, impacts, topics, and themes with relatively large datasets.

The aforementioned review work suggest that a review study is beneficial and can assist novice researchers in gaining existing knowledge in one specific field efficiently and effectively [21]. Meanwhile, review studies also remind us that it is necessary and critical to continually update the existing knowledge of e-learning in higher education, in order to keeping up with these constant changes.

3. Materials and Methods

This study harnessed bibliometric analysis to provide an overview of existing knowledge in the e-learning field within higher education. The bibliometric analysis can describe publication information regarding the distribution of papers, ranks of authors, institutions, journals, and disciplines [29]. Additionally, it allows researchers to conduct scientific mapping with co-occurrence analyses, so that major research themes, research trends, and collaboration networks can be identified. Compared with the traditional literature review providing a deep exploration of the scientific production, bibliometric methods can handle a large number of productions with a graphical illustration and introduce a structured, clear, and repeatable examination procedure [30]. Based on the established practices of bibliometric methodology studies [30,31], this section mainly consists of research design, data collection, and data analysis.

3.1. Research Design

This section involved the selection of a bibliographic database, the identification of search criteria, and the selection of analyzing software tools.

3.1.1. Database Selection

To offer a larger and extensive picture of e-learning studies in higher education, the Scopus database, developed by Elsevier in 2004, was selected for the study. Firstly, the Scopus database provides huge and reliable data on global academic research with multidisciplinary and peer-reviewed documents [32]. Secondly, the Scopus database offers a broader time coverage, which is preferable for performing evolution and citation analysis [33]. Another consideration for selecting Scopus lies in its full bibliographic information of the literature provided by the database such as author, title, abstract, publishing year, citation, etc. [29]. Lastly, the various data formats exported from Scopus can be chosen based on specific studies to meet the requirements of the majority of bibliometric analysis software tools [34].

3.1.2. Search Criteria

Given the transdisciplinary and multidisciplinary nature of e-learning [2], scholars from different specialized fields might have their preferential terms over other concepts for e-learning. Therefore, to achieve a more comprehensive representation of the research in the e-learning field, it is crucial to determine synonym equivalents of e-learning before searching the literature [2,35]. Based on the research objectives of the study, “e-learning” and “higher education” were two critical concepts that needed to be mainly focused on. Therefore, for the first step, synonym expressions for e-learning for the study were identified by referring to a wide review of the pertinent literature [2,21,24,25,27]. The search queries related to e-learning were (“e-learning” OR “online learning” OR “distance learning” OR “mobile learning” OR “web-based learning” OR “web learning” OR “Internet learning” OR “virtual learning” OR “e-teaching” OR “online teaching” OR “distance teaching” OR “mobile teaching” OR “web-based teaching” OR “web teaching” OR “Internet teaching” OR “virtual teaching” OR “e-training” OR “online training” OR “distance training” OR “mobile training” OR “web-based training” OR “web training” OR “Internet training” OR “virtual training” OR “e-education” OR “online education” OR “distance education” OR “mobile education” OR “web-based education” OR “virtual education” OR “MOOC” OR “online open course” OR “massive open online course” OR “massively open online course”). Secondly, the synonym expression queries for higher education were determined, including (“higher education” OR “higher institution” OR “university” OR “college” OR “tertiary education”). Thirdly, the search equation was finally created by combining the search words or phrases “e-learning” and “higher education” by making use of Boolean operators “OR” and “AND”.

Only the publications from 2019 to 2021 were selected, based on the research objective and research questions. As stated earlier in this study, e-learning was considered multidisciplinary and cross-disciplinary, hence all topic areas were included. Regarding published source types, only journal papers were included, because they are subjected to a peer-review procedure [2], which is thought to yield more credible results [36]. This search also included literature in all languages.

3.1.3. Software Selection

The software tools Biblioshiny and VOSviewer were chosen for the analysis in this study. Biblioshiny was mainly used to describe bibliographic data, to investigate wide categories in analytics and graphs ranging from individual contributions to the social network [37]. VOSviewer, as a robust tool for mapping and visualizing network structure data, was mainly used to map and structure topics and themes in this study.

3.2. Data Collection

The data collection was carried out on May 15, 2022. A total of 4326 documents were initially found, based on the search equation provided in the research design section. Among these, 3447 publications were then excluded at this stage in line with the search criteria: the publication time was not from 2019 to 2021 (2814); the source type was not a journal (690), and the document type was not an article (43). The remaining 879 records from the Scopus database were exported in “CSV” (comma-separated value) format for further eligibility verification. After removing some records due to unidentified bibliometric information, duplicates, and irrelevance, the final 877 entries were saved for the following bibliometric analysis. As illustrated in Figure 1, the entire data collection tracking procedure was carried out utilizing the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) technique, using a four-phase flow diagram [38].

3.3. Data Analysis

In terms of data analysis, the study first presented basic characteristics of the dataset, then provided quantitative results by analyzing an annual publication pattern of e-learning, determining the most productive and influential authors, articles, journals, and countries, using the Biblioshiny and VOSviewer software. In addition to the performance analysis, this study displayed primary network characteristics, such as keyword co-occurrence, cluster, and thematic evolution analysis. The visualization findings provide a comprehensive image of the field and a structural comprehension of the most influential linkages in e-learning research trends in higher education.

The main information from the collected dataset is shown in

Table 1. Main information of the data collection.

Description of Main Information	Results
Documents	877
Timespan	2019–2021
Sources	405
Average citations per document	7.361
Author’s keywords	2197
Authors	2723
Authors of single-authored documents	158
Authors of multi-authored documents	2565
Single-authored documents	163
Authors per document	3.1
Co-authors per document	3.29
Collaboration Index	3.59

. The analysis period covers three years, from 2019 to 2021, with 877 scientific productions from 405 sources on e-learning. Looking at the authorship pattern, there are a total of 2723 authors from these documents with an average of 3.1 authors per document. 163 documents out of 877 were written by a single author, i.e., approximately 81% of these documents were written by more than one author, which is confirmed by the 3.29 co-authors per document and 3.59 collaboration index. Regarding citations, 480 documents received an average number of citations of 7.36. In addition, 2197 authors’ keywords were observed in the dataset.

4. Results

The bibliometric analysis results based on the three study topics are presented in the following section.

4.1. RQ1: What Were the Main Distribution Patterns of E-Learning Documents in Higher Education between 2019 and 2021?

The development evolution of the overall situation can be reflected in the annual distribution of document numbers [39]. From 2019 to 2021, many scientific productions were published in different forms, and the number of publications continued to increase exponentially, with an annual growth rate of 72.08%, as shown in Figure 2. The number of documents was 154 in 2019 in the Scopus database, with a sharp growth in the number of productions observed in the following 2020 and 2021, with 267 publications and 456 publications respectively.

The studies grouped the documents based on their subject area. As shown in Figure 3, the research on e-learning in higher education emerged in various subject areas. However, it predominantly fell into the social science field, which accounted for 42% (n = 631) of the total documents. It was followed by computer science (n = 249;17%) and engineering (n = 84; 6%). The subject areas of arts and humanities, business, management and accounting, and psychology, respectively accounted for approximately 5% of the total documents. There were fewer documents in the subject areas of veterinary, neuroscience, physics and astronomy, and earth and planetary sciences, which accounted for less than 1% of the total. As Gurcan et al. [2] reported, the research was grouped under each of these subject areas, because some research was conducted in more than one discipline.

4.2. RQ2: Who or What Were the Most Contributing in Terms of Journals, Articles, Authors, and Countries of the E-Learning Field in Higher Education between 2019 and 2021?

4.2.1. Contributing Journals Based on Productivity and Impact

A total of 405 journals were identified in this dataset as publishing articles on e-learning in higher education. The top ten journals with the most publications are shown in Figure 4. Sustainability (Switzerland), with 34 articles, ranked top, according to the index of the most relevant journals, which was followed by Education and Information Technologies (n = 28) and International Journal of Emerging Technologies in Learning (n = 23). Although each of the top ten journals published more than ten articles, it was also noted that 66% (266 out of 405) of journals published only one e-learning-based article in higher education.

The number of citations, as one of the most used approaches to quantify the influence of an author, article, or journal [40] was utilized to estimate the source impact on e-learning in this study. The top 13 most cited sources with more than 100 citations are shown in Figure 5. The Journals with more than 200 citations included Education and Information Technologies (n = 523), Sustainability (Switzerland) (n = 453), Computers and Education (n = 225), and Computers in Human Behavior (n = 210). It is interesting to see from the results that some journals with fewer publications received more citations, such as the journal Children and Youth Services Review, receiving 156 citations with one published article.

4.2.2. Contributing Articles Based on Impact

According to the overall analysis results, 247 articles (28 percent) received no citations, and 456 (52 percent) received fewer than two citations out of 877 papers for this study. It was discovered that only a few research articles received the most citations, making them the primary reference studies.

Table 2. Top 10 most cited articles.

First Author	Title	Total Citations	Publication Year
Hasan, N. [41]	Impact of “e-Learning crack-up” perception on psychological distress among college students during COVID-19 pandemic: A mediating role of “fear of academic year loss”	156	2020
Abbasi, S. [42]	Perceptions of students regarding E-learning during COVID-19 at a private medical college	151	2020
Coman, C. [43]	Online Teaching and Learning in Higher Education during the Coronavirus Pandemic: Students’ Perspective	145	2020
Demuyakor, J. [44]	Coronavirus (COVID-19) and Online Learning in Higher Institutions of Education: A Survey of the Perceptions of Ghanaian International Students in China	129	2020
Rlzun, M. [45]	Students’ Acceptance of the COVID-19 Impact on Shifting Higher Education to Distance Learning in Poland	89	2020
Chung, E. [46]	Online Learning Readiness Among University Students in Malaysia Amidst COVID-19	87	2020
Shahzad, A. [47]	Effects of COVID-19 in E-learning on higher education institution students: the group comparison between male and female	86	2021
Isaac, O. [48]	Online learning usage within Yemeni higher education: The role of compatibility and task-technology fit as mediating variables in the IS success model	70	2019
Fawaz, M. [49]	E-learning: Depression, anxiety, and stress symptomatology among Lebanese university students during COVID-19 quarantine	68	2021
Almaiah, M.A. [50]	Applying the UTAUT Model to Explain the Students’ Acceptance of Mobile Learning System in Higher Education	67	2019

displays the top ten most-cited articles: the three most-cited articles in the field of e-learning from Hasan and Bao [41], Abbasi et al. [42], and Coman et al. [43], received 156, 151, and 145 total citations, respectively. Considering all those papers were published from 2019 to 2021, they can be considered as high quality and as having most critical impact on the e-learning environment, enabling their fast dissemination and exposure to be cited.

Overall, nearly all of the ten articles most cited from between 2019 and 2021, focused mainly on different perspectives of e-learning in the COVID-19 context. These documents addressed the issues of psychological distress of college students during the COVID-19 quarantine; college students’ perceptions towards e-learning during the period of COVID-19; online learning readiness and acceptance; and the influence of COVID-19 on college student learning. The research of Hasan and Bao [41], the most-cited publication, evaluated and analyzed the influence of e-learning crack-up beliefs on psychological distress among college students during the COVID-19 epidemic. This study provided insight into how “e-learning crack-up” and “fear of academic year loss” affect college students’ mental health. Three documents out of the ten most cited discussed the perceptions of college students in terms of e-learning during the COVID-19 pandemic, such as Abbasi et al. [42], Coman et al. [43], and Demuyakor [44].

4.2.3. Contributing Authors by Productivity and Impact

Lotka’s Law [51] was used to determine the author productivity distribution based on the frequency of papers published on e-learning in higher education from 2019 to 2021. Based on the results using Lotka’s Law, in

Table 3. Author productivity frequency by Lotka’s Law.

Documents Written	No. of Authors	Proportion of Authors
1	2584	0.949
2	120	0.044
3	15	0.006
4	4	0.001

, 2584 (94.9%) of the total 2723 authors published a single paper in this area, 120 (4.4%) published two articles, and 15 (0.6%) published three publications on e-learning. Four individual authors authored the most publications, accounting for 0.1% of the total number of researchers.

As seen in

Table 4. Top ten authors based on productivity and impact.

First Author	NP	PY Start	First Author	TC	PY Start
Almalah, M.A.	4	2019	Bao, Y.	156	2020
Ramayah, T.	4	2019	Hasan, N.	156	2020
Zhang, W.	4	2019	Abbasi, S.	151	2020
Zhang, Z.	4	2019	Ayoob, T.	151	2020
Al-Adwan, A.S.	3	2021	Malik, A.	151	2020
Aznar-Daz, I.	3	2020	Memon, S.I.	151	2020
Devisakti, A.	3	2019	Bularca, M.C.	145	2020
Hinojo-Lucena, F.J.	3	2020	Coman, C.	145	2020
Li, Y.	3	2020	Mesean-Schmitz, L.	145	2020
Mpungose, C.B.	3	2020	Ru, L.G.	145	2020

NP = Number of Publications; PY = Publication Year; TC = Total Citation.

, this study also revealed the authors who had published the most articles based on Lotka’s Law [51]. Almalah, M.A., Ramayah, T., Zhang, W. and Zhang, Z. were the four most productive authors. They were active authors in the subject of e-learning. In addition, a citation analysis of writers was performed, as shown in Table 4. The top ten writers in the study field, out of 877, garnered more than 100 citations for their articles. With 156 citations, Bao, Y. and Hasan, N. were named the most influential authors in the field of e-learning from 2019 to 2021, followed by Abbasi, S. Ayoob, T. Malik, A. and Memon, S.I. Four individual authors authored the greatest number of articles, accounting for 0.1% of the total number of experts.

4.2.4. Contributing Countries by Productivity and Impact

The scientific output of one country in one discipline can be used to evaluate the country’s influence within that particular academic field [38]. During the chosen period, 102 countries or regions published articles on e-learning. As shown in Figure 6, the top five countries by publication number, in descending order, were China (144), Malaysia (135), Indonesia (125), the United States (101), and Saudi Arabia (87). China was the most active country contributing to the e-learning field in terms of the number of publications. Based on the results, six Asian countries (i.e., China, Malaysia, Indonesia, Saudi Arabia, Jordan, and India) were in the top 10 countries for the number of scientific productions.

China (672), Malaysia (486), Saudi Arabia (235), Pakistan (188), Romania (183), Spain (177), United States of America (175), Hong Kong (172), Poland (150), and South Africa (149) were the top ten nations or regions in terms of total citations in Figure 7. The enormous number of citations scattered throughout different geographical areas confirms the global impact of academic research on e-learning. During the 2019–2021 timeframe, China, Malaysia, Saudi Arabia, and the United States took the lead in both the total number of publications and citations in the e-learning sector.

4.3. RQ3: What Were the Main Topics of Interest and Topic Evolution Patterns of the E-Learning Field in Higher Education between 2019 and 2021?

4.3.1. Popular Topics of Interest

“Keyword” gives an enhanced summation and refining of the essence of an article, and some academics have stated that keyword analysis may assist scholars easily in seeing patterns in certain subjects [52]. “Author keywords” provide a more thorough representation of an article’s content, and are thus frequently utilized as a unit of analysis in bibliometric studies. As a result, “author keywords” were chosen as a unit of analysis to determine the emerging topics and present theme evolution in this study. Keyword co-occurrence analysis is a useful tool for displaying the structure of scientific knowledge, identifying hotspots in a field, and identifying research trends [31]. VOSviewer software was utilized in this study to analyze the author keyword co-occurrence map to uncover rising topics in the e-learning sector. The frequency of occurrence of these terms was utilized to find the most dominating themes. With seven times occurrence as the minimal criterion, 53 out of 2191 author keywords met the minimum requirements, indicating their importance in the e-learning studies (Figure 8).

The findings indicated six subjects identified by six distinct cluster colors, generated by a substantial connection between the terms in each cluster. Six themes were found, and each subject was assigned an appropriate topic name after eliminating the core terms connected to the search query. The six topics (

Table 5. Topics and keywords.

Topics	Keywords
Technology acceptance factors	blended learning, coronavirus-learning, higher education, m-learning, mobile learning, mooc, Moodle, pedagogy, structural equation modeling, tam, technology acceptance, utaut
Perception	acceptance, distance education, education, ict, learning, online, pandemic, perception, teaching, university
Mental health during COVID-19	anxiety, college students, COVID-19, educational technology, online education, self-efficacy, self-regulated learning, stress, university students
E-teaching during COVID-19	attitudes, challenges, COVID-19 pandemic, motivation, online teaching, online teaching and learning, universities, virtual learning
University students	distance learning, interaction, online learning, satisfaction, student engagement, student satisfaction, students
Management and organization	e-learning, higher education institution, learning management, open and distance learning, readiness, virtual learning environment

) were “acceptance factors”, “perceptions”, “mental health”, “teachers”, “university students” and “management and organization”.

4.3.2. Theme Development Trends

Thematic evolution analysis was performed, to further investigate the dynamic shift in research themes in the e-learning sector. The temporal interval (2019–2021) was separated into two sub-periods in this study, i.e., 2020 was utilized as a cut-off point to divide the timeframe (2019–2021) into the 2019 sub-period and 2020–2021 sub-period. In each sub-period, author keywords were employed as analytic units, and the results were displayed in a theme diagram, as shown in Figure 9 and Figure 10.

The graphical representation allows for the determination of four theme typologies, based on Callon centrality and density. Callon centrality denotes a given topic’s importance in the overall collection, whereas Callon density represents the extent to which a specific topic has evolved [53]. Depending on their quadrant, themes in the upper-right quadrant are categorized as motor themes, meaning they are significant and created for the study field. According to Aria et al. [53], themes in the lower-right quadrant are seen as basic and transversal themes, indicating that they are important in one domain while being fundamental to a study field. Themes in the lower-left quadrant are marginal and less developed or emerging; themes in the upper-left quadrant are of relatively weak importance, but well developed in one domain [53].

Based on the strategic diagram in the first subperiod (Figure 9), the “case study” in the upper-left quadrant was a well-developed theme in this period, given its high density. Interestingly, some research methods such as “logistic regression” and “qualitative research” were gaining popularity from the upper-left quadrant as a niche theme moving to a motor theme. “Academic motivation”, “administration”, and ”pedagogical issues” in the upper-right quadrant were well-developed themes with high density and high centrality in this phase. It was observed that the journals published many studies focused on empirical methods to address the issues of e-learning. During this time, no new themes emerged or faded. Different uses of the terms of e-learning, such as web-based learning and online learning was observed in the lower-right quadrant. “Academic performance”, “educational technology”, “technology acceptance factors”, and “university students” appeared as general and transversal themes discussed in this period, with low density and high centrality.

In the second sub-period (2020–2021) (Figure 10), emergency online teaching, multi-group analysis, machine learning, information and communication technology, and community of inquiry, gained more focus from the researchers. It became highly developed due to its high density. In addition, there was a growing tendency for these topics from isolated or peripheral themes in the upper-left quadrant to move to the motor themes in the upper-right quadrant, which meant that these topics were becoming developed and important for e-learning. College students and higher education institutions began to become emerging hot topics again in this period of 2020–2021, with low density and low centrality. Digital competency, mobile learning, motivation, satisfaction, and e-learning were still the transversal themes in this stage, as they were in the first stage. It was worth noting that the research related to the COVID-19 context became important for the e-learning field, and focused on some general topics cutting across the different research areas of this field.

5. Discussion

Bibliometric analysis was used to conduct a review study of scholarly publications in the field of e-learning produced between 2019 and 2021. The outcomes of this study offered quantitative insights into e-learning research in the higher education context.

5.1. The Main Distribution Patterns of E-Learning Documents

Regarding the distribution pattern, the number of publications maintained an exponential annual growth from 2019 to 2021. The result was in line with previous review work that reported that studies into e-learning gained ground from the early 2000s and have shown steady growth over the years [54]. The possible reason that e-learning studies have boomed over the years is the development of information and communication technology. Moreover, in the higher education area, the increase in the opportunities and challenges of e-learning during the COVID-19 epidemic, aroused the interests of scholars, generating a large surge of academic research into the field under study [8].

In terms of subject areas, the findings showed that e-learning was attracting researchers from different backgrounds. Based on the study’s findings, the top five subject areas were social sciences, computer science, engineering, arts and humanities, business management, and accounting. The results were aligned with factors highlighted by Sobral [7], where a significant increase was obvious in the arts and humanities and medicine over the decades, suggesting them to be major subject areas for e-learning studies. At the same time, the current study’s findings revealed some rather peripheral subject areas for e-learning studies, such as veterinary studies, neuroscience, physics and astronomy, and earth and planetary sciences, which lacked reporting in previous research.

5.2. The Contributors Based on the Bibliometric Indictors

For the contributive journals in e-learning, it is worth noticing that the most productive journals do not fully correspond to the most impactful journals. Sustainability (Switzerland, out of a total of 405 journals, ranked top, with the highest number of publications in the area of e-learning, followed by Education and Information Technologies and International Journal of Emerging Technologies in Learning. This indicates that these journals offered a larger space for studies related to e-learning from 2019 to 2020. Education and Information Technologies, Sustainability (Switzerland), and Computers and Education were impactful journals, due to their high number of citations. These journals from different domains indicated that scholars from multidisciplinary fields took an interest in e-learning studies, which supports the analyses results of e-learning research.

Regarding the impactful articles, the articles published by Hasan and Bao [41], Abbasi et al. [42], and Coman et al. [43], were considered important articles in e-learning studies, according to the indicator of citation. Given that the articles were published during 2019 to 2021 in the study, it can be inferred that the articles with high citations were of a high level, and that the issues discussed by these articles were core topics in the e-learning field. A careful examination of these impactful articles revealed that issues related to the COVID-19 context were highly focused on, such as the psychological distress of college students during the COVID-19 quarantine [41]. The results of the current study further confirmed the results of earlier work, suggesting that in recent years e-learning studies in the COVID-19 environment have been in a more important place [55]. According to Mishra et al. [55], the entire education system from elementary to tertiary level across the globe collapsed during the lockdown period of the novel coronavirus disease 2019, so studies of online teaching and learning in higher education were critical during the COVID-19 lockdown period. At the same time, the results of most contributing articles were aligned with the results of the popular themes, which indicated the studies related to COVID-19 were hot topics during this stage of 2019 to 2021.

As far as contributing countries are concerned, China, Malaysia, and Saudi Arabia were the top countries, with the most publications and highest number of citations related to e-learning during this study period. The results differed from those of earlier research, in which the United States and the United Kingdom generally led in the number of document publications and total number of citations [26,56]. The result implied that China contributed to e-learning as the most active and influential country. The quantity and quality of research output showed Chinese researchers’ growing interest in e-learning, possibly due to e-learning’s increasing popularity recently in China, and China being the first country to transform all instructional activities from offline to online, to avoid the spread of COVID-19 during the lockdown period. Moreover, the results indicated that Asian researchers were more active in committing themselves to sharing ideas or practices at this stage of the e-learning field, which supported the result from Hwang et al. [21], showing researchers from Asia were more focused on e-learning than those from other learning contexts.

5.3. The Main Topics of Interest and Topic Evolution Patterns

The results through an analysis of the co-occurrence frequency of author keywords of the study revealed that “acceptance factors”, “perceptions”, “mental health”, “teachers”, “university students” and “e-learning management and organization” were emerging topics of interest discussed in the field of e-learning during 2019 to 2021. Most topics found in the current study were consistent with those of the existing research including Djeki et al. [56]; López-Belmonte et al. [57]; Rodrigues et al. [5]. With the greater presence of such keywords as “structural equation modeling”, “tam”, “technology acceptance”, “utaut” in an online context, it revealed that more empirical studies with some acceptance models or acceptance theories were carried out in the study period. Researchers created many models to predict and explain teachers and students’ attitudes toward using digital technology in their teaching and learning practices. The results were in line with Valverde-Berrocoso et al. [25], who identified the theory acceptance model as the most relevant theoretical model. For “e-learning teachers”, teacher education and professional development are always basic themes in this field. Although e-learning is not new terminology, many teachers lack the preparation for switching from offline to online instruction [58]. Therefore, professional teacher training was necessary for a successful shift to online mode during the epidemic [59]. E-learning has features that are different from traditional learning. Accordingly, the characteristics of e-learners are also different.

For this reason, some characteristics of e-learners, such as self-regulation, motivation, academic performance, affection, cognition, and demographics became scholars’ research interests [24]. The topic of “management and organization” was one of the most studied research themes. However, it was one of the least studied themes found in the systematic review study of Martin et al. [24]. It was suggested that more organizational-level topics such as equity and leadership needed research within online learning [24]. Meanwhile, some major topics found by other research were also developed in the study, such as “learning algorithms” and “adaptive learning” [2] and “artificial intelligence” and “machine learning” [60].

There were some changes in theme trend development between 2019 and 2021. More empirical research, including quantitative and qualitative methodologies was featured to address the issues in e-learning in 2019, and research interests focused on motivation, administration, and pedagogical issues. The results supported those of Valverde-Berrocoso et al. [25], showing that more empirical studies with methodologies such as case studies, structural equation modeling, or design-based research were carried out, to deal with the various problems of e-learning. The basic themes in this phase were academic performance, educational technology, technology acceptance factors, and university students. Meanwhile, in the stage 2020 to 2021, it was observed that themes such as machine learning, artificial intelligence, and information and communication technology in the e-learning field were booming, and becoming critical. These themes can be found in the study of many scholars, such as Rasheed and Wahid [61] and Kashive et al. [62]. Another remarkable change in research topic tendency during 2020 to 2021 was the fact that keywords such as a pandemic, COVID-19, and COVID, stood out. The new keywords revealed the influence of COVID-19 on e-learning, which was confirmed by previous studies [63,64] For example, the study of Marinoni et al. [64] reported the impact of COVID-19 on institutions and campus activities, based on a survey from 109 countries.

6. Conclusions, Limitation and Implication

A bibliometric analysis was adopted, to acquire a bigger picture of the current existing state of e-learning literature in higher education, by assessing 877 journal articles indexed in the Scopus database. The main findings of the study were presented as follows: a fast growth from 2019 to 2021 in the number of e-learning publications; Sustainability (Switzerland) and Education and Information Technologies were the most contributing journals, due to their high number of citations and their productivity; Almalah, Ramayah, and Zhang were core authors, based on Lotka’s Law; Asian countries played the leading roles in terms of publication numbers and citation indicators; “Acceptance factors”, “perceptions”, “mental health”, “teachers”, “university students”, and “e-learning management and organization” were interest topics; motivation, administration, and pedagogical aspects were focused on in 2019, while, in the stage 2020 to 2021, machine learning, artificial intelligence, and information and communication technology in the e-learning field were booming. The study contributed to the existing knowledge by adding bibliometric research on e-learning in higher education, from 2019 to 2021.

Limitations exist in the study, although some rigorous procedures have been conducted. Firstly, the data were determined by searching for the title of the documents for result accuracy and relevance. Therefore, this procedure might exclude some important studies. Secondly, due to the different formats of different databases, it is hard to combine a large amount of data from different databases, and so the study selected the Scopus database as the source of data. Added to all these factors, because theme mining in this bibliometric analysis was confined to author keywords rather than the entire text, some major concepts and theme developments may differ from data mining of the entire text. Given the constraints stated above, recommendations are made for further bibliometric review research efforts. Firstly, the data coverage may be expanded by including other databases for comparative studies, such as the Web of Science database. Similar studies can be carried out to examine the change tendency of the field at different stages. Secondly, it is hoped that the use of advanced topic modeling algorithms and analyzing tools will discover richer research findings in the e-learning field, by digging deeper into textual information within larger datasets.

Despite some limitations, the quantitative findings provided important insights into the existing state of e-learning development, and therefore it hopes to provide a reference for stakeholders such as future researchers, institutions, policymakers, and practitioners working in the e-learning field. With e-learning gaining growing popularity in more types of fields, it is important for a nation or educational system to prioritize investment in infrastructure construction for information and communication technology and the internet, to guarantee the smooth implementation of e-learning. For higher institutions, it is necessary to consider e-learning development as a long-term strategy for institution development in the long run, because of e-learning’s important role in higher education, such as in lifelong study for untraditional students. Moreover, the findings could help institutions to introduce sustainable policies related to e-learning, such as teachers’ professional development training in e-learning, for example. In addition, the study may serve as a comparison reference for interested scholars to carry out prospective research in the e-learning field. Lastly, as a multidisciplinary and interdisciplinary field, it is suggested that researchers try out various research methods and designs, to provide a deeper and more comprehensive insight into the constantly updated changes within e-leaning development in higher education, so that e-learning studies can continue to be sustainable.