Do We Perform Systematic Literature Review Right? A Scientific Mapping and Methodological Assessment

Abstract

Background: Systematic literature review (SLR) is increasingly utilized to maximize the element of rigor and minimize the individual bias of research synthesis. An analysis of the Web of Science (WoS) database indicates that 90% of the literature review studies using SLR have been published between 2012 and 2022. However, this progressive agenda is impaired by the lack of methodological consistency and rigorousness. To fill this gap, this paper aims at mapping the theoretical comprehension and practices of SLR and providing a stepwise approach to employing such a framework. Methods: A comprehensive narrative review is used in this paper to analyze the studies concerning the literature review typology and the structural assessment of the SLR. Furthermore, the methodological approach of the literature review studies that adopted the SLR and were published in the Logistics journal is assessed across a set of vital criteria associated with conducting an SLR. Results: There is a concrete link between the purpose of a review, i.e., to describe, test, extend, or critique, and the literature review type. There are 17 distinct literature review types, e.g., a narrative review, a bibliometric analysis, etc., which must be justified meticulously regardless of the SLR. The ambiguity in conceiving the SLR either as a toolkit or a review type, the lack of justification regarding the review purpose and type, and vague conceptual distinguishment between the bibliometric analysis, as a distinct review type, and the SLR framework, are only a few of the shortcomings observed in the analyzed papers. Conclusions: Given the significant role of SLR in elevating the element of rigor within the literature review studies, it is deemed essential to employ this framework by paying attention to two holistic factors: (1) theoretical distinction between the literature review purpose, the literature review type, and the SLR; (2) strict adherence to the SLR procedure with a high degree of accuracy and explicitness.

1. Introduction

“If I have seen further, it is by standing on the shoulders of Giants.” It is a famous metaphor stated by Isaac Newton (1642–1727 A.D.), in his letter to Robert Hooke, in 1675 [1]. Nevertheless, the use of this quote is originally attributed to the French philosopher Bernard of Chartres, in the 12th century A.D., who symbolized the scientists as dwarfs on the shoulders of giants, such that they have better sight and could see farther because they are raised higher (not because of more physical and intellectual capabilities) [2]. This statement or aphorism, which gained rich attention after the Newton’s letter, is a simile that depicts the progress of science as “discovering truth by building on previous discoveries” [3]. Aside from the scientific contributions of individuals, this metaphor goes hand in hand with the accumulation of knowledge (as an essential condition) and the practice of the literature review which are in favor of a better comprehension of the research breadth and the development of a scientific field [4,5]. This scientific approach is interchangeably known as research synthesis which has played an important role in the development of knowledge throughout the past two centuries within several fields, e.g., physics, medicine, sociology, etc. [6]. In this context, conducting a literature review seeks to illustrate the body of knowledge on a topic using the extant literature, which contributes to the field by elucidating new perspectives and altering/adding concepts [2,7]. A literature review may be accomplished for various purposes, such as elaborating on the theoretical background of a research work and an empirical study—equivalently known as a primary study; however, from a broader perspective, a literature review refers to a distinct research contribution, commonly known as a secondary study, which aims at reviewing and synthesizing the accumulative information of primary studies related to a specific research question [8].

In order to provide the research community with valid and reliable findings, conducting a literature review study in the absence of biased interpretations and information synthesis is a significant factor [9]. Given the adverse impact of human subjectivity, this agenda echoes the necessity of a rigorous and consistent methodology in conducting literature reviews with the same amount of ”conscious objectivity” as other scientific studies [7,10]. To that aim, scholars have widely discussed the practice of systematic literature review (SLR) which contribute to minimizing biased interpretations and providing more reliable findings [11,12]. This approach was initially introduced as systematic reviews in the 1970s within the medicine realm, and then it drew wide attention of academia in the late 1990s and was used to describe specific systematic approaches for research synthesis [4,6,13,14]. Over the years, many scientific domains have taken advantage of the SLR; however, the element of creativity and innovation, denoted as “art”, have engendered the variation in the typology and, thus, resulted in the lack of conceptual coherency associated with the employment of the SLR [4,14]. Although there is a consensus on the perception of the SLR as a set of procedures, academicians often incorporate it into their literature review studies with translucent explanations of their literature review type and methodology. More precisely, researchers within the philosophy of science as well as methodological studies, have referred to the SLR as a tool rather than a subset of literature review taxonomies [6,7]. However, this distinction has barely received attention in many review studies, and scholars mainly employ the SLR as a type of literature review (not as a tool); even though the essential principles of the SLR have been followed in those studies. This reveals a conceptual paradox and inconsistency associated with the use of SLR, despite the fact that there is a strong emphasis on the alignment between the methodological requirements and the objectives of the research synthesis [15,16]. To illuminate this blurry and interweaving conceptual link between the SLR and the literature review types, there are several studies which can be used to elaborate on either of those aspects, e.g., those by Paré et al. [4], Okoli and Schabram [7], Xiao and Watson [9], Templier and Paré [16], Cooper [17]. However, the incremental trend of publications employing SLR shows inadequate and vague explanations of the selected methodology in conjunction with the review goal and the review type. Thus, it is deemed essential to put more effort into this direction, and this study sets the primary objective to stress the distinction between the typology of literature review and SLR and to present a practical instruction for the SLR application. To realize this objective, particularly with a focus on engineering and management, the following questions were developed: (1) What is the SLR, and how can researchers benefit from the SLR framework? (2) How do researchers use SLR in the literature review studies?

To answer these research questions, we primarily sought to elucidate the scientific contribution of SLR to the domain of research synthesis and offer a transparent and practical comprehension of this concept. This was followed by establishing an overarching standardized guideline that would suit the scholars of the engineering and management fields for employing SLR in the literature review studies. To further validate the established guideline, a comparative assessment of the literature review studies, which employed SLR and were published in the Logistics journal was then performed. Logistics has become a rapidly developed and heavily focused industry, where SLR has been increasingly used to provide academic researchers, practitioners, and decision makers with critical insights toward technology adoption, sustainability, and circular economy. However, the lack of methodological consistency may hinder the replicability of the research and the rigor of the conclusion obtained. Thus, our research aims specifically at identifying and analyzing the common pitfalls of these SLR studies. This can help researchers, particularly in logistics, to effectively avoid them and better use SLR in future literature review studies to derive replicable and reliable insights, which may further enhance the sustainable practices in logistics industry.

The remainder of this paper is organized as follows. Section 2 explains the research design and method to achieve the study goal. Section 3 concisely illustrates the philosophy of the research synthesis and the accumulation of knowledge, which is followed by the discussions regarding the literature review types. Section 4 presents a thorough and lucid theoretical and methodological perception of the SLR, by investigating the conceptual fundaments and proposing a solid and smooth framework for the application of this methodology. Section 5 examines the methodological approach of the literature review papers employing SLR—selected from the Logistics journal—compared to the devised procedures and aspects surrounding SLR. Section 6 concludes the paper.

2. Research Method

The first research question is concerned with delivering a transparent conceptual and methodological comprehension of the SLR, and the associated discussions and findings in this regard are presented in Section 3 and Section 4. This agenda, first and foremost, necessitates gaining the knowledge surrounding the philosophy of research synthesis which is considered as the theoretical realm of SLR. Thus, we initially refer to the books and journal papers that contribute to the domain of philosophy of science in order to trace the theoretical footsteps of the research synthesis. Moreover, and according to the blurry perception of the SLR either as a tool or a as a literature review type—this debate was highlighted in Section 1—we conduct a narrative review to initially elaborate on the typology of literature review, and furthermore, to elucidate the SLR practice in the domain of the literature review studies. In response to the first research question, we secondly seek to study the SLR principles from the methodological standpoint. Therefore, we investigated the scientific motivation of employing the SLR in a literature review study, which was then followed by a comprehensive narrative review to assess the methodological framework of using the SLR. In this regard, we benefit from studies regarding the employment of SLR in several fields, e.g., computer science [8], urban planning [9], information systems [7], and so forth, and, thus, present a more versatile guideline that suits the engineering and management fields. It is noteworthy to mention that studying the philosophy of the research synthesis reveals the overarching exercise of science mapping—equivalently known as the bibliometric analysis—as one of the prominent approaches to performing a literature review study, which is widely discussed and practiced in conjunction with SLR. Given the multifaceted attention to SLR within the scope of this study, we conducted a narrative review to study the typology of the bibliometric analysis and complement the preceding discussions pertaining to the literature review types. In this regard, the methodological framework of the bibliometric analysis is also studied, and by taking advantage of the proposed SLR guidelines, we compiled a concise instruction on the delivery of a rigorous bibliometric analysis.

In pursuit of addressing the second research question, we focused on the Logistics journal and provided an assessment of the current SLR practices from the published literature review studies in Section 5. This concern demanded the establishment of a standard protocol to accomplish an objective evaluation. For this purpose, we utilized the SLR guidelines proposed in response to the first research question and produced a list of criteria which reflect the fundamental steps and aspects associated with employing the SLR. Hence, we present a cross-dimensional assessment by comparing the targeted review studies—published in the Logistics—based on the selected criteria. It is worthwhile to highlight that the entire process of retrieving the relevant documents from the online database of the Logistics and analyzing the essential data, were accomplished in accordance with the proposed SLR principles.

3. Accumulation of Knowledge and Literature Review

3.1. The Philosophy of Research Synthesis

Thomas S. Kuhn in his book of 1962, titled The Structure of Scientific Revolutions, expounded the evolution and progression of science from a philosophical standpoint and defined several decisive and phenomenal terms and concepts. Kuhn [18] emphasized the accumulation of knowledge as an essential component of scientific development and referred to normal science as “research [which is] firmly based upon one or more past scientific achievements, achievements that some particular scientific community acknowledges for a time as supplying the foundation for its further practice.” Among other concepts that he introduced and expatiated on, i.e., paradigms, puzzle solving, anomaly, etc., whose interpretation does not fall within the scope of this paper, this comprehension of the scientific progress embraces the element of research synthesis. As also highlighted by Small [19] in his speech for the ASIS Award in 1999, Kuhn’s study elucidated the significance of revealing the growth of science by investigating the snapshots of paradigms [1]. This channel of expanding the science boundaries relies on the means of conveying the knowledge gained by humans to the successor generations of the time, without compromising the significance of specific and technical scientific contributions of individuals. Robert King Merton, an American scholar in sociology, acknowledged this in his book titled On The Shoulders of Giants, where he studied the historical roots of this famous metaphor. In this regard (inspired by Bernard of Charters), Merton [2] argued the scantiness of successors’ bright minds to know more, and he put emphasis on transferring the accumulated knowledge in order to establish a foundation for them to see farther. This concept leads us to the realm of research synthesis, which is not a new academic effort in the history of science, and the research community equivalently recognizes it as the “standalone review”, as articulated by Chalmers et al. [6].

The term ”standalone” necessitates the discussion regarding the categorization of literature reviews, which entails three sorts of contribution according to the scopes of studies [7]: the first category serves as a section of a primary study, e.g., a journal article, commonly denoted as a literature review, which aims at presenting and investigating the background of the scientific contributions related to the topic of the research; the second category corresponds to a distinct chapter of a master’s/doctoral dissertation that is written in pursuit of an educational degree; the third category, recognized as the standalone literature review, is a thorough and complete research work, which is mainly published as a journal-length article to evaluate and summarize the extant knowledge and evidence surrounding the topic. Notably, the last described category plays an important role in ”fostering fields” and forms a key knowledge source for new scholars and academicians, so that review studies with high quality are potentially turned into the ”core” papers of the research area [4,16,20,21]. For this reason, this paper only concerns this sort of literature reviews, which is also referred to as the secondary study [8]. Thus, the term ”literature review” in the remainder of this study refers to the standalone literature review. It is noteworthy to mention that the differentiation between the first and the second category of the review studies is justified based on the expertise of student authors and experienced scholars, in addition to the scopes and the purposes of such studies, particularly, doctoral programs, as discussed by Boote and Beile [22], Hüttner [23], Maxwell [24], Rempel and Davidson [25].

3.2. Literature Reviews and Typology

A literature review inherently utilizes the data and information extracted from the retrieved materials to produce a trustworthy collective picture of the recorded research works following various goals, i.e., to develop a theory, to provide a conceptual background for subsequent studies, to illustrate the breadth of the research within the domain of interest, to review the application of a theory or model, and so forth [4,17]. Hart [26] referred to the literature review as “the selection of available documents (both published and unpublished) on the topic, which contains information, ideas, data and evidence written from a particular standpoint to fulfill certain aims or express certain views on the nature of the topic and how it is to be investigated, and the effective evaluation of these documents in relation to the research being proposed.” This definition sheds light on three important aspects of conducting a literature review, i.e., the purpose of the study, the selection of the documents, and the effective assessment of the data and information, which are interconnected to one another and shape the grounds for discussion regarding the typology of literature reviews. These aspects were elaborated on by Paré et al. [4] in a methodological review study, in which they distinguished the literature review types based on seven dimensions: the overarching goal of the review, scope of the review question, search strategy, nature of the primary sources considered for the search strategy, explicitness of the study selection, quality appraisal, and methods for synthesizing the information. According to the evaluation of 20 studies, including the ones by Rousseau et al. [10], Tricco et al. [14], etc., they developed a typology spectrum embracing nine sorts of literature reviews, i.e., a narrative review, a descriptive review, a scoping review, a meta-analysis, etc. In another study conducted by Templier and Paré [16], which was mainly inspired by Paré et al. [4], Rousseau et al. [10], Cooper [17], etc., the authors set the boundaries between the literature review types by emphasizing the review goal and the rigor of the methodology. In this regard, they proposed four types of literature reviews, namely, a narrative review, a developmental review, a cumulative review, and an aggregative review, which are differentiated mainly based on three measures: the variation in research findings (input), the type of research questions (process), the expected outcomes (output). In a thorough review study regarding the methodology and typology of the literature review, Xiao and Watson [9] investigated 22 papers and presented a quaternary categorization that was mainly developed according to the purpose of the review: to describe, test, extend, or critique. Given the explicit articulation of their proposed typology, we based our discussion regarding the literature review types on their findings particularly in conjunction with the goal of the review.

3.2.1. Literature Review Types

The most common approach of a review is to “describe”, which seeks to illustrate the status quo of a topic or concept and summarize the extant literature pertinent to the research question, rather than to expand the investigated domain. According to the procedures and techniques that correspond to data extraction and synthesis, the category of the descriptive review entails five types of literature reviews:

• Narrative Review: It is by far one of the most utilized and recognized approaches for conducting literature reviews. However, it is being referred to as a nonrigorous and inefficient—in terms of the invested time and resources—review type, which may potentially be impacted by the reviewer’s subjectivity [27,28], not to mention that the data extraction and information synthesis are accomplished without the compliance with any standard and systematic framework. Notably, some studies interchangeably use narrative review to refer to descriptive (with the purpose to describe) reviews [4,16].
• Textual Narrative Review: Compared to a narrative review, this type of the literature review incorporates a more standard approach to the data collection process by paying attention to different characteristics of each document, and it also benefits from the coding strategy to facilitate more systematic information synthesis [29,30].
• Meta-Summary: This type of review accommodates a quantitative element by calculating the frequency and intensity of “the effect size” through the categorization of findings, also denoted as vote counting [30]. The primary predecessor in this context, is the thematic grouping of the information derived from each piece of the included literature. Thereafter, the frequency of the effect size is calculated by dividing the number of documents associated with each theme by the total number of documents, which is expressed as percentage [31]. Accordingly, the intensity of the effect size aims to examine the reliability of calculations; for instance, to investigate which documents had the most contribution to the findings, which is determined by the division of the number of findings in a certain document by the total number of findings [31].
• Meta-Narrative: Inspired by the notion of normal science from Kuhn [18], Greenhalgh et al. [32] elaborated on a meta-narrative by paying attention to investigating the research traditions associated with the same problem. Among the six steps considered for this review, the “mapping” and the “synthesis” phases are of particular importance that distinguish this approach. The former mainly aims to identify the key elements of the research paradigm, i.e., conceptual, theoretical, methodological, etc., and to unveil the key actors and milestones pertaining to the tradition; the latter, on the other hand, articulates the core aspects of the problem and presents a “narrative account” of the contribution that each identified aspect has made to the research tradition [32].
• Scoping Review: A scoping review mainly puts forward a wide and inclusive image of the research domain, which in turn, necessitates a high degree of data extraction from relevant documents [33]. It is worthwhile to note that the comprehensiveness criterion is satisfied at the expense of the quality of utilized research documents [4]. Such an approach is beneficial for shedding light on the breadth of a concept, available evidence surrounding an issue, as well as potential research gaps [9,34].

The purpose of the second category of the review types is to “test”. The purpose of such a review is to find the answer to a particular question regarding the extant literature or to examine a specific hypothesis. In this regard, four review types underpin this approach which are mainly differentiated based on the literature type, i.e., qualitative, quantitative, and mixed, that is supposed to be investigated and analyzed.

• Meta-Analysis: This is a quantitative approach which was initiated by Glass [35], and it seeks to perform a statistical examination based on the data collected from the retrieved articles. A major form of this approach is the meta-regression which investigates the relationship between the dependent variables—a summary statistic derived from each article—and the independent variables—could be characteristics of the method or utilized data—associated with the collected empirical research items [36].
• Bayesian Meta-Analysis: It is a literature review type that incorporates qualitative studies into a meta-analysis procedure and aims at determining the significance of some factors—identified by experts—on an outcome [37,38,39]. As explained by Roberts et al. [39], the potential factors that impact the outcome are initially ranked by the judgment of experts, which are then revised according to the qualitative literature. While there is a prior probability pertinent to each factor based on the score received from different experts, they are then merged with the quantitative evidence—coded data derived from the quantitative literature—to generate a posterior probability.
• Realist Review: A useful instrument to assess the exerted policies, and it seeks to find the best compromise between “what works for whom, under what circumstances/conditions, and how” [9]. According to the testing nature of this approach, various conditional sentences form the grounds for this analysis, such as “if A, then B, or in the case of C, D is unlikely to work” [40].
• Ecological Triangulation: It seeks to “determine what evidence across cases (articles) do theory, method, and the analysis of persons and conditions support interventions with positive results” [41]. Thus, it is similar to the realist review, and the data extraction utilizes similar questions which are then presented and organized in the form of a matrix consisting of different attributes, for example, one axis representing the study participants while the other marking the different contextual attributes [9].

To “extend” is the purpose of the third category of the literature review types. This category of reviews sets a high-level objective that utilizes the recorded data and research works to generate new aspects or expand the boundaries of the studied subject, equivalently known as “theory-building” [9]. As emphasized by Xiao and Watson [9], studies incorporated into these reviews are supposed to be similar enough to generate reliable and credible third-order constructs. In a similar manner to that of testing reviews, this purpose of review is supported by specific review types which are categorized according to the type of the literature used.

• Meta-Ethnography: This type of literature review has evolved from a technique of qualitative research synthesis, which in turn, shapes the grounds for further review types associated with the review purpose of extend [9,42]. As proposed by Noblit and Hare [43], meta-ethnography is approached by revealing the relationship between the concepts that are derived from each study. In particular, the studies are translated to one another; a process that is recognized as “unique syntheses, because they protect the particular, respect holism, and enable comparison. An adequate translation maintains the central metaphors and/or concepts of each account in their relation to other key metaphors or concepts in that account” [43].
• Thematic Synthesis: In this review type, similarly to the meta-ethnography, the data extraction and information synthesis are conducted thematically based on a principle, where the analytical themes are derived from clustered information [44]. In order to satisfy the extension aim, Thomas and Harden [44] referred to this synthesis process as “It may be, therefore, that analytical themes are more appropriate when a specific review question is being addressed (as often occurs when informing policy and practice), and third-order constructs should be used when a body of literature is being explored in and of itself, with broader, or emergent, review questions”.
• Meta-Interpretation: Rather than a research question, this literature review type deals with a research area to access a wide range of studies and discovers a range of contrasting studies [9,45]. Therefore, more literature searches will be conducted until the authors observe the “theoretical saturation” pertinent to the subject [45].
• Meta-Study: This novel approach, as presented, operationalized, and elaborated on by Paterson et al. [46] and Zhao [47], is a composition of three methods: a metadata analysis, a meta-method, and a meta-theory. Similarly to the meta-ethnography, a metadata analysis seeks to analyze a wide range of studies, however, from a quantitative approach and by studying the results of the data analysis research works [47]. While a data analysis processes “raw data”, a metadata analysis incorporates the “processed data” into the investigation which is denoted as “the analysis of analyses” [35,47]. The major focus of the meta-method is directed towards the methodological approach that was selected by the authors of the collected research articles to establish a connection between the outcomes, ideology, and methods [46]. Beyond the aims of earlier approaches, a meta-theory is an established philosophical approach that investigates the theoretical assumptions, the underpinning paradigms, and the quality of the theories considered in each study [42,46,47]. The composition of the outlined approach within the framework of a meta-study is not bounded to a standard structure, and it is rather dynamic and iterative [42].
• Critical Interpretive Synthesis: This literature review type, by adding critical attribute to the meta-ethnography, incorporates extensively a broad range of studies without any particular “quality appraisal” to investigate any potential impact of the research traditions on the evidence and outcomes presented in the studies [48]. Given the qualitative inherence of this approach, it is iterative and exploratory rather than being confined to a standard procedure [48].
• Framework Synthesis: The term “framework” is emanated from the coding structure of this literature review type which is articulated based on the devised research question and prior to commencing the review process [49,50]. Thus, through the investigation of the literature, the developed conceptual structure/codes would be further modified and extended to generate the ultimate list of factors [50]. Even though this approach is suggested for qualitative research, it is argued by scholars that it should be applied to other literature types as well [9].

The purpose of the last category of literature reviews is to “critique” which is accomplished through using only one recognized review type, the so-called critical review. The aim of this approach is to compare the scientific findings against the corresponding established list of criteria and make a judgment across a value stream of acceptable or not acceptable [9].

The inferences developed by a literature review study are potentially influenced by several technical dynamics, such as the tools utilized for data synthesis, the choice of database for document retrieval, the date of literature search, and so forth [34]. However, from the tactical perspective, the process of literature review is a function of the review type, which is in turn determined by the purpose of review. In this regard, and inspired by Xiao and Watson [9], we shed light on 16 literature review types that were articulated and distinguished with respect to the purpose of the review, namely, to describe, test, extend, or critique. Although there are some review types that have been more frequently used by scholars, e.g., scoping review, meta-analysis, narrative review, etc., it is common to consider a hybrid approach in order to better satisfy the concerns of the review [4,9]. For instance, the scoping review is a comprehensive method that is founded on the breadth of the extant literature rather than the depth, which necessitates the assessment of a wide span of documents to identify the extent or range of research activities, or research gaps within the topic of interest [4,33,51]. On the other hand, the meta-analysis entails a statistical study of the incorporated research items following several aims, i.e., precise estimation of an incident or phenomenon, explanation or cause-and-effect investigation, calculation of the effect size of an element or component corresponding to a particular outcome, etc. [4,7,9,52]. Thus, it is advantageous to derive more rigorous findings through the combination of available methods, for example, scoping review and meta-analysis, which implies a composition of both qualitative and quantitative reviews.

3.2.2. A Rigorous Literature Review

The development of review types can help researchers to accomplish a literature review following a more scientific framework. This effort to study the typology is aligned with the emphatic statement by Cooper [17] which signifies that a literature review is not only a procedure for analyzing the extant knowledge but, more importantly, also a scientific method that demands equal attention from academia as the methods pertaining to primary studies. This agenda is deemed even more essential, considering the fact that conducting a literature review is prone to human misinterpretation—based on the authors’ individual background and subjectivity—which potentially leads to a biased study [28]. Thus, from a methodological standpoint, it is significant to enhance the element of rigor in conducting a literature review, which can minimize the risk of biased inferences [53]. To better conceive this vital aspect, a rigorous literature review study is collectively characterized by four major criteria as follows [16,54,55,56]:

• Internal Validity: It seeks to ensure that the established procedure for accomplishing the literature review perfectly complies with the concern of the study, and precisely addresses the devised research question(s) [57]. This could be reflected and measured by the decisions regarding searched sources, considered keywords for the search process, qualifications for appraisal of the collected documents, temporal interval of the retrieved articles, and so forth.
• Objectivity: This criterion emphasizes the elimination of individuals’ subjectivity throughout the review process, particularly concerning the drawn conclusions. Although this is a reflection of an unbiased literature review, objectivity solely stresses ascertaining that the ultimate results and findings are derived from the collected data and information, and not emanated from the author’s imagination [56]. This is mainly achieved by consistent compliance with a well-established review protocol throughout the review process [16].
• External Validity: This criterion underlines the generalizability of the results in accordance with the investigated domain or theory [16]. The immediate implication of this measure is reflected in the review type with the purpose “extend” the research where the main goal is set to push the conceptual frontiers of the investigated theory or topic; however, this is not the sole target, and other review types are expected to satisfy this metric by inclusive coverage of the previous research works. In this context, a major prerequisite consideration is paying attention to the choices of sources for the literature search as well as the search strategy [58].
• Reproducibility: A literature review without explicit, traceable, and repeatable reported procedures lacks methodological quality. This is simply fulfilled by articulating and documenting the entire process of the literature review, including but not limited to, devising the research question, search strategy, data collection, and information synthesis, so that it is replicable while generating the same results [59,60].

Despite the essence of a decision regarding the appropriate review type, conducting a literature review study requires a benchmark of reliability, known as rigor. A rigorous literature review puts the main emphasis on the process of literature review, which mainly implies a sound, comprehensive, and precise methodological approach. Although some literature review types are accompanied by methodological recommendations, such as meta-narrative or meta-interpretation, to satisfy the qualifications of a rigorous study, there is a lack of consistency in the proposed frameworks. In addition, the majority of studies that prioritize a rigorous approach for literature review are specifically targeting the medical realm, and a limited number of studies are directed towards engineering and management fields. This discussion necessitates the adoption of a systematic approach to conducting a literature review to satisfy the element of rigor, which is equivalently recognized as the SLR. There is a particular interplay between the attributes of a rigorous literature review and SLR. This interconnection, however, is a booster factor to the theoretical confusion within the literature review studies that employed the SLR. This confusion is reflected in the recognized taxonomies of the SLR: to conceive this systematic approach whether as a literature review type or as a tool. To address this issue, we primarily put the typology of the literature review into discussion which was established on the basis of the review purpose. Moreover, it is necessary to further elaborate on the procedural approach towards the adoption of the SLR in a literature review study. In this regard, the shared characteristics and procedures attributed to the SLR and a rigorous literature review not only facilitate a better comprehension of the SLR as a tool but also shape the grounds for providing the research community with a more solid and consistent structural procedure for employing SLR. Thus, we mapped the methodological contribution proposed for the SLR in Section 4, which was complemented by the thorough articulation of an overarching procedural approach to adopting this tool.

4. Conducting a Systematic Literature Review

Reviewing the typology of literature review depicted that this contribution is not an identical process across different studies, and it varies depending on the purpose of the review. Although this classification is vital to produce review articles more effectively and efficiently, each contribution must satisfy the criteria of a rigorous literature review in order to minimize the subjective bias and provide readers with reliable inferences. In this context, internal validity, objectivity, external validity, and reproducibility are considered the prominent attributes of a rigorous literature review. In practice, the incorporation of rigor elements into a literature review study implicates the review process, which is realized by translating the outlined criteria into practical steps and integrating them with the review procedure. For this purpose, Fink [61] identified four properties associated with the methodological approach to conducting a literature review: systematic in terms of a procedural structure to conduct the study, explicit in terms of describing and articulating the involved steps throughout the review, comprehensive in terms of accurately incorporating all relevant materials into the assessment and synthesis steps, and reproducible in terms of enabling other researchers to conduct the same procedure. These attributes lay the foundation for establishing and developing the methodological discussions concerning SLR. In this regard, Okoli and Schabram [7] presented an operative definition of the SLR: “a systematic, explicit, comprehensive, and reproducible method for identifying, evaluating, and synthesizing the existing body of completed and recorded work produced by researchers, scholars, and practitioners.” Systematic literature review (SLR)—interchangeably referred to as systematic review—serves as a procedural framework for identifying, evaluating, and interpreting the extant primary research works [8]. The most eminent advantage of the SLR is the well-defined structure and steps that prevent biased information and data synthesis throughout the process of the literature review [8,34]. Thus, literature review studies that benefit from such a systematic and rigorous approach are, based on their principle, recognized as SLR [7]. As emphasized by Petticrew and Roberts [62], the SLR is merely identified as a procedural toolkit for conducting a literature review study—rather than being recognized as a distinct method. In this regard, it is vital to avoid using the term “systematic” to dichotomize the review studies as systematic or nonsystematic [7]. In fact, a review study could be measured as more or less systematic depending on the depth of compliance with the core principles attributed to SLR [7,10].

According to the utilities of the SLR and benefits delivered by this systematic approach, the number of studies surrounding this topic has witnessed a dramatic increase throughout the last couple of decades, encompassing literature reviews employing SLR and studies related to the methodological development of this framework, not to mention that the former type of studies bears the major proportion. To observe this trend, we opted for using the online database of Web of Science (WoS) which is reported as one of the principal sources of accessing research materials, similarly to Scopus, SienceDirect, and so forth [63]. It is noteworthy to outline that WoS has demonstrated considerable strengths in enabling the scholars for conducting graphical and detailed citation analysis [64]. In this regard, a preliminary search was performed on 2 January 2023, to access the documents that entail the keyword “systematic literature review” or “systematic review” in any of their title, abstract, and list of keywords, which resulted in 305,100 items. After applying the filters of document type (journal and proceedings papers), language (English), and publication year (published by the end of 2022), the derived list was further narrowed down to 259,584 research papers. Utilizing the metadata registered for each research item, the collected papers are refined according to the publication year, and the results are presented in Figure 1. The primary implication refers to the rudimentary SLR footsteps that began in 1965. This fact signifies the preliminary attempts of academia to employ/study this concept in the framework of “research synthesis” in the late 1960s, as was also discussed by Chalmers et al. [6]. However, such a systematic approach was not as developed as the contemporary practice, and the aggregated number of studies published between 1965 and 1996 is only 257. Thanks to the development of online databases and widespread information availability via the Internet platform, the number of publications maintained an incremental trend, which yielded 90.16% of the articles published in the last decade (from 2012 to 2022). It is also of significance to highlight the radical leap in the number of publications, starting from 2019, which indicates the incremental attention of academia to this approach.

SLR was initiated within the medical fields, mainly in response to the essence of evidence-based research synthesis [4,6,13,14]. While this is an indicator of the pioneering role of medicine in this realm, it is beneficial to reflect the SLR exercise across different academic fields. To that aim, WoS assigns each article—either literature review studies using SLR or studies contributing to the development of the SRL framework—to a pertinent subject area. Based on this information, the collected articles are divided into 244 distinct fields. It is evident that medical fields have the most prominent impact on SLR in comparison to other subject areas, constituting 69 classes (out of 244) which is equivalent to 80.5% of articles. The dominant role of medical fields is also observed by ranking the subject areas based on their associated number of publications. The statistics indicate that medical fields have dominated the first 50 subject areas of the list by occupying 43 classes, not to mention that the first 25 classes are entirely medicine-related, including medicine general internal, surgery, clinical neurology, oncology, just to name a few. According to the scope of this study, the remainder of the classifications is separated into two groups. The first group, comprising 11.6% of the publications, mainly corresponds to applied sciences, social and environmental sciences, linguistics, ethics, and agriculture, while the second group encompasses an array of engineering and management topics, accounting for 7.9% of the publications.

Table 1. The scientific categories contributing to the systematic literature review studies, according to the results derived from WoS.

Main Category	Proportion of Total	Top Five Subcategories	Proportion within Main Category
Medicine	80.5%	Medicine General Internal	7.7%
		Surgery	6.3%
		Public Environmental Occupational Health	5.5%
		Clinical Neurology	4.8%
		Oncology	4.2%
Applied sciences, social and environmental sciences, linguistics, ethics, and agriculture	11.6%	Sport Sciences	14.3%
		Education Educational Research	6.8%
		Green Sustainable Science Technology	5.1%
		Environmental Studies	4.8%
		Food Science Technology	3.5%
Engineering and management	7.9%	Computer Science Information Systems	10.6%
		Management	10.2%
		Business	6.8%
		Computer Science Theory Methods	6.3%
		Engineering Electrical Electronic	6.3%

depicts the discussed scientific categories along with their respective proportion and the corresponding top five subcategories.

Further elaboration revealed that, compared to the other classes of the second group, computer science and civil engineering had the highest number of articles. In addition, the majority of studies contributing to the methodological development of the SLR were accomplished within the realm of the outlined classes, which are, in turn, limited in number. This limitation was also evident in Section 3.2, where it was shown that the majority of studies derived and utilized almost a similar list of papers to review the literature review types. In this regard, we conduct a narrative review throughout the remainder of this section to provide a thorough articulation of the SLR and supply the researchers with an explicit instruction for the adoption of this procedural framework in the literature review studies, particularly in engineering and management.

4.1. Employing the SLR Framework in a Literature Review

In their study to review the history of research synthesis, Chalmers et al. [6] discussed the rise of concern regarding methodological issues and the risk of bias interpretations in research synthesis. In this regard, the research community acknowledged that “methodological rigor” in conducting literature review studies is as important as primary research [58,65]. A rigorous literature review must satisfy four major criteria: internal validity, objectivity, external validity, and reproducibility. This critical agenda was translated into the context of the SLR based on four fundamental elements: systematic, explicit, comprehensive, and reproducible. Hence, research works surrounding the study and the development of the SLR framework must adhere to the core quaternary criteria outlined.

The urge for methodological rigor was not only a concern of scholars, but it was also a vital agenda in the field of medicine and incited a variety of organizations to contribute to this domain. For instance, The Cochrane Collaboration published the Handbook for Systematic Review following the primary objective to “help people make well-informed decisions about health care” [52]; or the handbook published by the Australian National Health and Medical Research Council [66]. These efforts within the domain of medicine and sociology inspired Kitchenham and Charters [8] to propose three underpinning phases for employing SLR within the realm of computer science: planning the review, conducting the review, and reporting the review. This trinary thematic agenda primarily emphasizes the “need assessment”—throughout the planning phase—prior to conducting a literature review study, which is realized by identifying the research questions and developing a review protocol that articulates the review procedures. The latter phase is the most resource-intensive part of the review which concisely relates to the collection and synthesis of the extant literature. Ultimately, the review report must disseminate the results in a systematic, comprehensive, and structured manner. Notably, there is a lack of consensus regarding the main steps of the SLR, or in other words, the articulation of the SLR phases. However, this does not justify magnifying the question “which guideline is correct?” The variations emanate from the context and subject area of such studies, e.g., medicine, civil engineering, computer science, linguistics, etc., and more importantly, from the scholars’ attempts to articulate the SLR steps with higher or lower degree of complexity. For instance, Kitchenham and Charters [8] identified twelve steps—ten mandatory and two optional—and Xiao and Watson [9] proposed eight steps to approach the fulfillment of the discussed phases within the field of urban planning. Following the operative definition of the SLR and its quaternary criteria, Okoli and Schabram [7] presented four modules—composed of eight distinct steps—to provide the research community of information systems with an SLR guideline: planning, selection, extraction, and execution. In the “planning” module, the researchers are supposed primarily to identify the purpose of the literature review, as well as to oversee the strategical protocol to guide the processes of the study. The “selection” module consists of building a comprehensive collection of articles, which are further filtered and selected according to certain criteria such as the publication year, language, and so forth. Throughout the “extraction” module, the author(s) eliminate articles of inadequate quality and derive the necessary data and information from individual research items. Utilizing appropriate techniques and approaches, the author(s) accomplish the synthesis of the collected information within the “execution” module. In another study, and from a more holistic perspective, Templier and Paré [16] emphasized the element of rigor and recommended six general steps for the researchers to follow (regardless of the study field) in conducting a literature review study: formulating the problem, searching the literature, screening for inclusion, assessing the quality, extracting the data, analyzing and synthesizing the data.

Methodological studies surrounding SLR fundamentally adhere to the main principles of a rigorous literature review. Although these studies differ in the degree of articulation, they bear significant similarities in their strategical and tactical approach, regardless of their contextual domain. Through an in-depth assessment, this section seeks to present an accumulative and concise procedural framework, which was generalized and accommodated for the engineering and management area. To that aim, the underlying principle is set to articulate the steps according to the quaternary criteria suggested by Okoli and Schabram [7], which act as the main structural framework for the SLR procedure. Notably, an exhaustive instruction on employing the SLR does not fit into the scope of a journal article, even though this study puts forward an inclusive guide. In this study, our focus was on providing the research community with a stepwise guide at a higher abstraction level, which ultimately contributes to delivering a more transparent and practical comprehension of the SLR, particularly as a procedural tool in conducting literature review studies. As also emphasized by Kitchenham and Charters [8], the SLR steps are not rigid in adaptation, even though the process follows a linear path. In fact, by progressing upon the SLR flowchart the author(s) may encounter unforeseen aspects that are worth being incorporated into the review process. Hence, it is advised to consider any necessary refinement within any of the proposed steps in order to upbuild the review quality and reliability. Figure 2 illustrates the main steps which are clustered into four modules.

The planning module underpins the entire review process and seeks to ensure high efficiency and quality of the literature search and information synthesis. In practice, the author(s) must determine the strategies for the SLR steps to be explicit, comprehensive, and reproducible. The entire procedure is designed with transparent structure and architecture to satisfy the “systematic” element of the SLR quaternary criteria [7]. The literature selection module constitutes a comprehensive collection and explicit refinement of the most relevant and high-quality studies. The major effort is invested in the data extraction and synthesis module, in which the author(s) need to derive and synthesize the appropriate data according to the main concern of the review. The documenting and reporting module, aside from preparing the manuscript pertaining to the entire literature review study, emphasizes explicit reporting of the entire SLR process.

4.1.1. Problem Formulation

The primary step is the identification of the main problem that is supposed to be addressed by the SLR. In practice, this agenda is initially realized by “defining the purpose of the review” [7]. This is one of the most driving elements considering the pivotal impact of the review purpose on the decision regarding the literature review type. The procedure of a literature review study is a function of the literature review type, which determines several critical aspects surrounding the review process, e.g., data types to be collected, document retrieval process, etc.

The identified problem should be further articulated by the research question(s) which must be precise—reflecting the problem explicitly and correctly—and be accurate—driving the SLR towards the most relevant documents and information. To ensure the novelty and to justify the necessity of the review, Kitchenham and Charters [8] suggested studying former systematic reviews carried out on the topic of interest. This effort also provides researchers with the opportunity to refine the research questions by having a wider outlook towards the problem [67]. For instance, decide whether the devised research question(s) reflect the review’s concerns correctly, whether the research question(s) are specific enough to result in an efficient process of article collection, and so forth [9].

4.1.2. Developing the Review Protocol

Review protocol is recognized as a strategical plan that sheds light on the roadmap of a literature review study. An explicit and comprehensive plan leads to tackling the risk of biased assessment and increasing the review quality and reliability [7,8]. In effect, the review protocol should sufficiently elaborate on the approach and tactics required to satisfy the SLR steps [9]. The review protocol should be conceived by all of the authors evenly and from the same perspective, and be externally validated, as emphasized by Okoli and Schabram [7]. More importantly, in order to prevent disrupting the explicitness and ensure the reproducibility of an SLR, any changes considered during the review should be reflected in the review protocol and the report [7,61].

4.1.3. Literature Search

The retrieved material for the accomplishment of a literature review has a determining role in the quality of the findings and analyses. The most prominent approach in this context is the systematic search, which entails several considerations and requires an explicit searching strategy. To that aim, the primary step is the identification of the keywords that favor accessing the most relevant documents, which must be selected according to the devised research question(s). In this regard, Kitchenham and Charters [8] argued that breaking the research question down into meaningful and distinct conceptual subjects paves the way for better identification of pertinent keywords. This approach emphasizes the significance of determining the tactics for combining the identified keywords, which ultimately engender the “search string”. A well-structured search string assists in accessing a larger pool of relevant articles, which increases the search efficiency. In this regard, Mohamed Shaffril et al. [68] discussed the techniques that assist in constructing a multifaceted search string, and we elaborated on three of them as the most eminent ones:

• Phrase Searching: In order to best exploit the potential of online search engines, one ought to pay attention to the underlying differences of searching a single word or a collocation of words. In order to find the exact matches, phrasal keywords must be placed between two double quotation marks (“), which form a binding constraint so that the search results comply with the order of words constituting that phrase. “agile supply chain”, “industry 4.0”, “public infrastructure”, just to name a few, are instances that necessitates considering double quotations. It is worthwhile to note that small and capital letters would have an equal impact, and punctuation marks, e.g., hyphen, slash, space, etc., are treated similarly by the search engines.
• Truncation: From the linguistic standpoint, terms may take various forms. In most cases, this causes changes to their ending part, including variations related to singular or plural, noun or adjective, and so forth. In this regard, the possible variants of the primary keywords are specified by replacing the ending altering section of the word with an asterisk (*) symbol. For instance, the asterisk symbol in “financ*” takes the forms of either “e” or “ial” in order to refer to “finance” or “financial”, respectively.
• Boolean Operators: This technique is particularly helpful in combining the primary keywords and their variations in the most meaningful and operational manner. The use of Boolean operators (AND, OR, AND NOT) enables the researchers to narrow down the search list effectively associated with the topics of research questions. The basic rule suggests combining distinct topics with “AND” in order to access the correct research items and extend the population of documents by using “OR” to incorporate the conceptually equal terms or the synonyms of the keywords within each subtopic.

Although determining the keywords and their combinations is vital, it is significant to assess the applicability and accuracy of the search strategy. For this purpose, two practical recommendations are of critical importance [8]: (1) consider various combinations of keywords and trial searches in order to figure out the optimal search string; (2) benchmark the approved keywords against a list of primary and acknowledgeable research works to ensure the reliability of the search process. The author(s) need to derive the optimal balance between the degree of exhaustiveness and precision, even though exhaustiveness is more important at this stage [69]. This implies that broad keywords lead to more irrelevant results. It is also suggested to consider the utilization of filters provided by the online databases to better narrow down the initial list of articles, such as language, publication year, document types, subject area, and so forth. Following the identification of the keywords and the search strategy, the author(s) encounter the decision regarding the selection of online database(s). In a comprehensive study, Gusenbauer and Haddaway [63] examined the performance of 28 available online databases across 27 criteria including, but not limited to, the formulation of queries or search strings, correct interpretation of the search string by the search engine, and retrieval capabilities. The results reveal that only 14 databases fulfill the essential requirements, and they fall within the category of principal databases, i.e., Scopus, Web of Science (WoS), PubMed, Science Direct, etc., as opposed to other 14 that fit into the supplementary category, i.e., Google Scholar, IEEE Xplore, Springer Link, etc. The majority of online databases enable users to locate the touch point of the search string, and this implies the identification of the fields that are supposed to be scanned by the search engine. To reduce the probability of retrieving irrelevant research items, it is suggested to adjust the scanning fields to title, abstract, and list of keywords. However, this may vary across the online databases and each database may provide these options with different explanations and terms.

Backward and forward searches are further considerations to enrich the retrieved pool of the articles with more relevant research works [70,71]. In addition, research topics with the limited extant literature could benefit from incorporating the “gray literature” into the list of documents [7,9]. Although an all-inclusive and exhaustive literature search is not particularly suggested for all types of literature review—for instance, in the scoping review, as discussed in Section 3.2.1, the goal is to illustrate the breadth of a topic, and, thus, it demands the collection of a vast number of research items, as suggested by Levy and Ellis [70]—the rule of thumb is to stop searching when similar documents are retrieved by repeating the search process.

4.1.4. Quality Appraisal

The overall goal of this step is to ensure that the remainder steps of an SLR utilize the most relevant and high-quality materials that are suitable to address the devised research question(s) [7,61]. This is accomplished in two stages: the coarse sieve and fine sieve.

The main goal of the coarse sieve stage—also referred to as the first screening, early screening, or practical screening—is to eliminate the articles with weak thematic relevance to the research question(s) and use a list of predefined criteria for inclusion/exclusion [7,9]. In order to neutralize the bias effect, the criteria are supposed to be identified during the review protocol development, even though it is advised to consider refining the criteria during the process if necessary [8]. The vital consideration in this stage, as also highlighted by Okoli and Schabram [7], is the degree of criteria simplicity so that it is possible to determine the suitability of papers by reading their “title” and “abstract”. The underlying principle is the practicality of the criteria in classifying the articles, to distinguish the articles which do not address the devised research question(s) [7,8]; for instance, including studies that are associated with a certain geographical part or country, excluding studies with irrelevant methodological approach according to the scopes of the research question(s), excluding studies with noncompliant achievements and goals to the scope of review, and so forth. During this process, the author(s) are recommended to consider two fundamental policies to accomplish this stage with a higher degree of inclusiveness and reliability: in the case the author(s) are in doubt regarding the incorporation of an article, a wise decision would be to add it to the list of the approved research items [7]; at least two authors should be engaged in this stage provided that they resolve the discrepancies by discussion or consultation with an external expert [8,16,67,72].

The fine sieve stage—also referred to as the second screening—is the final stage of selecting the most suitable material for the analysis, and it is accomplished by reading the full text of the articles that were approved in the previous stage. The most significant challenge is that there is no consensus on how to approach this stage and what measures can thoroughly determine the sufficient quality of an article [7,9]. Despite the fact that this stage varies fundamentally across the literature review studies, each piece of the collected material should be measured against a set of standards that are established during the review protocol development. As suggested by Okoli and Schabram [7], the quality of papers could also follow a ranking scheme. The rule of thumb is to reject the studies that bear insufficient similarity or homogeneity in terms of the methodological quality [7,9]. To that aim, authors are provided with some checklists that are developed to facilitate better completion of this stage [68]: the Critical Appraisal Skills Programme (CASP) which investigates the methodological shortages across ten measures [73] and the Mixed Methods Appraisal Tool (MMAT) developed for systematic reviews that are based on mixed studies [74] are two of the instances. In line with this approach, Kitchenham and Charters [8] argued that bias and validity issues of an empirical study could be concisely investigated with respect to four aspects: the design, conduct, analysis, and conclusions. In addition, the fine sieve stage is not suggested for the literature review types that aim to illustrate the breadth of a topic, such as descriptive or critical reviews [9,75]. It is worthwhile to remind of the viability of the inclusion/exclusion criteria—that were applied in the coarse sieve stage—during the second stage, so that the studies of limited compliance with those criteria could be removed from the list of the approved papers. Similar to the coarse sieve stage, it is suggested that the engagement of more than one author is considered to decide on the final list of the approved articles [28,67].

4.1.5. Data Extraction

Although SLR has been articulated differently across the extant guides, the data extraction has been a consistent step in most of those methodological studies. This stage is founded on the final list of research materials that have been approved through the preceding steps, and the core objective in this context is to extract the relevant information and data with respect to the devised research question(s). However, there is limited technical and generic recommendations and guidelines concerning the accomplishment of this stage which is potentially emanated from the variation in the type of data to be synthesized in each literature review study [7].

From a procedural perspective, the data extraction is the basis of information synthesis and influences the ultimate inferences derived from the analyzed articles. According to the correlation between the literature review type and the data synthesis, the preliminary guide for accomplishing the data extraction stage is the review type [9]. This implies that the type of literature review is the initial determinant of the data extraction strategy, which in turn guides the researchers to identify the required information and data to be extracted from the collected documents. For instance, conducting a meta-analysis review necessitates the collection of quantitative data that best serve the meta-regression, i.e., data associated with the dependent and independent variables. In this context, and regardless of the literature review type, data coding is widely discussed as a practical approach for deriving the relevant data. Coding the data serves as a systematic mechanism for storing the information pertaining to each research item and facilitates conducting more efficient information synthesis that leads to higher reliability of findings and conclusions [7,9,76]. Hence, it is crucial to establish a thorough, comprehendible, and transparent coding strategy that can best serve the literature review type. Data coding is critical to minimize the adverse impacts of the bias and objectivity elements. In this regard, it is also suggested to conduct the data coding individually by at least two researchers [8,28,72]. Although developing a coding form assists in finding the relevant data and information with more comfort, it should not act as an obstacle to reading the entire text of each article. As a result, the data extraction process must rely on the entire manuscript rather than specific parts that suit the established codes to a larger extent, e.g., conclusions, main results, etc. [9,77].

4.1.6. Data and Information Synthesis

Information synthesis, similar to the data extraction, echoes in the essence and impact of the literature review type. The fundaments in this context depend on the nature of the primary studies and the literature reviews, that is, either qualitative or quantitative [7]. In general, the synthesis of information may include a variety of means to present the findings and enables the authors and the readers to better conduct and understand the analyses and interpretations of information, including, but not limited to, charts and tables [9]. One of the most established analysis approaches is the quantitative synthesis of quantitative studies, which is also known as the meta-analysis. However, the interpretation and comprehensive description of data precede the quantitative analysis [8,10,62]. This approach, referred to as triangulation, commences with a narrative synthesis which aims at providing an outlook of the collected data using comparisons, tables, and so forth [7,62]. This is followed by deriving the best evidence and meta-analysis techniques. On the other hand, there is no consensus on approaching the synthesis of data within a qualitative review study, and a degree of subjectivity is inevitable in this regard [10]. However, as a rule of thumb, the authors are urged to utilize cognitive tools and discover preliminary correlations between the clusters of the extracted data. Although this approach intrinsically entails some extent of subjectivity, Okoli and Schabram [7] argued that an explicit and well-described SLR contributes to methodological transparency, which facilitates the reproducibility of research and further improvement of the adopted method.

4.1.7. Review Report

The systematic, explicit, comprehensive, and reproducible elements were discussed earlier as the main characteristics of the SLR. From a methodological perspective, the entire procedure and steps of the SLR are designed to satisfy the systematic element; however, the other elements are technically fulfilled by the precise completion of the individual SLR steps [7]. This signifies the important role of meticulous documenting of each step in order to deliver a review study that provides readers with an explicit, comprehensive, and reproducible report. Hence, it is critical to pay equal attention to the review report and the other steps of the SLR. Although there are numerous guidelines in this respect [26,78], some of the most important factors are concisely reflected below:

• The review protocol should be explained thoroughly, including any revision that was applied during the SLR process.
• The exact date of the literature search must be reported in the review due to the continuous update of the online databases. The same search string would result differently if used at a later time within the same online database.
• The rationale of the selected inclusion/exclusion criteria in the coarse sieve stage must be scientifically justified and individually explained [79], not to mention the importance of reporting and explaining the reasons for the filters applied within the database during the literature search, e.g., the publication year, document type, subject area, etc.
• The fine sieve stage embraces more descriptive criteria in order to eliminate the articles that do not address the devised research question(s). Thus, the author(s) are supposed to provide sufficient explanations regarding the selected measures.
• The “literature selection” commences with collecting a wide range of articles, and numerous research items are eliminated in this phase. It is of significance to keep a precise track of the records at each step, i.e., the literature search and quality appraisal. To that aim, it is recommended not only to document the statistics in the review report but, more importantly, to take advantage of the visualization aids and flow diagrams to illustrate the entire phase with a concise numerical and descriptive outlook. In this context, the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) were developed by Moher et al. [80] in 2009 to provide the research community with a checklist for conducting systematic reviews, denoted as the PRISMA statement, and a platform to illustrate the literature selection phase as outlined above, denoted as the PRISMA flow diagram. These tools were updated in 2020 [81], and scholars can freely access them at http://www.prisma-statement.org (accessed on 17 August 2023).
• It should be indicated in the “title” of a journal article that the literature review study is based on the SLR framework [8].
• The author(s) of a literature review study should consider harnessing the potential of the SLR and provide the research community with a detailed list of the identified research gaps and future agendas [7,62].

4.2. Science Mapping and Bibliometric Analysis

In pursuit of studying the rudimentary thoughts towards the quantitative assessment of science progress, Chen [1] shed light on the collocation of three concepts that are worth mentioning: the Matthew effect, the scientific obliteration, and the invisible college. Robert K. Merton [82] introduced the “Matthew effect” by highlighting a remarkable point in his explanation of this concept: “But what you note is the acknowledgment at the end of the paper to the senior person for his ‘advice and encouragement.’ So you will say: ‘This came out of Greene’s lab, or se and-so’s lab.’ You remember that, rather than the long list of authors.” This quote was highlighted and echoed by Chen [1], and he described it such that “the rich get richer and the poor get poorer.” In effect, established and well-known researchers—without compromising the scientific value of their contribution—are more likely to receive citation and credits compared to nonestablished researchers. Another progressive and implicit phenomenon in the research community is the scientific obliteration, as emphasized by Merton [82]: “The transmitters may be so familiar with its origins that they mistakenly assume these to be well known. Preferring not to insult their readers’ knowledgeability, they do not cite the original source or even refer to it.” This signifies the process by which the researchers do not cite the origin of a finding since it is being taken by the research community for granted [1]. Price [83]—a pioneer researcher in the field of the “science of science” [1]—recognized the invisible college as the informal and implicit network among the scholars and scientists of a field.

The abovementioned concerns demonstrate the phenomena and praxes under the veins of scientific communities. In this regard, Kuhn [18]—in the postscript of his book—proposed using the linkage between citations as a means of problem solving. This is recognized as the “science mapping” which was defined by Small [84] as “a spatial representation of how disciplines, fields, specialties, and individual papers or authors are related to one another as shown by their physical proximity and relative locations, analogous to the way geographic maps show the relationships of political or physical features on the Earth.” The main objective of the science mapping is “to identify the intellectual structure of a domain” [85], and it is supported by various quantitative examinations. In an analytical study, Boyack and Klavans [86] investigated three major citation-based approaches to the science mapping, namely the bibliographic coupling [87], the co-citation analysis [88,89], and the direct citation (also referred to as inter-citation) analysis [90], and the results signify higher performance accuracy on bibliographic coupling. The science mapping, however, is not limited to the outlined examinations and according to different units of analysis, i.e., words, authors, etc., other approaches are widely used by researchers, such as the analysis of keywords and authorships, denoted as the co-word analysis and the co-authorship analysis, respectively [1,91]. Notably, the distinction between the science mapping and the meta-analysis is underpinned by the difference between the metadata and primary data. Based on their principle, metadata—also written as meta-data—constitute the information created, stored, and shared to describe objects, files, and so forth, and it is possible to obtain knowledge by interacting with metadata: metadata is the “data about data” [92]. Given that a research document is as a unique object, it embraces a plethora of information including the title, abstract, author affiliation, and funding information, the so-called bibliographic metadata [93]. On the other hand, primary data refer to the information derived from evidence-based independent studies [35,52]. As discussed earlier in Section 3.2.1, the meta-analysis is a type of literature review utilizing the evidence derived from the primary studies.

The advancements of quantitative methods within the scope of science mapping, along with the escalation of information availability, led to the advent of a new discipline rooted in the library science, interchangeably known as scientometrics/informetrics/bibliometrics [94]. In this regard, the “bibliometric analysis” is constructed based on two core techniques [95,96]: the science mapping and the performance analysis. While the former was explained earlier in this section, the latter aims at evaluating the performance of academic actors—mainly the authors but also institutions, countries, and journals—based on their academic contributions in terms of the number of publications and received citations [97]. Despite comprising quantitative metrics, the performance analysis necessitates descriptive assessments, which serves as a background of the influential research constituents to complement the findings from the science mapping and provide academia with a comprehensive bibliometric analysis [95]. Total publications, total citations, and h-index are only a few of metrics to approach the performance analysis [95].

Bibliometric Analysis: Typology and Procedure

The science mapping—and the bibliometric analysis in the later stages—emerged as a means of problem solving, particularly in response to the essence of harnessing the implicit potentials of academic social networks [1,18]. The bibliometric analysis enables researchers to answer specific research questions that are difficult to approach via other literature review types, and, in this context, Aria and Cuccurullo [98] identified three general purposes that could be pursued by using bibliographic metadata: (1) to demonstrate the intellectual structure of a research field; (2) to examine the research front—denoted as a conceptual structure by the authors—of a research field, (3) to generate and illustrate the social network of a scientific community.

Given the transparent aspects of the objectives of conducting a bibliometric analysis study, it is beneficial to map its contribution within the context of the literature review typology, as it provides academia with a thorough and, more importantly, a distinct method of literature review. In order to better realize this distinction, we refer to the typology of literature reviews which was elaborated on in Section 3.2.1, presenting four categories of the review purposes: to describe, test, extend, or critique. A comparison between the aims and scopes of the outlined categories indicates that the bibliometric analysis fits the descriptive review framework (with the purpose to describe). In effect, the bibliometric analysis is added to the list of literature review types pertaining to the category with the purpose to “describe” and would be peered with the meta-summary and the meta-narrative, just to name a few. This results in extending the list of literature review types from 16 to 17, and in this regard, all the review types discussed in this paper are summarized in

Table 2. The summary of the literature review types with respect to the review purpose.

Review Purpose	Review Types
Describe	1. Narrative Review	2. Meta-Narrative
	3. Textual Narrative Review	4. Scoping Review
	5. Meta-Summary	6. Bibliometric Analysis
Test	7. Meta-Analysis	8. Realist Review
	9. Bayesian Meta-Analysis	10. Ecological Triangulation
Extend	11. Meta-Ethnography	12. Meta-Study
	13. Thematic Synthesis	14. Critical Interpretive Synthesis
	15. Meta-Interpretation	16. Framework Synthesis
Critique	17. Critique

.

The quantitative features of the bibliometric analysis are potentially in favor of reducing the subjectivity level. However, it does not immune this type of literature review from biased interpretations, and it is deemed necessary to incorporate the element of rigor in such a study. To that aim, Zupic and Čater [99] investigated and synthesized the guidelines associated with 81 bibliometric studies and proposed a five-tier workflow to conduct a rigorous bibliometric analysis: (1) research design; (2) compilation of bibliometric data; (3) analysis; (4) visualization; (5) interpretation. By putting more emphasis on the choice of the analysis techniques, Donthu et al. [95] recommended four steps to conduct a rigorous bibliometric analysis: (1) define the aim and scope of the study, (2) choose the appropriate techniques, (3) collect data, (4) conduct the bibliometric analysis and report the findings. There are other approaches to accomplishing a bibliometric analysis; however, they are either based on more conventional computation techniques, such as the study by Börner et al. [100], or are part of the outlined workflows, such as the study by Cobo et al. [101]. Although commenting on the discussed workflows or instructions for conducting a bibliometric analysis is beyond the scope of this study, it is beneficial to deliver some of the insights from the SLR framework into this realm. On the one hand, a rigorous bibliometric analysis should, first and foremost, comply with the criteria of internal validity, objectivity, external validity, and reproducibility. On the other hand, the outlined elements of rigor were further elaborated on and practically translated into the SLR framework, in which the SLR framework was recognized as a procedural toolkit that embraces four key properties [7,61]: systematic, explicit, comprehensive, and reproducible. In this regard, Section 4.1 presented an articulation of the SLR processes to satisfy the four-tier properties of this framework. Hence, according to the conceptual link between a rigorous literature review and an SLR, we assimilate the main principles and steps of the SLR to the proposed workflows [95,99], and present a realigned procedure for delivering a rigorous bibliometric analysis:

• Problem Formulation. The main agenda of this step consists of identifying the main objective of the study and research questions. The research questions should be precise to best reflect the concern of the study. Ramos-Rodríguez and Ruíz-Navarro [97] suggested that the scope of the study—which is specified by the research questions—should be wide enough to ensure access to a large volume of the bibliographic metadata. As a rule of thumb, Donthu et al. [95] suggested reformulating the study scope in the case of retrieving fewer than 300–500 research items.
• Review Protocol. The mission of this step is to illustrate the entire procedure of the review, and in the context of a bibliometric analysis, it embraces two vital decisions: the bibliometric methods (or techniques) and software.
  As emphasized by Zupic and Čater [99], bibliometric methods are individually suitable to address distinct types of research questions. For instance, “investigating the intellectual structure of emerging literature” is answered by the bibliographic coupling, while “investigating the intellectual structure of literature X” is answered by the co-citation analysis. In this regard, Donthu et al. [95] argued that a bibliometric analysis intends to highlight three temporal aspects of a research field: past, present, and future. In this regard, they suggested considering, among other things, the co-citation analysis, the bibliographic coupling, and the co-word analysis. Hence, it is critical to identify the research questions and bibliometric methods accordingly.
  Another determining element in this type of literature review is the utilized software. The development of tools has been among the major drivers of the increased number of review studies employing a bibliometric analysis [95]. In a thorough study, Cobo et al. [101] compared nine eminent tools with versatile and distinct capabilities, including BibExcel [102], CiteSpace II [103,104], and VOSviewer [105]. The results of their study indicate that there is no “single solution” in the choice of the software, and the bibliometric analysis tools differ in the visualization, mathematical computation, and processing of raw data. Thus, the authors suggested choosing a combination of tools to best serve the research community with an all-inclusive bibliometric analysis; otherwise, the choice of the software must meet the aims and scope of the study. In a recent study, Pessin et al. [106] introduced the term “smart bibliometrics” which attempts to facilitate a more dynamic and intelligent platform, and ultimately to “automize the entire process of bibliometric analysis”. This tool is provided in the Microsoft Power BI software, and the free version of it is accessible via the following link: https://bit.ly/3GgYUZh (accessed on 17 August 2023).
• Literature Search. This stage is equivalent to the data collection and seeks to extract relevant and sufficient data. The entire process in the context of a bibliometric analysis is relatively identical to employing the SLR framework in other literature review types, including the identification and combination of keywords, the application of appropriate filters, and the choice of the database (see Section 4.1.3). However, according to the key role of the bibliographic metadata in this review type and the lack of data consistency, the authors must pay attention to some technical considerations. For example, the name of a journal could be cited in slightly different forms; in the case of using initials, the name of two distinct authors with similar last names and different first names—provided that the initials of their first names are the same—would appear the same. Thus, it is essential to refine the raw data before the analysis [99]. In addition, when using multiple databases, there is a high probability of having duplicate records in the dataset which should be converted into single records [95].
• Data Synthesis and Report Preparation. The interpretation of the outputs pertaining to the quantitative examinations, i.e., the bibliographic coupling, the co-citation analysis, the co-word analysis, etc., demands the highest level of effort required to deliver a rigorous bibliometric analysis. In this regard, authors with a higher seniority level are potentially better equipped with the necessary background knowledge and the level of expertise to process the particular linkages and derive fruitful inferences. However, it is of significance to utilize the knowledge for further discoveries, rather than considering it as a package of “preconceptions” [99]. In addition, the authors should make use of different sorts of visualizations, tables, and so forth, in order to not only demonstrate the results in comprehendible and transparent forms but also to support the interpretations with visual aids [95]. Last but not least, we put emphasis on producing a thorough and transparent report that not only presents in-depth interpretations and assessments but also satisfies the elements of explicitness, comprehensiveness, and reproducibility. In the context of the bibliometric analysis, descriptive assessments must be accompanied by proper visualizations. Mentioning the “bibliometric analysis” or “science mapping” in the title of the article, specifying the date of data collection, explicit and meticulous explanation of the review protocol, or equivalently, the entire procedure for the bibliometric analysis, are some of the most important agenda items that demand the attention of the authors during the preparation of the report.

5. Further Discussions and Overview of the Logistics

Incorporating the element of rigor was discussed as the most important criterion in delivering a literature review study. Although a thorough articulation of the SLR process was presented in the previous section, a substantial emphasis was placed on the distinction between the typology of literature review studies and conducting a literature review using SLR. Based on the review purpose, four categories of literature review were studied which encompassed 16 types of literature review. This list was extended to 17 by incorporating the bibliometric analysis into the category with the purpose to “describe”. As a hybrid approach, a bibliometric analysis can be utilized in combination with other methods of literature review depending on the aims and scope of research questions. The core tactical outcome of this package of knowledge corroborates the identification of literature review type concerning the study goal especially prior to the utilization of the SLR framework. Thereafter, from a technical standpoint, it is deemed necessary to follow the SLR procedure with a high degree of accuracy.

In this section, we utilized this transparent roadmap for delivering a rigorous literature review and performed a compliance test by analyzing the literature review papers that adopted the SLR and were published in the Logistics. This assessment, however, does not seek to initiate a critique attitude towards the reliability of findings presented in those articles. Nevertheless, the main aim was to capture and benchmark the methodology employed in each review study given the claim of using the SLR. Therefore, it is primarily deemed necessary to establish a standard protocol to facilitate the cross-dimensional assessment of the selected review studies against the critical and compulsory facets of SLR. In this regard, we opted for producing a list of criteria that reflects the principal steps of the SLR, which were discussed in Section 4. As a matter of fact, the criteria selection process seeks to establish the SLR elements/steps to be approached consistently regardless of the literature review type and, ultimately, to generate a standard and generalized assessment protocol that serves the engineering and management fields. In this regard, the SLR steps which are a function of the study’s context are disregarded, and, thus, the assessment protocol is composed of context-free criteria. For this purpose, among the seven steps of the SLR, the “data extraction” and “data and information synthesis”, Step 5 and Step 6, respectively, are disregarded for the inclusion given the fact that these steps are entirely dependent on the literature review type, i.e., a meta-summary, meta-analysis, etc., and the questions/context of the individual review studies. Hence, the outlined list of criteria embraces the remaining steps of the SLR: problem formulation, review protocol development, literature search, quality appraisal, and review report preparation. The selected criteria are then translated into practical steps to enrich the assessment protocol with executable dimensions and measurable metrics. For this purpose, we considered the practical guidelines associated with the SLR framework which were discussed throughout Section 4.1.

It is noteworthy that given the ambiguity associated with the theoretical application of the bibliometric analysis—a translucent mapping of the bibliometric analysis with respect to SLR and the typology of literature reviews was thoroughly discussed and addressed in Section 4.2—the assessment protocol warrants the inclusion of the “bibliometric analysis” as another criterion. Ultimately, after the inclusion of this criterion, the final list of criteria encompasses 17 measurable metrics, presented in

Table 3. The list of major compulsory elements/steps of the SLR for compliance test.

Code	Criteria
A	Problem Formulation
A1	Is there any specific research question devised to lead the SLR process?
B	Developing the Review Protocol
B1	Is the review protocol explained explicitly?
C	Literature Search
C1	Is the search query explained sufficiently and demonstrated from a technical standpoint?
C2	Is the process of searching/scanning literature specified accurately (e.g., titles, abstract, …)?
C3	Is there an explanation of the applied filters (e.g., language, publication year, subject area, etc.)?
D	Quality Appraisal
D1	Is the coarse sieve stage (the first screening) conducted correctly and sufficiently?
D2	Is the fine sieve stage (the second screening) conducted correctly and sufficiently?
E	Review Report Preparation (Reproducible and Comprehensive)
E1	Does the title contain “systematic literature review” or express the intention of using the SLR framework?
E2	Is the exact date of the literature search reported?
E3	Is the rationale for the applied filters regarding the literature search explained and justified?
E4	Is the rationale for the inclusion/exclusion criteria (quality appraisal) explained and justified?
E5	Is the number of retrieved items elaborated on throughout the literature search and quality appraisal?
E6	Is there any flowchart (e.g., PRISMA) used to represent the literature search and quality appraisal?
E7	Is there any elaboration on the future agenda?
F	Bibliometric Analysis
F1	Does the review study embrace any element/examination of the bibliometric analysis?
F2	Is the review on the sole basis of the bibliometric analysis?
F3	Is the review combined with the bibliometric analysis with clear distinction of the applied methods?

. It is noted that each metric is assigned a unique code to be used for the assessment and to enable a smoother demonstration of the compliance test.

5.1. Literature Search and Quality Appraisal

A preliminary search within the online database of the Multidisciplinary Digital Publishing Institute (MDPI) showed that the keyword “systematic literature review” appeared in 19,750 articles, published in 428 MDPI journals. While the number of journals and the number of articles is a function of time, we completed the search process on 7 June 2023. Given the fact that such a broad search process scans all partitions of a paper, e.g., references, affiliations, authors, titles, etc., we refined the search results by setting the scanning process to “full text”, which reduced the number of records to 4563. While the aim of this section necessitates the retrieval of papers that utilize the SLR as a means of conducting a literature review study, we then applied the filter “article types”, where “review” and “systematic review” were selected as the desired types to reduce the number of papers to 2337. It is worth discussing that the time temporal of the papers began in 1996, but 96.15% of the papers—equivalent to 2247—were published after 2018, which signifies an immense attention of academia to SLR throughout the last five years. Ultimately, we confined the search results to the papers published by the Logistics using the “journals” filter. The retrieved list encompassed 19 papers mainly published after 2020, except for two papers that were published in 2017 and 2019.

According to the scope of this section, one record was eliminated throughout the coarse sieve stage—reading the title and abstract of the papers, because this study employed the SLR to review the literature review papers, known as a tertiary study [8]. Although the SLR procedure is also accommodated to conduct a tertiary study, we disregarded this record to ensure a high degree of consistency between the collected studies. The fine sieve stage—evaluating the full text of the papers—commences according to two major inclusion/exclusion criteria: (1) the SLR framework should be employed as the main means of conducting the review study; (2) there should be sufficient methodological explanation regarding the applied procedure. It is observed that the remaining papers from the preceding stage satisfy the outlined criteria. As a result, 18 papers remain for further assessment. The literature search and quality appraisal steps are illustrated in Figure 3 using the PRISMA tool [107].

5.2. The Results of the Compliance Test

The first impression from the assessment of the collected articles is the lack of consistency associated with the applied methodology. It is explicitly observed that the literature review studies employ the SLR procedures differently from both tactical—the number and sequence of steps—and technical—the accuracy in following each step—perspectives. This shortage of methodological coherency and consistency not only leads to a translucent conceptual perception and divergent terminologies associated with SLR, but, more importantly, it implicates the necessity of more standard guidelines for the SLR processes. Notably, despite nine studies (No. 1, 3, 7, 8, 9, 10, 12, 13, and 15 in

Table 4. Assessing the methodological approach of the review papers employing the SLR.

No.	Author(s)	Year	A1	B1	C1	C2	C3	D1	D2	E1	E2	E3	E4	E5	E6	E7	F1	F2	F3
1	Tozanli et al. [109]	2017	✓	✓	✗	✗	✓	✓	✓	✗	✗	✗	✓	✓	✓	✓	✓	✗	✗
2	Silva et al. [110]	2019	✗	✓	✗	✓	✓	✓	✓	✗	✗	✗	✓	✓	✓	✓	✓	✗	✗
3	Rejeb et al. [111]	2020	✓	✗	✓	✓	✓	✓	✗	✗	✗	✓	✗	✗	✗	✓	✓	✗	✗
4	Ritola et al. [112]	2020	✓	✓	✓	✗	✓	✓	✗	✓	✗	✓	✓	✗	✗	✓	✗	◯	◯
5	Varese et al. [113]	2020	✗	✓	✓	✓	✓	✓	✓	✗	✓	✓	✓	✗	✓	✗	✗	◯	◯
6	Abideen et al. [114]	2021	✓	✓	✓	✓	✓	✓	✗	✓	✗	✗	✗	✗	✗	✓	✓	✗	✓
7	Cerqueira-Streit et al. [115]	2021	✓	✓	✓	✗	✓	✓	✗	✓	✗	✓	✓	✓	✓	✓	✓	✗	✗
8	Frederico [116]	2021	✓	✓	✗	✓	✓	✓	✗	✓	✗	✓	✗	✗	✓	✓	✓	✗	✗
9	Lamzaouek et al. [117]	2021	✗	✗	✗	✗	✗	✓	✗	✗	✗	✗	✗	✓	✓	✗	✗	◯	◯
10	Schroeder and Lodemann [118]	2021	✓	✗	✗	✓	✗	✓	✗	✓	✗	✗	✓	✓	✗	✗	✗	◯	◯
11	Sharma et al. [119]	2021	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✓	✗	◯	◯
12	Zafarzadeh et al. [120]	2021	✓	✓	✓	✗	✓	✓	✓	✓	✗	✓	✓	✓	✓	✓	✗	◯	◯
13	Debnath et al. [121]	2022	✓	✓	✓	✓	✓	✓	✓	✗	✗	✗	✗	✓	✓	✗	✗	◯	◯
14	Jafari et al. [122]	2022	✓	✓	✓	✓	✓	✓	✓	✗	✗	✓	✓	✓	✓	✓	✓	✗	✓
15	Malik et al. [123]	2022	✓	✓	✓	✗	✓	✓	✓	✓	✗	✓	✓	✓	✓	✓	✓	✗	✓
16	Phan et al. [124]	2022	✓	✓	✓	✗	✓	✓	✓	✓	✓	✓	✓	✓	✓	✗	✗	◯	◯
17	Mansouri et al. [125]	2023	✗	✓	✗	✓	✓	✗	✗	✓	✗	✓	✗	✗	✗	✓	✓	✗	✗
18	Tikwayo and Mathaba [126]	2023	✗	✓	✗	✓	✓	✓	✓	✓	✗	✓	✓	✓	✓	✗	✓	✗	✓

(✓) Implies a “positive answer”; (✗) implies a “negative answer”; (◯) implies “not applicable”.

) using the same reference for adopting the SLR—a study accomplished by Tranfield et al. [108]—it is evident that the level of the methodological adherence and rigor is not the same in those studies. This is an indicator of limited attention from academia to the core practice of SLR as a tool for conducting a rigorous literature review.

The underpinning motivation of conducting a systematic literature review, equivalently known as the review purpose, was expatiated in Section 3.2 within the context of the typology of the literature review. The main concern of this discussion, along with highlighting 16 types of literature review—this list was extended to 17 by incorporating the bibliometric analysis, was to emphasize the distinction between the SLR as a procedural toolkit and a literature review type. To that end, the authors of a literature review study are urged to decide on the review type according to the main purpose of the review while benefiting from the SLR to ensure the reliability of findings and delivering a rigorous study. In this regard, none of the collected research items addressed this concern sufficiently, even though the main objective of the literature review is explained in each paper. In addition, the analyzed articles suffer from a lack of communication between a pertinent review method and the study goal. For instance, Silva et al. [110] defined their study goal as “So, the intention of this paper is […] to conduct an analysis of the state-of-art related to SSCM (sustainable supply chain management) topic, considering all dimensions of sustainability, through a systematic literature review”. As observed, the SLR framework is identified as the main review method, without any justification corresponding to the review type in conjunction with the review purpose. They explained the review method as follows: “The systematic literature review was chosen, since this procedure establishes well-defined and structured criteria within the process of searching and analyzing information about the topic, in this case considering three main factors: The year of publication, number of citations and impact factor, under a multicriteria decision aid perspective”. This primarily indicates that the authors intended to perform a quantitative research synthesis—without referring to a proper methodology, e.g., a metanarrative—and more importantly, reveals that the authors recognized the SLR explicitly as a review type, not as a tool. In practice, the assessment of the collected papers signified that the decision on the literature review typology paled in comparison with the pervasive utilization of the SLR framework, and, thus, the authors considered the SLR as a fundamental review type in conducting a literature review study, such as in [111,117,119], and so forth. Hence, we emphasize that the authors of the literature review studies should articulate the review purpose and, accordingly, expound the review type and the proper approach before the employment of the SLR framework. According to the collected sample, the majority of papers accomplished a scoping review which is one of the most considered review types with respect to the review purpose to “describe”.

Another important conceptual issue refers to the vague distinction between the main review method/type and the bibliometric analysis. A literature review study may benefit from a hybrid approach, such as enriching the qualitative research synthesis, e.g., a scoping review, by the findings from the bibliometric analysis. However, the findings and the applied steps pertaining to each approach deserve a high degree of elaboration. As depicted in Table 4, the examinations of the bibliometric analysis contributed to ten review studies, while only four of them had adequate explanation of the applied approaches.

Apart from the holistic and tactical issues discussed in this section, there are also technical shortcomings in the accomplishment of the SLR steps within the analyzed articles. The following points shed light on some of the most significant issues:

• The exact date of the literature search process is reported only in two studies. Although some of the review papers reported the rough date, e.g., late December, it is recommended to document the exact date of the literature search.
• The number of retrieved items is not explicitly reported in all studies in accordance with each filter or each inclusion/exclusion criterion. To that end, we suggest that the authors take advantage of comprehensive standard flowcharts—such as Figure 3—not only to better illustrate the entire document retrieval process but also to elaborate on the numerical impact of the applied filters and inclusion/exclusion criteria.
• The rationale for the applied filters in the literature search and the inclusion/exclusion criteria pertaining to the quality appraisal demand more elaboration. This concern should be addressed precisely by the authors to ensure the reliability and validity of such filters and criteria. For instance, the publication year associated with the literature search requires theoretical justification. Furthermore, and as a means of inclusion/exclusion criteria, some authors explained the elimination of papers by the fact that they were “not within the scope of the review”. However, this rationale is not technically sufficient and demands more elaboration. The lack of methodological rigor, the lack of empirical evidence, and weak contribution to the topic of interest, are only a few examples that could help the authors in articulating the inclusion/exclusion criteria.
• Some papers are published with limited access. Thus, the number of documents that are not retrieved must be reported and disregarded for further analysis. In this regard, it is recommended not to rely on reading the abstract of such papers instead of the full text.
• It is observed that some of the collected papers suffer from insufficient and vague explanations associated with the quality appraisal step. Although the assessment of a retrieved research document in a review study would differ according to the context of the study, the authors are supposed to pay attention to some of the general rules and document them explicitly in the review report; for instance, the basis of the scanning process in the coarse sieve stage with regard to the different sections of a paper demands clarification, e.g., methodology, utilized data, conclusion, abstract. In this context, we observed that in some papers, the fine sieve stage (the second screening) is performed by reading only the introduction and conclusion sections instead of the full text. Hence, the authors of the literature review papers are encouraged to report the quality appraisal phase with a high degree of detail and transparency.
• Suggesting a future research agenda is among the primary contributions of a literature review study and it should be accomplished with a high level of articulation, particularly compared to an empirical and evidence-based research. This dimension, however, has not received adequate attention in some of the analyzed studies. It is noteworthy to remind of the potential of research synthesis of expanding the cognitive boundaries of a scientific field, and we encourage the authors to invest time in elaborating on the future research agenda.
• Although the majority of papers referred to a review protocol for conducting their study using the SLR, they suffered from a weak link between the structure of the report and the outlined steps. In fact, it is deemed necessary to optimize the study report by elevating the contextual communication between the SLR steps and the organization of the paper. Thus, it is recommended to not only employ a thorough and explicit review protocol but also to adhere to it in preparing the manuscript and reporting the results of each SLR step.

6. Conclusions

Literature review studies, by providing collective and analytical assessments of the earlier scholarly efforts, have a substantial role in presenting the current research landscape and proposing the future research agenda of scientific fields. Given the contribution of the SLR framework to this sphere, the analysis of the WoS database in this study revealed a radical increase throughout the last decade in the number of the literature review studies using the SLR framework, as well as of the studies surrounding the methodological development of the SLR framework, accounting for 90.16% of the total publications in this context since 1965. Despite the promising facet of this trend which is in favor of boosting the element of rigor, it was observed that most of the literature review studies employing SLR suffer from the translucent theoretical perception of the SLR and the lack of methodological consistency and rigorousness. Although there are scientific contributions that support the methodological adoption of the SLR, this study put forward a wider conceptual and practical outlook of this concept. Hence, the primary objective was set to deliver a lucid theoretical mapping of the SLR and a versatile protocol for using its framework. To that end, we sought the answers to two research questions devised for this study:

• Research Question 1: We initially gave critical accounts of the conceptual perception of the SLR and emphasized its application as a procedural tool, not as a literature review type. This step was followed by illuminating the connection between the major elements of a rigorous literature review and an SLR, and we conducted a narrative review to establish a standardized guideline for adopting the SLR in the realm of engineering and management.
• Research Question 2: We outlined the critical steps of an SLR which are not substantially affected by the review type and established an assessment protocol to evaluate the methodological approach of the literature review studies that employed the SLR and were published in the Logistics journal. The processes of this assessment, i.e., literature search, document retrieval, quality appraisal, etc., were accomplished by taking advantage of the SLR guidelines which were recommended in response to the first research question.

The scientific mapping of the SLR elucidated an underpinning principle of conducting a literature review study: the identification of the review purpose and the review type as the fundamental step that precedes the adoption of the SLR. In this regard, the SLR serves as a procedural toolkit that equips the authors for accomplishing a literature review study with methodological consistency, reliability, and rigorousness. As a result, the adherence to the SLR steps ensures the element of rigor associated with the methodological approach of the literature review studies. Evaluating the current practices of the SLR framework, particularly in the field of logistics, revealed that not only there is a lack of transparency in addressing the review purpose and review type, but, evidently, there is no consistency in the methodological approach of the papers that employ the SLR. This may, however, significantly compromise the replicability of the research and the rigor of the conclusions. Given the fact that digitalization, data-driven technology adoption, and the triple bottom line have been increasingly becoming the norm to form the future human-centric, sustainable, and resilient logistics transformation [122,127,128], robust SLR studies are of fundamental importance to sift through vast amount of the literature to identify the most significant and impactful research trends. However, the methodological inconsistency may hinder the use of new technological enablers and the adoption of sustainability measures. In this regard, our standardized guideline can help researchers in logistics to enhance the methodological rigorousness, transparency and replicability, quality and relevance, and knowledge synthesis of the literature review studies through the adoption of the SLR. Based on the comparative assessment of the literature review papers published in the Logistics journal, this paper proposed a series of recommendation to better guide the future SLR studies, and some of them are highlighted below; notably, from a broader perspective, these recommendations may also be applicable to other fields within the realm of engineering and management.

• Reporting the exact date of the literature search.
• Articulating the rationale for the search filters and inclusion/exclusion criteria.
• Explicitness in documenting the review protocol in the review report.
• Consistency between the paper structure and the review protocol and procedure.
• Adequate methodological transparency with respect to the quality appraisal.

Although it is imperative to enrich the literature review studies with a transparent and consistent methodological approach, some of the SLR steps are intrinsically flexible and also could be impacted by the literature review type, such as the data extraction and data synthesis. Therefore, the suggested guidelines embrace a degree of genericity for versatile application. The other aspect of research limitations in this study refers to the methodological assessment of the literature review papers using the SLR. The reflected points are confined to the Logistics journal, and, thus, considering the literature review papers of another journal would potentially lead to different results. Last but not least, it is important to develop future research following the incremental application of technology and innovative tools in the sphere of the literature review studies. In this context, we put forward a multidimensional assessment of the potential contribution pertinent to artificial intelligence (AI) by not only focusing on the enabling factors and opportunities provided by AI, but also investigating the possible threats and pitfalls.