Next Article in Journal
Bridging the Gap between Physical and Circuit Analysis for Variability-Aware Microwave Design: Modeling Approaches
Previous Article in Journal
Research on Secure Communication on In-Vehicle Ethernet Based on Post-Quantum Algorithm NTRUEncrypt
Previous Article in Special Issue
Cefuzz: An Directed Fuzzing Framework for PHP RCE Vulnerability
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Effectiveness Evaluation of Different IDSs Using Integrated Fuzzy MCDM Model

1
Department of Computer Science, College of Computers and Information Technology, Taif University, P.O. Box 11099, Taif 21944, Saudi Arabia
2
Department of Information Technology, Babasaheb Bhimrao Ambedkar University, Lucknow 226025, Uttar Pradesh, India
3
Department of Information Technology, College of Computers and Information Technology, Taif University, P.O. Box 11099, Taif 21944, Saudi Arabia
4
Department of Computer Science and Engineering, Babu Banarasi Das University, Lucknow 226028, Uttar Pradesh, India
*
Author to whom correspondence should be addressed.
Electronics 2022, 11(6), 859; https://doi.org/10.3390/electronics11060859
Submission received: 3 February 2022 / Revised: 28 February 2022 / Accepted: 7 March 2022 / Published: 9 March 2022
(This article belongs to the Special Issue Advances on Networks and Cyber Security)

Abstract

:
Cyber-attacks are becoming progressively complicated; hence, the functional issues of intrusion-detection systems (IDSs) present ever-growing challenges. Failing to detect intrusions may jeopardize the trustworthiness of security services, such as privacy preservation, authenticity, and accessibility. To fight these risks, different organizations nowadays use a variety of approaches, techniques, and technologies to safeguard the systems’ credibility. Establishing policies and procedures, raising user awareness, implementing firewall and verification systems, controlling system access, and building computer-issue management groups are all examples of safeguarding methods. There is a lack of sufficient emphasis on the effectiveness of intrusion-detection systems. In enterprises, IDS is used to analyze the potentially dangerous activities taking place within the technological settings. The selection of efficient IDS is a challenging task for organizations. This research evaluates the impact of five popular IDSs for their efficiency and effectiveness in information security. The authors used the fuzzy analytical hierarchy process (AHP) and fuzzy technique for order performance by similarity to ideal solution (TOPSIS)-based integrated multi-criteria decision-making (MCDM) methodology to evaluate the efficacy of the popular IDSs. The findings of this research suggest that most of the IDSs appear to be highly potential tools. Even though Snort is extensively deployed, Suricata has a substantial advantage over Snort. Suricata uses multi-threading functionality in comparison to Snort to boost the processing performance.

1. Introduction

Communication networks are an integral part of our lives in the digital age. They are a privilege to the digital world. They introduce the rest of the world relatively close to all of us. The challenge of intrusion became much more prevalent with the network. Hackers are always present in the virtual environment. Business organizations could be vulnerable to cyber-attacks. It is necessary to take significant action to analyze network attacks and return them to normal. As a result, intrusion-detection systems (IDS) are important in a network security solution. IDS is a solution that detects network attacks. IDS assists anyone in detecting network traffic. IDS can further initiate an immediate alert. This will assist the IT workforce in dealing with such challenges. However, it would not stop a threat from impacting system applications. It analyzes daily network activities by using the guidelines. The security management system receives an alert through IDS in case any malicious activity is detected over the network. As a result, it is easier for the security expert to identify such suspected activities. The traffic information is moved in bulk and then analyzed for any unusual behavior in the data. The team explores threats by using known attack signatures, as well as trends [1,2,3,4,5,6].
IDS is a mechanism that detects discrepancies in capturing attackers before they cause serious damage to the network system. On the user’s computer, a host-based IDS is installed. The network is monitored by a network-based IDS. During the normal assignment, IDSs search for signatures from attack patterns. These abnormalities are reported to the knowledge base and, after that, analyzed at the guidelines and application layers based on the policy and procedures. IDS could be used as software or as a network security device. Since the IDSs only need to identify the risks, these systems are positioned outside of band on the communications infrastructure, indicating that the transmitter and the recipient of data do not communicate in real-time. IDS systems commonly use a Test Access Point (TAP) or Switch Port Analyzer (SPAN) port for the analysis of a copy of the Inline Traffic Stream, so that IDS does not affect the performance of the Inline Network. This has been established initially by IDS, since the required intrusion analysis at the time cannot be conducted at a pace that is consistent with the elements on the network infrastructure’s direct communication line [7,8,9,10]. Figure 1 shows the general architecture of IDS.
An efficient IDS and mitigation systems are important for an organization’s normal system functioning. Traditional defense technology has become increasingly inefficient as a result of attacker tactics, such as obfuscation techniques, metamorphism, and polymorphism, which increase malware’s resistance. Discrepancies are detected by the IDSs to capture attackers that can cause significant harm to the organization. They could be a network or even a host. Due to new attacks developing almost daily, IDSs are crucial in identifying and responding to potential system intrusions. IDSs should change and continually adjust to all of these new threats and assault techniques. The challenge on which scholars have been researching for decades is how to construct effective, efficient, and responsive IDSs [11,12,13,14,15].
The evaluation of IDSs is a popular and challenging task in research at present. Choosing an efficient IDS to secure a business network must not be undertaken lightly, rapidly, or without a thorough grasp of the technology, solutions, and potential consequences. The decision-making process can be broken down into several stages, which include determining the requirement, gaining a broad awareness of IDSs, more in-depth knowledge of the network, and also determining policy and processes by evaluating various IDS solutions [16]. This paper evaluates the effectiveness of different IDSs, using the integrated fuzzy analytical hierarchy process (AHP) technique for order performance by similarity to the ideal solution (TOPSIS) model.
The remainder of this paper is structured as follows: The study outlines relevant research efforts in this domain in Section 2. Section 3 discusses the overview of several types of popular IDSs. Section 4 discusses the integrated fuzzy AHP-TOPSIS technique. Section 5 of this work contains the findings, comparisons of the findings, and a sensitivity analysis. Section 6 closes with the summary and conclusions.

2. Related Research Studies

Upendran and Gopinath [16] proposed an improved entropy-based TOPSIS approach to recommend one or more options from a large set of alternatives. To decrease the number of the network traffic sample, they applied five feature selection strategies. To measure the calculation time, as well as intrusion-detection time, classification techniques, such as Artificial Neural Network (ANN), Naive Bayes, and Support Vector Machine (SVM) are employed. Their suggested TOPSIS approach is used to monitor the effectiveness of feature selection in order to improve intrusion detection.
Hu et al. [17] evaluated two well-known open-source intrusion-detection systems, Snort and Suricata, as well as their comprehensive comparative standards, to develop a better understanding of drop rates, as well as identification efficiency on 100 Gb/s networks. Furthermore, they investigated critical parameters (such as system resource utilization, packet processing power, packet drop frequency, and identification accuracy) which constrain IDSs’ application to high-speed networks. Moreover, they also discussed a complete investigation to demonstrate the effectiveness of IDSs utilizing various setups, traffic levels, and flows. They outlined the difficulties of utilizing open-source IDSs in strong networks, offered solutions to assist network managers in addressing identified concerns, and also presented some suggestions for establishing novel IDSs that may be employed in high-speed networking.
Imoize et al. [18] suggested an expansive and cost-based intrusion-detection system. Based on this approach, an objective metric driven by information theory is introduced, and a package for determining the intrusion-detection capabilities of an intrusion-detection system (IDS), given specified input parameters, is constructed in Java. For each operational IDS, the decision-making methodology is applied to evaluate the projected costs and the capacity to detect false-positive rates.
Saber et al. [19] developed a testing method to monitor the effectiveness of the IDS elements and their impacts on the whole system. The assessment is based on several tests. In addition, the effect of the implementation parameters of IDS was studied. They also developed the IDS SNORT on systems with various technical features, and they have constructed a network to produce a series of experiments to evaluate the performance of a higher bandwidth network deployment.
Shiaeles et al. [20] introduced a DDoS detection technique based on developing a fuzzy estimator for the mean packet inter-arrival rate. They separated the task into two concerns: the first was detecting the DDoS incident in progress, and the second was identifying the offender’s IP addresses. They set strong real-time limits on the first assignment and more flexible restrictions on address recognition. They also demonstrated through practical assessment that the identification can be accomplished within better real-time restrictions and by utilizing fuzzy estimators rather than crisp statistical classifiers.
Schrötter et al. [21] developed a standard set for assessing intrusion-detection systems in IPv6 settings. This standard is used to compare the popular intrusion-detection systems, such as Snort, Zeek, and Suricata. Furthermore, an IPv6 Plugin Suite was also provided and assessed, which improved Snort by detecting stateful attacks. Their evaluation results indicated the current ability to identify IPv6 connection attacks.
As intrusion-detection systems are particularly built to function on specific systems and situations, they are challenging to implement. Therefore, there is a significant need to evaluate the effectiveness of different IDSs based on their approaches and rules that are present in the most recent version of the IDSs. This fundamental challenge is more succinctly expressed as the intrusion-detection-assessment challenge, and its solution is typically dependent on a number of parameters in a hierarchical structure. Prior studies, however, suggested that no single model was ideal for all issues. Using a highly integrated fuzzy-based strategy may yield greater results than other approaches. To the extent of our knowledge, our work is the first study that examines several IDSs systematically employing an integrated fuzzy multi-criteria decision-making (MCDM)-based technique. The effectiveness of five major intrusion-detection systems, including Zeek, Suricata, Security Onion, OSSEC, and Snort, is examined in this research. The effectiveness is evaluated by using a hierarchical structure based on the MCDM solution.

3. Different Types of Intrusion-Detection Systems

3.1. Zeek

Zeek [22,23] is a network intrusion-detection system that runs on Unix (IDS). Zeek analyzes network traffic and recognizes intrusion attempts, depending on the type and substance of the traffic. Zeek IDS was previously known as Bro. Zeek identifies intrusions by matching network traffic to rules that describe undesirable events. These rules may define activities (for example, certain hosts interacting to specific services), what actions warrant alerting (for example, attempts to a particular number of distinct hosts constitute a “scan”), or signatures characterizing known attacks or exposure to security issues. If Zeek discovers anything of interest, the direction could be to either write a log entry or run an operating system function. Zeek is designed for high-speed (Gbps) and increased intrusion detection. Zeek IDS can perform competently while operating on commercially accessible PC hardware by intelligently exploiting packet filtering methods, and so can provide an economically efficient way of monitoring a site’s Internet communication.

3.2. Suricata

Suricata was designed in 2010 by the OISF (Open Information Security Foundation), with funding from the US Department of Homeland Security [24]. Suricata’s design is quite close to snort’s, except that, instead of using a single thread to process packets, Suricata uses many threads [25]. This enables Suricata’s distinctive feature, which is to maximize the capacity to obtain packets. Snort was single-threaded; therefore, when packets exceeded the capacity to obtain bandwidth, Snort disregarded them. Thus, multi-threading is a useful capability of Suricata. Suricata has many detection algorithm threads.

3.3. Security Onion

Security Onion [26] is a Linux-based intrusion-detection approach that includes several IDS that are both host-oriented (HIDS) and network-oriented (NIDS). Security Onion can collect and analyze an extensive range of data. This comprises information on the host, connection, session, resource, alerting, and standards. Security Onion can be deployed as a solo implementation with a server and detector, or with a master server and many detectors that allow the platform to be expanded as needed. Numerous gateways and tools are present for system management and information analysis, including Sguil, Snorby, Squert, and Enterprise Log Search and Archive (ELSA). It offers host-based identification in the form of OSSEC HIDS, as well as network-based detection via Snort, Suricata, and Zeek NIDS. Security Onion is particularly customizable because it may be set as a master server with numerous sensors or as an independent or hybrid installation. The information gathered by Security Onion is saved in log files and also in a Sguil database, which includes a new interface for recording and analysis.

3.4. OSSEC

OSSEC is an open-source intrusion-detection system built by Daniel B. Cid, who sold the product to Trend Micro in 2008 [27]; however, the project remained a free and open source. The most recent stable release is 2.9.3. It is made up of numerous services and modules, each of which has its own distinct set of intrusion-detection capabilities. HIDS has numerous elements, and OSSEC combines them all to provide certain fundamental advantages. OSSEC ensures that security compliance standards are met. Many consumers, primarily corporate clients, demand that the companies with whom they do business have particular security compliances, including Payment Card Industry Data Security Standard (PCI DSS), Health Insurance Portability and Accountability Act (HIPAA), and so on. Analyzing logs and evaluating them for suspicious activity is a one-way OSSEC that helps firms comply with various security standards [28].

3.5. Snort

Snort fills an essential “ecological niche” in network security by serving as a cross-platform lightweight network intrusion-detection program that can be used to monitor tiny TCP/IP networks, as well as discover a wide range of abnormal network traffic and also explicit cyberattacks. It can offer controllers adequate information to make informed judgments on how to proceed in the event of suspicious behavior. Snort could also be quickly implemented to cover any gaps in a network’s protection measures, such as when a big threat occurs and corporate security companies take their time releasing new attack identification signatures. Snort is beneficial when it is not cost-effective to install commercial NIDS sensors. Current commercialized intrusion-detection systems cost thousands or tens of thousands, or perhaps even hundreds of thousands of dollars in extreme situations [29,30]. Snort was created to encounter the requirements of a typical compact network intrusion-detection system. It has progressed into a compact, versatile, and high-performing technology that is used on both large and small networks around the globe. It has met its initial design aims and is a perfectly capable solution to corporate intrusion-detection systems in locations where installing standard production systems would be prohibitively expensive.
Table 1 shows the comparison of the five popular intrusion-detection methods.

4. Methods

4.1. Identification of Evaluation Criteria and Alternatives

MCDM is a discipline of combinatorial optimization in which the alternatives are evaluated to identify the best alternative that meets a set of various and frequently contradictory parameters. MCDM is a critical component of the decision-making principle and operational investigation. It is frequently regarded as trustworthy. It is a set of strategies and techniques for integrating various and contradictory parameters into a decision-making process. Furthermore, MCDM might be regarded as a systematic approach for evaluating and selecting between possibilities. It seeks to divide an issue into smaller components, analyze and evaluate each part, and then aggregate those components to determine the appropriate reasonable alternative from a range of options based on a stated set of parameters. In uncertain, unclear, fuzzy, or risky contexts, MCDM tries to help decision-makers to tackle contradictory real-world statistical and/or qualitative subjective multi-criteria challenges, and to select best-fit options from a group of options. Figure 2 shows the hierarchy for the assessment of some popular intrusion-detection systems (IDS) in this research study. Based on an analysis of the relevant literature, as well as insight from seventy-seven security experts, the four considerable factors at level one, as well associated sub-factors at level two, in the current method that make a significant contribution to the assessment of multiple IDSs were evidently identified and designed. The four main criteria to evaluate the different popular IDSs are Types, Audit source location, Targets, and Protected system denoted as M1, M2, M3, and M4 respectively. The significant criterion Types contain the different types of IDSs. The IDSs may be open-source, closed-source, or freeware, denoted by M11, M12, and M13, respectively. Different IDSs are classified by the type of input data they evaluate at the audit source location. Audit procedures on a host log file, network packets, application log files, or sensor alerts denoted by M21, M22, M23, and M24, respectively, produced by other IDSs can all be used as input data. The targets’ criterion shows the IDS capability to detect potential attacks against specific targets. These targets may be application, network, or host, as denoted by M31, M32, and M33, respectively. The protected system criteria show the approach of intrusion detection. It can be HIDS, NIDS, or hybrid types denoted by M41, M42, and M43, respectively. The five alternatives, namely Zeek, Suricata, Security Onion, OSSEC, and Snort, are represented by S1, S2, S3, S4, and S5, respectively.

4.2. Fuzzy AHP-TOPSIS Methodology

Problems with decision-making are likely a consequence of putting too much emphasis on analogical reasoning that has worked previously. When attempting to make a judgment or decision, researchers use a predictive model, which is a heuristic algorithm or guiding principle. These strategies can help in making better decisions by reducing our cognitive strain, and they can also lead to mistakes. However, AHP is unable to resolve the fundamental uncertainty and inaccuracies in a decision-maker’s response to genuine statistical information. Because the real world is so indistinct, researchers noted that experts have combined the fuzzy theory with AHP to investigate obscure real-world problems [31,32,33]. Furthermore, while the AHP method is typically based on a highly volatile scale of decisions, the fuzzy AHP does have deficiencies [23,24]. As a result, a combined AHP and TOPSIS fuzzy method is a special process that could aid in the efficacious evaluation of options. Moreover, the fuzzy AHP-TOPSIS technique is as follows:
Fuzzy AHP: Fuzzy AHP is a popular methodology for resolving difficult selection challenges. Every complicated topic can be investigated by using remarkable categorized ranges of objectives, i.e., hierarchy. With the help of fuzzy AHP, the problem is differentiated into a tree form to describe it. Figure 2 shows how to make a tree shape. This tree shape was created with the help of experts’ opinions [34]. The triangular fuzzy number (TFN) is then constructed from the hierarchical structure. A pair-wise comparison of each group of categorized goals is critical because of the impact of one criterion on other criteria.
Transforming linguistic numbers into crisp numbers, as well as TFN, is the next step. The TFN is used in this research, and it ranges from 0 to 1 [35]. The quantitative simplification of TFN membership functions, as well as their capabilities to handle with fuzzy data, is driving this implementation [23]. In addition, linguistic values are categorized as equally important or weakly important, and crisp values are categorized as 1, 2, …, 9. Furthermore, if the membership functions of a fuzzy number M on F are recognized, it is referred to as TFN:
µ a   ( x ) = a [ 0 , 1 ]
µ a ( x ) = { x m i l l m i l x [ l , m i ] x m i u u m i u x [ m i , u ]
In the triangular membership function, l, mi, and u represent the lower, middle, and upper limits, respectively. A TFN is shown in Figure 3.
A TFN can be written as (l, mi, u). Specialists assigned numerical scores to the elements that influence the values using a measure that is highlighted in Table 2.
The numerical methods (3–6) are used to convert numeric values into TFNs that are identified as (lij, miij, and uij), where lij is lower significance, miij is middle significance, and uij is uppermost significance events. Moreover, TFN (ɳij) is represented as follows:
Φ i j = ( l i j ,   m i i j ,   u i j )
where l i j   m i i j   u i j
l i j = m i n ( J i j d )
m i i j = ( J i j 1 , J i j 2 , J i j 3 ) 1 x
a n d       u i j = m a x ( J i j d )
In Equations (3)–(6), Jijd specifies the proportional position of the values among two elements that is quantified by specialist d, where i and j indicate a pair of elements being decided by specialists. Φij is calculated according to the geometric mean of a specialist’s observations for a particular assessment. The geometric mean is capable of appropriately joining and demonstrating the consent of specialists and designates the lowest and highest marks, respectively, for the relative consequence among the two elements. Additionally, Equations (7)–(9) provide the joint TFN values. Take into account the two TFNs, M1 = (l1, mi1, u1) and M2 = (l2, mi2, u2). The procedures of actions on them are as follows:
( l 1 ,   m i 1 ,   u 1 ) + ( l 2 ,   m i 2 ,   u 2 ) = ( l 1 + l 2 ,   m i 1 + m i 2 ,   u 2 + u 2 )
( l 1 ,   m i 1 ,   u 1 ) × ( l 2 ,   m i 2 ,   u 2 ) = ( l 1 × l 2 ,   m i 1 × m i 2 ,   u 1 × u 2 )
( l 1 , m i 1 , u 1 ) 1 = ( 1 u 1 , 1 m i 1 , 1 l 1 )
With the support of the equation, a fuzzified pair-wise comparative matrix in the state of n × n matrix is formed after acquiring the TFN variables for every pair of comparisons (10).
A d ˜ = [ k ˜ 11 d k ˜ 12 d     k ˜ 1 n d k ˜ 21 d k ˜ 22 d     k ˜ 2 n d         k ˜ n 1 d k ˜ n 2 d k ˜ n n d ]
where k i j k ˜ symbolizes the dth decision-makers’ priority of the ith standards over the jth criteria. If more than one responsible party is prevalent, the equation is used to calculate the average of every judgment-maker’s priorities (11).
k ˜ i j = d = 1 d . k ˜ i j d
The next step is to use the equation to modify the pair-wise comparative matrixes for all elements in the hierarchical order based on the averaged priorities (12).
A ˜ = k 11 ˜     k 1 n ˜         k n 1 ˜     k ˜ n n  
The fuzzy geometrical mean and fuzzy weights of each element are then described by using the geometrical mean method, as demonstrated in Equation (13).
p ˜ i = ( j = 1 n . k ˜ i j ) 1 n ,   i = 1 ,   2 ,   3   , n
The next step is to use the equation to calculate the identified element’s fuzzy weight (14).
w ˜ i = p ˜ i ( p ˜ 1 p ˜ 2 p ˜ 3     p ˜ n ) 1
Equations are also used to determine the average and normalized weight criteria (15–16).
M i = w ˜ 1 w ˜ 2     w ˜ n n
N r i = M i M 1 M 2     M n
Additionally, the Centre of Area (COA) method is used to estimate the BNP (best non-fuzzy performance) number of the fuzzy weights of each quantity with the support of Equation (17).
B N P w D 1 = [ ( u w 1 l w 1   ) + ( m i w 1 l w 1   ) ] 3 + l w 1
Fuzzy TOPSIS: TOPSIS sees multi-standard decision-making problems, with m options as a geometrical configuration with m points inside the n-dimensional problem area. The strategy used in this article for TOPSIS is principally founded on the notion that a designated possibility is the relatively short and farthest range from the positive-ideal solution, as well as the negative-ideal solution, correspondingly, for best possible and lowest ideal solutions [36]. Specialists have a difficult time assigning a particular performance rating to an alternative based on criteria. TOPSIS uses fuzzy numbers rather than precise numerals to demonstrate the relative importance of criteria in order to be consistent with the actual fuzzy situation. Furthermore, the fuzzy AHP-TOPSIS method is most suitable for handling group decision-making issues in fuzzy settings. The following is the fuzzy AHP-TOPSIS methodology:
The first step is to determine the weights of the evaluation criteria. With the support of equations, this research utilized fuzzy AHP to arrive at fuzzy choice weights (1–16). Moreover, with support of Equation (18), as well as Table 3, the researcher designed the fuzzy decision matrix and selected the best linguistic aspects as criteria possibilities.
C 1       C n K ˜ = A 1 A m [ x ˜ 11 x ˜ 1 n x ˜ m 1 x ˜ m n ]
where x ˜ i j = 1 D ( x ˜ i j 1     x ˜ i j d     x ˜ i j D ) , and x ˜ i j d is the assessment of the alternative Ai in terms of factor Cj effectiveness estimated by the dth specialist, as well as x ˜ i j d = ( l i j d , m i i j d ,   u i j d ).
The subsequent step is to use an expression to normalize the fuzzy decision matrix (19). The normalized fuzzy assessment matrix denoted by P ˜ is represented as follows.
P ˜ = [ p ˜ i j ] m × n
After that, using equations, one may complete the normalizing procedure by using Equation (20).
p ˜ i j = ( l i j u j + , m i i j u j + , u i j u j + ) ,   u j + = m a x { u i j ,   i = 1 ,   2 ,   3 , n }
Otherwise, we can establish the highest anticipated level, u j + , and j = 1, 2, …, n is equivalent to 1; or else, the worst is 0. Furthermore, the normalized p ˜ i j remains to be TFNs. For TFNs, the normalization procedure can be achieved in an analogous way. The subjective fuzzy normalized assessment matrix ( Q ˜ ) is measured with the support of Equation (21).
Q ˜ = [ q ˜ i j ] m × n i = 1 , 2 ,   , m ;   j = 1 ,   2 ,   3   , n
where q ˜ i j = p ˜ i j w ˜ i j also, at that point, describes the Fuzzy Positive-Ideal Solution (FPIS), as well as Fuzzy Negative-Ideal Solution (FNIS). The subjective normalized fuzzy assessment matrix specifies that the components q ˜ i j are normalized positive TFN and their choices fit in to the closed interval [0, 1]. Afterward, we can designate the FPIS A+ (aspiration levels) and FNIS A (the worst levels), as presented in Equations (22) and (23).
A + = ( q ˜ 1 ,       * q ˜ j ,       * q ˜ n , * )
A = ( q ˜ 1 ,       * q ˜ j ,       * q ˜ n , * )
where q ˜ 1 * = ( 1 , 1 , 1 )     w ˜ i j = ( L w j , M w j , H w j ) and q ˜ i j = ( 0 , 0 , 0 ) , j = 1 ,   2 ,   3   , n . For computing the space of every alternative solution from FPIS, as well as FNIS, the spaces ( d ˜ i + a n d   d ˜ i ) of every alternative solution from A+ and A can be assessed with the area compensation procedure, as presented in Equations (24) and (25).
d ˜ i + = j = 1 n . d ( q ˜ i j , q ˜ i j * ) i = 1 , 2 ,   , m ;   j = 1 , 2 , 3   , n
d ˜ i = j = 1 n . d ( q ˜ i j , q ˜ i j * ) i = 1 , 2 ,   , m ;   j = 1 , 2 , 3   , n
The next stage is to identify the closeness coefficients (absolute gaps’ degree) and construct the alternative solutions for achieving the aspiration degrees in each element. Ying-Chyi Chou et al. recommended that C C ˜ i is cleared to estimate the point of the fuzzy gap based on the fuzzy closeness coefficients to increase the alternatives solution [37]. Once the d ˜ i + and d ˜ i of every alternative solution is assessed, the comparisons to the ideal clarification are computed. This procedure resolves the equation’s similarity to the best choice (26).
C C ˜ i = k ˜ i k ˜ i + + k ˜ i = 1 k ˜ i + k ˜ i + + k i   ,   i = 1 , 2 ,   , m
where k ˜ i k ˜ i + + k ˜ i is demarcated as the fuzzy satisfaction amount in the ith alternative solution, and k ˜ i + k ˜ i + + k ˜ i is the fuzzy-gap amount in the ith alternative solution. Based on the ranks of alternatives, the solution is accomplished. The subsequent step is to evaluate the various IDSs by using their contributing qualities.

5. Numerical Data Analysis

5.1. Statistical Findings

It should be necessary to evaluate the effectiveness of different intrusion-detection systems [33,34]. A condition that is hard to ascertain is unsuitable for the job. Prohibitively expensive evaluations still would not yield the anticipated outcomes. The gap of cost-benefit analysis is advantageous toward a better assessment, as long as it includes the necessary standards for reliability and bias. A common method for estimating effectiveness is to evaluate the IDS numerous times and then determine a “significance level of measurement”. An independent evaluation is better than one that is biased. Measuring bias, on the other hand, is challenging. In this research, we used an integrated fuzzy AHP-TOPSIS based methodology to evaluate the different popular IDSs. First, seventy-seven researchers and cybersecurity specialists with different IDS experiences were consulted for each parameter set and relevant technologies. These 77 decision-makers are composed of 30 cybersecurity specialists with more than 12 years of experience and 47 researchers from various security organizations with 10 years of IDS research experience. The specialists were required to submit and assess their viewpoints in a collaborative online setting, and they were given information on the degree of the variables in relation to the various groups, as well as linguistic standards. To begin, the decision-maker creates a pair-wise comparison matrix containing the parameters. The decisions are considered valid, since the consistency ratio (CR) is less than 0.1 or near to it. Table 4 demonstrates the combined fuzzify pair-wise assessment matrix at the initial first level. The fuzzy-aggregated pair-wise comparison matrix at the second level for Types, Audit Source Location, Targets, and Protected System is presented in Table 5, Table 6, Table 7 and Table 8. For every second-level phase, the global weights were calculated. The statistical findings were tabularized in Table 9, Table 10, Table 11, Table 12 and Table 13. Table 14 shows the overall weights and ranking of different factors. Table 15 shows the evaluators’ subjective intelligence findings in linguistic terminology, and Table 16 shows the fuzzy-decision matrix with normalized decisions. Furthermore, Table 17 demonstrates the fuzzy-decision matrix with weighted normalization. In addition, with the support of the hierarchical structure, Table 18 and Figure 4 illustrate the comprehensive and the ultimate relative closeness of the different alternatives.
Depending upon the value of satisfaction degree shown in Figure 4, it is concluded that the initial ranking of the efficient IDS using the integrated fuzzy AHP-TOPSIS approach is S2 > S5 > S3 > S4 > S1 (\>“means\higher to”). Therefore, S2, which is Suricata, is considered the preferable and effective IDS.

5.2. Comparative Analysis

The use of multiple techniques on almost the same data delivers contradictory conclusions. The investigators use a variety of complementary ways to test the correctness of the research outcome. In this investigation, the researcher applied a hybrid fuzzy AHP-TOPSIS-based approach to evaluate the effectiveness of different alternatives. The data-acquisition and -assessment process for that sample in fuzzy AHP-TOPSIS is similar to the traditional AHP-TOPSIS approach. As a result, fuzzification and defuzzification are required for the fuzzy-AHP-TOPSIS. Furthermore, the data for fuzzy AHP-TOPSIS are captured in their initial statistical values and afterward transformed into fuzzy value sets. The outcomes obtained through the traditional method were well correlated with those obtained by using the fuzzy procedure. The comparison analysis’ results were not highly different and varied from each other, but the accuracy of the outcomes was varied. The traditional approaches for the effectiveness evaluation of different IDSs are insufficient for working with the inaccurate or ambiguous quality of linguistic evaluations. Therefore, integrated fuzzy multi-criteria decision-making strategies are developed to tackle this challenge. Moreover, the fuzzy approach confirms the findings of the classic strategy, thus enhancing the validity of the ranking among the five methods. Table 19 and Figure 5 show the comparison of outcomes with the fuzzy and traditional AHP-TOPSIS approach.

5.3. Sensitivity Analysis

The responsiveness was evaluated by altering the parameters to test the validity of the gathered data. During the same statistical analysis, the sensitivity of the obtained weights (factors involved) was evaluated. Throughout the investigation, at the final (second) step, 13 variables were picked so that sensitivities can be investigated by using 12 experiments. In each trial, the degree of satisfaction (CC-i) was found by adjusting the weights of every parameter, while the weight of the other element remained stable, using the Fuzzy-AHP-TOPSIS approach. Reported consequences are shown in Table 20 and Figure 6.
Table 18 represents the real weights of this research investigation in the first row. Taking the performances into account, we see that the alternative (S2) has a high level of satisfaction (CC-i). Twelve experiments were carried out. The results reveal that, after twelve tests, Alternative-2 (S2) consistently retains a high level of satisfaction (CC-i). The performance results demonstrate that the rankings of the alternatives are weight-dependent.

6. Conclusions

In today’s modern environment, network security is significant for small, as well as large, organizations. Modern security challenges that are increasingly complex and advanced are always being developed. Online security issues are becoming more prevalent, and an IDS can assist in defending your organization from malicious activities. An IDS simply analyzes network traffic and notifies the system administrator if any suspicious activity is detected. This paper discusses the five most widely used intrusion-detection systems. Thereafter, the functionality of these five popularly used intrusion-detection systems, including Zeek, Suricata, Security Onion, OSSEC, and Snort, are effectively compared with the help of a hybrid fuzzy-logic-based approach and discussed in this study. All of the IDSs appear to be highly potential tools. Even though Snort is extensively deployed, it is important to note that Suricata is the new-generation multi-threaded application with a broader range of features. Suricata also has a significant benefit over Snort in that it does not need many processes to handle an upsurge in network traffic. The type of connection or attack to be monitored will determine the implementation of a multispectral IDS. The recommendation to network administrators is based on a generic network design and frequent network obstructions; however, this may vary depending on the network infrastructure of the business entity. The target for future research work is to deploy similar research with other MCDM models. The real-world dataset would be reviewed, and final assessments would be easily measured without the assistance of any other third entity.

Author Contributions

Conceptualization, H.A., M.T.J.A., A.A. (Abdullah Alharbi), W.A. and M.A.; methodology, H.A., M.T.J.A., A.A. (Abdullah Alharbi), A.A. (Alka Agrawal) and R.K.; software, M.T.J.A., A.A. (Alka Agrawal) and R.K.; validation, M.T.J.A., A.A. (Abdullah Alharbi), A.A. (Alka Agrawal) and R.K.; formal analysis, H.A., M.T.J.A., A.A. (Alka Agrawal) and R.A.K.; investigation, H.A., M.T.J.A., A.A. (Abdullah Alharbi), W.A., M.A., D.P., A.A. (Alka Agrawal), R.K. and R.A.K.; resources, H.A., M.T.J.A., A.A. (Abdullah Alharbi), W.A. and M.A.; data curation, M.T.J.A., A.A. (Abdullah Alharbi), A.A. (Alka Agrawal) and R.K.; writing—original draft preparation, M.T.J.A., A.A. (Abdullah Alharbi), W.A., M.A., D.P., A.A. (Alka Agrawal) and R.A.K.; writing—review and editing, M.T.J.A., A.A. (Abdullah Alharbi), W.A., D.P., A.A. (Alka Agrawal) and M.A.; visualization, M.T.J.A., A.A. (Abdullah Alharbi), W.A., M.A., D.P. and R.A.K.; supervision, H.A., M.T.J.A., A.A. (Alka Agrawal) and R.A.K.; project administration, H.A., A.A. (Abdullah Alharbi), W.A. and R.A.K.; funding acquisition, H.A., A.A. (Abdullah Alharbi), W.A. and R.A.K. All authors have read and agreed to the published version of the manuscript.

Funding

The project was funded by Researchers Supporting project number (TURSP-2020/306), Taif University, Taif, Saudi Arabia.

Acknowledgments

This research was supported by Taif University Researchers Supporting Project number (TURSP-2020/306), Taif University, Taif, Saudi Arabia.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Sen, S. A survey of intrusion detection systems using evolutionary computation. In Bio-Inspired Computation in Telecommunications; Morgan Kaufmann: Burlington, MA, USA, 2015; pp. 73–94. [Google Scholar]
  2. Ansari, M.T.J.; Pandey, D.; Alenezi, M. STORE: Security threat oriented requirements engineering methodology. J. King Saud Univ.-Comput. Inf. Sci. 2018, 34, 191–203. [Google Scholar] [CrossRef]
  3. Ali, M.; Shiaeles, S.; Bendiab, G.; Ghita, B. MALGRA: Machine learning and N-gram malware feature extraction and detection system. Electronics 2020, 9, 1777. [Google Scholar] [CrossRef]
  4. Swann, M.; Rose, J.; Bendiab, G.; Shiaeles, S.; Savage, N. A comparative study of traffic generators: Applicability for malware detection testbeds. J. Internet Technol. Secur. Trans. 2021, 8, 705–713. [Google Scholar] [CrossRef]
  5. Shiaeles, S.N.; Papadaki, M. FHSD: An improved IP spoof detection method for web DDoS attacks. Comput. J. 2015, 58, 892–903. [Google Scholar] [CrossRef] [Green Version]
  6. Ansari, T.J.; Pandey, D. An Integration of Threat Modeling with Attack Pattern and Misuse Case for Effective Security Requirement Elicitation. Int. J. Adv. Res. Comput. Sci. 2017, 8, 16–20. [Google Scholar]
  7. Ansari, M.T.J.; Baz, A.; Alhakami, H.; Alhakami, W.; Kumar, R.; Khan, R.A. P-STORE: Extension of STORE methodology to elicit privacy requirements. Arab. J. Sci. Eng. 2021, 46, 8287–8310. [Google Scholar] [CrossRef]
  8. Alosaimi, W.; Ansari, M.T.J.; Alharbi, A.; Alyami, H.; Seh, A.H.; Pandey, A.K.; Agrawal, A.; Khan, R.A. Evaluating the Impact of Different Symmetrical Models of Ambient Assisted Living Systems. Symmetry 2021, 13, 450. [Google Scholar] [CrossRef]
  9. Ji, H.; Wang, Y.; Qin, H.; Wang, Y.; Li, H. Comparative performance evaluation of intrusion detection methods for in-vehicle networks. IEEE Access 2018, 6, 37523–37532. [Google Scholar] [CrossRef]
  10. Magán-Carrión, R.; Urda, D.; Díaz-Cano, I.; Dorronsoro, B. Towards a reliable comparison and evaluation of network intrusion detection systems based on machine learning approaches. Appl. Sci. 2020, 10, 1775. [Google Scholar] [CrossRef] [Green Version]
  11. Hussein, S.M. Performance evaluation of intrusion detection system using anomaly and signature based algorithms to reduction false alarm rate and detect unknown attacks. In Proceedings of the 2016 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA, 15–17 December 2016; pp. 1064–1069. [Google Scholar]
  12. Nikolopoulos, K.; Punia, S.; Schäfers, A.; Tsinopoulos, C.; Vasilakis, C. Forecasting and planning during a pandemic: COVID-19 growth rates, supply chain disruptions, and governmental decisions. Eur. J. Oper. Res. 2021, 290, 99–115. [Google Scholar] [CrossRef]
  13. Alhakami, W.; Binmahfoudh, A.; Baz, A.; Alhakami, H.; Ansari, M.T.J.; Khan, R.A. Atrocious Impinging of COVID-19 Pandemic on Software Development Industries. Comput. Syst. Sci. Eng. 2021, 36, 323–338. [Google Scholar] [CrossRef]
  14. Ansari, M.T.J.; Khan, N.A. Worldwide COVID-19 Vaccines Sentiment Analysis through Twitter Content. Electron. J. Gen. Med. 2021, 18, em329. [Google Scholar] [CrossRef]
  15. Attaallah, A.; Ahmad, M.; Ansari, M.T.J.; Pandey, A.K.; Kumar, R.; Khan, R.A. Device security assessment of Internet of healthcare things. Intell. Autom. Soft Comput. 2020, 27, 593–603. [Google Scholar] [CrossRef]
  16. Upendran, V.; Gopinath, R. Feature selection based on multi-criteria decision making for intrusion detection system. Management 2021. [Google Scholar] [CrossRef]
  17. Hu, Q.; Yu, S.Y.; Asghar, M.R. Analysing performance issues of open-source intrusion detection systems in high-speed networks. J. Inf. Secur. Appl. 2020, 51, 102426. [Google Scholar] [CrossRef]
  18. Imoize, A.L.; Oyedare, T.; Otuokere, M.E.; Shetty, S. Software intrusion detection evaluation system: A cost-based evaluation of intrusion detection capability. Commun. Netw. 2018, 10, 211–229. [Google Scholar] [CrossRef] [Green Version]
  19. Saber, M.; Belkasmi, M.G.; Chadli, S.; Emharraf, M.; El Farissi, I. Implementation and Performance Evaluation of Intrusion Detection Systems under high-speed networks. In Proceedings of the 2nd International Conference on Big Data, Cloud and Applications, Tetouan, Morocco, 29–30 March 2017; pp. 1–6. [Google Scholar]
  20. Shiaeles, S.N.; Katos, V.; Karakos, A.S.; Papadopoulos, B.K. Real time DDoS detection using fuzzy estimators. Comput. Secur. 2012, 31, 782–790. [Google Scholar] [CrossRef] [Green Version]
  21. Schrötter, M.; Scheffler, T.; Schnor, B. Evaluation of Intrusion Detection Systems in IPv6 Networks. In Proceedings of the 16th International Joint Conference on e-Business and Telecommunications (ICETE 2019), Prague, Czech Republic, 26–28 July 2019; pp. 408–416. [Google Scholar]
  22. Haas, S.; Sommer, R.; Fischer, M. Zeek-osquery: Host-network correlation for advanced monitoring and intrusion detection. In IFIP International Conference on ICT Systems Security and Privacy Protection; Springer: Cham, Switzerland, 2020; pp. 248–262. [Google Scholar]
  23. Paxson, V. Bro: A system for detecting network intruders in real-time. Comput. Netw. 1999, 31, 2435–2463. [Google Scholar] [CrossRef]
  24. Garcia-Teodoro, P.; Diaz-Verdejo, J.; Maciá-Fernández, G.; Vázquez, E. Anomaly-based network intrusion detection: Techniques, systems and challenges. Comput. Secur. 2009, 28, 18–28. [Google Scholar] [CrossRef]
  25. Park, W.; Ahn, S. Performance comparison and detection analysis in snort and suricata environment. Wirel. Pers. Commun. 2017, 94, 241–252. [Google Scholar] [CrossRef]
  26. Burks, D. Security Onion. 2012. Available online: Securityonion.blogspot.com (accessed on 17 January 2022).
  27. Cid, D.B. Log Analysis Using OSSEC. 2017. Available online: http://www.academia.edu/8343225/Log_Analysis_using_OSSEC (accessed on 17 January 2022).
  28. Anafcheh, A. Intrusion Detection with OSSEC. 2018. Available online: https://www.theseus.fi/bitstream/handle/10024/150030/ali-anafcheh-thesis.pdf?sequence=1 (accessed on 17 January 2022).
  29. Roesch, M. Snort: Lightweight intrusion detection for networks. In Proceedings of the LISA ’99: 13th Systems Administration Conference, Seattle, WA, USA, 7–12 November 1999; Volume 99, pp. 229–238. [Google Scholar]
  30. Saaty, T.L. How to make a decision: The analytic hierarchy process. Eur. J. Oper. Res. 1990, 48, 9–26. [Google Scholar] [CrossRef]
  31. Hwang, C.L.; Yoon, K. Methods for multiple attribute decision making. In Multiple Attribute Decision Making; Springer: Berlin/Heidelberg, Germany, 1981; pp. 58–191. [Google Scholar]
  32. Chen, S.J.; Hwang, C.L. Fuzzy multiple attribute decision making methods. Fuzzy Mult. Attrib. Decis. Mak. 1992, 375, 289–486. [Google Scholar]
  33. Ansari, M.T.J.; Al-Zahrani, F.A.; Pandey, D.; Agrawal, A. A fuzzy TOPSIS based analysis toward selection of effective security requirements engineering approach for trustworthy healthcare software development. BMC Med. Inform. Decis. Mak. 2020, 20, 236. [Google Scholar] [CrossRef] [PubMed]
  34. Rose, J.R.; Swann, M.; Bendiab, G.; Shiaeles, S.; Kolokotronis, N. Intrusion Detection using Network Traffic Profiling and Machine Learning for IoT. In Proceedings of the 2021 IEEE 7th International Conference on Network Softwarization (NetSoft), Tokyo, Japan, 28 June–2 July 2021; pp. 409–415. [Google Scholar] [CrossRef]
  35. Kumar, R.; Khan, A.I.; Abushark, Y.B.; Alam, M.M.; Agrawal, A.; Khan, R.A. An integrated approach of fuzzy logic, AHP and TOPSIS for estimating usable-security of web applications. IEEE Access 2020, 8, 50944–50957. [Google Scholar] [CrossRef]
  36. Kumar, R.; Khan, A.I.; Abushark, Y.B.; Alam, M.M.; Agrawal, A.; Khan, R.A. A knowledge-based integrated system of hesitant fuzzy set, ahp and topsis for evaluating security-durability of web applications. IEEE Access 2020, 8, 48870–48885. [Google Scholar] [CrossRef]
  37. Abushark, Y.B.; Khan, A.I.; Alsolami, F.J.; Almalawi, A.; Alam, M.M.; Agrawal, A.; Kumar, R.; Khan, R.A. Usability Evaluation through Fuzzy AHP-TOPSIS Approach: Security Requirement Perspective. CMC-Comput. Mater. Contin. 2021, 68, 1203–1218. [Google Scholar] [CrossRef]
Figure 1. General architecture of intrusion-detection system (IDS).
Figure 1. General architecture of intrusion-detection system (IDS).
Electronics 11 00859 g001
Figure 2. Hierarchy for the effectiveness evaluation of different IDSs.
Figure 2. Hierarchy for the effectiveness evaluation of different IDSs.
Electronics 11 00859 g002
Figure 3. Triangular fuzzy numbers.
Figure 3. Triangular fuzzy numbers.
Electronics 11 00859 g003
Figure 4. Satisfaction Degree of CC-i.
Figure 4. Satisfaction Degree of CC-i.
Electronics 11 00859 g004
Figure 5. Comparison of the results with traditional approach.
Figure 5. Comparison of the results with traditional approach.
Electronics 11 00859 g005
Figure 6. Sensitivity analysis.
Figure 6. Sensitivity analysis.
Electronics 11 00859 g006
Table 1. Comparison of the five intrusion-detection methods.
Table 1. Comparison of the five intrusion-detection methods.
IDS TechniquesProviderTypeOperating SystemLicenseNetwork Traffic
ZeekVern PaxsonNIDSUnix/Linux/MacBSD LicenseIPv4
SuricataOISFNIDSWin/Unix/MacGNU General Public License (version 2)IPv4/IPv6
Security OnionSecurity Onion Solutions, LLCNIDS, HIDSLinuxGNU General Public License (version 2)IPv4/IPv6
OSSECDaniel B. Cid et al.HIDSWin/Unix/Linux/MacGNU General Public License (version 2)IPv6
SnortCisco SystemNIDSWin/Unix/LinuxGNU General Public License (version 2)IPv4/IPv6
Table 2. TFN scale.
Table 2. TFN scale.
Saaty Scale DefinitionFuzzy Triangle Scale
1Equally important(1, 1, 1)
3Weakly important(2, 3, 4)
5Fairly important(4, 5, 6)
7Strongly important(6, 7, 8)
9Absolutely important(9, 9, 9)
2Intermittent values between two adjacent scales(1, 2, 3)
4(3, 4, 5)
6(5, 6, 7)
8(7, 8, 9)
Table 3. Linguistic scales for the rating.
Table 3. Linguistic scales for the rating.
Linguistic VariableCorresponding Triangular Fuzzy Number
Very poor (VP)(0, 1, 3)
Poor (P)(1, 3, 5)
Fair (F)(3, 5, 7)
Good (G)(5, 7, 9)
Very good (VG)(7, 9,10)
Table 4. Level 1 fuzzify consolidated pair-wise comparative matrix.
Table 4. Level 1 fuzzify consolidated pair-wise comparative matrix.
M1M2M3M4
M11.000000, 1.000000, 1.0000001.000000, 1.515700, 1.9331000.489600, 0.637200, 1.0000000.415200, 0.574300, 1.000000
M2-1.000000, 1.000000, 1.0000000.574300, 0.665700, 0.8022000.303900, 0.393600, 0.566100
M3--1.000000, 1.000000, 1.0000001.000000, 1.319500, 1.551800
M4---1.000000, 1.000000, 1.000000
Table 5. Pair-wise fuzzy multi-criteria decision comparative matrix for types.
Table 5. Pair-wise fuzzy multi-criteria decision comparative matrix for types.
M11M12M13
M111.000000, 1.000000, 1.0000000.237552, 0.287963, 0.3675260.342154, 0.447785, 0.824763
M12 -1.000000, 1.000000, 1.0000000.661454, 1.172563, 1.693686
M13--1.000000, 1.000000, 1.000000
Table 6. Pair-wise fuzzy multi-criteria decision comparative matrix for audit source location.
Table 6. Pair-wise fuzzy multi-criteria decision comparative matrix for audit source location.
M21M22M23M24
M211.000000, 1.000000, 1.0000000.694154, 0.895356, 1.1124850.234596, 0.287864, 0.3641680.711256, 0.954163, 1.351257
M22-1.000000, 1.000000, 1.0000000.493154, 0.642362, 1.2414350.271354, 0.351565, 0.521635
M23--1.000000, 1.000000, 1.0000001.085484, 1.329762, 1.558235
M24---1.000000, 1.000000, 1.000000
Table 7. Pair-wise fuzzy multi-criteria decision comparative matrix for targets.
Table 7. Pair-wise fuzzy multi-criteria decision comparative matrix for targets.
M31M32M33
M311.000000, 1.000000, 1.0000000.665365, 1.172384, 1.6974651.157663, 1.447254, 1.704365
M32-1.000000, 1.000000, 1.0000001.007762, 1.524765, 1.934368
M33--1.000000, 1.000000, 1.000000
Table 8. Pair-wise fuzzy multi-criteria decision comparative matrix for protected system.
Table 8. Pair-wise fuzzy multi-criteria decision comparative matrix for protected system.
M41M42M43
M411.000000, 1.000000, 1.0000001.197856, 1.588385, 2.1564650.491541, 0.642285, 1.009958
M42-1.000000, 1.000000, 1.0000000.224165, 0.295684, 0.427969
M43--1.000000, 1.000000, 1.000000
Table 9. Pair-wise comparative matrix with defuzzification and local weight of parameters at level 1.
Table 9. Pair-wise comparative matrix with defuzzification and local weight of parameters at level 1.
M1M2M3M4Weights
M11.0000001.4912000.6910000.6410000.214422
M20.6706001.0000000.6770000.4143000.159049
M31.4472001.4771001.0000001.2977000.312280
M41.5601002.4137000.7706001.0000000.314249
C.R.= 0.015241
Table 10. Defuzzified pair-wise comparative matrix with parameter local weight for types.
Table 10. Defuzzified pair-wise comparative matrix with parameter local weight for types.
M11M12M13Weights
M111.0000001.1735400.4945640.275854
M120.8525501.0000001.1725470.328627
M132.0243400.8535451.0000000.395519
C.R. = 0.0488003
Table 11. Defuzzified pair-wise comparative matrix with parameter local weight for audit source location.
Table 11. Defuzzified pair-wise comparative matrix with parameter local weight for audit source location.
M21M22M23M24Weights
M211.0000000.8926541.1735540.9945470.246313
M221.1212421.0000000.6915260.3725460.182575
M230.8525621.4472561.0000001.2985410.272112
M241.0066242.6883540.7704351.0000000.299000
C.R. = 0.034904
Table 12. Defuzzified pair-wise comparative matrix with parameter local weight for targets.
Table 12. Defuzzified pair-wise comparative matrix with parameter local weight for targets.
M31M32M33Weights
M311.0000001.1725411.3636520.388975
M320.8533451.0000001.4912240.355978
M330.7337540.6707251.0000000.255047
C.R. = 0.002506
Table 13. Defuzzified pair-wise comparative matrix with parameter local weight for protected system.
Table 13. Defuzzified pair-wise comparative matrix with parameter local weight for protected system.
M41M42M43Weights
M411.0000001.6332440.6918440.322565
M420.6124771.0000000.3034570.356224
M431.4472473.3003471.0000000.321211
C.R. = 0.0052045
Table 14. Overall weights and level of different factors.
Table 14. Overall weights and level of different factors.
Level 1 MethodsLocal Weights of Level 1Level 2 MethodsLocal Weights of Level 2Overall WeightsOverall Ranks
M10.214422M110.2758540.0591499
M120.3286270.0704658
M130.3955190.0848086
M20.159049M210.2463130.03917612
M220.1825750.02903813
M230.2721120.04327911
M240.2990000.04755610
M30.312280M310.3889750.1214691
M320.3559780.1111653
M330.2550470.0796467
M40.314249M410.3225650.1013664
M420.3562240.1119432
M430.3212110.1009405
Table 15. Evaluators’ subjective intelligence findings in linguistic terminology.
Table 15. Evaluators’ subjective intelligence findings in linguistic terminology.
S1S2S3S4S5
M115.3600, 7.3006, 8.73005.5500, 7.5500, 8.91000.6400, 2.2700, 4.27005.3600, 7.3600, 8.73004.1800, 6.0900, 7.6400
M123.7300, 5.5500, 7.27004.4500, 6.4500, 8.18001.6400, 3.5500, 5.55003.5500, 5.5500, 7.36005.0000, 7.0000, 8.4500
M132.3600, 4.2700, 6.27005.3600, 7.3006, 8.73005.5500, 7.5500, 8.91000.6400, 2.2700, 4.27005.3600, 7.3600, 8.7300
M214.8200, 6.8200, 8.55003.7300, 5.5500, 7.27004.4500, 6.4500, 8.18001.6400, 3.5500, 5.55003.5500, 5.5500, 7.3600
M225.5500, 7.5005, 9.27002.3600, 4.2700, 6.27002.4500, 4.2700, 6.27001.3600, 3.3600, 5.36004.4500, 6.4500, 8.1800
M234.2700, 6.2700, 8.18004.8200, 6.8200, 8.55004.6400, 6.6400, 8.55000.8200, 2.6400, 4.64004.4500, 6.4500, 8.2700
M245.3600, 7.3006, 8.73005.5500, 7.5500, 8.91000.6400, 2.2700, 4.27005.3600, 7.3600, 8.73005.7300, 7.7300, 9.2700
M313.7300, 5.5500, 7.27005.3600, 7.3006, 8.73005.5500, 7.5500, 8.91000.6400, 2.2700, 4.27005.3600, 7.3600, 8.7300
M322.3600, 4.2700, 6.27003.7300, 5.5500, 7.27004.4500, 6.4500, 8.18001.6400, 3.5500, 5.55003.5500, 5.5500, 7.3600
M335.3600, 7.3006, 8.73005.5500, 7.5500, 8.91000.6400, 2.2700, 4.27005.3600, 7.3600, 8.73004.4500, 6.4500, 8.1800
M413.7300, 5.5500, 7.27004.4500, 6.4500, 8.18001.6400, 3.5500, 5.55003.5500, 5.5500, 7.36004.4500, 6.4500, 8.2700
M422.3600, 4.2700, 6.27002.4500, 4.2700, 6.27001.3600, 3.3600, 5.36004.4500, 6.4500, 8.18005.7300, 7.7300, 9.2700
M434.8200, 6.8200, 8.55004.6400, 6.6400, 8.55000.8200, 2.6400, 4.64004.4500, 6.4500, 8.27005.1800, 7.1800, 8.8200
Table 16. Fuzzy decision matrix with normalized decisions.
Table 16. Fuzzy decision matrix with normalized decisions.
S1S2S3S4S5
M110.3800, 0.6000, 0.80000.5400, 0.7500, 0.92000.5200, 0.7400, 0.93000.4200, 0.6900, 0.99000.5200, 0.7400, 0.9400
M120.5200, 0.7400, 0.94000.3800, 0.6000, 0.80000.5400, 0.7500, 0.92000.5200, 0.7400, 0.93000.4200, 0.6900, 0.9900
M130.3800, 0.6000, 0.80000.5200, 0.7400, 0.94000.5400, 0.7500, 0.92000.5200, 0.7400, 0.92000.2000, 0.4700, 0.7700
M210.3800, 0.6000, 0.80000.5400, 0.7500, 0.92000.5200, 0.7400, 0.93000.4200, 0.6900, 0.99000.5400, 0.7500, 0.9400
M220.5200, 0.7400, 0.94000.3800, 0.6000, 0.80000.5400, 0.7500, 0.92000.5200, 0.7400, 0.93000.4200, 0.6900, 0.9900
M230.3800, 0.6000, 0.80000.5200, 0.7400, 0.94000.5400, 0.7500, 0.92000.5200, 0.7400, 0.92000.2000, 0.4700, 0.7700
M240.3800, 0.6000, 0.80000.5400, 0.7500, 0.92000.5200, 0.7400, 0.93000.4200, 0.6900, 0.99000.5400, 0.7500, 0.9400
M310.5200, 0.7400, 0.94000.3800, 0.6000, 0.80000.5400, 0.7500, 0.92000.5200, 0.7400, 0.93000.4200, 0.6900, 0.9900
M320.3800, 0.6000, 0.80000.5200, 0.7400, 0.94000.5400, 0.7500, 0.92000.5200, 0.7400, 0.92000.2000, 0.4700, 0.7700
M330.3800, 0.6000, 0.80000.5400, 0.7500, 0.92000.5200, 0.7400, 0.93000.4200, 0.6900, 0.99000.5400, 0.7500, 0.9400
M410.5200, 0.7400, 0.94000.5400, 0.7500, 0.92000.3800, 0.6000, 0.80000.5400, 0.7500, 0.92000.5200, 0.7400, 0.9300
M420.3800, 0.6000, 0.80000.3500, 0.5800, 0.81000.5200, 0.7400, 0.94000.5400, 0.7500, 0.92000.5200, 0.7400, 0.9200
M430.5200, 0.7400, 0.92000.4600, 0.6700, 0.86000.3800, 0.6000, 0.80000.3500, 0.5800, 0.81000.4200, 0.6900, 0.9900
Table 17. Fuzzy decision matrix with weighted normalization.
Table 17. Fuzzy decision matrix with weighted normalization.
S1S2S3S4S5
M110.00000, 0.00200, 0.009000.00200, 0.00700, 0.022000.00200, 0.00700, 0.024000.00100, 0.00500, 0.018000.00300, 0.01100, 0.03600
M120.00300, 0.01200, 0.041000.00000, 0.00200, 0.009000.00200, 0.00700, 0.022000.00200, 0.00700, 0.024000.00100, 0.00500, 0.01800
M130.00300, 0.01200, 0.042000.00300, 0.01200, 0.041000.00300, 0.01200, 0.041000.00500, 0.01600, 0.048000.00500, 0.01600, 0.04900
M210.00000, 0.00200, 0.009000.00200, 0.00700, 0.022000.00200, 0.00700, 0.024000.00100, 0.00500, 0.018000.00200, 0.00900, 0.03800
M220.00300, 0.01200, 0.041000.00000, 0.00200, 0.009000.00200, 0.00700, 0.022000.00200, 0.00700, 0.024000.00100, 0.00500, 0.01800
M230.00300, 0.01200, 0.042000.00300, 0.01200, 0.041000.00300, 0.01200, 0.041000.00500, 0.01600, 0.048000.00500, 0.01600, 0.04900
M240.00000, 0.00200, 0.009000.00000, 0.00200, 0.009000.00200, 0.00700, 0.022000.00200, 0.00700, 0.024000.00100, 0.00500, 0.01800
M310.00300, 0.01200, 0.041000.00300, 0.01200, 0.041000.00300, 0.01200, 0.041000.00500, 0.01600, 0.048000.00500, 0.01600, 0.04900
M320.00000, 0.00200, 0.009000.00000, 0.00200, 0.009000.00200, 0.00700, 0.022000.00200, 0.00700, 0.024000.00100, 0.00500, 0.01800
M330.00300, 0.01200, 0.041000.00300, 0.01200, 0.041000.00300, 0.01200, 0.041000.00500, 0.01600, 0.048000.00500, 0.01600, 0.04900
M410.00000, 0.00200, 0.009000.00200, 0.00700, 0.022000.00200, 0.00700, 0.024000.00100, 0.00500, 0.018000.00200, 0.00900, 0.03800
M420.00300, 0.01200, 0.041000.00300, 0.01200, 0.041000.00500, 0.01600, 0.048000.00500, 0.01600, 0.049000.00100, 0.00500, 0.01800
M430.00300, 0.01200, 0.042000.00300, 0.01200, 0.042000.00200, 0.01000, 0.037000.00200, 0.00900, 0.038000.00100, 0.00500, 0.01800
Table 18. Coefficients of closeness to the aspired level throughout the alternatives.
Table 18. Coefficients of closeness to the aspired level throughout the alternatives.
Alternativesd + id − iGap Degree of CC+iSatisfaction Degree of CC-i
Alternative 1S10.04525640.05565470.62356520.3954740
Alternative 2S20.05645540.03536250.36554740.6586950
Alternative 3S30.04754580.05554740.56958570.4585660
Alternative 4S40.04535670.04635620.46857450.4458570
Alternative 5S50.04522650.04255550.45366520.4695850
Table 19. Comparison of the results of classical and fuzzy AHP-TOPSIS methods.
Table 19. Comparison of the results of classical and fuzzy AHP-TOPSIS methods.
Methods/AlternativesS1S2S3S4S5
Fuzzy-AHP-TOPSIS0.39547400.65869500.45856600.44585700.4695850
Traditional-AHP-TOPSIS0.37854100.64535200.44458600.43245500.4585700
Table 20. Sensitivity analysis.
Table 20. Sensitivity analysis.
ExperimentsWeights/Alternatives S1S2S3S4S5
Experiment-0Original WeightsSatisfaction Degree (CC-i)0.39547400.65869500.45856600.44585700.4695850
Experiment-1M110.43542540.60125240.48956850.47754740.4956526
Experiment-2M120.47652850.70256520.52856570.52454120.5346525
Experiment-3M130.52253650.53454740.39526530.34574250.3858547
Experiment-4M210.34125240.38552680.42456230.37563520.4152334
Experiment-5M220.37785690.41847590.52125420.55263540.5348549
Experiment-6M230.36452560.38326540.34526350.39658750.3856368
Experiment-7M240.48002150.49769650.37765380.42365870.4165365
Experiment-8M310.32854520.55635980.36856590.52526350.5363524
Experiment-9M320.52563560.53326540.48569650.34526350.3863897
Experiment-10M330.34363520.38536520.42765660.37534160.4163526
Experiment-11M410.37856950.41832650.39652350.35352770.3863524
Experiment-12M420.36457580.38526530.38385740.49653520.4963564
Experiment-13M430.48563650.49635260.54856840.52542910.5458473
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Alyami, H.; Ansari, M.T.J.; Alharbi, A.; Alosaimi, W.; Alshammari, M.; Pandey, D.; Agrawal, A.; Kumar, R.; Khan, R.A. Effectiveness Evaluation of Different IDSs Using Integrated Fuzzy MCDM Model. Electronics 2022, 11, 859. https://doi.org/10.3390/electronics11060859

AMA Style

Alyami H, Ansari MTJ, Alharbi A, Alosaimi W, Alshammari M, Pandey D, Agrawal A, Kumar R, Khan RA. Effectiveness Evaluation of Different IDSs Using Integrated Fuzzy MCDM Model. Electronics. 2022; 11(6):859. https://doi.org/10.3390/electronics11060859

Chicago/Turabian Style

Alyami, Hashem, Md Tarique Jamal Ansari, Abdullah Alharbi, Wael Alosaimi, Majid Alshammari, Dhirendra Pandey, Alka Agrawal, Rajeev Kumar, and Raees Ahmad Khan. 2022. "Effectiveness Evaluation of Different IDSs Using Integrated Fuzzy MCDM Model" Electronics 11, no. 6: 859. https://doi.org/10.3390/electronics11060859

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop