CNAIS: Performance Analysis of the Clustering of Non-Associated Items Set Techniques ^†

Abstract

Mining technologies depend upon their outcomes, focusing only on certain data features within the database. They select only certain features related to the process from diverse integrated data resources and transform them into a form suitable for mining tasks. Different implementations of mining techniques run on data sources, which may be of considerable volume, to extract different knowledge outcomes suitable for various analyses and decision-making processes. The proposed study provides the design and development of the Clustering of Non-Associated Items set (CNAIS) within a transactional database. The development of the algorithm and its application to the data set are described and the results are noted. Comparisons with state-of-the-art methods show that CNAIS exhibits better performance.

1. Introduction

Depending on their results, mining technologies concentrate primarily on specific aspects of a database. They take only specific process-related aspects from integrated data resources that contain a variety of data and convert them into a format that is appropriate for mining operations. Different mining techniques are applied to data sources with potentially enormous volumes in order to extract diverse knowledge results suitable for diverse analyses and decision-making processes. These knowledge results are assessed and represented visually using a variety of methods appropriate to the domain, such as tabular forms, decision tree forms, graphs, rules, charts, data cubes, and multi-dimensional graphics. These are categorized into descriptive and prescriptive viewpoints on data mining. According to the general qualities found in a data repository, descriptive mining summarizes or characterizes massive amounts of data. Prescriptive mining is the process of predicting and inferring information from past data. Both forms of data mining include a variety of methods, such as association, clustering, the categorization of data items, exploring outliers, regression and trending analytics, and machine learning techniques. Finding missing values, choosing the right features, and concentrating on outlier detection are just a few of the difficulties involved in mining massive amounts of data. Other challenges involve clustering techniques to find patterns from complex/distributed data, analyzing high dimensional data, identifying imbalanced classes in the classification, protecting data privacy, and leaving data sets with useless data due to algorithm logic.

Mining huge volumes of data is not a simple task, but throws many challenges across broad categories, such as finding missing values, apt feature selection, and focusing on outlier detection. Also, other challenges include methods for clustering high-dimensional data, identifying imbalanced classes in the classification, ensuring data privacy, extracting patterns from complex/distributed data, leaving data unused in data sets due to algorithm logic, etc. Association rule mining (ARM) is one of the widely utilized data mining methodologies introduced by this study [1]. ARM finds useful correlations in data items, frequent patterns in datasets, and associations among items or casual structures involved in the transaction databases or data repositories [2]. Association rule mining involves the extraction of interesting associations or correlation relationships among various features within a given large set of data items. Day-to-day activities that generate massive amounts of data are continuously collected and stored. Many industries are becoming interested in mining association rules from their databases. Companies rely on decision making processes, such as cross-marketing, market basket analyses, and loss leader analyses, etc., to develop their business strategies. They use association mining to identify correlations in data items among the huge amounts of business transaction records generated daily [3,4]. In the process of mining, interesting relationships among items in a given data set may be found. Rule support and confidence are two measures of rule interestingness that reflect the usefulness and certainty of discovered rules, respectively. Rules are said to be strong when they satisfy both a minimum support threshold (min_support) and a minimum confidence threshold (min_confidence). With experience, such thresholds can be set by users or domain experts.

2. Related Works

The Apriori algorithm proposed, which is used for frequent itemset extraction from transaction data, is a commonly used technique in Market Basket Analysis. It is used to analyze sales in super markets or any sales related business to find associations between the items purchased by customers so that mangers can make better decisions related to product forecasting in terms of sales and profits [5].

There are several steps involved in this process, as mentioned below:

• Market Basket Analysis to make decisions regarding product stock and purchasing;
• Recommender Systems to recommend items that are frequently used by customers;
• Fraud Detection for identifying abnormal transactions;
• Network Intrusion Detection to identify any abnormal behaviors in access patterns;
• Medical Analysis to identify diseases by observing patterns;
• Text Mining to mine texts with frequent phrases or words;
• Web usage mining to identify frequent visitors to web sites and how are they used.

As an important data mining technique similar to classification, clustering involves grouping a set of data objects when the groupings are unknown. In clustering, grouping produces classes or clusters of objects, where [6] objects within a cluster share many traits but differ greatly from those in other clusters. Dissimilarities are taken into consideration when describing objects based on their attribute values. Distance measurements, centroids, etc., are frequently used. Clustering has applications across many disciplines, including data mining, statistics, biology, and machine learning [7,8,9,10].

The grouping of data points based on similar characteristics or nearness among data values is termed clustering. It involves extracting data points that exhibit similar features within a dataset, resulting in the placement of these points into the same cluster.

2.1. K-Means Clustering Technique

K-Means is a well-known clustering approach that is used in machine learning and data analysis to divide a dataset into discrete groups or clusters based on their similarities. K-Means clustering aims to group data points that are close to each other while keeping data points from other clusters reasonably separated. The K-Means clustering algorithm works as follows: K initial cluster centroids are picked at random or on purpose from the dataset. These centroids serve as the focal points for each cluster [11,12,13,14].

For huge datasets or a large number of clusters, K-Means can be computationally expensive [15,16,17,18,19,20]. Outliers can have a major impact on K-Means outcomes. A single outlier may cause the centroid to be pulled away from the main cluster, resulting in poor clustering performance. Techniques such as outlier detection and the use of robust versions of K-Means can help to reduce this problem. K-Means produces predictable results, which means that running the method numerous times yields the same clusters each time. However, if you are seeking various clusterings to examine the structure of the data, this can be a drawback. You can mitigate this by utilizing K-Means variants, such as K-Means++ or Mini-Batch K-Means. K-Means is primarily intended for numerical data, and it may not perform well with categorical or mixed data types unless adequate preprocessing or distance metric selection is used.

To overcome these constraints, it is critical to evaluate the individual properties of your data, as well as the aims of your clustering task. Other clustering techniques, such as hierarchical clustering, DBSCAN, Gaussian Mixture Models (GMMs), or spectral clustering, may be more suited and successful depending on the nature of the data. Furthermore, in some circumstances, preprocessing steps, including careful analysis of the distance metric and initialization techniques, may increase the performance of K-Means.

2.2. Cross Industry Standard Process in Data Mining (CRISP-DM)

The phases of Cross Industry Standard Process in Data Mining (CRISP-DM) are shown in Figure 1.

3. Methodology

In studying the above association and clustering techniques, we propose the Clustering Technique of Non-Association rule Items Set (CNAIS) method over the transaction data sets method, which considers non-association rule item sets to cluster them along with associated rule-based sets or independently to prevent the loss of data items or rows in the database, which might occur as unused data after undergoing filtering in either ARM or clustering processes after long computations.

The proposed algorithm, Clustering Technique of Non-Association rule Items Sets over transaction data sets (CNAIS), works in three phases as shown in Algorithm 1:

(a)

Extracting Non-Associative Rule Sets (S_Na) from the transaction dataset (T_d);

(b)

Computation of the threshold value and the application of the clustering technique over items of both S_A and S_NA to form clusters, which will enable us to group transactions (T_i) in the T_d according to the T-value;

(c)

Based on different T-Values we can prioritize transactions (T_i) in the T_d.

In Phase 1, we used the function ENAIS (T_d, S_A, and S_NA), as shown below.

Algorithm 1: ENAIS (Td, SA, and SNA)

Inputs: Transaction Database Td containing set T{t1,t2,t3…tn} Returns: SA,SNA  1. Begin;  2. For each Ti in Td perform   2.1 Count occurrences of Ik   2.2 Find SupportCount (Sc)   2.3 SA = [set of items ≥ Sc]   2.4 SNA= [set of items < Sc]   End of step 2;  3. Repeat until count_itemset(SA) = Prev_Count for each item in SA    (Loop until no more items satisfy)   3.1 Prev_count = count_itemset(SA)    (Item set count before generating the next new item sets)   3.2 Calculate SupportCount(Sc)   3.3 Obtain Subset Sk that satisfies the Support-count(Sc)   3.4 Combine items in SA = Sk U SAk-1    (For the generation item sets of size k that satisfy Support-count (Sc));  4. SNA=Td − SA;  5. Return SA,SNA.

In Phase 2, we used the function GenerateCluster(S_A,S_NA) to create clusters which contained fewer outliers as shown in Algorithm 2.

Algorithm 2:GenerateCluster(,SA,SNA)

Inputs: SA = { ia1,ia2,….ian} is the set of associated item sets, which is a subset of I. SNA = { ina1,ina2,….inan} is the set of non-associated item sets, which is a subset of I. Returns: Clusters K1..Km(contains SA,SNA) 1. Begin; 2. Select attribute Ait, which will be used as the T-Value(Tv) for the selection of the item from SA,SNA; 3. Arrange in ascending order of Ait in SA and find Count Cn; 4. Find the median of values MSA; 5. Repeat steps 3 and 4 for SNA and find MSNA; 6. Tv= MSA + MSNA/2; 7. Choose TV as the threshold value for creating clusters; 8. Initialize the points randomly k1…km… as the number of clusters required from Ait of SA,SNA such that the value is nearest to that of the support count (+ or - Sc%) and the TV (threshold value) from SA and SNA data points as the medoids; 9. For each of the points chosen from step 8, create from n objects in SA and 10. For all the other non-medoids in each km, compute the cost (distance as computed using the Manhattan/Euclidean method) from the initial medoid; 11. In each kith cluster, compare each medoid to that of the ki+1th cluster, select the minimum distance (values related to SC% of threshold value Tv), and form clusters with SA and SNA; 12. Compute the total cost of min medoids in kith and assign to Dki+1 ie Dki = Ʃ(min(ki)) and Dki+1 = Ʃ(min(ki+1)); 13: Compute S1 = |Dk − Dk+1|; 14. Repeat the steps for the kith and ki+2th clusters from step 11 to step 13 and find S2; 15. Compute Z = S1-S2; 16. If (z < 0) then:   Swap the initial medoids to the next random medoid and repeat steps 8 to 15 until the clusters do not change, or clusters ki, ki+1, ki+2 are perfect; Return ki..km clusters; End.

In Phase 3, we used the Algorithm 3 to generate clusters K₁…K_m, where 1…m are priority-based clusters that are created based on the threshold value.

Algorithm 3: PriorityClusterGenerate(k1..Km)

Input: Clusters K1...Km formed using SA and SNA based on TV which depends on Ait Output: Cluster graphs with non-associated items selected and priority-based selection. 1. Begin; 2. For each Pair(Ki,Ki+1),   (a) Compute x = Ʃ|kimed + ki+1med| and y = ki+2med   (b) Plot scatted graph Gi; 3. Gi gives S=non-associated items selected for SA and SNA satisfying the threshold   value TV    Prioritize the cluster points based on Tv; 4. Plot the graph using clusters Ki/Ki+1 Ki/Ki+2. Threshold value Tv clusters are    formed with different points with Tv + Sc% of Ait; End.

4. Results and Discussion

Data Preparation

The sample Ttansaction dataset included features such as TransId, considering purchases of up to six items on different dates. The items dataset with prices was considered from

Table 1. Items available in the hotel which were present in the transaction file.

Item No.	Item Name	Price
1	BIRIYANI	300
2	PIZZA	270
3	COOL-DRINK	85
4	FISH-FRY	200
5	GULAB-JAM	70
6	HALWA	60
7	TANDOORI	250
8	VEG-RICE	200
9	MUTTON-FRY	230
10	SANDWICH	120
11	LEMON-SODA	70
12	ICE-CREAM	150
13	SAMOSA	50
14	CHICKEN-FINGERS	150

.

Some of the frequently purchased items were selected from the above transactions and are noted in

Table 2. Frequently purchased items in the transactions given.

Transaction ID	Item Sets
10016	Biryani, Pizza, Cool drink, Fish fry, and Ice-Cream
31001	Gulab jam, Biryani, Tandoori, Halwa, Sandwich, Mutton fry, and Lemon soda
90121	Pizza, Cool drink, Biryani, Veg rice, Samosa, and Ice-Cream
50091	Biryani, Gulab jam, and Halwa
69091	Veg Rice, Lemon soda, Cool drink, Ice-Cream, Pizza, and Biryani
10909	Sandwich, Chicken fingers, Mutton fry, and Cool drink

.

CNAIS was implemented using Python. Figure 2 shows the clusters formed with non-associated items added with their threshold values. Figure 3 shows the cluster of points showing transaction IDs with the corresponding purchase amounts. Figure 4 shows the cluster of points showing transaction IDs with the corresponding purchase amounts and their threshold values.

Generally, only the associated items were considered and the non-associated item sets were ignored, even though more processing costs were incurred to obtain the final result. Considering the non-associated items sets not only allows for the inclusion of cost-effective non-associated items, but also maximizes the processing power that might have been wasted when only finding the associated items.

Figure 2 and Figure 3 display two clusters of associated items and non-associated item clusters, which completely ignore non-associated clusters. If we consider both clusters and utilize the threshold value, which is the base value (such as the median of the highest and lowest non-associated sets), we can consider non-associated items. This approach enables the inclusion of some transactions or customers for potential benefits. Additionally, some items with high costs purchased by customers in the non-associated cluster are valuable and could be considered as per the business rules.

This shows that one or more customers are added from the non-associated item sets marked by the threshold value and that some of the customer IDs can be considered along with the associated items sets.

The CNAIS algorithm was executed using an Intel Pentium 5 machine with 16 GB of RAM on Windows 10 OS. The transactions were stored for each transaction ID for a period of time and the results and graphs are provided. Since CNAIS returns the associated items, the non-associated item sets, and the total price of all the transactions, this algorithm may use more time in milliseconds when compared to the ordinary Apriori algorithm.

Table 3. Execution time in milliseconds.

Algorithm	ItemSet Mining		CLUSTER	Total-Time (ms)
CNAIS + CLUSTER	432.78	AIS, NAIS are generated	401.2	833.98
APRIORI + CLUSTER	410.13	Only AIS	409.2	819.33
	AIS-Associated Item Sets NAIS–Non-Associated Item Sets

shows the execution time.

The Figure 5 compares the execution times of the different techniques.

5. Conclusions

Association rule mining and clustering techniques were studied with examples and algorithms. The proposed CNAIS algorithm is given along with sample datasets. The results and analyses are displayed with figures and tables and we have shown the aim of the algorithm, considering one or more non-associated item sets or transactions in the graphs. The proposed study focuses on the design and development of the Clustering of Non-Associated Items Set (CNAIS) technique within a transactional database. The development of the algorithm and its applications within datasets are given and the results are noted. Comparisons with state-of-the-art methods show that CNAIS exhibits better performance.