Affirmative Ant Colony Optimization Based Support Vector Machine for Sentiment Classification

Abstract

Sentiment analysis is part of contextual text mining, which detects, extracts and supports an organization in understanding their brand or service in social sentiment while monitoring the reviews provided by customers in online shops. The rise of online shopping and digitalization is practically achieved, and the quality of products is tough for users to judge. There is no model to find out about the same or unlike a set of people with similar sentiment analysis concerning online product evaluations. In this paper optimization-based classification algorithm is proposed namely, Affirmative Ant Colony Optimization Based Support Vector Machine (AACOSVM) to classify sentiments provided by customers in online shopping. This paper provides a new Ant Colony Optimization method via providing a novel pheromone model for support vector machine optimization parameters in two steps. The first one is statute of state transition, and the second step is statute of state updates. They aim to allow the ants to use the fake pheromone path to pick parameters and to motivate ants to create subsets having the least classification mistakes. The proposed work includes product review datasets from Amazon to assess the performance of the AACOSVM against existing classifiers, namely, Entropy-Based Classifier (EBC) and Enhanced Feature Attention Network (EFAN). Various review datasets are accessible at Amazon for various items. This research effort has identified a dataset from DVDs, books, kitchen appliances and electronics from the many multiple available review datasets. It utilizes the natural foraging behavior of ants towards searching for food to identify and classify the sentiments present in the product reviews. AACOSVM is evaluated using two standard data mining performance metrics, namely F-Measure and Classification Accuracy. Results indicate that the proposed classification algorithm AACOSVM achieves better F-Measure and Classification Accuracy than the EBC and EFAN classifiers.

1. Introduction

Sentiment Analysis (SA) is a technique for determining the feelings of groups and individuals, including a segment of a brand’s audience or a single client in contact with a customer service agent [1]. Sentiment analysis observes conversations and assesses linguistic and voice expressions to measure attitudes, views, and feelings about a company, service or product, or issue using a scoring method. Sometimes SA is also known as opinion mining [2,3]. The sentiment analysis is an essential component of the global speech analysis system that determines views or sentiments. SA is frequently powered by an algorithm that scores the words spoken together with voices that might reveal the underlying sentiments of a person regarding a subject. SA enables a more objective study of elements that generally are difficult to quantify or generally subjectively judged.

In internet shopping, SA is a powerful tool that allows users to monitor views and emotions between different client segments, such as those who contact a particular number of representatives in shifts, consumers calling on a specific concern, service or product lines and other separate groups [4]. SA may be automated, entirely relying on human analysis or a combination of both. In certain situations, SA is automated mainly by a degree of human supervision that drives master knowledge and enhances algorithms and procedures, especially in early deployment phases. Language is complicated and SA is similarly complex as a method of quantification and measuring of language. What is relatively easy for people to measure subjectively is face-to-face communication. For example, whether a person is happy or sad, smiley or angry with the subject matter at hand, must be transformed into objective, quantifiable scores that account for the numerous nuances in the human language in a debate [5,6]. Here in this study, SA is the practice of evaluating the reviews of online product datasets from Amazon provided by customers to identify whether they are good, negative, or neutral. Thus, sentiment analysis helps to determine how people feel about a product.

The sentiment is a feeling-led attitude, opinion, or judgment. SA examines the feelings of people concerning specific entities. The Internet is an informative place for feelings. From the user’s point of view, users can submit their own material on various social media, such as discussion boards, internet forums, or social networking websites [7]. From a researcher’s point of view, many social media sites provide their Application Software which urges stakeholders, developers and researchers to collect and analyze data deeply. For example, Twitter presently offers three versions of APIs [8], (i) REST API, (ii) Search API and (iii) Streaming API. The developers may collect status and user information through the REST API; the Search API provides developers with the possibility of querying certain Twitter material. The Streaming API can obtain Twitter contents in real-time. In addition, developers can mix the APIs to construct their apps. Therefore, sentimental analysis with the help of vast internet data appears to be firmly based [9].

These forms of internet data, however, have many faults that may prevent the sentiment analysis process. The first defect is that while people can submit their information freely, they cannot enhance the consistency of their ideas. For example, internet spammers publish spam on forums instead of providing subject pertinent thoughts [10]. Some spams are useless, while others include irrelevant views, which are also known as false viewpoints. The second issue is that online data are not always available as a matter of principle. An essential fact is like a tag of something, such as a specific opinion that indicates good, negative, or neutral [11,12].

The Amazon Product Review Dataset is indeed one of the essential truth datasets available to the public. The corpus has 1,6 billion reviews. Each communication is marked with the emoticons found inside the message. Here in this research work, the Affirmative Ant Colony Optimization Based Support Vector Machine (AACOSVM) is proposed to classify sentiments provided by customers to determine how people feel about a product. This research work provides a new Ant Colony technique via providing a novel pheromone model for support vector machine optimization parameters in two steps, the first one is statute of state transition which allows for the ants to use the fake pheromone path to pick parameters. The objective of every artificial ant throughout this ant colony method is to construct a solution subset. The Ants create solutions using a probabilistic decision-making policy to go across neighboring countries. The second step is statute of updates which use to motivate ants to create subsets with the least classification mistakes. This update statute applies solely to the subset of parameters that caused the least error in the latest iteration. This statute of updates increases the pheromone level in the optimal parameter subset. So, the ant that discovers the optimal solution can put pheromones on the set of specified parameters. This decision and the use of the statute of updates will help guide the search which searches the neighborhood with the best state produced by the current algorithm iteration. The statute of updates is only conducted when all the ants have established their answers. The use of the statute of updates increases the level of pheromone.

The rest of the paper is structured as follows. Section 2 presents the related work. Section 3 focuses on the details of an affirmative Ant colony optimization optimizing-based support vector machine while Section 4 outlines the dataset. Section 5 presents the performances metrics. Finally, Section 5 offers conclusions and future directions.

2. Related Work

Prospect Theory [13] is proposed to analyze the relationship among the sentiments and their rating. The loss aversion and the sensitivity of diminishing samples were validated to calculate the sensitivity. The results implicate the utilization of absolute and relative measures for the cognitive bias service recovery. The Freezing Technique [14] is proposed for learning sentiment-based vectors from Long Short Term Memory (LSTM) and Convolutional Neural Network(CNN). It integrates various deep learning methods from which it is observed where the clustered documents work better with the ensemble technique. This method worked better for different datasets. Learning Method [15] is proposed to train the embedding aspects based on the relationship between terms and categories. A cosine metric measure is introduced to study the alleviating limitations. It is used to initialize the existing models to solve the aspect-category sentiment analysis task. The result shows that the embedding aspect of the proposed technique improved the analysis of sentiments efficiently. Deep Study [16] is performed to analyze and explore the primarily utilized sentiments to detect opinions based on the subject element. Various categories of techniques are employed for classifying the text and it is based on the views which are either positive or negative. A two-step process was portrayed as the preliminary step. Public Sentiment Discovery [17] is proposed as a technique to mine Twitter data to analyze the sentiments towards the prediction of stock movement. In this method, textual messages are clustered before it is processed. Apart from that, data mining algorithms are used to elaborate the word list to classify the sentiments.

Propagating Sentiment Signal [18] is proposed to analyze the sentiments in Twitter customer tweets. Estimated polarity tweets are identified based on sentiments where supervised annotation feasibility and the tweet’s polarity are also estimated. Trained data is identified as sufficient for handling the polarity reputation. A Two-Pass classifier [19] is proposed to predict the satisfaction level of a drug. A combination of a Support Vector Machine (SVM) and Artificial Neural Network (ANN) is used to review the customer comments collected in the health care domain. The essential features were extracted from every review and feature vectors are generated. The two-pass classifier is applied for predicting whether the review relies on positive or negative. Implicit Aspect Extraction [20] is proposed to extract the sentiment features at the aspect level. The aspects are specified with explicit words and retrieved through text formats. This method makes utilization of different techniques to identify the implicit characteristics which were classified selected approaches. The issues and limitations of the aspect retrieval are also presented. The Multi-modal Joint Sentiment Topic Model [21] is proposed to analyze the sentiments present in weakly supervised blogs. Latent Dirichlet Allocation is applied to explore the hidden topics and sentiment in messages based on the emotion and personality of microblog users. The experiment shows the effectiveness of unsupervised approaches for measuring the level of accuracy. Enhanced Feature Attention Network (EFAN) [22] is proposed to increase the classification accuracy of target-dependent sentiments. The enhanced representation of features, position-based features of word and speech features were studied. A multi-view network is developed to (i) study the target words modeling (ii) increase target words based on sentiments and context. Experimental studies were performed to validate the model’s efficiency, and the results indicate better accuracy than previous models.

Features Opinions Extraction [23] is proposed to identify the features related to product opinions using text mining concepts. The customer’s opinions are identified and the tweets about such products were used in feature extraction. For validating the effectiveness of feature extraction, customer reviews of different products are tested. Hybrid Ensemble Scheme [24] is proposed to prune the clusters in the text sentiment classification. The ensemble classifiers are applied in separate clusters for the prediction of features. This scheme is tested using balanced datasets and compared with Bagging, Random Subspace and AdaBoost algorithms. The results proved to increase the efficiency and validity of the proposed scheme. The Arabic Language Sentiment Analysis Model [25] is proposed to analyze the feelings and opinions present in different languages, including Arabic. It measures users’ sentiments at all stages wherein the importance of corpus in annotated form was performed. It helped in regulating the phonetic level along with metonymy levels. BilSTM Model [26] is proposed to increase the attention of multi-polarity towards analyzing the implicit sentiments. Initially, the difference between the sentiments and words is identified and a restriction mechanism was adopted to ensure the optimization performance. Experiments were carried out on two exact sentiment datasets and it proved the accurate capturing of sentiment polarities-based features.

Entropy-Based Classifier (EBC) [27] is proposed to perform the feature and opinion classification. It was performed with the source domain for the prediction. Comparison with different product reviews present in various domains is presented where it makes use of modified maximum entropy and bipartite graph clustering. To ensure the effectiveness, the EBC is evaluated with domain-specific and independent words with the help of the SentiWordNet dataset.

In deep learning, attention is one of the most powerful concepts used for improving the performance of the neural network [28,29]. Attention effectively displays mechanisms on “attend to” a specific part of the input sequence, which can be said to be of higher importance [30,31,32,33,34,35].

This proposed method namely, Affirmative Ant Colony Optimization Based Support Vector Machine (AACOSVM) can classify sentiments provided by customers in online shopping by utilizing the way of the natural foraging behavior of ants towards searching for food to identify and classify the sentiments present in the product reviews where it follows different types of rules for optimization to achieves a better performance. However, no optimization or rules are followed in some of the models, such as EBC and EFAN. Furthermore, these models use a sequential classification to obtain better accuracy.

3. Affirmative Ant Colony Optimizing-Based Support Vector Machine

Sentiment Analysis (SA) is the use of evaluating the reviews of product data to determine whether data are positive, negative, or neutral. This would assist to monitor product sentiment in customer feedback and understand customer needs. A Support Vector Machine (SVM) method is used to review the customer comments collected from the dataset. The essential features are extracted from every review and feature vectors are generated. In the following sections, we aim to explain the proposed technique of Affirmative Ant Colony Optimization Based Support Vector Machine (AACOSVM) which helps to classify the sentiments present in the product data.

3.1. Support Vector Machine

The fundamental method of the Support Vector Machine (SVM) is to map input data training data by applying the Q mapping function to an enhanced dimensionality feature. Let the training set used for classification be TS = $\left\{u_a,v_a\right\}|u_aϵD,v_aϵ\left(+1 or-1\right), a=\left(1,2,3,\dots ,e^{-1},e\right)\}$ where $u_a$ represents the vectors of input and $v_a$ represents the $u_a$ label. The target function $\left(tf\right)$ of classification is expressed as Equation (1).

$$tf=\{\begin{array}{c}\min Q\left(x\right)=0.5\left(x·x\right)+PP{\displaystyle {\displaystyle \sum }_{a=1}^e}{\lambda }_a\\ s.t v_a\left(x·Q\left(c_i\right)+f\right)\le \left(1-{\lambda }_a\right), a=1,2,3,\dots ,e^{-1},e, {\lambda }_a<0\end{array}$$

(1)

where a = 1, 2, 3,…, e⁻¹, e, ${\lambda }_a<0$. The penalty parameter is indicated as PP, and ${\lambda }_a$ indicates slack variable having a value greater than 0.

The task of building the optimum hyperplane has, therefore, become the following problem in quadratic programming:

$$qp=\{\begin{array}{cc}max& E\left(z\right)={\displaystyle {\displaystyle \sum }_{a=1}^e}{z}_a-0.5{\displaystyle {\displaystyle \sum }_{a,b}}{z}_a{z}_b{v}_a{v}_{b}KF\left(u_a{u}_b\right)\\ s.t& {\displaystyle {\displaystyle \sum }_{a=1}^e}{z}_a{v}_b=0, 0<z<PP, a=1,2,3,\dots e^{-1},e\end{array}$$

(2)

The function of taking a decision is expressed as Equation (3).

$$df\left(c\right)=\left[{\displaystyle \sum }_{a=1}^e{v}_a{z}_{a}KF\left(c_i · c\right)+f\right]$$

(3)

The most often utilized SVM kernel functions are as follows:

Radial Basis Function based Kernel

$$RBFKF\left(c,c_i\right)=\exp \left(\frac{-{\left(c-c_i\right)}^2}{2g^2}\right)$$

(4)

Polynomial based Kernel

$$PKF\left(c,c_i\right)={\left(\left(c ·c_i\right)+d \right)}^e$$

(5)

Linear Kernel

$$LKF\left(c,c_i\right)=\left(c ·c_i\right)$$

(6)

This research concentrates on RBFKF for its good functionality and widespread use.

3.2. Affirmative Ant Colony Optimization (AACO)

Ant algorithms are optimizing algorithms based on the drilling behavior of genuine wild ants. When they seek nourishment, they start to randomly explore the area around the nest. When an ant finds a food source, it analyses the quantity and quality of food and brings them back to the nest. During the return journey, the ant deposits on the ground a chemical pheromone trail. The amount of pheromone deposited depends on the amount and food quality and leads other ants into the food source. The indirect connection between the axes through pheromone trails allows them to determine the quickest routes between their nest and food. This feature of actual ant colonies is used for solving challenging issues of combinatory optimization in artificial ant colonies. Artificial ants probabilistically construct solutions in ant colony algorithms by taking into consideration dynamic artificial pheromone trails. Affirmative Ant Colony Optimization algorithm’s (AACO) key component is the pheromone model, comprising the transition and update rule, which is utilized for probabilistically sampling the search area. You may define the AACO issue as follows:

Algorithm 1 depicts the structure of a fundamental AACO algorithm. The issues present in AACO are provided at the beginning, and certain variables are initialized. A productive heuristic approach for developing probabilistic solutions is the fundamental component of any AACO algorithm. Ants use a predefined pheromone framework to build probabilistic solutions to the optimization issue at each iteration. The solutions are then utilized to update the pheromones. Daemon movements can sometimes be used for creating centralized activities that a single ant cannot execute. Examples include applying local searches to the solutions designed or gathering global information to determine if it is or is not beneficial to deposit additional pheromones to bias the search procedure from a non-locally based standpoint.

Algorithm 1: Pseudocode of Fundamental AACO Algorithm

Input: Issues present in AACO algorithm I = (ValSet, U, Y)

Output: Identification of best solution

Method:

Initialization Construction of solution While termination terms are not fulfilled Scheduling of Activities Construction of ant-based solution Update of Pheromone Actions of daemons Final Scheduling of Activities End while

3.3. Affirmative Ant Colony Optimization Based Support Vector Machine

3.3.1. Parameters

SVM performance refers mainly to the capacity to accurately categorize the unknown data samples using SVM by learning from examples, which is also known as generalization ability. Continuous regularization (PP) and kernel function variables have a significant effect on SVM generalization capability. PP kernel function parameters, such as the RBF kernel bandwidth g will impact the mapping process of the data space and modify the sample distribution difficulty in higher dimensional functionality space. PP establishes a compromise between fitting error minimization and classification margin maximization. Since each parameter’s value, too large or too little, prevents SVM generalization, optimizing parameters is necessary to obtain a strong capacity for generalization in practice. This research article offers an AACO method to optimize PP and g parameters automatically.

3.3.2. Objective Parameter Optimization Function

Support vector machine parameter optimization aims to employ optimized techniques that only explore a small subset of potential values to identify the parameters that minimize the generalization error. This study calculates the absolute validation error. The estimate is uneven and its variance is reduced by reducing the complexity present in the training dataset. If there would be a validation set $ValSet=\left\{\left(u_a^{\prime },v_a^{\prime }\right)|u_a^{\prime }\subseteq FS,v_a^{\prime }\subseteq LS, a=1,2,3,\dots ,e^{-1},e\right\}$ with the FS set of features, the LS set is a label, the objective parameter optimization function for SVM is explored via Equation (7).

$$minimize Y=\left(\frac{1}{e}\right)\times {\displaystyle \sum }_{a=1}^e\left(SF\times \left(-v_a^{\prime }\right)\times DF(u_a^{\prime }\right))$$

(7)

where SF represents the step function: if a > 0 then SF(a) = 1 else SF(a) = 0. DF represents support vector machine decision function.

3.3.3. Novel Pheromone Model

Artificial ants probably develop solutions by taking dynamic artificial pheromone trails into account. The core component of the AACO Algorithm is the pheromone model, comprising the statutes of (i) state transition and (ii) updating, which will be used to test the search space probabilities. This research article provides a new AACO method via providing a novel pheromone model for SVM optimization parameters as follows.

3.3.4. Statute of State Transition (SST)

SST allows for the ants to use the fake pheromone path to pick parameters. The objective of every artificial ant throughout this AACO method is to construct a solution subset. The Ants create solutions using a probabilistic decision-making policy to go across neighboring countries. The rule for the state changeover is as follows:

$${\mathrm{SST}}_{ab}=\frac{{\beta }_{ab}}{{{\displaystyle \sum }}_{a=1}^N{\beta }_{ab}}$$

(8)

3.3.5. Statute of State Updates (SSU)

The objective of SSU is to motivate ants to create subsets having the least classification mistakes. This update statute applies solely to the subset of parameters that caused the least error in the latest iteration. This SSU increases the pheromone level in the optimal parameter subset. So, the ant that discovers the optimal solution can put pheromones on the set of specified parameters. This decision and the use of the SST will help guide the search which searches the neighborhood with the best state produced by the current algorithm iteration. The SST is only conducted when all the ants have established their answers. The use of the SSU increases the level of pheromone.

$${\beta }_{ab}^{new}=\left(1-ec\right){\beta }_{ab}^{old}+\frac{PI}{e^{ObFun}}$$

(9)

where ObFun represents the objective function (i.e., Y) in Equation (7), ec represents the coefficient of evaporation, PI represents the intensity of pheromone.

SSU aims to supply the solution set with higher pheromone levels that create fewer classification mistakes that would make selecting them more desirable to future ants. In other words, these suitable parameters lead to fewer classification mistakes and a greater likelihood that the ants would in the future, choose a solution.

4. Dataset and Performance Metrics

4.1. Dataset

A dataset is a collection of related, discrete items of data related which may access be in combination or individually or managed as an entire entity. A dataset is organized inside the database into some type of data structure. A dataset might contain a gathering of business data (names, contact information, sales, salaries and so forth). The database itself can be thought of as a dataset within certain data, such as sales data for a specific department of a company.

This paper includes a product review dataset from Amazon to assess the performance of the AACOSVM against existing classifiers, namely EBC [27] and EFAN [22]. Various review data sets are accessible at Amazon for various items. This research effort has identified a dataset from DVDs books, kitchen appliances and electronics from the many multiple available review datasets. The number of records accessible in the selected datasets is shown in

Table 1. Count of Records in Dataset.

Amazon Product Review Big Dataset	Total
DVD	142,875
Book	146,284
Kitchen appliances	72,849
Electronics	88,137

, the comparative analysis of F-Measure is shown in

Table 2. Comparative Analysis of F-Measure.

Domain Name	F-Measure (%) AACOSVM	F-Measure (%) EFAN	F-Measure (%) EBC
DVD	78.3	63.1	55.2
Book	75.6	65.4	59.7
Kitchen appliances	74.5	66.9	58.2
Electronics	76.9	63.0	51.9
Avrage of F-Measure (%)	76.33	64.60	56.25

and the comparative analysis of accuracy is shown in

Table 3. Comparative Analysis of Accuracy.

Domain Name	Accuracy (%) AACOSVM	Accuracy (%) EFAN	Accuracy (%) EBC
DVD	77.8	62.6	54.4
Book	75.1	65.1	59.3
Kitchen appliances	73.8	66.4	57.9
Electronics	76.6	62.5	51.4
Avrage of Accuracy (%)	75.83	64.15	55.75

.

4.2. Performance Metrics

This research utilizes the F-Measure and Classification Accuracy to measure the performance of AACOSVM against existing classifiers, namely EBC [27] and EFAN [22]. F-Measure and Classification Accuracy are computed using four variables of confusion matrix namely: (i) False Positive (FalPos) (ii) False Negative (FalNeg) (iii) True Positive (TruPos) (iv) True Negative (TruNeg).

FalPos—Outcome of imprecise prediction of positive class.

FalNeg—Outcome of imprecise prediction of negative class.

TruPos—Outcome of precise prediction of positive class.

TruNeg—Outcome of precise prediction of negative class.

4.2.1. F-Measure Analysis

It is a statistical examination of classification accuracy. Higher results indicate the adequate performance of the classifier. It is expressed mathematically as Equation (10).

$$\mathrm{F-Measure}=\frac{2TruPos}{\left(2TruPos+FalPos+FalNeg\right)}$$

(10)

In Figure 1, the x-axis is indicated with the product review dataset from amazon, and the y-axis is indicated with F-Measure which is measured in terms of percentage. Figure 1 makes a clear representation that the proposed classifier AACOSVM attains better classification accuracy than EBC and EFAN and the average of F-Measure for these products is 75.83% as shown in Table 2. By following two different types of rules for optimization, AACOSVM achieves a better F-Measure than EBC and EFAN. No optimization or rules are followed in EBC and EFAN which leads to poor F−Measure.

4.2.2. Classification Accuracy

It is the number of adequately predicted cases versus the number of forecasts. It is represented mathematically as Equation (11).

$$\mathrm{Classification} \mathrm{Accuracy}=\frac{\left(TruPos+TruNeg\right)}{\left(TruPos+TruNeg+FalPos+FalNeg\right)}$$

(11)

In Figure 2, the x-axis is indicated with the product review dataset from amazon, and the y-axis is indicated with Classification Accuracy which is measured in terms of percentage. In Figure 2, it is simple to realize that AACOSVM performs better than EBC and EFAN. Enhanced optimized classification helps AACOSVM to obtain better results in Classification Accuracy than EBC and EFAN and the average Classification Accuracy for these products is 76.33% as shown in Table 3. Due to sequentially performing classification, EBC and EFAN attain low Classification Accuracy than AACOSVM.

5. Conclusions and Future Work

The Affirmative Ant Colony Optimization Based Support Vector Machine (AACOSVM) is proposed to classify the sentiments present in the amazon product review dataset. AACOSVM adopts ant characteristics to identify the sentiments in the dataset. To make better classification, two different types of statutes are used, which are (i) State Transition and (ii) State Updates. AACOSVM is evaluated against the existing classification algorithms Entropy-Based Classifier (EBC) and Enhanced Feature Attention Network (EFAN)with F-Measure and Classification Accuracy metrics. Results indicate that AACOSVM achieves better results in classifying the sentiments present in the amazon product review dataset than the EBC and EFAN classifiers. Therefore, the proposed optimization-based classification algorithm determines the sentiments about a product and identifies the features related to product opinions better than other algorithms.

In the future, we will work on better optimization techniques to achieve increased F-Measure and Classification Accuracy. Additionally, the computational and storage complexities of the proposed algorithm will be analyzed. Moreover, we will conduct an examination of hierarchical-attention, self-attention, and self-attention mechanisms under the deep learning infrastructure, such as different dropouts. It constitutes the main engine that can operate in future work.