Issues and Solutions in Deep Learning-Enabled Recommendation Systems within the E-Commerce Field

Abstract

In recent years, especially with the (COVID-19) pandemic, shopping has been a challenging task. Increased online shopping has increased information available via the World Wide Web. Finding new products or identifying the most suitable products according to customers’ personalization trends is the main benefit of E-commerce recommendation systems, which use different techniques such as rating, ranking, or reviewing. These recommendations can be formed using different techniques and approaches, particularly using the technology of intelligent agents, and specific interfaces or personal agents can be used to model this type of system. These agents usually use the techniques and algorithms of Artificial Intelligence internally. A recommendation system is a prediction system that has been created to help the user to select the proper product for them, and to reduce the effort spent in the search process using advanced technology such as deep learning techniques. We investigate all studies using a standard review process for collecting and retrieving data from previous studies and illustrate their relevant accuracy and interpretability along with pros and cons helpful to business firms to adopt the most legitimate approach. The study’s findings revealed that recommendation problems are solved better by using deep learning algorithms such as CNN, RNN, and sentiment analysis, especially for popular problems such as cold start and sparsity.

1. Introduction

Most e-commerce marketing success is due to the application of recommendation systems that personalize the customer experience in the e-commerce sector. The e-commerce recommendation system can assist platforms in providing the most relevant data to customers at the most suitable time, enhancing consumer loyalty, and providing a more comfortable purchasing experience. An example of a recommendation system is presented in Figure 1. Accuracy of the recommendation outcomes is a critical aspect in deciding whether the RSS succeed or failed [1].

In previous years, especially during the years of the pandemic (COVID-19), as well as at present, the popularity of internet shopping has increased, and this has resulted in an explosion of data and information on the World Wide Web [2]. This creates some difficulty for users when choosing the product that is useful to them and that matches their tastes, behaviors, and estimations, because there are hidden items that simply cannot be found. Therefore, there must be a connecting link or bridge to connect users and items by structuring similarities between users or between items or between items and users together. For this purpose, the multi-directional recommendation system was born, started to grow, and played an important role in many areas such as economics, learning, social media, information systems, etc., and designing a predictive model that can estimate prediction function is the main goal of such systems [3].

Currently, in the global e-commerce market, we can observe a gradual increase in the number of online stores that introduce advanced personalization mechanisms. Until recently, they were a novelty, available only to e-commerce giants such as Amazon.com, which was one of the first retailers to use personalized recommendation techniques in its store. In recent years the number of vendors who have started to develop offerings has increased significantly, therefore both large and smaller online stores can afford the personalization of their offers.

Many reports and studies, including “E-commerce in Poland 2010. E-shops.”, indicate that personalized recommendations are key to increasing sales. Voivici’s research shows that in the United States more than half of online store owners (57%) offer recommendations that match the client’s profile, and 25% intend to introduce them to their business. On the other hand, in Poland, only every third e-shop offers simple recommendations for products such as “customers who bought also bought”. Personalized product recommendations bring great benefits, both for customers who get what they are looking for right away and for sellers who have a chance to increase revenue. In addition, long-term benefits are also significant, such as a higher degree of customer engagement, which will result in customer loyalty in the long run. Intelligent systems of personalized recommendations automatically tell us what we may like, try to adapt the content of the website to our preferences when we view it, and assess our preferences using many complex algorithms and techniques, including artificial intelligence.

Personalized recommendation systems are not only used in online stores, they are also used wherever we want to increase parameters, such as sales, the user’s stay on the website, their satisfaction with the right choices, and tailoring the offer individually to the needs of a specific customer.

The latest research conducted by Gemius in 2010 shows that maintaining a regular, satisfied customer requires five times lower financial outlays than acquiring a new customer. Having a loyal, satisfied customer is worth its weight in gold, so it is worth trusting and investing in personalized recommendation systems. One of the most important factors for the success of e-stores is intelligent e-commerce support systems because they reliably and accurately help to recommend product ranges to specific users.

An important barrier is the fact that not every store owner has specialist knowledge that will make the recommendation system high-quality and bring the expected results. By introducing a low-quality recommendation system, we will be recommending products to the customer that are off-limits. The cost of building a dedicated system for the store, which would use methods similar to those used by, for example, Amazon.com, will be associated with very large expenses on research and development, which is a significant problem for owners of most online stores. Systems may be an alternative that offer high-quality personalized recommendations in the form of SaaS (Software as a Service). An example of such a system is the Quartic system, a product introduced by the Polish company Data Force Group.

These recommendations can be formed using different artificial intelligence techniques and approaches, particularly the technology of deep learning, that may be used to model this type of system. Nowadays, the abundance of online shops and the large numbers of products makes exploring all products almost impossible. Clients face significant difficulty in exploring products that are irrelevant to their needs. Recommendation systems have been developed to calculate such probabilities or to determine the extent to which a product and a user match each other. A smart referral system is a powerful selling tool for any e-commerce platform [3]. As early as 2006, 35% of Amazon’s revenue should have been generated by product recommendations. Netflix held a $1 million prize money competition to improve its referral system in the same year, prompting many researchers to address the issue and develop new recommendation algorithms. In e-commerce, typical data records consist of historical customer transactions, so here the focus is on the forecast of purchase probabilities instead of valuation points. However, studies of such sparse implicit ratings have been reported, but rarely performed due to the lack of publicly available e-commerce records. As an outcome, RS has become a functional field of research, and significant advances were made as far as expanding the forecast and positioning into the precision of recommendations is concerned [4].

Recommendation systems are dependent mostly on user tendencies and past cooperation [5]. The proposed models are, for the most part, arranged into cooperative sifting and content-based recommendation frameworks based on the kinds of information available [6].

Deep learning has gained huge promotion and, over the past decade, an accomplishment of intense learning in numerous application areas. The scholarly world and businesses have been rushing to implement profound answers about how to incorporate a more extensive scope of uses based on its ability to unveil numerous unpredictable undertakings [7].

The proposed method differs from other similar work proposed by [6,8] [NO_PRINTED_FORM]. The proposed work analyzes sentiment analysis and deep learning approaches specific to e-commerce systems.

The goal of this review is to identify approaches used in building e-commerce recommendation systems. Other studies focused on cross-domain recommendation systems to figure out why each technique based on deep learning is used in recommendation systems. It also attempts to provide awareness of the solutions that are based on deep learning that have been provided to contemporary difficulties in the e-commerce field.

Contribution

This work contributes to the literature by the following:

• Identifying deep learning techniques used to build e-commerce recommendation systems.
• Identifying issues and possible solutions to e-commerce recommendation systems.

The rest of the paper is organized as follows: Section 2 introduces the necessary background concepts of recommendation systems and popular deep learning algorithms. Section 3 reviews the previous literature studies in the e-commerce field, Section 4 provides an overview and discussion of the results. Section 5 concludes the paper and present future works.

2. Background

2.1. Introduction to Recommendation Systems in E-Commerce

E-commerce is defined as the process of promoting products over the internet through electronic devices. E-commerce can be divided into three trends [9]. The substantial influence COVID-19 has on e-commerce is seen in retail sales [10]. Global retail website traffic peaked at 14.3 billion visits in March 2020 [10]. At the same time, online sales in Turkey soared by 98.3 percent across the country. Ecommerce exploded in Turkey, like it did in the rest of the world, at a time when people were being advised to stay at home, indicating extraordinary e-commerce growth during the 2020 shutdown [11]. According to a 2019 KPMG report, Turkey’s retail industry is worth 1.1 trillion Turkish liras (£108 billion) and is expected to rebound in 2020. With nearly 1.9 million people employed in this industry, the sector is one of the most important engines for the economy. According to the report, users in Turkey used online channels for purchasing at a rate of 34 percent in 2019, up from 29.2 percent in 2018. According to the Turkish Statistics Institute, retail sales volume in Turkey increased by 4.2 percent in October 2020 compared to the previous month [12].

As e-commerce activity continues to proliferate after COVID, so too are online retailers searching for smart and productive ways to give shoppers a dynamic buying experience tailored to their fluctuating needs and intentions. The discovery of new and deep methods has increased the creation of recommendations for shoppers to increase commercial profits and save time for the user to obtain the desired products in the fastest time and without effort. Data mining is a technique of searching large amounts of data to obtain useful information or to discover items by investigating interrelationships between other items. The most popular types of recommendation systems are a famous example of data analysis [13]. Recommendation systems have already been a key component of early e-commerce companies like Amazon and CDNOW. In the beginning, there was a strong need to help people find the right products through recommendations and customer feedback. Since its inception, Amazon has developed a variety of recommendation features, from algorithmic approaches, such as collaborative filtering, to non-algorithmic approaches, such as non-personalized recommendations. On the one hand, the use of a recommendation system aims for mass customization by providing each user with services tailored to their needs. For this, Amazon carefully stores the transactions of its customers. Regarding click-through rate metrics, personalized recommendations showed better results than non-personalized recommendations [14]. In addition, personalization is seen as a way to build long-term relationships with customers [13]. Recommendation systems are still very important in e-commerce today and continue to grow [15].

2.2. Similarity Concept in the Recommendation System

One of the main concepts used in the recommendation system is similarity. This consists of finding products that are comparable to prior products that the user has acquired or that are similar to each other (nearest neighbor). For example, on Amazon, we may recommend to the target user the products that other similar users have already purchased. Using the evaluations and preferences of all users, the similarity of these persons to the target user can be calculated. Thus, it will be possible to make recommendations to the target user for products that have not yet been purchased, according to their similarities with other users, and the ratings of these items. The success or accuracy of the method used in the recommendation system is dependent on the precision of the calculation of similarity in the prediction. The calculation of similarity between items and users is one of the important steps of the collaborative filtering algorithm and in selecting people identical to the target user. The similarity of elements can be determined by binary search and prediction methods, as well as vectorization models. Classification, clustering, and regression algorithms are most commonly used in similarity learning [16,17,18,19,20,21].

2.3. Recommendation System Approaches

A recommendation algorithm calculates the most suitable products for the user [22]. There are different approaches to this, mainly including content-based recommendations that might suggest similar products based on the consumer’s specific interests. However, because it is not personalized, it is unable to find new and exciting features for the customer [23]. However, association rule-based learning can discover new areas of interest for the user [24,25,26], and Content-Based Filtering and Collaborative Filtering algorithms are combined in a hybrid recommendation technique, which increases the effectiveness of a particular algorithm. However, it is ineffective for a variety of problems and applications [27]. However, the most widely disseminated are Collaborative Filtering and Content-Based algorithms. Collaborative Recommendation Systems try to exploit collective intelligence. They attempt to forecast the valuation of something a user would purchase and provide a product that has not been valued yet [9,11,13,17,21,28,29].

2.4. Issues of Recommendation Systems

The researchers presented alternate solutions to recommendation difficulties such as accuracy, sparsity, and the cold-start problem by exploiting deep learning’s efficacy in extracting hidden characteristics and associations. These difficulties are briefly described in this section. The rating’s precision forecasts, product estimation, item ranking, and users’ review prediction are the most discussed difficulties of recommendation systems. As a result, to improve the preferred ability and efficacy, a recommendation system must meet a certain level of forecast accuracy [1]. The problem of sparsity is unique to collaborative recommendation systems based on similarity [30].

2.5. Deep Learning

Deep learning has gained popularity in recent years. It has also witnessed major advancements in natural language and the analysis of image processing, as well as in the field of predicting useful recommendations through recommendation systems. Deep learning-based recommendation systems can handle large amounts of data and are faster than standard recommendation algorithms. In this section, we explain that recommendation systems usually employ deep learning techniques such (CNN, RNN, LSTM, Gru, DMF, and DNN) algorithms and autoencoders and attention mechanism techniques [28,31,32,33,34,35,36,37,38].

2.6. Sentiment Analysis in E-Commerce

SA (Sentiment Analysis), also recognized as Opinion Mining (OM), is an activity of Natural Language Processing (NLP) and is process of extracting and evaluating peoples’ opinions, sentiments, attitudes, and viewpoints toward particular entities, such as topics, products, and services [39]. Emerging internet-based applications such as social media platforms, the World Wide Web, blogs, etc., have led people to produce huge numbers of reviews of products, services, and other regular activities.

Sentiment analysis is divided into two main approaches, lexicon-based and machine learning approaches. Each section is divided into other subsections, as Figure 2 shows.

Sentiment analysis is divided into three main approaches to analyze the text, as shown in Figure 2 above. Most used techniques in the machine learning-based approach lie under the supervised learning type, whereas CNN-based techniques are used to observe behaviors and make a special connection while learning text. Finally, the semantic technique is also widely used as a sub-category of the lexicon corpus-based approach.

Example of Studies Using Deep Learning in Sentiment Analysis

We can describe the approaches that use sentiment analysis as follows:

• Bidirectional LSTM and Simple Embedding

An extension of conventional LSTMs, called a bidirectional LSTM, can enhance model performance on sequential classification issues. Bidirectional LSTMs train two LSTMs rather than one on the input sequence in situations in which all style interface of the input pattern is known. The first is on an earlier version of the input sequence, and the next is on a reversed copy. This can give the network extra context and lead to a quicker and even more thorough learning process for the problem. In [39,40,41], LSTM has identified DL as the key feature extraction of the approach used in customer sentiment analysis.

2.

Convolutional Neural Networks (CNNs)

Convolutional Neural Networks (CNNs) are machine intelligence or artificial intelligence (AI) algorithms built on multi-layer neural networks that can recognize and classify objects, as well as detect and segment objects in images. The benefit of CNNs over other classification algorithms (such as SVM, K-NN, Random-Forest, and others) would be that CNNs learn the best aspects to represent the artifacts in the images and have a high generalization capacity, being able to accurately classify new examples with only a small number of examples in the training set. In [40], CNN has identified DL as the key feature extraction component of the approach used in customer sentiment analysis.

3.

Average Pooling.

To order to construct a down-sampled (pooled) feature map, the average value for regions of a feature map is calculated using the average pooling method. It is frequently applied following a convolutional layer. It adds a tiny bit of translation invariance, which means that changing the image’s size slightly has little impact on the values of the majority of pooled outputs. In [42], average pooling has identified DL as the key feature extraction component of the approach used in customer sentiment analysis.

4.

Gated Recurrent Uni (GRU)

Cho et al. (2014) presented rectified linear units (GRUs) as a gating technique for recurrent neural networks. The GRU has a number of parameters fewer than an LSTM because it doesn’t have an output gate, but it is similar to an LSTM with a forget gate. It was discovered that GRU and LSTM performed similarly on some polyphonic music modeling, speech signal modeling, and processing of natural language tasks. On some smaller, less frequently used datasets, GRUs have been proven to perform better. In [43] GRU has identified DL as the key feature extraction of the approach used in customer sentiment analysis.

3. Literature Studies

Recently, deep learning has seen major advancement in natural language and image processing analysis, as well as in the field of predicting useful recommendations through recommendation systems. In the case of recommendation systems, there are several problems such as accuracy, sparsity, and the cold-start problem, which have occurred due to technological development in different areas. To overcome these issues, different approaches are used in deep learning algorithms such as CNN, RNN, LSTM, Gru, DMF, and DNN-based recommendation methods. Also, autoencoder and attention mechanism feature extraction techniques are used to overcome such issues. Recommender systems mostly benefit from these algorithms in analyzing (rating, ranking, and behavior of user) [44,45,46,47,48,49,50,51].

3.1. LSTM Used Studies

This model is made up of four layers (an embedding layer, adaptive attention layer, LSTM layer, and output layer). In embedding layer, a created memory matrix was created at sequence time by encoding the basket information of the user into two-dimensional levels. The adaptive attention layer has consisted of intra-basket and inter-basket attention mechanisms, where the intra-basket part is designed to establish a basket with a variety of user interests, and the intra-basket part is presented to represent the diversity of items between baskets, as well as the change of items within the basket over time, and then to choose items close to the item to be predicted using the max-pooling function. The LSTM algorithm is used to investigate dynamically adjusting candidate items inter-basket, and intra-basket adaptive user representations. Then, in the output layer, there is an “aggregated the display of relevant elements that describe a certain user’s preferences using softmax function to predict item that is purchased in the next basket” [52]. To resolve the issue, (2020) Liu et. al. introduced a creative Next Basket Attribute-Aware Multi-Level Attention Based Recommendation Model (NbRAM) by adding a sequence layer using LSTM based on same the data set in the previous study (Ta-Feng and JingDo). In this study, they used the same metrics for evaluating performance for this model, adding precision and HitRate measures. According to the results of experiments on two models, Ta-Feng and JingDo conducted this work focused on ranking prediction. NbRAM surpasses the previous IIAAN approach in terms of recall (0.227), F1-scores (0.125), and NDCG (0.167), precision (0.087), and HitRate (0.291), also N-Top = 5, especially with the JingDo dataset [53]. Figure 3 describes the general architecture of the proposed model [54].

3.2. DNN Used Studies

DNN combined with a collaborative filtering approach also was employed to predict recommendations as it can retrieve the latent features with more precision. In the structure of this work, the feature of the user-item is taken from the rating matrix by encoding with a Neural Collaborative Filtering (NCF) approach of this matrix into two dimensions according to the ID feature of the user and the feature of an item. Then, the two features with a missing feature vector are concatenated together using the previous steps, called Quadric Polynomial Regression, as the input data for a hidden layer using the DNN model. This model proved to be effective using the evaluation metrics, as compared to nine baseline algorithms, it performed higher prediction accuracy with MovieLens-1M (MAE: 0.6586, RMSE: 0.9357) [52]. Figure 4 shows a DNN network applied for rating reviews.

3.3. CNN Used Studies

Images and appearance of clothing products often conveys useful information, and recommendation system developers began to focus on visual recommendation systems. The SBPR (Style-aware Bayesian Personalized Ranking) model was proposed to concentrate on the connection between a client and a thing to demonstrate clients’ inclinations at the style level. Each dataset was handled by separating implied criticism. The relationships between particular channel reactions in each layer of this organization were obtained utilizing a convolutional neural organization model (CNN) utilizing VGG nets to secure style credits. The VGG-19 organization was used, with 16 convolutional and 5 pooling layers, and the maximum pooling layer was changed to the normal pooling layer. In light of information from Amazon.com, Tradesy.com, and a recycled attire exchange site, the model was prepared to utilize information from the dress, shoes, and adornments classes. Investigations of troublesome public datasets show the adequacy of the recommendation innovation and its capacity to get a handle on client style inclinations, as well as its capacity to tackle the virus start issue [53].

Likewise, to build a general idea of quality, a separate suggestion framework is proposed, with different suggestion methods for dynamic and inactive clients. An information expansion of a client utilizing styleNet-CNN with an information diagram is created to uncover the possible relationship between articles and clients. This study was centered around the Amazon-style dataset. The test results in view of hit rate, coverage, and time- consumption of KG show that by utilizing an information increase calculation to further develop information quality, the factorization machine model produces higher suggestion precision (81.25%), the built information diagram can reduce the virus start issue for proposal, and the separated suggestion procedure has accomplished better proposals for both dynamic and inert customers [55]. Figure 5 shows the different components of the approach to generate a recommendation to customers.

3.4. RNN Used Studies

Recurrent networks (or RNNs for Recurrent Neural Networks) are neural organizations in which data can result in two bearings, from the profound layers to the primary layers. In this, they are nearer to the genuine working of the sensory system, which is anything but a single-direction road. These organizations have repetitive associations, as in they keep data in memory: they can take into account at a time, t, a specific number of past states. Consequently, RNNs are especially reasonable for applications including setting, and all the more especially for the handling of fleeting arrangements such as learning and signage, for example, when the data form an arrangement and are not free of one another. In any case, for applications including long-time contrasts (ordinarily the classification video arrangements), this “transient memory” isn’t adequate. Without a doubt, “exemplary” RNNs (basic repetitive neural organizations or vanilla RNNs) are simply ready to remember the supposed close past and start to “neglect” after around fifty emphases. This two-way move of data makes their preparation substantially more muddled, and it is only as of late that viable strategies have been grown, such as LSTM (Long Short-Term Memory). These enormous “transient memory” networks have remarkably reformed voice acknowledgment by machines (speech recognition) or the agreement and age of text (natural language processing). According to a hypothetical perspective, RNNs have a lot more prominent potential than traditional neural organizations: theoretically to recreate any calculation. Notwithstanding, this does not provide any insight concerning how to assemble them for this function, practically speaking [56].

RNNs are numerical capacities with a few boundaries and are flexible. The relationship dates from the first automata proposed in 1943 by Warren McCulloch and Walter Pitts. As in the neurons of the mind where associations are made, vanish, or are supported by various upgrades and produce an activity, the fake (or formal) neural organizations change boundaries (called synaptic loads regarding the natural working of the cerebrum) in view of information to give the most ideal response.

3.5. NLP Used Studies

Another study applied TF-IDF, which is a kind of vectorization technique using in NLP (natural language process), in which two-way personal review generation models were created based on an explanation by using LSTM, RNN, Encoder, Decoder, and attention mechanism techniques using two datasets (Video and Toys) from Amazon 5-score and employing other two data sets (Beer and Yelp). This experiment was assessed using RMSE, NDCG@K, and DeltaRMSE for measuring recommended rating performance, predicted ranking performance, and improvements of machine-generated reviews, respectively [57]. Figure 6 shows the different phases to process text in the CNN-LSTM model.

The Authors in [58] proposed UltraGCN to improve the existing GCN technique by omitting parts of messaging layers that cause overload on the training process. However, the core way that GCN works is to aggregate and group similar information, which is the main cause for the delay in the training phase. The authors managed to omit the layers that will reduce the transaction of such messages.

3.6. GRU Used Studies

A group of researchers (2020) developed the session recommendation system using a multi-head attention mechanism and sequential behavior model (SBM) based on the GRU algorithm. The fundamental goal of SR is to forecast a user’s next click based on anonymized session data from the past. This model has consisted of three layers: the embedding layer, the sequential layer, and the prediction layer. In the embedding layer, each item’s features, and which items are clicked with each session have been embedded in the embedding layer as a high dimension. Then, in SBM based on GRU, there are two gates (the rest gate and update gate). In the prediction layer the output of item prediction that is clicked by the user in the next session is obtained. According to the results from the implementation of this methodology based on the YOOCHOOSE1 and DIGINETICA2 datasets by using the measurements, R@K (which represents the ratio of cases where the desired item is among the first k items in all test cases) and MRR@K (which is the average of the reciprocal ranks of the desired items) concluded that the proposed model outperformed all baselines [55].

The majority of current referral systems only utilize one form of user activity data. The Neural Multitasking Recommendation (NMTR) model is proposed to process the existing numerous behavioral recommendation models’ flaws (lack of behavior semantics, irrational embedding learning, inability to model complex interactions). This model had a choice of three NCF units as interaction functions, the respective methods were named NMTR-GMF, NMTR-MLP, and NMTR-NeuMF. The architecture of the NMTR model is created from four parts (the embedding layer, hidden layer, encoding hidden layer, and update layer). ID user and ID item have shared features in the embedding layer. In the hidden layer, interactions between latent features for user–item using three neural network units of Neural Collaborative Filtering (NCF) were gathered. Then, the predictions as encoding the different sequential behaviors were cascaded. Finally, in the update layer, the joint prediction for multiple types of behaviors were obtained.

3.7. Autoencoders Studies

Another study used a knowledge graph to solve the cold-start problem and improve the accuracy of the recommendation system, as using the Autoencoder Neural Network in the hidden layer also processed the sparsity problem based on the word2vec approach. The edges that are between the KG and Autoencoder nodes of the system are labeled by using a Semantics-Aware Autoencoder (SemAuto). The method’s details are investigated, and it is demonstrated how the SemAuto model can be used to produce content-based descriptions for suggested goods. “Both the point and binary description styles were tried, leveraging different sorts of information available in DBpedia, to see how the efficiency of the proposed description varied depending on the selected information used to feed SemAuto (categorical and factual)” [59]. The proposed method is compared with two kinds of Matrix Factorization (MF) algorithms, Weighted Regularized Matrix Factorization (WRMF) and Bayesian Personalized Matrix Factorization (BPMF), and another state-of-the-art method, which is a single-layer autoencoder. The Precision, Recall, F-1 score, NDCG, and aggregate diversity measurements are used to measure the performance of this work. Experiments were undertaken on three datasets (MovieLens 20M, Amazon Digital Music, and LibraryThing). In terms of accuracy, SemAuto was found to outperform baselines based on all evaluation metrics only with the Amazon Digital Music data set for different numbers of neighbors by k(40,50) [59].

A denoising autoencoder (DAE) is a type of autoencoder used in another study. In this study, user rating trend information was employed to predict the rating for recommending the first N items using rating-trend collaborative denoising autoencoder (UT-CDAE) technology. Coding the users’ rating trends is a crucial concern when dealing with faulty records. A user with higher rating behavior must be learned to learn the hidden features of other users. The rating trend of users is incorporated in this proposed model by employing two additional node vectors, High Tendency User Node (HTUN) and Low Tendency User Node (LTUN), in a simple DAE architecture. Low-value ratings are encoded by one of the node vectors, while high-value ratings are encoded by the participation of both node vectors. The study was based on measurement methods for scaling ratings of TOP-N recommendations were accuracy (P @ N), recall (R @ N), MAP, NDCG, NDCG @ N, and MRR. The proposed method (UT-CDAE) was compared to the performance of the basic noise removal autoencoder (DAE) and collaborative noise removal autoencoder (CDAE) for the ML-100K and ML-1M datasets in terms of several sequencing evaluation criteria. The performance of UT-CDAE is compared to three recent deep learning-based approaches (Caser, Neural Collaborative Filtering (NCF), and Enhanced Collaborative Auto-encoder (ECAE)) to demonstrate its utility for the ML-1M dataset. Aside from competing methodologies such as DAE and CDAE, in terms of several assessment criteria, the UT-CDAE’s performance and robustness in proposing top-N recommendations to users have increased [60].

3.8. Collaborative Filtering Studies

Collaborative filtering is a strategy and matrix decomposition is a method to implement the strategy. Collaborative filtering relies primarily on personalized recommendations based on user experience and suggestions with comparable qualities or interests. Through recommendation systems, it is beneficial to gather individuals with similar interests or characteristics and provide their feedback to users in the same cluster for reference, in order to satisfy the mentality of people to refer to the opinions of others before making decisions. Collaborative filtering technology must include the following aspects: (1) a process of comparing and collecting the interest preferences of each user; (2) a lot of user information to predict personal interest preferences; (3) statistics on the degree of relevance of interest preferences to develop recommendations for users who have the same interest preferences [57].

Recommendation services are always based on a volume of data. Depending on the nature of this data, a distinction is also made between the different types of systems. Typically, we can distinguish between content-based systems and so-called collaborative ones. There are also recommendation services adapted to different contexts and those which include in their calculation dated histories or demographic data of the users. Content-based recommendation systems suggest similar items or content that the user has previously searched for, viewed, purchased, or rated positively. For this, the system must be able to determine similarities between objects. Content analysis is therefore carried out. In the case of streaming services for music, the system, for example, evaluates the songs, taking into account their structure, to find similar ones. The collaborative technique bases the advice on the monitoring of users who share common behavior. When a group of people has previously expressed a strong interest in a certain object, the algorithm will continue to promote it [61,62,63,64]. Exact information about the object is not even necessary here. Amazon uses this process extensively. Recommendation services use different learning methods. Most of the time, model or memory-based methods are used. The memory-based method uses all stored classification data and identifies similarities between users or objects. The result serves as a forecast basis for combining objects that have not yet been mined. On the other hand, model-based recommendation services work with machine learning principles. Based on the data, the system should create a mathematical model capable of forecasting user interest in a given product.

Another approach is item-based collaborative filtering combined with an attention mechanism neural network proposed by the authors in [65]. This technique often offers high accuracy which is why it is suggested to be used in recommending items. However, the authors focused on hidden factors that might lead to better identification of item similarity, thus improving user’s experience.

The contrastive learning (CL) approach is proposed by authors of [66], that addresses the sparsity challenge in recommendation systems, as it follows the self-generated data from the customer side that the recommendation system needs. The proposed method visualizes the distribution between items and users which will result in showing graph augmentation and adding pre-configured noise to show contrastive visualization.

4. Findings and Discussion

Recommendation systems in the field of e-commerce are possible through the application of different models of deep learning techniques, and according to the results of different models, it is clear that each model has its own development and drawbacks. However, most of the models presented in the research paper gave similar results in the analysis of previous data and the detection of new elements and users and in solving the common problems in RS such as sparsity, cold-start, accuracy, and many others.

4.1. Issues of Recommender Systems

The main challenges for recommender systems, whether they are based on deep learning models or machine learning, are:

• Accuracy

Providing accurate recommendations is one of the main issues recommendation systems suffer from. The quality of data and the approach used will determine the accuracy of the particular system.

• Scalability

The rapid growth of data in e-commerce applications makes it difficult for traditional recommendation systems to generate recommendations, as the traditional ML algorithms had a linear relation to a number of products and users. For example, Amazon recommends to more than 20 million clients an existing 18 million products.

• Synonymy

Sometimes the same product is called by different names which will generate some conflict in many recommendation systems, such as collaborative filtering-based approaches. For example, a “TV” can be called “Television“ and recommendation systems treat them as a different product.

• Shilling assaults

This challenge represents malicious users that provide incorrect harmful information about products such as fake and negative or positive reviews. This attack mostly affects collaborative filtering techniques, but not product-based collaborative filtering approaches.

• Privacy

Data privacy is a major issue in any recommendation system, as most users are concerned when sharing their data for security and privacy reasons. However, the more data is fed into the recommendation system, the better the recommendation becomes. Collaborative filtering techniques are more popular because of privacy issues, as they tend to store the users’ sensitive data, such as ratings and reviews, in central repositories. Therefore, any leak with these repositories results in privacy issues.

• Latency problem

Newly added products to an e-commerce application suffers from delays in recommending these products to users, as no rating is given to them. Therefore, there is a long delay in recommending a newly added product, which is a frequent event.

• Context awareness

Context-based recommenders aggregate all details such as location, language, preferences, etc., thus, the performance of recommendations can be affected greatly depending on the data collected for each user to construct his/her context circumstances.

• Cold-start problem

This issue affects recommending new items in the magazine or annotating new user profiles. The recommender usually depends on existing data such as product ratings or user preferences. Newly introduced data are fresh and recommended based on that, which results in inaccurate results.

4.2. Possible Solutions to Recommender Systems Issues

In order to deal with the continuous challenges faced by the relevant approaches in the recommendation systems, there are some solutions:

• Improving accuracy:

Different scholars have tested different models to improve the overall accuracy. Salakhutdinov et al. show that using RBM models in conjunction with Singular Value Decomposition (SVD) results in predictions that are more accurate than Netflix’s suggested viewing a list of recommendation systems, such as sparsity and scalability. Similarly, Collaborative Denoising Autoencoder (CDAE), which Wu et al. (2016) suggest for a top-N recommendation, is created by recreating the dense form of user preferences. They show how CDAE’s three primary elements—the translating function, transfer functions, and corruption level—affect the accuracy performance.

• Scalability:

Boltzmann machines are used by Truyen et al. (2009) to extract latent factors from users and items in order to generate predictions over huge datasets. In order to increase the scalability of RBM-based CF, Louppe (2010) suggests a number of solutions including distributed computing, method ensembles, and distributed processing with shared memory. According to the author, parallel computing is more effective than other methods in terms of the level of recommendations [67].

• Cold-start problems and sparsity:

Unger et al. (2016) created low-dimensional hidden representations of context factors derived from sensors using autoencoders to address sparsity in context-aware recommender systems brought on by adding high context features [68,69]. Furthermore, different studies, such as [70,71], attempted to apply a social approach using an attention mechanism to eliminate the “cold-start problem”.

• Context-aware problem:

Different studies have attempted to address this issue using user attribution, text information, and other context data as the auxiliary source to determine the context in an environment [69,72]. However, three techniques are possibly used to eliminate this issue: facial expression detection, speech recognition and interpretation, and analyzing physiological signals.

• Privacy solution:

The best technique to eliminate this issue is applying cryptography and omitting third-party or peer users to keep the data private and only used for what is intended. Ref. [69] is an example of an improved cryptography solution.

• Shilling assaults solution:

Different approaches can effectively detect these attacks such as behavior monitoring and hit ratio. To categorize the attacks there are three criteria used to identify possible assaults such as the intention of attacking, the necessary knowledge to issue such an attack, and the possible size of attack [73,74].

• Synonymy solution:

The most appropriate technique to overcome this issue is using descriptive text rather than terms on a product by using an ontology, SVD (single value decomposition), and LSI (latent semantic indexing) [75].

• Latency solution:

To address this issue generalization techniques are applied to users to reduce computing. Furthermore, collaborative filtering approaches based on the model are also used to improve performance [76,77,78].

In this section the results of a group of previous literature studies on this topic are presented in

Table 1. Finding summary.

Study	Advantage	Method	Disadvantage
[45]	Achieves better performance using the MAE and RMSE measures with the three datasets and achieves higher prediction accuracy, and this model proves to be effective.	OPR combined with DNN	The number of experiments in all three datasets is about 20. This is a big effort, and removing one of the best results will reduce the efficiency of the results, randomizing the data
[30]	Extracts deep features from hidden issues extracted from textual user reviews to develop a recommendation system.	CNN	Did not perform better compared to the MF and HFT models in the movies and TV product category.
[46]	Added information to an ads platform to improve recommendation performance for cold-start users of an online shopping space	NLP-Based CNN	It is complex and inflexible due to the interconnection of two models (GMF and MLP) in the hidden layer.
[47]	Broke the groundwork in both the rating estimation and the item ranking problem, and explanation experiments show that it can produce both compelling and readable explanations for recommendations.	CNN	Extensive information on the proposal was not extracted, and the opaque estimation process and expensive to train.
[48]	Significantly improves accuracy and interpretability in the top N recommendation task compared to the most advanced methods. Effective for extracting high-level features from various low-level data, especially unstructured data such as images, text, and audio.	Multimodal IRIS	Requires a Big Data storage repository that holds unrefined information and unstructured data is in various formats, so it needs to be converted to uniformity, all of which are required for additional work, bid, capacity, and time. In the optimization process, rendering with interactive user-item pairs was rated higher by the model than non-interactive user-item pairs.
[52]	The IIAAN outperforms single-component models in all situations. It shows that considering the in-cart and between-baskets at the same time helps predict the next basket.	Attention-based RNN	Does not use IIAAN category information. When the size is greater than 100, the model does not benefit from a larger size. Overfitting can be a possible cause.
[49]	Experiments on two general datasets show that the proposed model is superior to traditional and existing deep learning-based recommendation models, and the next basket proposal is to recommend users one basket of products at a time.	CNN-LSTM	Constraints in application areas, each user from the embedding size 100 is recommended 5 items that are not in the same basket, which means product diversification for several users, not for the same basket and the same user. This can cause sparsity and cold-start issues.
[53]	In performing the work with its strengths, the SBPR may be aware of idiosyncratic features to detect users’ preferences.	CNN	The constraints in application domains are personalized for implicit feedback in this study. Not using point sort with binary sort in the sort model.
[70]	With the strengths of the study, the data augmentation algorithm proposed in this paper is effective, and the application of the infographic is also possible.	CNN-LSTM	By evaluating the limitations in the application areas, and whether it is a clothing product, 100 randomly sampled products were selected from the processed product and the process was repeated 10 times.
[60]	From the strengths of the study, a new approach is presented to provide accurate, diversified, and explainable recommendations. Also, this approach can be used to study data quality for knowledge bases in recommendation scenarios. Finding the best number of features to calculate explanations is not within the scope of this article	Autoencoder Neural Network	While removing all items from unplanned datasets in DBpedia leads to skipping some important elements of the recommendation, there is a disadvantage in these experiments that is not present in DBpedia.
[62]	Provides additional editing flexibility, which helps to estimate the improved top N recommendations. Results showed that the proposed model outperforms state-of-the-art noise-canceling autoencoder models. Provides an opportunity to learn a user’s high trending percentage. Patterns of co-liking by users are also well learned during the coding process when trained on the user.	Collaborative filtering strategy (UT-CDAE)	Ignores constraints in application domains, ratings below 4, which may affect the number of recommendations. The second constraint of this study is that it randomly divides the data into training data and test the rest.

:

The trending of sentiment analysis usage in the past years is growing as shown in Figure 7.

Figure 8 shows a summary of approaches used in implementing e-commerce recommendation systems. However, some of these approaches keep struggling due to some issues mentioned earlier.

5. Conclusions and Future Work

We conclude from this study that the notion of recommendation systems and their application in the sphere of internet commerce are extremely important, particularly in light of the current COVID-19 pandemic, which has sparked a lot of interest in this subject. Systems have been developed that help consumers purchase in more convenient ways by providing recommendations and advice based on artificial intelligence approaches, the most notable of which is deep learning. There are a variety of deep learning models that have helped the development and efficiency of recommendation systems. However, they have several flaws due to their complexity in dealing with unstructured data, such as user reviews, which require NLP processing and time to build various algorithms.

The detailed review shows that domain adaption is a key factor, which must be focused on by analysts when building models for scenarios that have different meanings in another domain. Such a challenge can be resolved if DL approaches are used, and models such as RNN, CNN, and LSTM can provide better results for estimating customer reviews. Despite that, this review offers a large dataset under empirical analysis in which DL shows better results; however, future scholars can conduct empirical analysis to determine if new hybrid models can be helpful in offering better accuracy to organizations. Another theoretical gap is also identified in this study, requiring further empirical investigation about why and how words used in one domain differ in another domain.

Evidently, recommendation systems have helped companies and shopping sites increase earnings while also assisting users in obtaining their needs in a simple and timely manner. The results of the research papers above were appraised. Deep learning algorithms clearly have a good approach to discovering recommendations as well as the ability to generate valuable electronic reviews, but one of their limitations is that the more sparse the data that enters the system, the less accurate the system becomes. In future work, we will propose adopting an optimization recommendation system according to changing user considerations based on sequential and representation learning and choosing an optimal feature selection method.