Sparse Data Recommendation by Fusing Continuous Imputation Denoising Autoencoder and Neural Matrix Factorization

Abstract

In recent years, although deep neural networks have yielded immense success in solving various recognition and classification problems, the exploration of deep neural networks in recommender systems has received relatively less attention. Meanwhile, the inherent sparsity of data is still a challenging problem for deep neural networks. In this paper, firstly, we propose a new CIDAE (Continuous Imputation Denoising Autoencoder) model based on the Denoising Autoencoder to alleviate the problem of data sparsity. CIDAE performs regular continuous imputation on the missing parts of the original data and trains the imputed data as the desired output. Then, we optimize the existing advanced NeuMF (Neural Matrix Factorization) model, which combines matrix factorization and a multi-layer perceptron. By optimizing the training process of NeuMF, we improve the accuracy and robustness of NeuMF. Finally, this paper fuses CIDAE and optimized NeuMF with reference to the idea of ensemble learning. We name the fused model the I-NMF (Imputation-Neural Matrix Factorization) model. I-NMF can not only alleviate the problem of data sparsity, but also fully exploit the ability of deep neural networks to learn potential features. Our experimental results prove that I-NMF performs better than the state-of-the-art methods for the public MovieLens datasets.

1. Introduction

In the era of information explosion, big data exhibits a rich value and great potential, which brings transformative development to human society, but it also generates the serious “information overload” problem. Recommender systems are an effective way to alleviate the problem of “information overload”, having been widely adopted by many online services, including E-commerce, online news, and social media sites [1]. Recommender systems can help determine which information to offer to individual consumers and allow online users to quickly find the personalized information that fits their needs [2]. Collaborative Filtering (CF) is a successful approach commonly used by many recommender systems [3]. CF [4,5] is based on the user’s past interaction records (such as ratings) to simulate the user’s preferences for the item. The scarcity of original interaction records has always been a difficult point for CF, which is the problem of data sparsity.

As the most popular approach among various CF techniques, Matrix Factorization (MF) [6,7], which projects users and items into a shared latent space, using a vector of latent features to represent a user or an item, has become a standard model for recommendation due to its scalability, simplicity, and flexibility [8,9]. Most previous research [3,7,10,11,12,13,14,15] on MF did not change the nature of linearity, such as integrating it with neighbor-based models [7], combining it with topic models of item content [3], and extending it to factorization machines [15] for a generic modelling of features. Despite the effectiveness of MF for CF, such a linear model is insufficient to understand the complex and non-linear relationship between users and items [16]. Furthermore, the problem of data sparsity is difficult to solve in MF.

Of course, there has been some advanced research [1,2,17,18] on MF that essentially changed the linear structure of MF by combining the idea of deep learning. The combination of deep learning and MF mainly uses the implicit feedback data to learn the hidden vector of the user(item) by using the deep learning method, thereby predicting the user’s preference for the item based on the hidden vector. Among existing models, an outstanding performance was shown for the NeuMF (Neural Matrix Factorization) model proposed in [1]. The NeuMF model combines matrix factorization and a multi-layer perceptron to achieve a good performance.

MF based on deep learning can be regarded as a kind of nonlinear generalization of the traditional hidden factor model. Its obvious advantage is the introduction of nonlinear feature transformations in the process of learning the hidden vector of the user(item), which has a better performance than traditional MF. However, this method still cannot improve the problem of data sparsity in MF.

In this paper, in order to alleviate the problem of data sparsity, we propose a new CIDAE (Continuous Imputation Denoising Autoencoder) model based on the Denoising Autoencoder (DAE). The CIDAE model performs regular continuous imputation for the missing parts of the original data and trains the imputed data as the desired output, which is different from the DAE algorithm that randomly adds noise to the input data. The advantage of CIDAE is that it alleviates the problem of data sparsity by using the idea of imputation.

Then, this paper optimizes the advanced NeuMF model based on the training process. The NeuMF model introduces the idea of deep neural networks into matrix factorization. The NeuMF model first proposes the GMF (Generalized Matrix Factorization) model and then combines the GMF model with the MLP (Multi-Layer Perceptron) model. We name the optimized NeuMF model the O-NeuMF-p model. Additionally, our experimental results prove that the accuracy of O-NeuMF-p is higher than that of NeuMF.

Finally, we fuse CIDAE and O-NeuMF-p with reference to the idea of ensemble learning. We name the fused model the I-NMF (Imputation-Neural Matrix Factorization) model, which combines the advantages of CIDAE and O-NeuMF-p. The I-NMF model can not only alleviate the problem of data sparsity, but also fully exploit the ability of deep neural networks to learn potential features, and I-NMF is also robust. Furthermore, our experimental results confirm that the performance of I-NMF is better than CIDAE, O-NeuMF-p, and current advanced recommendation algorithms for the public MovieLens (ML) [19] datasets.

In summary, our main contributions of this work are outlined as follows:

• We propose a new CIDAE model based on DAE. The advantage of CIDAE is that it alleviates the problem of data sparsity by using the idea of imputation.
• By optimizing the training process of the NeuMF model, we improve the accuracy and robustness of NeuMF. We name the optimized NeuMF model the O-NeuMF-p model.
• We fuse CIDAE and O-NeuMF-p with reference to the idea of ensemble learning. Additionally, we name the fused model the I-NMF model. The I-NMF model can not only alleviate the problem of data sparsity, but also fully exploit the ability of deep neural networks to learn potential features, and I-NMF is also robust.

This paper is organized as follows. In Section 2, we present the architecture and details of the CIDAE model. In Section 3, we briefly introduce the NeuMF model and explain in detail how we optimize its training process. In Section 4, we propose the idea of fusing CIDAE and O-NeuMF-p, and present the architecture and details of the I-NMF model. In Section 5, we evaluate all the models mentioned above using the public MovieLens datasets, and compare the results with current advanced recommendation algorithms. Concluding remarks with a discussion of some future work are in the final section.

2. CIDAE: Continuous Imputation Denoising Autoencoder

Our CIDAE consists of three parts: learning from the original data; continuous imputation for the missing parts of the original data; and learning from the imputed data. Figure 1 shows the structure of CIDAE.

2.1. Part 1: Learning from the Original Data

In the first part, there are many existing models [10,11,12,13,14] to choose from for learning the original data. In this paper, we chose the traditional autoencoder (AE) [20].

For each user $u_i$, some items in the original data are of interest to $u_i$, while the remaining items are the missing parts of the original data. The goal is to predict the items that are of interest to $u_i$ from the missing parts of the original data.

In this part, labels of the items that the $u_i$ is interested in are set to 1, and labels of the remaining items are set to 0. Similar to [16,21], labels for all items constitute the initial value of the user vector ${\mathit{u}}_{\mathit{i}}$ ($u_{ij}=0$ or $1$).

According to the AE, we use ${\mathit{u}}_{\mathit{i}}$ as the input and get the reconstructed ${\mathit{u}}_{\mathit{i}}$ by encoding and decoding. The specific process is as follows:

Encoding: Map ${\mathit{u}}_{\mathit{i}}\in {ℝ}^{\mathit{n}}$ to the $d$-dimensional hidden layer by encoding function $f$.

$$\mathit{y}=f\left({\mathit{u}}_{\mathit{i}}\right)=\sigma \left(\mathit{W}{\mathit{u}}_{\mathit{i}}+\mathit{b}\right)$$

(1)

where $\mathit{W}\in {ℝ}^{\mathit{d}\times \mathit{n}},\mathit{b}\in {ℝ}^{\mathit{d}}, \mathit{y}\in {ℝ}^{\mathit{d}},n$ is the total number of items and $\mathsf{\sigma }$ is a non-linear mapping function, e.g., a sigmoid function.

Decoding: Map $\mathit{y}\in {ℝ}^{\mathit{d}}$ to the $n$-dimensional space by decoding function $g$ for reconstructing ${\mathit{u}}_{\mathit{i}}$. Let ${\tilde{\mathit{u}}}_{\mathit{i}}$ denote the reconstructed vector of ${\mathit{u}}_{\mathit{i}}$.

$${\tilde{\mathit{u}}}_{\mathit{i}}=g\left(\mathit{y}\right)=\sigma \left({\mathit{W}}^{\prime }\mathit{y}+{\mathit{b}}^{\prime }\right)$$

(2)

where ${\mathit{W}}^{\prime }\in {ℝ}^{\mathit{n}\times \mathit{d}},{\mathit{b}}^{\prime }\in {ℝ}^{\mathit{n}}, {\tilde{\mathit{u}}}_{\mathit{i}}\in {ℝ}^{\mathit{n}}$. We hope that ${\tilde{\mathit{u}}}_{\mathit{i}}$ and ${\mathit{u}}_{\mathit{i}}$ are as similar as possible, and the minimum objective function of AE is as follows:

$$\underset{\mathit{W},{\mathit{W}}^{\prime },\mathit{b},{\mathit{b}}^{\prime }}{argmin}{\displaystyle \sum }_{i=1}^m\mathcal{L}\left({\mathit{u}}_{\mathit{i}}, {\tilde{\mathit{u}}}_{\mathit{i}}\right)+\frac{\lambda }{2}({||\mathit{W}||}_F^2+{||{\mathit{W}}^{\prime }||}_F^2+{||\mathit{b}||}_F^2+{||{\mathit{b}}^{\prime }||}_F^2)$$

(3)

$$\mathcal{L}\left({\mathit{u}}_{\mathit{i}}, {\tilde{\mathit{u}}}_{\mathit{i}}\right)=-\frac{1}{n}{\displaystyle \sum }_{j=1}^n{u}_{ij}log{\tilde{u}}_{ij}+\left(1-u_{ij}\right)log(1-{\tilde{u}}_{ij})$$

(4)

where $m$ is the total number of users and $\mathcal{L}$ is the loss function. Furthermore, we choose the cross entropy as the loss function. $\lambda$ is a regularization parameter to prevent overfitting. Just like the traditional autoencoder, we optimize the objective function by using the gradient descent method.

2.2. Part 2: Continuous Imputation on the Missing Parts of the Original Data

In this part, CIDAE performs continuous imputation for the missing parts of the original data by using ${\tilde{\mathit{u}}}_{\mathit{i}}$. Let ${\widehat{\mathit{u}}}_{\mathit{i}}\in {ℝ}^{\mathit{n}}$ denote the imputed vector of ${\mathit{u}}_{\mathit{i}}$. For each ${\widehat{u}}_{ij}$ in ${\widehat{\mathit{u}}}_{\mathit{i}}$, we can use the following formula to calculate:

$${\widehat{u}}_{ij}=\{\begin{array}{c}\hfill 1, u_{ij}=1\\ \hfill min\left\{1,f\left({\tilde{u}}_{ij}\right)\right\}, u_{ij}=0\end{array}$$

(5)

where $f\left(x\right)$ can be any continuous function, and it can be a linear or nonlinear function; for example, $f\left(x\right)=kx$ or $f\left(x\right)=ktanx$, where $k$ is a hyper-parameter. Obviously, the larger the value of $k$, the larger the proportion of data that is imputed to 1. There are many choices for $f\left(x\right)$, which shows that CIDAE has great flexibility.

2.3. Part 3: Learning from the Imputed Data

The traditional Denoising Autoencoder (DAE) first adds noise to the input data and then reconstructs the noise-added data. CIDAE performs regular imputation of the output data, which is different from DAE.

The specific process is similar to the Part1, as we still choose ${\mathit{u}}_{\mathit{i}}\in {ℝ}^{\mathit{n}}$ as the input and obtain the reconstructed ${\mathit{u}}_{\mathit{i}}$ by encoding and decoding. In Part 3, let ${\overline{\mathit{u}}}_{\mathit{i}}\in {ℝ}^{\mathit{n}}$ denote the reconstructed vector of ${\mathit{u}}_{\mathit{i}}$. We want the value of ${\overline{\mathit{u}}}_{\mathit{i}}$ to be as similar as possible to ${\widehat{\mathit{u}}}_{\mathit{i}}$, not ${\mathit{u}}_{\mathit{i}}$, which is different from the Part1. The minimum objective function of CIDAE is as follows:

$$\underset{W,W^{\prime },b,b^{\prime }}{\mathrm{argmin}}{\displaystyle \sum }_{i=1}^m\mathcal{L}\left({\widehat{\mathit{u}}}_{\mathit{i}}, {\overline{\mathit{u}}}_{\mathit{i}}\right)+\frac{\lambda }{2}({||W||}_F^2+{||W^{\prime }||}_F^2+{||b||}_F^2+{||b^{\prime }||}_F^2)$$

(6)

$$\mathcal{L}\left({\widehat{\mathit{u}}}_{\mathit{i}}, {\overline{\mathit{u}}}_{\mathit{i}}\right)=-\frac{1}{\mathrm{n}}{\displaystyle \sum }_{\mathrm{j}=1}^{\mathrm{n}}{\widehat{u}}_{ij}\log {\overline{u}}_{ij}+\left(1-{\widehat{u}}_{ij}\right)\log \left(1-{\overline{u}}_{ij}\right)$$

(7)

Similar to [22], we introduce a hyper-parameter ($\mathsf{\alpha }$) in order to distinguish between the imputed data and the original data. We can distinguish the reconstruction error between the imputed data and the original data by using $\mathsf{\alpha }$ as a weight. So, the final minimum objective function of CIDAE is as follows:

$$\underset{W,W^{\prime },b,b^{\prime }}{\mathrm{argmin}}{\displaystyle \sum }_{i=1}^{m}K\left({\widehat{\mathit{u}}}_{\mathit{i}}, {\overline{\mathit{u}}}_{\mathit{i}}\right)+\frac{\lambda }{2}\left({||W||}_F^2+{||W^{\prime }||}_F^2+{||b||}_F^2+{||b^{\prime }||}_F^2\right)$$

(8)

$$\mathrm{K}\left({\widehat{\mathit{u}}}_{\mathit{i}}, {\overline{\mathit{u}}}_{\mathit{i}}\right)=\mathsf{\alpha }\left({\displaystyle \sum }_{\{j|u_{ij}=0\}}\mathcal{L}\left({\widehat{u}}_{ij},{\overline{u}}_{ij}\right)\right)+\left({\displaystyle \sum }_{\{j|u_{ij}=1\}}\mathcal{L}\left({\widehat{u}}_{ij},{\overline{u}}_{ij}\right)\right)$$

(9)

In this way, CIDAE can adjust the loss weight of the imputed data through $\mathsf{\alpha }$. Just like the traditional denoising autoencoder, we optimize the objective function of the CIDAE by using the gradient descent method.

It is worth emphasizing that CIDAE trains the imputed data ${\widehat{\mathit{u}}}_{\mathit{i}}$ as the desired output. On the one hand, the imputation method can alleviate the sparseness of the original data. On the other hand, a well-founded imputation is more meaningful than random noise. In addition, the CIDAE model, like the DAE algorithm, can increase the robustness of the model by introducing disturbances.

3. Optimization of the NeuMF Model

In this section, we briefly introduce the NeuMF model [1], and then explain in detail how to optimize NeuMF from the training process. The NeuMF model consists of GMF (Generalized Matrix Factorization) and MLP (Multi-Layer Perceptron). Figure 2 shows the structure of NeuMF.

3.1. GMF (Generalized Matrix Factorization)

GMF has four layers: input layer, embedding layer, inner layer, and prediction layer (output layer). The input to GMF is a sparse user ID and item ID. The embedding layer is a fully connected layer that can project sparse vectors into dense vectors. The vector obtained by the embedding layer can be regarded as the hidden vector of the user (item). The hidden vector of user $u$ is denoted as ${\mathit{p}}_{\mathit{u}}$, and the hidden vector of the item $v$ is denoted as ${\mathit{q}}_{\mathit{v}}$. The mapping formula of the inner layer is as follows:

$$\phi {\mathit{p}}_{\mathit{u}},{\mathit{q}}_{\mathit{v}}={\mathit{p}}_{\mathit{u}}\odot {\mathit{q}}_{\mathit{v}}$$

(10)

where $\odot$ represents the element-wise product of vectors. The operation of the last prediction layer (output layer) is

$${\widehat{y}}_{uv}=a_{out}\left({\mathit{h}}^{\mathit{T}}\left({\mathit{p}}_{\mathit{u}}\odot {\mathit{q}}_{\mathit{v}}\right)\right)$$

(11)

$$a_{out}=\sigma \left(x\right)=\frac{1}{\left(1+e^{-x}\right)}$$

(12)

where $a_{out}$ is the activation function. According to the literature [1], we adopt the sigmoid function $\sigma \left(x\right)$ as $a_{out}$. $\mathit{h}$ is the weight of the prediction layer and is trained by the loss function.

3.2. MLP (Multi-Layer Perceptron)

MLP is designed to explore the interaction between users and items’ latent features. Different from collaborative filtering of GMF, MLP is more flexible. MLP is not limited to the vector inner product, and can deeply learn the potential interaction between the user’s hidden vector ${\mathit{p}}_u$ and the item’s hidden vector ${\mathit{q}}_v$. The specific process of MLP is as follows:

$$\begin{gathered} {\mathit{z}}_1={\varphi }_1\left({\mathit{p}}_{\mathit{u}},{\mathit{q}}_{\mathit{v}}\right)=\left[\begin{array}{c}{\mathit{p}}_{\mathit{u}}\\ {\mathit{q}}_{\mathit{v}}\end{array}\right] \\ {\varphi }_2\left({\mathit{z}}_{\mathbf{1}}\right)=a_2\left({\mathit{W}}_{\mathbf{2}}^{\mathit{T}}{z}_1+{\mathit{b}}_{\mathbf{2}}\right) \\ {\varphi }_L\left({\mathit{z}}_{\mathit{L}-\mathbf{1}}\right)=a_L\left({\mathit{W}}_{\mathit{L}}^{\mathit{T}}{\mathit{z}}_{\mathit{L}-\mathbf{1}}+{\mathit{b}}_{\mathit{L}}\right) \\ {\widehat{y}}_{uv}=\sigma \left({\mathit{h}}^{\mathit{T}}{\varphi }_L\left({\mathit{z}}_{\mathit{L}-\mathbf{1}}\right)\right) \end{gathered}$$

(13)

where ${\mathit{W}}_{\mathit{x}}$ is the weight matrix of each layer, ${\mathit{b}}_{\mathit{x}}$ is the bias vector of each layer, and $a_x$ is the activation function of each layer. According to the literature [1,23], we adopt the Rectifier (ReLU) as $a_x$. The ${\widehat{y}}_{uv}$ is the output of the last prediction layer and $\sigma$ (sigmoid function) is the activation function of the prediction layer. $\mathit{h}$ is the weight of the prediction layer and is trained by the loss function. MLP selects the four-layers ($L=4$) tower structure, and halves the layer size for each successive higher layer. Together with the final prediction layer, MLP has a total of five layers.

3.3. The NeuMF Model

NeuMF is a comprehensive model that combines GMF with MLP. NeuMF combines the training of GMF and MLP by adding the “NeuMF layer”. The specific combination process is as follows:

$$\begin{gathered} {\varphi }^{GMF}={\mathit{p}}_{\mathit{u}}^{\mathit{G}}\odot {\mathit{q}}_{\mathit{v}}^{\mathit{G}} \\ {\varphi }^{MLP}=a_L\left({\mathit{W}}_{\mathit{L}}^{\mathit{T}}\left(a_{L-1}\left(\dots a_2\left({\mathit{W}}_{\mathbf{2}}^{\mathit{T}}\left[\begin{array}{c}{\mathit{p}}_{\mathit{u}}^{\mathit{M}}\\ {\mathit{q}}_{\mathit{v}}^{\mathit{M}}\end{array}\right]+{\mathit{b}}_{\mathbf{2}}\right)\dots \right)\right)+{\mathit{b}}_{\mathit{L}}\right) \\ {\widehat{y}}_{uv}=\sigma \left({\mathit{h}}^{\mathit{T}}\left[\begin{array}{c}{\varphi }^{GMF}\\ {\varphi }^{MLP}\end{array}\right]\right) \end{gathered}$$

(14)

where ${\varphi }^{GMF}$ is the output result of the inner layer of GMF, and ${\varphi }^{MLP}$ is the output of MLP after four layers. The “NeuMF layer” can be understood as a combination of GMF and MLP from the prediction layer. $\sigma$ (sigmoid function) is the activation function of the “NeuMF layer”. $\mathit{h}$ is the weight of the “NeuMF layer” and is trained by the loss function.

3.4. The NeuMF-p Model

NeuMF-p(pre-training) refers to the model that initializes NeuMF. NeuMF is initialized by pre-training GMF and MLP. The experiment in paper [1] shows that NeuMF-p is better than NeuMF. The specific initialization process of NeuMF is described in the following.

Firstly, GMF and MLP are separately pre-trained. Pre-training at this time means that the GMF(MLP) model is first randomly initialized, and the model is then trained by the loss function until the model converges. Secondly, the parameters of the pre-trained GMF and the pre-trained MLP are the initialization parameters of NeuMF-p. Finally, $h$ is the weight of the “NeuMF layer”, and the initialized value is as follows:

$$\mathit{h}\leftarrow \left[\begin{array}{c}\gamma {\mathit{h}}^{\mathit{G}\mathit{M}\mathit{F}}\\ \left(1-\gamma \right){\mathit{h}}^{\mathit{M}\mathit{L}\mathit{P}}\end{array}\right]$$

(15)

where ${\mathit{h}}^{\mathit{G}\mathit{M}\mathit{F}}$ is the weight of the prediction layer of GMF and ${\mathit{h}}^{\mathit{M}\mathit{L}\mathit{P}}$ is the weight of the prediction layer of MLP. The $\gamma$ is a hyper-parameter and the value of $\gamma$ is set to $0.5$ [1].

Considering that the experiment [1] shows that NeuMF-p is better than NeuMF, this paper chooses to optimize the training process of NeuMF-p directly. Here is a brief introduction to the loss function of NeuMF-p and the original training process. The loss function of NeuMF-p is as follows:

$$\begin{gathered} L=-{\displaystyle \sum }_{\left(u,v\right)\in y}\log {\widehat{y}}_{uv}-{\displaystyle \sum }_{\left(u,v\right)\in y^{-}}\log (1-{\widehat{y}}_{uv}) \\ =-{\displaystyle \sum }_{\left(u,v\right)\in y\cup y^{-}}{y}_{uv}\log {\widehat{y}}_{uv}+\left(1-y_{uv}\right)\log (1-{\widehat{y}}_{uv}) \end{gathered}$$

(16)

This is the loss function of NeuMF-p, also known as the log loss or binary cross-entropy loss, which can be optimized by the SGD (Stochastic Gradient Descent) algorithm. The value of the target $y_{uv}$ is $0$ or $1$, which can be regarded as a label. If $y_{uv}=1$, it means that there is an interaction between user $u$ and item $v$. If $y_{uv}=0$, it means that there is no interaction between user $u$ and item $v$. By using the sigmoid function as the activation function of the output layer, ${\widehat{y}}_{uv}$ can be controlled to output in the range of $\left(0,1\right)$, where $y$ represents the set of observed interactions in dataset $Y$ and $y^{-}$ represents the set of counterexamples in dataset $Y$. The counterexample in the original paper [1] refers to the data of unobserved interactions in dataset $Y$.

3.5. The O-NeuMF-p Model

We name the optimized NeuMF-p model the O-NeuMF-p model. This paper optimizes the training process of NeuMF-p from the counterexample label. The original paper [1] records the label $y_{uv}$ in the counterexample set $y^{-}$ as $0$, which we think is unreasonable. Recommender systems focus on the data that has no interaction between users and items. This means that counterexamples are the focus of the research. If the label of the counterexample is marked as $0$ when training, it is not conducive to learning the interaction in $y^{-}$.

This paper refers to the idea of DAE, adding noise to the original data and reconstructing the data. We change the label in the counterexample set $y^{-}$ from $0$ to a random number. The specific change formula is as follows:

$$if\left(u,v\right)\in y^{-},y_{uv}=random\left(0,r\right)$$

(17)

where $r$ is a hyper-parameter and can be an arbitrary number between $\left[0,1\right]$. If $r=0$, then $y_{uv}=random\left(0,0\right)=0$. In this case, it is back to the original situation, where the label $y_{uv}$ in the counterexample set $y^{-}$ is marked as $0$. If $r=1$, then $y_{uv}=random\left(0,1\right)$. This means that the label $y_{uv}$ in $y^{-}$ is a random number from $0$ to $1$.

The idea of adding random numbers can make the label $y_{uv}$ in $y^{-}$ no longer equal to $0$, which is equivalent to adding a disturbance to the data of $y^{-}$. On the one hand, O-NeuMF-p is more robust than NeuMF-p. On the other hand, O-NeuMF-p is more focused on the learning of the $y^{-}$ data, which is exactly what we want. In addition, our experimental results show that under the same dataset, O-NeuMF-p has a higher recommendation accuracy and quality than NeuMF-p. This confirms the advantage of introducing random numbers in O-NeuMF-p from the side.

4. I-NMF: Fusion of CIDAE and O-NeuMF-p

On the one hand, although O-NeuMF-p performs better than NeuMF-p, it still does not solve the problem of data sparsity. On the other hand, although CIDAE can alleviate the problem of data sparsity, CIDAE only utilizes the idea of a single-layer neural network(AE) and does not fully exploit the ability of deep neural networks to learn potential features. Therefore, this paper fuses CIDAE and O-NeuMF-p with reference to the idea of ensemble learning. We name the fused model the I-NMF (Imputation-Neural Matrix Factorization) model, which combines the advantages of CIDAE and O-NeuMF-p. The I-NMF model can not only alleviate the problem of data sparsity, but also fully exploit the ability of deep neural networks to learn potential features, and I-NMF is also robust. Figure 3 shows the structure of I-NMF.

It is worth noting that there was a paper [24] combining the AE algorithm and the MF algorithm. The paper [24] uses the AE’s feature layer as the initial value of the hidden vector of MF, but the combined model is still linear, like MF. The idea of this paper is not the combination of the feature layer, but purely the combination of the prediction layer, which guarantees the integrity of CIDAE and O-NeuMF-p, so that both models can play their own advantages.

Specific Fusion Process

In data set $\mathcal{O}$, $m$ is the total number of users, and $n$ is the total number of items. Additionally, the data of $\mathcal{O}$ is divided into a training set $\mathcal{X}$ and test set $\mathcal{T}$.

On the one hand, the input of CIDAE trained with $\mathcal{X}$ is ${\mathit{u}}_{\mathit{i}}\left({\mathit{u}}_{\mathit{i}}\in {ℝ}^{\mathit{n}},i\in \left[1,m\right]\right)$, and the output is ${\overline{\mathit{u}}}_{\mathit{i}}$. On the other hand, the input of O-NeuMF-p trained with $\mathcal{X}$ is a group of user IDs and item IDs ($\left(u_i,v_j\right)\left(i\in \left[1,m\right],j\in \left[1,n\right]\right)$), and the output is the predicted value ${\widehat{y}}_{u_i{v}_j}({\widehat{y}}_{u_i{v}_j}\in \left(0,1\right))$.

For each user $u_i$ in $\mathcal{O}$, taking the top-N recommendation as an example, our goal is to use only the data in $\mathcal{X}$ and recommend the top-N items that may be of interest in the missing data. The accuracy and quality of the top-N recommendation is calculated by a comparison with the data in $\mathcal{T}$.

For each user $u_i$, firstly, we use the trained CIDAE to get the vector ${\overline{\mathit{u}}}_{\mathit{i}}\left({\overline{\mathit{u}}}_{\mathit{i}}\in {ℝ}^n\right)$ of the user $u_i$. Then, we use the trained O-NeuMF-p to get $n$ predicted values ${\widehat{y}}_{u_i{v}_j}\left(j\in \left[1,n\right]\right)$ of the user $u_i$. The vector consisting of these $n$ predicted values is denoted as ${\overline{\mathit{y}}}_{\mathit{i}}\left({\overline{\mathit{y}}}_{\mathit{i}}\in {ℝ}^n\right)$. In this way, we have the prediction vectors ${\overline{\mathit{u}}}_{\mathit{i}}$ and ${\overline{\mathit{y}}}_{\mathit{i}}$ corresponding to these two models for each user $u_i$. The prediction vector for each user $u_i$ in I-NMF is denoted as ${\mathit{x}}_{\mathit{i}}\left({\mathit{x}}_{\mathit{i}}\in {ℝ}^{\mathit{n}}\right)$. The value of ${\mathit{x}}_{\mathit{i}}$ is determined by the prediction vectors ${\overline{\mathit{u}}}_{\mathit{i}}$ and ${\overline{\mathit{y}}}_{\mathit{i}}$, and the specific formula is as follows:

$${\mathit{x}}_{\mathit{i}}=\beta {\overline{\mathit{y}}}_{\mathit{i}}+\left(1-\beta \right){\overline{\mathit{u}}}_{\mathit{i}}$$

(18)

where $\beta$ is a hyper-parameter, and $\beta \in \left[0,1\right]$. The introduction of $\beta$ can adjust the fusion ratio of CIDAE and O-NeuMF-p. When $\beta =0$, CIDAE is used for recommendation alone. Conversely, when $\beta =1$, O-NeuMF-p is used for recommendation alone.

We give the recommended top-N items based on the ranking of the value of ${\mathit{x}}_{\mathit{i}}$ in the missing data of the training set $\mathcal{X}$. It is worth noting that the range of the value of ${\mathit{x}}_{\mathit{i}}$ is not necessarily between $0$ and $1$, but this does not affect the resolution of the top-N recommendation. Since our goal is to recommend top-N items, we only need to care about the ranking of the value of ${\mathit{x}}_{\mathit{i}}$.

Our experimental results show that the accuracy and quality of the top-N recommendation of I-NMF is higher than that of CIDAE and O-NeuMF-p, and is also better than some of the current advanced recommendation algorithms. This confirms from the side that I-NMF can combine the advantages of the two models, and I-NMF achieves good recommendation results by using the idea of ensemble learning.

5. Experiments

There are three sets of experiments in this paper. They are the experiment of CIDAE, the experiment of O-NeuMF-p, and the experiment of I-NMF. In this section, we answer the following questions through the three sets of experiments:

• Q1: Is CIDAE better than DAE and AE in the top-N recommendation?
• Q2: Is O-NeuMF-p better than NeuMF-p in terms of the accuracy and quality of the recommended results? What is the degree of optimization of NeuMF-p?
• Q3: Is I-NMF better than CIDAE and O-NeuMF-p in terms of the accuracy and quality of the recommended results?
• Q4: Is I-NMF better than some of the current advanced recommendation models?

5.1. Experimental Datasets

We evaluate all models using the public MovieLens (ML) [19] datasets. We specifically select the ML-100K dataset and the ML-1M dataset in MovieLens (ML). These two datasets are commonly used for evaluating the performance of recommender systems [25,26]. The specific information of the two datasets is shown in

Table 1. Statistics of the evaluation datasets.

Datasets	Ratings	Items	Users	Sparsity
ML-100K	100,000	1682	943	93.70%
ML-1M	1,000,209	3706	6040	95.53%

. It is worth emphasizing that our experiments rely very little on datasets. Specifically, we only use user IDs, item IDs, and ratings.

Each rating in the two datasets is an integer from 1 to 5. Since this paper discusses the top-N recommendation, we only need to focus on whether the user is interested in the item. So, similar to [1,16], we convert the rating to 0 or 1, indicating whether the user has rated the item.

5.2. Performance Metrics

We use Hit Ratio (HR), Normalized Discounted Cumulative Gain (NDCG) [27], and Mean Average Precision (MAP) to evaluate the performance of models. The value of HR can directly reflect the accuracy of the ranking. The larger the HR value, the higher the accuracy of the ranking. NDCG can account for the position of the hit by assigning higher scores to hits at top ranks [1]. The larger the NDCG value, the higher the ranking quality [2]. Mean average precision calculates the mean of users’ average precision. The larger the MAP value, the better the effect of the model.

For each model, we calculate HR@N (N = 5, 10, 15), NDCG@N (N = 5, 10, 15), and MAP@N (N = 5, 10, 15) for each test user and report the average score. It is worth noting that N can be any value. In our paper, the reason we set the value of N to 5, 10, and 15 is to have a span to show the recommended effect of our model under different values of N.

5.3. Experimental Method

To ensure the reliability of the experimental data, we introduce five-fold cross-validation. In five-fold cross-validation, the original sample is randomly partitioned into five equal sized subsamples. Of the five subsamples, a single subsample is retained as the validation data for testing the model, and the remaining four subsamples are used as training data. The cross-validation process is then repeated five times, with each of the five subsamples used exactly once as the validation data. The five results can then be averaged to produce a single estimation. The advantage of this method over repeated random sub-sampling is that all observations are used for both training and validation, and each observation is used for validation exactly once.

We use five-fold cross-validation for all algorithms (including comparison algorithms) on the ML-100K dataset and the ML-1M dataset. Therefore, the experimental results of each algorithm are the average of five-fold cross-validation. It is worth noting that all our models’ parameter experiments are performed only in a certain fold cross-validation process to avoid overfitting.

In order to prove that our algorithm is statistically better than other algorithms, rather than the result of experimental fluctuations, we compare the relative increase rate of the experimental results with the standard deviation of the five-fold cross-validation of our algorithm. For a more intuitive expression, we record the increase rate of the A algorithm relative to the B algorithm as A%B (specific calculation method is (A − B)/B). Furthermore, we record the standard deviation of the five-fold cross-validation of the A algorithm as A-SD. If the relative increase rate (A%B) is greater than the standard deviation (A-SD), our algorithm (A) is statistically superior to another algorithm (B).

5.4. Performance Comparison

The models used for comparison are as follows:

• ItemPop: This model judges the popularity of the item by calculating the number of interactions, and then recommends items with high popularity. ItemPop is a non-personalized recommendation model [28].
• ItemKNN [4]: This model is a standard item-based collaborative filtering model. The item recommended by this model is similar to the item that the user once liked.
• BPR [28]: BPR is the state-of-the-art method for recommendation based on implicit feedback. This model optimizes the MF model based on Bayesian theory and gives personalized rankings. It is a highly competitive baseline for item recommendation.
• WRMF [12,29]: WRMF is a recommendation algorithm based on weight matrix, and it is an optimization algorithm of the MF model. The characteristic of this model is that it assigns different weights to the positive sample and the negative sample.
• CDAE [30]: It is a collaborative denoising autoencoder for collaborative filtering with implicit feedback. CDAE additionally plugs a user node to the input of autoencoders for reconstructing the user’s ratings.
• ExpoMF [31]: Exposure MF (abbreviated as ExpoMF) is a new probabilistic approach that directly incorporates user exposure to items into collaborative filtering. The exposure is modeled as a latent variable and the model infers its value from data.

5.5. Experiment of the CIDAE Model

In this part, firstly, we compare the CIDAE model with the DAE algorithm and the AE algorithm on the ML-100K dataset and the ML-1M dataset. Then, we analyze the influence of parameters of CIDAE on experimental results using the ML-100K dataset.

Table 2. Performance comparison of CIDAE, DAE, and AE based on HR@N, NDCG@N, and MAP@N. (N = 5, 10, 15).

Datasets	Models	HR@5	HR@10	HR@15	NDCG@5	NDCG@10	NDCG@15	MAP@5	MAP@10	MAP@15
ML-100K	CIDAE	0.421306	0.355360	0.314747	0.447365	0.392567	0.356907	0.339666	0.255331	0.210620
	DAE	0.381883	0.326142	0.291124	0.402781	0.356766	0.326358	0.297374	0.223106	0.184123
	AE	0.386891	0.327394	0.292639	0.409233	0.360020	0.329322	0.300240	0.223058	0.183858
	CIDAE%AE	0.088952	0.085419	0.075549	0.093179	0.090405	0.083765	0.131316	0.144688	0.145561
	CIDAE-SD	0.007510	0.004360	0.003198	0.008413	0.005948	0.004636	0.006023	0.002972	0.002202
ML-1M	CIDAE	0.402267	0.344528	0.309183	0.423420	0.375963	0.345131	0.322861	0.247092	0.206846
	DAE	0.348828	0.304988	0.276929	0.364341	0.328476	0.304359	0.269107	0.207003	0.174379
	AE	0.350492	0.304143	0.275353	0.368488	0.330032	0.304874	0.270459	0.205960	0.172438
	CIDAE%AE	0.147721	0.132783	0.122857	0.149075	0.139169	0.132046	0.193753	0.199708	0.199540
	CIDAE-SD	0.001563	0.001495	0.001195	0.001480	0.001417	0.001243	0.001452	0.001454	0.001150

shows that the performance of the CIDAE model is better than the DAE algorithm and the AE algorithm on both datasets. Additionally, it can be found in Table 2 that CIDAE performs better on the ML-1M dataset. Compared with the AE algorithm, the advantage of CIDAE is that it can alleviate the problem of data sparsity by using the idea of imputation. Compared with the DAE algorithm, CIDAE does not add noise randomly, but performs regular continuous imputation on the output data. Therefore, the recommended ranking of CIDAE achieves a higher accuracy and better quality.

We also analyze the influence of parameters of CIDAE on experimental results on the ML-100K dataset. The specific parameters are as follows:

• $f\left(x\right)$: Imputation function;
• $k$: Hyper-parameter in imputation function $f\left(x\right)$;
• $N_i$: Number of iterations;
• $d$: Number of hidden nodes;
• $\mathsf{\alpha }$: The loss weight of the imputed data.

Figure 4 uses NDCG@10 as an indicator to show the performance of various parameters on CIDAE in the ML-100K dataset. Specifically, Figure 4a shows that different imputation functions $f\left(x\right)$ have different experimental results, but the difference is not obvious. So, we finally chose $f\left(x\right)=kx$ because its form is relatively simple. Conversely, the value of $k$ can significantly affect the experimental results and we finally chose $f\left(x\right)=1.4x$ because it works best. Figure 4b shows that when the number of iterations is $400$, the accuracy of CIDAE is converged. Figure 4c shows that CIDAE achieves the highest accuracy when the number of hidden nodes is $200$. So, we finally chose $N_i=400$ and $d=200$. Figure 4d shows that when $\mathsf{\alpha }=0.1$, CIDAE shows the highest accuracy. In addition, we can see that the accuracy is the worst when $\mathsf{\alpha }=1$. This confirms that it is meaningful to calculate the loss of the imputed data separately with the weight $\mathsf{\alpha }$.

5.6. Experiment of the O-NeuMF-p Model

In this part, firstly, we compare O-NeuMF-p with NeuMF-p on the ML-100K dataset and the ML-1M dataset. Then, we count the degree of optimization of NeuMF-p. Finally, we analyze the influence of the parameter of O-NeuMF-p on experimental results on both datasets.

Table 3. Performance comparison of O-NeuMF-p and NeuMF-p based on HR@N, NDCG@N, and MAP@N. (N = 5, 10, 15).

Datasets	Models	HR@5	HR@10	HR@15	NDCG@5	NDCG@10	NDCG@15	MAP@5	MAP@10	MAP@15
ML-100K	O-NeuMF-p	0.403139	0.341947	0.304335	0.427511	0.375740	0.342528	0.324252	0.244377	0.202115
	NeuMF-p	0.390153	0.332802	0.297135	0.412432	0.364942	0.333371	0.308308	0.233447	0.193468
	O-NeuMF-p %NeuMF-p	0.033284	0.027478	0.024231	0.036562	0.029586	0.027469	0.051713	0.046820	0.044698
	O-NeuMF-p -SD	0.006162	0.002480	0.002021	0.006155	0.003796	0.002892	0.007397	0.004844	0.003379
ML-1M	O-NeuMF-p	0.401836	0.349859	0.315786	0.420715	0.377941	0.348670	0.321740	0.250935	0.212394
	NeuMF-p	0.376824	0.332281	0.302629	0.392953	0.356452	0.331099	0.294546	0.230894	0.196529
	O-NeuMF-p %NeuMF-p	0.066374	0.052902	0.043474	0.070650	0.060286	0.053067	0.092326	0.086797	0.080725
	O-NeuMF-p- SD	0.005920	0.005609	0.005026	0.005747	0.005484	0.005080	0.005579	0.004942	0.004452

shows that the performance of O-NeuMF-p is better than NeuMF-p on both datasets. Furthermore, it can be found in Table 3 that the optimization effect of NeuMF-p is more obvious on the ML-1M dataset. The O-NeuMF-p model is more focused on the learning of counterexamples than the NeuMF-p model, and is more robust. Additionally, the experimental results confirm that the idea of introducing random numbers into the label $y_{uv}$ in $y^{-}$ is beneficial and improves the accuracy of NeuMF-p.

We also analyze the effect of the hyper-parameter $r$ controlling the range of random numbers on the accuracy of O-NeuMF-p. It is worth noting that when $r=0$, the O-NeuMF-p model is transformed into the NeuMF-p model.

Figure 5a shows that when $r=0.7$, O-NeuMF-p achieves the highest accuracy on the ML-100K dataset. Figure 5b shows that when $r=0.6$, O-NeuMF-p achieves the highest accuracy on the ML-1M dataset. Furthermore, it can be seen that on the ML-1M dataset, the value of $r$ has little effect on the value of NDCG@10. It is worth emphasizing that for all values of $r$, the value of NDCG@10 of O-NeuMF-p is higher than that of NeuMF-p on the ML-100K dataset and the ML-1M dataset. This proves that O-NeuMF-p is more accurate than NeuMF-p on both datasets.

5.7. Experiment of the I-NMF Model

In this part, firstly, we test the performance of I-NMF and compare it with CIDAE and O-NeuMF-p on the ML-100K dataset and the ML-1M dataset. Then, we analyze the influence of the parameter of I-NMF on the experimental results on both datasets. Finally, we compare I-NMF with current advanced recommendation algorithms on both datasets. The experimental results prove that our I-NMF model performs better. In addition, we also count the running time of I-NMF and comparison algorithms.

Table 4. Performance comparison of I-NMF, CIDAE, and O-NeuMF-p based on HR@N, NDCG@N, and MAP@N. (N = 5, 10, 15).

Datasets	Models	HR@5	HR@10	HR@15	NDCG@5	NDCG@10	NDCG@15	MAP@5	MAP@10	MAP@15
ML-100K	I-NMF	0.437091	0.368832	0.324451	0.463643	0.406837	0.368565	0.359102	0.271408	0.224247
	CIDAE	0.421306	0.355360	0.314747	0.447365	0.392567	0.356907	0.339666	0.255331	0.210620
	O-NeuMF-p	0.403139	0.341947	0.304335	0.427511	0.375740	0.342528	0.324252	0.244377	0.202115
	I-NMF%CIDAE	0.037467	0.037911	0.030830	0.036388	0.036349	0.032663	0.057221	0.062963	0.064696
	I-NMF-SD	0.005536	0.002196	0.002866	0.005829	0.003351	0.003575	0.004329	0.001878	0.001698
ML-1M	I-NMF	0.422925	0.365302	0.328502	0.442975	0.396000	0.364304	0.343087	0.267112	0.225634
	CIDAE	0.402267	0.344528	0.309183	0.423420	0.375963	0.345131	0.322861	0.247092	0.206846
	O-NeuMF-p	0.401836	0.349859	0.315786	0.420715	0.377941	0.348670	0.321740	0.250935	0.212394
	I-NMF% O-NeuMF-p	0.052483	0.044140	0.040269	0.052909	0.047784	0.044839	0.066348	0.064468	0.062334
	I-NMF-SD	0.003211	0.003311	0.002990	0.003393	0.003400	0.003020	0.003694	0.003469	0.002841

shows that the performance of I-NMF is better than CIDAE and O-NeuMF-p on both datasets. On the one hand, the imputation ability of CIDAE can alleviate the sparseness of the original data. On the other hand, O-NeuMF-p uses the learning ability of deep neural networks to learn the interaction between users and items. Moreover, both CIDAE and O-NeuMF-p have a good robustness. It can be seen from the experimental results that CIDAE and O-NeuMF-p can complement each other to achieve a higher accuracy. Additionally, it is worth discussing that although CIDAE performs better on the ML-100K dataset, O-NeuMF-p performs better on the ML-1M dataset. This confirms the advantage of the powerful learning ability of deep neural networks for large datasets. In addition, this paper provides a new fusion idea; not only the combination of CIDAE and O-NeuMF-p, but also the combination of different types of recommendation algorithms.

We also analyze the effect of the hyper-parameter $\beta$ controlling the fusion ratio on the accuracy of I-NMF. When $\beta =0$, CIDAE is used for recommendation alone. Conversely, when $\beta =1$, O-NeuMF-p is used for recommendation alone. Figure 6a shows that when $\beta =0.6$, I-NMF achieves the highest accuracy on the ML-100K dataset. At this time, I-NMF is better than CIDAE and O-NeuMF-p. Figure 6b shows that when $\beta =0.8$, I-NMF achieves the highest accuracy on the ML-1M dataset. It is worth emphasizing that for all values of $\beta$, I-NMF is better than CIDAE and O-NeuMF-p on the ML-1M dataset. This also confirms that CIDAE and O-NeuMF-p can complement each other very well.

Finally, we compare I-NMF with current advanced recommendation algorithms on both datasets.

Table 5. Performance comparison of I-NMF and advanced recommendation algorithms based on HR@N, NDCG@N, and MAP@N. (N = 5, 10, 15).

Datasets	Models	HR@5	HR@10	HR@15	NDCG@5	NDCG@10	NDCG@15	MAP@5	MAP@10	MAP@15
ML-100K	I-NMF	0.437091	0.368832	0.324451	0.463643	0.406837	0.368565	0.359102	0.271408	0.224247
	ItemPop	0.213072	0.189456	0.168726	0.222822	0.203229	0.186199	0.141576	0.104503	0.084945
	BPR	0.252880	0.217062	0.194386	0.265752	0.236420	0.216823	0.180987	0.131805	0.107027
	ItemKNN	0.353631	0.293864	0.256773	0.378404	0.328200	0.295540	0.279124	0.204424	0.165260
	WRMF	0.358073	0.306407	0.274969	0.379183	0.336027	0.308229	0.276228	0.207112	0.171358
	CDAE	0.381714	0.325527	0.290687	0.402744	0.356477	0.326134	0.297378	0.223183	0.184031
	ExpoMF	0.410737	0.343349	0.304449	0.434643	0.379345	0.345066	0.331008	0.248299	0.204915
	I-NMF% ExpoMF	0.064161	0.074218	0.065699	0.066722	0.072470	0.068098	0.084873	0.093068	0.094341
	I-NMF-SD	0.005536	0.002196	0.002866	0.005829	0.003351	0.003575	0.004329	0.001878	0.001698
ML-1M	I-NMF	0.422925	0.365302	0.328502	0.442975	0.396000	0.364304	0.343087	0.267112	0.225634
	ItemPop	0.210982	0.182530	0.167163	0.221308	0.197872	0.184095	0.152467	0.112520	0.092966
	BPR	0.290552	0.248252	0.222520	0.306210	0.271397	0.248911	0.221773	0.164860	0.135679
	ItemKNN	0.343951	0.287240	0.252039	0.365079	0.318339	0.287692	0.271287	0.201249	0.164336
	WRMF	0.386441	0.338752	0.308109	0.403729	0.364616	0.338220	0.299831	0.232759	0.197326
	CDAE	0.397328	0.340984	0.306816	0.419981	0.372899	0.342745	0.321560	0.247175	0.208099
	ExpoMF	0.401107	0.346231	0.312267	0.420421	0.375562	0.346169	0.324212	0.251804	0.212747
	I-NMF% ExpoMF	0.054395	0.055081	0.051991	0.053646	0.054420	0.052387	0.058218	0.060794	0.060574
	I-NMF-SD	0.003211	0.003311	0.002990	0.003393	0.003400	0.003020	0.003694	0.003469	0.002841

shows that the performance of I-NMF is best on both datasets. On the ML-100K dataset, compared to the state-of-the-art method of ExpoMF, on average, I-NMF obtains 6.8%, 6.9%, and 9.1% relative improvements in HR, NDCG, and MAP metrics, respectively. On the ML-1M dataset, compared to the state-of-the-art method of ExpoMF, on average, I-NMF obtains 5.4%, 5.3%, and 6.0% relative improvements in HR, NDCG, and MAP metrics, respectively. These prove that the accuracy of our I-NMF model is very high.

In addition, we also count the running time (a fold cross-validation process) of I-NMF and comparison algorithms in

Table 6. Running time of I-NMF and comparison algorithms.

Datasets	Time	I-NMF	ItemPop	BPR	ItemKNN	WRMF	CDAE	ExpoMF
ML-100K	Train Time	2.4 min	1 s	11 min	5 s	5.5 s	1.3 min	2.2 min
	Test Time	18.2 s	1 s	2.6 s	36.7 s	2.5 s	1.4 min	2.7 s
ML-1M	Train Time	51.3 min	5 s	111 min	1.4 min	1.2 min	26.6 min	16.4 min
	Test Time	2 min	28 s	40 s	25 min	40 s	66 min	38 s

. It is worth noting that our experiment is run under the CPU (Processor Intel Core i7-6700k, Memory 16GB), and if it can use the GPU to run, it may be faster.

Some parameters of our I-NMF model can affect the running time, such as: $N_i$ (Number of iterations) and $d$ (Number of hidden nodes) in CIDAE, the number of layers of MLP in O-NeuMF-p, and so on. But the hyper-parameters $\beta$ (controlling the fusion ratio), $r$ (controlling the range of random numbers), and $\mathsf{\alpha }$ (the loss weight of the imputed data), and so on, do not affect the runtime. The training time in our model is relatively large. However, during prediction, the running time is negligible, where it only takes 18.2 s for the ML-100K dataset and 2 min for the ML-1M dataset. In addition to this, we can optimize the training time in the future.

6. Conclusions and Future Work

Firstly, we propose a new CIDAE model based on DAE. The advantage of CIDAE is that it alleviates the problem of data sparsity by using the idea of imputation. Additionally, our experimental results show that CIDAE is more accurate than the AE algorithm and the DAE algorithm.

Then, we optimize the advanced NeuMF model based on the training process. O-NeuMF-p is more focused on the learning of counterexamples than NeuMF-p, and is more robust. Furthermore, the experimental results confirm that the idea of introducing random numbers into the label $y_{uv}$ in $y^{-}$ is beneficial and improves the accuracy of NeuMF-p.

Finally, this paper fuses CIDAE and O-NeuMF-p with reference to the idea of ensemble learning. I-NMF can not only alleviate the problem of data sparsity, but also fully exploit the ability of deep neural networks to learn potential features, and I-NMF is also robust. It can be seen from the experimental results that CIDAE and O-NeuMF-p can complement each other to achieve a higher accuracy. Moreover, our experimental results prove that I-NMF performs better than current advanced recommendation algorithms on both datasets.

It is worth emphasizing that this paper provides a new fusion idea; not only the combination of CIDAE and O-NeuMF-p, but also the combination of different types of recommendation algorithms.

In the future, there are two directions to extend our work. On the one hand, we can try more ways to combine different kinds of recommendation algorithms to achieve a better accuracy. On the other hand, we can try to incorporate auxiliary extra data into recommender systems, such as social relations, review text, type of items, personal information of users, and so on. We can further improve the accuracy of the model by adding extra valuable data.