# A Knowledge Concept Recommendation Model Based on Tensor Decomposition and Transformer Reordering

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

- The integrity of the heterogeneous data of the online course is preserved by modeling the student, knowledge concept, and interaction data through the creation of a tensor, and the overall data are analyzed comprehensively in multiple (student, knowledge concept, cognitive level, knowledge concept achievement, and student–system interaction) dimensions using a tensor-based higher-order singular value decomposition to uncover latent information between the data.
- The transformer encoder layer is used to capture sequential information between knowledge concepts and to fusion personalized student characteristics, enabling more accurate knowledge concept recommendations.
- Extensive experiments are conducted on two real datasets, and the experimental results demonstrate the advantages of the TTRKRec proposed in this paper compared to several state-of-the-art knowledge concept recommendation models.

## 2. Related Work

## 3. Correlation Definition

#### 3.1. The Degree of Student–System Interaction

#### 3.2. The Degree of Student–Teacher Interaction

#### 3.3. Tensor and Tensor Calculations

**Definition**

**1.**

**Definition**

**2.**

#### 3.4. Tensor Construction and Fusion

#### 3.5. HOSVD

#### 3.6. Transformer-Based Knowledge Concept Embedding

#### 3.6.1. Sequential Information Encoding

#### 3.6.2. Multi-Head Attention

#### 3.6.3. Residual Connection

#### 3.6.4. Feedforward Networks

## 4. TTRKRec Model

#### 4.1. Recommendations Based on Tensor Decomposition

#### 4.2. Reordering Based on Transformer

## 5. Experiment

#### 5.1. Experimental Dataset

#### 5.2. Evaluation and Baselines

#### 5.3. Implementation Details

#### 5.4. Experimental Results

#### 5.5. Ablation Studies

#### 5.6. Case Study

#### 5.7. Discussion

## 6. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Wu, B.; Wang, Y. Formation mechanism of popular courses on MOOC platforms: A configurational approach. Comput. Educ.
**2022**, 191, 104629. [Google Scholar] [CrossRef] - Uddin, I.; Imran, A.S.; Muhammad, K.; Fayyaz, N.; Sajjad, M. A Systematic Mapping Review on MOOC Recommender Systems. IEEE Access
**2021**, 9, 118379–118405. [Google Scholar] [CrossRef] - Lin, L. Learning information recommendation based on text vector model and support vector machine. J. Intell. Fuzzy Syst.
**2021**, 40, 2445–2455. [Google Scholar] [CrossRef] - Li, Y.; Li, Q.; Meng, S.; Hou, J. Transformer-Based Rating-Aware Sequential Recommendation; Springer International Publishing: Cham, Switzerland, 2022; pp. 759–774. ISBN 0302-9743. [Google Scholar]
- Jena, K.K.; Bhoi, S.K.; Malik, T.K.; Sahoo, K.S.; Jhanjhi, N.Z.; Bhatia, S.; Amsaad, F. E-Learning Course Recommender System Using Collaborative Filtering Models. Electronics
**2022**, 12, 157. [Google Scholar] [CrossRef] - Shen, D.; Jiang, Z. Online Teaching Course Recommendation Based on Autoencoder. Math. Probl. Eng.
**2022**, 2022, 8549563. [Google Scholar] [CrossRef] - Zhu, Y.; Lu, H.; Qiu, P.; Shi, K.; Chambua, J.; Niu, Z. Heterogeneous teaching evaluation network based offline course recommendation with graph learning and tensor factorization. Neurocomputing
**2020**, 415, 84–95. [Google Scholar] [CrossRef] - Pu, Y.; Chen, H. Exercise Recommendation Model Based on Cognitive Level and Educational Big Data Mining. J. Funct. Space
**2022**, 2022, 3845419. [Google Scholar] [CrossRef] - Diao, X.; Zeng, Q.; Li, L.; Duan, H.; Zhao, H.; Song, Z. Personalized learning path recommendation based on weak concept mining. Mob. Inf. Syst.
**2022**, 2022, 2944268. [Google Scholar] [CrossRef] - Liu, Y.; Zhang, Y.; Zhang, G. Learning path recommendation based on Transformer reordering. In Proceedings of the 2020 5th International Conference on Information Science, Computer Technology and Transportation (ISCTT), Shenyang, China, 13–15 November 2020; pp. 101–104. [Google Scholar]
- Gong, J.; Wang, S.; Wang, J.; Feng, W.; Peng, H.; Tang, J.; Yu, P. Attentional Graph Convolutional Networks for Knowledge Concept Recommendation in MOOCs in a Heterogeneous View. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual, 25–30 July 2020; pp. 79–88. [Google Scholar]
- Zhao, Z.; Yang, Y.; Li, C.; Nie, L. GuessUNeed: Recommending courses via neural attention network and course prerequisite relation embeddings. ACM Trans. Multimed. Comput. Commun. Appl. TOMM
**2020**, 16, 1–17. [Google Scholar] [CrossRef] - Shi, D.; Wang, T.; Xing, H.; Xu, H. A learning path recommendation model based on a multidimensional knowledge graph framework for e-learning. Knowl. Based Syst.
**2020**, 195, 105618. [Google Scholar] [CrossRef] - Wang, F.; Zhang, L.; Chen, X.; Wang, Z.; Xu, X. A personalized self-learning system based on knowledge graph and differential evolution algorithm. Concurr. Comput.
**2022**, 34, e6190. [Google Scholar] [CrossRef] - Huang, G.; Yuan, M.; Li, C.; Wei, Y. Personalized knowledge recommendation based on knowledge graph in petroleum exploration and development. Int. J. Pattern Recognit. Artif. Intell.
**2020**, 34, 2059033. [Google Scholar] [CrossRef] - Yang, X.; Huan, Z.; Zhai, Y.; Lin, T. Research of Personalized Recommendation Technology Based on Knowledge Graphs. Appl. Sci.
**2021**, 11, 7104. [Google Scholar] [CrossRef] - Wang, X.; Jia, L.; Guo, L.; Liu, F. Multi-aspect heterogeneous information network for MOOC knowledge concept recommendation. Appl. Intell.
**2022**, 16, 132. [Google Scholar] [CrossRef] - Gong, J.; Wang, C.; Zhao, Z.; Zhang, X. Automatic Generation of Meta-Path Graph for Concept Recommendation in MOOCs. Electronics
**2021**, 10, 1671. [Google Scholar] [CrossRef] - Ye, B.; Mao, S.; Hao, P.; Chen, W.; Bai, C. Community enhanced course concept recommendation in MOOCs with multiple entities. In Proceedings of the Knowledge Science, Engineering and Management: 14th International Conference, KSEM 2021, Tokyo, Japan, 14–16 August 2021; pp. 279–293. [Google Scholar]
- Ling, Y.; Shan, Z. Knowledge Concept Recommender Based on Structure Enhanced Interaction Graph Neural Network; Springer International Publishing: Cham, Switzerland, 2022; pp. 173–186. ISBN 0302-9743. [Google Scholar]
- Liang, Z.; Mu, L.; Chen, J.; Xie, Q. Graph path fusion and reinforcement reasoning for recommendation in MOOCs. Educ. Inf. Technol.
**2022**, 28, 525–545. [Google Scholar] [CrossRef] - Sun, Z.; Zhang, X.; Li, H.; Xiao, Y.; Guo, H. Recommender systems based on tensor decomposition. Comput. Mater. Contin.
**2020**, 66, 621–630. [Google Scholar] [CrossRef] - Hong, M.; Jung, J.J. Multi-criteria tensor model consolidating spatial and temporal information for tourism recommendation. J. Ambient Intell. Smart Environ.
**2021**, 13, 5–19. [Google Scholar] [CrossRef] - Liu, H.; Ding, J.; Yang, L.T.; Guo, Y.; Wang, X.; Deng, A. Multi-dimensional correlative recommendation and adaptive clustering via incremental tensor decomposition for sustainable smart education. IEEE Trans. Sustain. Comput.
**2019**, 5, 389–402. [Google Scholar] [CrossRef] - Shou, Z.; Wen, Y.; Chen, P.; Zhang, H. Personalized Knowledge Map Recommendations based on Interactive Behavior Preferences. Int. J. Perform. Eng.
**2021**, 17, 36–49. [Google Scholar] - Zhu, H.; Liu, Y.; Tian, F.; Ni, Y.; Wu, K.; Chen, Y.; Zheng, Q. A Cross-Curriculum Video Recommendation Algorithm Based on a Video-Associated Knowledge Map. IEEE Access
**2018**, 6, 57562–57571. [Google Scholar] [CrossRef] - Liu, H.; Yang, L.T.; Chen, J.; Ye, M.; Ding, J.; Kuang, L. Multivariate Multi-Order Markov Multi-Modal Prediction with Its Applications in Network Traffic Management. IEEE Trans. Netw. Serv. Manag.
**2019**, 16, 828–841. [Google Scholar] [CrossRef] - Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, A.; Polosukhin, I. Attention is all you need. arXiv
**2017**, arXiv:1706.03762. [Google Scholar] - Yu, J.; Luo, G.; Xiao, T.; Zhong, Q.; Wang, Y.; Feng, W.; Luo, J.; Wang, C.; Hou, L.; Li, J. MOOCCube: A large-scale data repository for NLP applications in MOOCs. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Virtual, 5 July 2020; pp. 3135–3142. [Google Scholar]
- Mnih, A.; Salakhutdinov, R.R. Probabilistic matrix factorization. Adv. Neural Inf. Process. Syst.
**2007**, 20, 1257–1264. [Google Scholar] - Li, L.; Lin, F.; Xiahou, J.; Lin, Y.; Wu, P.; Liu, Y. Federated low-rank tensor projections for sequential recommendation. Knowl. Based Syst.
**2022**, 255, 109483. [Google Scholar] [CrossRef] - Zhu, B.; Yeung, C.H.; Liem, R.P. The impact of common neighbor algorithm on individual friend choices and online social networks. Phys. A Stat. Mech. Its Appl.
**2021**, 566, 125670. [Google Scholar] [CrossRef] - Mishra, R.; Kumar, P.; Bhasker, B. A web recommendation system considering sequential information. Decis. Support Syst.
**2015**, 75, 1–10. [Google Scholar] [CrossRef]

**Figure 2.**Tensor fusion and simplification. (

**a**) Integrated student–knowledge concept tensor; (

**b**) simplified integrated tensor.

Models | Paper Numbers | Advantages | Limitations |
---|---|---|---|

Knowledge graph-based recommendation models | [13,14,15,16] | Making recommendations interpretable | Problems with missing relationships or entities |

Recommendation models based on heterogeneous information networks | [17,18,19] | Achieves a more accurate representation of students and knowledge concepts | Over-reliance on meta-path similarity |

Graphical neural-based recommendation models | [11,20,21] | High ability to extract time-series features | Requires some complex design to apply to heterogeneous information |

Recommendation models based on tensor decomposition | [22,23,24] | Suitable for the representation and extraction of potential features in high-dimensional data spaces | Weak ability to capture semantic and sequential information |

Tensor | Components |
---|---|

Student | student ID, stage assessment score, cognitive level |

Knowledge concept | student ID, knowledge concept ID, knowledge concept score, knowledge concept learning time |

Interaction | student ID, knowledge concept ID, student–system interaction, student–teacher interaction |

Baselines | Description |
---|---|

PMF | This is a classical matrix decomposition model with a probability distribution. For knowledge concept recommendations, the method decomposes the student knowledge concept rating matrix and makes recommendations based on predicted scores [30]. |

ACKRec | This is a graph convolutional neural network model with an attention mechanism that transforms data into several adjacency matrices and feeds them into the model to generate embeddings of different entities [11]. |

Multi-HIN | This is a knowledge concept recommendation model based on a multifaceted heterogeneous information network that can naturally use rich heterogeneous context-aided information for dynamic node identification and can effectively discover and aggregate student interests [17]. |

FedSeqRec | This is a new horizontal federation framework for sequential recommendations that use low-rank tensor projections to model users’ long-term preferences [31]. |

ITCA-PR | This is a tensor decomposition-based learning resource recommendation method that can recommend personalized learning resources in different contexts [24]. |

Dataset | Model | AUC | NDCG@5 | NDCG@10 | MRR |
---|---|---|---|---|---|

MOOCCube | PMF | 0.8532 | 0.2584 | 0.2908 | 0.2562 |

ACKRec | 0.9232 | 0.4635 | 0.5170 | 0.4352 | |

FedSeqRec | 0.9692 | 0.3472 | 0.3984 | 0.3294 | |

Multi-HIN | 0.9315 | 0.4182 | 0.5130 | 0.4140 | |

ITCA-PR | 0.9079 | 0.4053 | 0.4584 | 0.4028 | |

TTRKRec | 0.9441 | 0.5011 | 0.5715 | 0.4512 | |

Online | PMF | 0.8514 | 0.2923 | 0.3318 | 0.2912 |

ACKRec | 0.8858 | 0.3820 | 0.4015 | 0.3511 | |

FedSeqRec | 0.8731 | 0.3515 | 0.3884 | 0.4028 | |

Multi-HIN | 0.8974 | 0.4255 | 0.4697 | 0.4291 | |

ITCA-PR | 0.8910 | 0.4212 | 0.4654 | 0.4021 | |

TTRKRec | 0.9241 | 0.4862 | 0.5113 | 0.4315 |

Recommend List | Real Learning Record | |||
---|---|---|---|---|

TTRKRec | Multi-HIN | ACKRec | ITCA-PR | |

LinkList | Order List | Data object | Substring | Top |

Top | Queue | Last-in first-out | Topological sequences | Last-in First-out |

Bottom | Adjacency table | Rear | Top | LinkList |

Last-in First-out | Top | Array | Full binary tree | Search |

Queue | Binary tree | Sequential strings | First-out First-in | Top |

Binary Trees | Graph traversal | Binary tree | Queue | Queue |

Sequential storage | Postorder traversal | Queue | Hash functions | Binary tree |

tree | Hash Tables | Inorder traversal | Search | Array |

Graph traversal | Array | Hash Tables | Sort | Graph traversal |

Preorder traversal | Sort | Efficiency | Graph traversal | Inorder traversal |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Shou, Z.; Chen, Y.; Wen, H.; Liu, J.; Mo, J.; Zhang, H. A Knowledge Concept Recommendation Model Based on Tensor Decomposition and Transformer Reordering. *Electronics* **2023**, *12*, 1593.
https://doi.org/10.3390/electronics12071593

**AMA Style**

Shou Z, Chen Y, Wen H, Liu J, Mo J, Zhang H. A Knowledge Concept Recommendation Model Based on Tensor Decomposition and Transformer Reordering. *Electronics*. 2023; 12(7):1593.
https://doi.org/10.3390/electronics12071593

**Chicago/Turabian Style**

Shou, Zhaoyu, Yishuai Chen, Hui Wen, Jinghua Liu, Jianwen Mo, and Huibing Zhang. 2023. "A Knowledge Concept Recommendation Model Based on Tensor Decomposition and Transformer Reordering" *Electronics* 12, no. 7: 1593.
https://doi.org/10.3390/electronics12071593