Bias Assessment Approaches for Addressing UserCentered Fairness in GNNBased Recommender Systems
Abstract
:1. Introduction
 Review of the stateoftheart concerning the evaluation and mitigation of bias in the field of machine learning, especially in the areas of recommender systems and GNNbased methods.
 Study of the adequacy of the metrics for evaluating fairness bias that affects users of recommender systems regarding discrimination between different groups.
 Examination of the behavior of GNNbased recommendation methods against the aforementioned fairness bias and comparison with other recommendation approaches.
 Analysis of the relationship between the values of the fairness metrics and the classic metrics for evaluating the quality of the recommendation lists (precision, recall, mean reciprocal rank, etc.) since the mitigation of biases usually results in a worsening of the recommendation quality evaluated by these measures.
 RQ1: Can the findings reported in the literature in the general context of machine learning be extended to the specific field of recommender systems?
 RQ2: Do the performances of GNNbased recommendation methods against biases depend on dataset characteristics and sensitive attributes?
 RQ3: Are all bias evaluation metrics appropriate for assessing usercentered fairness in recommender systems in all application domains?
 RQ4: Do less biasprone methods always provide lowerquality recommendations?
2. StateoftheArt
2.1. Bias and Fairness in Machine Learning (ML)
2.2. Bias and Fairness in GNNBased Models
2.3. Bias and Fairness in Recommender Systems
2.4. Bias and Fairness in GNNBased RS
3. Study of Performance against Bias of Recommendation Methods
3.1. Methodology
3.2. Benchmark Datasets
3.2.1. MovieLens 100k
3.2.2. LastFM 100k
3.2.3. Book Recommendation 100k
3.3. Recommendation Methods
 Collaborative filtering (CF).CF approaches are implemented based on ratings given to items by users, which enclose the user preferences. The recommendation algorithms predict the ratings that users would give to items not rated by them by calculating user or item similarities. These similarities indicate that similar ratings given to a certain item by two different users can also happen for new items. Items can be recommended to a user based on the previous items consumed or rated by the user. In CF approaches, ratings are represented in a user–item rating matrix that is used to find similarities among users and items. A drawback related to these approaches is that user and item features are often not available since recommendations in CF are provided by only using the feedback of other users. CF techniques can be implemented in two different ways, as can be seen below:
 Userbased: This technique is used to predict the items that a user might desire on the basis of ratings given to that item by other users who share similar preferences with the target user [1].
 Itembased: This technique predicts the rating of a user for an item on the basis of the ratings given by the user to similar items. The similarity of the items is calculated from the values of the ratings they receive from users. Itembased approaches are usually more reliable, faster, and do not need to be updated frequently, but the results are sometimes worse than those of userbased methods [1].
Figure 10 shows the differences between userbased and itembased approaches.The CF methods used: ItemKNN: It is a widely used algorithm belonging to the itembased group, where the similarity between items is computed based on the ratings given by the users. Customers usually are more likely to consume items with the same characteristics as those previously consumed by them, which is the main idea behind this method. It follows a modelbased approach involving two important components. The first is in charge of inducing a model which captures the relations between different items, and the second component uses the model to provide recommendations for a user. The response time to the user of this method is quite short because the model is already created at the recommendation time. In addition, it provides good accuracy compared with other similar CF methods [78,79,80,81].
 Neural collaborative filtering model with the interactionbased neighborhood (NNCF): This is a CF method where complex interactions between users and items are modeled by means of deep learning, although neighborhood information is used to improve the performance. The results of traditional methods, such as simple linear factorization, can be enhanced by NNCF, giving the best support to the complex interactions among users and items. Another advantage of this method is the possibility to obtain highquality user–item embeddings. [82,83,84].
 Matrix Factorization:Recently the usage of MF methods in RS has increased significantly thanks to their advantages and the capability of reducing the storage size [1,85]. The purpose of MF models is to address the sparsity problem of the rating matrix through its transformation into two more compact matrices of latent factors of users and items. The inner product of these matrices encloses user preferences for items [1,86].Used MF approaches:
 Deep matrix factorization (DMF): This method benefits from neural network architecture by constructing a user–item matrix containing explicit ratings and nonpreference implicit feedback. The matrix is the input for learning common lowdimensional space for the deep structure learning architecture. To optimize this method, a new loss function based on binary cross entropy is introduced, which considers explicit and implicit feedback. Compared with other conventional models, DMF shows better accuracy in topK recommendations by using implicit feedback, hence, reconstructing the ratings of users through learning hidden structures from explicit ratings. In addition, DMF supports twochannel structures for combining side information from both users and items. Several studies show that DMF methods provide good accuracy and high efficiency [87,88,89].
 Neural collaborative filtering (NeuMF): In this method, a neural network architecture replaces the inner product of user–item interaction in CF models. Neural networkbased collaboration (NCF) is an approach that generalizes matrix factorization and can be improved by using nonlinear kernels. To do so, a multilayer perceptron is used to learn the user–item interaction function. The use of general NCF in NeuMF can be useful for the combination of different models and using the side information. Studies indicate that acceptable accuracy of deep neural networks comes from their good capacity and nonlinearity [85,90].
 GNNbased.Graph learning (GL), machine learning applied to graph structure data, is a rapidly developing technology with good capabilities [23]. Graph Learningbased recommender system(s) (GLRS) are introduced by using relational data in this structure [8]. In realworld systems and applications, objects are often connected either implicitly or explicitly, forming a graph structure. In the field of RS, users, items, attributes, and context are the objects of the structure. They are strongly connected and influence each other via various relations. Using graph techniques can significantly improve the quality of RS. In addition, GL has both a great capacity to learn complex relations and a strong potential for seizing knowledge encapsulated in different types of graphs [56].Diverse relations in RS can be comprehended with entities, including users, items, and attributes. A wide variety of graphs can be used to represent these entities. To implement a conventional RS, user, item, and interaction between them are sufficient, but to evaluate various metrics and enhance the fairness and accuracy of the model, other information regarding users and/or items can be used. This information can be categorized into two main groups, namely user–item interaction data (user ratings, clicks, purchases…), and side information data (user and item attributes). The first group can be further categorized according to whether the interactions are sequential or general [56]. Additional subclassifications of each group are shown in Table 3.Information regarding the type of interaction that happens between users and items represented in the user–item matrix. Data from the mentioned interaction can be either explicit or implicit. The first type occurs when users express their opinions about items directly (e.g., ratings on items). Implicit information is induced from the actions of the user in the interaction with the system (e.g., click, view) [56,91].The GNN Methods used.
 LightGCN: LightGCN is a simplified variant of the graph convolution network (GCN) containing the most essential components of GCN for recommendation tasks. This method involves the linear propagation of the user and item embeddings on the user–item interaction graph. Moreover, the calculation of the final embedding is performed by using the weighted sum of the embeddings that are learned throughout all layers [92]. LightGCN adopts the same symmetric normalization used in standard GCN for controlling the expansion size of embeddings by means of graph convolution operations. Several studies have indicated the good performance of LightGCN in comparison to traditional approaches [93,94].
 Neural graph collaborative filtering (NGCF): The method represents user–item interactions as a graph structure, which is used to create highorder connectivity and generate embeddings on it. In this structure, the collaborative signal is explicitly transferred through the process of embedding [56]. In addition, multiple embedding propagation layers with concatenated outputs are used to finally predict items to be recommended. NGCF is known for its good performance with respect to model optimization [95,96].
 Selfsupervised graph learning (SGL): SGL is a version of GCN optimized for improving accuracy and robustness. This method is very noise tolerant and outperforms other models in this matter. An enhanced classical supervised recommendation task with support for the selfsupervised task is used in this method to reinforce the learning of node representation via selfdiscrimination. Various views of a node can be generated in this structure, which maximizes the agreement between the different views of the same node compared to the views of other nodes. Many strides show that the SGL method performs very well in RS tasks with respect to the accuracy of the results [21,97,98,99].
 Disentangled graph collaborative filtering (DGCF): This method is designed to focus more on user–item relationships by extracting factors and producing disentangled representations. DGCF models distribution for each user–item interaction and iteratively filters the intentaware interaction graphs and representations while maintaining the independence of different intents. Although executing DGCF is very timeconsuming, this method outperforms many stateoftheart models [100,101,102].
3.4. Evaluation Metrics
Metric Name  Description 

Average Popularity  Measures the average popularity of the items included in the recommendations. It is calculated as follows, where $\varphi (i)$ is the number of interactions or ratings on item i in the training data [107].
$$AveragePopularity@K=\frac{1}{U}\sum _{u\in U}\frac{{\sum}_{i\in {R}_{u}}\varphi (i)}{{R}_{u}}$$

Gini index [51]  Measures the diversity of recommended items. The calculation is shown below, where $P(i)$ indicates the number of occurrences of the item i and the recommendation list, which is indexed in nondecreasing order [108].
$$GiniIndex@K=\frac{{\sum}_{i=1}^{I}(2iI1)P(i)}{I{\sum}_{i=1}^{I}P(i)}$$

Item Coverage  Represents the percentage of recommended items over all items [109]
$$ItemCoverage@K=\frac{{\cup}_{u\in U}\widehat{R}(u)}{I}$$

Differential fairness (DF) for gender (a sensitive attribute) [103,110]  Measures the bias in the recommendations received by the protected groups. An algorithm or mechanism $M(x)$ is $\u03f5$differentially fair with respect to $(A,\theta )$ if for all $\theta \in \Theta $ with $x\sim \theta $, and $y\in Range(M)$. For all $({s}_{i},{s}_{j})\in A\times A$ where $P({s}_{i})>0$, $P({s}_{j})>0$. ${s}_{i},{s}_{j}\in A$ are tuples of all protected attribute values.
$${e}^{\u03f5}\le \frac{{P}_{M,\theta}(M(x)=y{s}_{i},\theta )}{{P}_{M,\theta}(M(x)=y{s}_{j},\theta )}\le {e}^{\u03f5}$$

Value Unfairness [105,106,111]  Computes instability in signed estimation error over user types
$${U}_{val}=\frac{1}{I}\sum _{j=1}^{I}({E}_{g}{\left[y\right]}_{j}{E}_{g}{\left[r\right]}_{j})({E}_{\neg g}{\left[y\right]}_{j}{E}_{\neg g}{\left[r\right]}_{j})$$
$${E}_{g}{\left[y\right]}_{j}:=\frac{1}{i:((i,j)\in X)\wedge {g}_{i}}\sum _{i:((i,j)\in X)\wedge {g}_{i}}{y}_{ij}$$

Absolute Unfairness [105,106]  Measures differences in absolute estimation error over groups of users
$${U}_{abs}=\frac{1}{I}\sum _{j=1}^{I}{E}_{g}{\left[y\right]}_{j}{E}_{g}{\left[r\right]}_{j}{E}_{\neg g}{\left[y\right]}_{j}{E}_{\neg g}{\left[r\right]}_{j}$$

NonParity Unfairness [105,106]  It is the absolute difference between the rating average of protected and unprotected groups
$${U}_{par}={E}_{g}\left[y\right]{E}_{\neg g}\left[y\right]$$

4. Results of the Experimental Study
5. Discussion of Results
 RQ1: Can the findings reported in the literature in the general context of machine learning be extended to the specific field of recommender systems? Although the literature states that GNN methods are more prone to bias than other classical techniques, the same cannot be said in the area of recommender systems, since some of these methods perform well against bias while maintaining the accuracy of the recommendations.
 RQ2: Does the performance of GNNbased recommendation methods against biases depend on dataset characteristics and sensitive attributes? The study showed that the fairness metrics present irregular results for the different datasets. For example, the tested algorithms yield totally different values for the genderrelated unfairness metrics in the MovieLens and LastFM datasets, with the gender imbalance being very similar in both datasets. This reveals that the bias in the results is highly dependent on other characteristics of the data. This irregular behavior occurs with different sensitivity attributes.
 RQ3: Are all bias evaluation metrics appropriate for assessing usercentered fairness in recommender systems in all application domains? The literature review has allowed us to compile the most commonly used bias metrics, some of which do not assess the quality of the results but rather the similarity of the results themselves for different groups. Because these groups are formed on the basis of sensitive attributes such as gender or age, which were shown to influence preferences, these metrics are not appropriate in the field of recommender systems whose objective is to predict user preferences. In most of the application domains of RS, such as recommendations of movies, music, etc., preferences change according to these attributes. In fact, some methods use them to generate better recommendations.
 RQ4: Do less biasprone methods always provide lowerquality recommendations? Although the decrease in biases generally results in low values of quality metrics such as precision, NDCG, etc., this does not always happen. Some of the GNNbased recommendation methods present good values both for these last metrics and for the bias metrics, presenting in some cases better behavior against bias than classical methods.
6. Conclusions and Future Work
Author Contributions
Funding
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
 Ricci, F.; Rokach, L.; Shapira, B. Recommender Systems: Techniques, Applications, and Challenges. In Recommender Systems Handbook; Springer: New York, NY, USA, 2022; pp. 1–35. [Google Scholar]
 Zheng, Y.; Wang, D.X. A survey of recommender systems with multiobjective optimization. Neurocomputing 2022, 474, 141–153. [Google Scholar] [CrossRef]
 PérezMarcos, J.; MartínGómez, L.; JiménezBravo, D.M.; López, V.F.; MorenoGarcía, M.N. Hybrid system for video game recommendation based on implicit ratings and social networks. J. Ambient Intell. Humanized Comput. 2020, 11, 4525–4535. [Google Scholar] [CrossRef]
 Lin, S.; Wang, J.; Zhu, Z.; Caverlee, J. Quantifying and Mitigating Popularity Bias in Conversational Recommender Systems. arXiv 2022, arXiv:2208.03298. [Google Scholar]
 Chen, J.; Dong, H.; Wang, X.; Feng, F.; Wang, M.; He, X. Bias and debias in recommender system: A survey and future directions. arXiv 2020, arXiv:2010.03240. [Google Scholar] [CrossRef]
 Boratto, L.; Marras, M. Advances in Biasaware Recommendation on the Web. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, Virtual Event, Israel, 8–12 March 2021; pp. 1147–1149. [Google Scholar]
 MisztalRadecka, J.; Indurkhya, B. BiasAware Hierarchical Clustering for detecting the discriminated groups of users in recommendation systems. Inf. Process. Manag. 2021, 58, 102519. [Google Scholar] [CrossRef]
 Gao, C.; Wang, X.; He, X.; Li, Y. Graph neural networks for recommender system. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, Tempe, AZ, USA, 21–25 February 2022; pp. 1623–1625. [Google Scholar]
 Di Noia, T.; Tintarev, N.; Fatourou, P.; Schedl, M. Recommender systems under European AI regulations. Commun. ACM 2022, 65, 69–73. [Google Scholar] [CrossRef]
 Fahse, T.; Huber, V.; Giffen, B.V. Managing bias in machine learning projects. In Proceedings of the International Conference on Wirtschaftsinformatik, online. 9–11 March 2021; pp. 94–109. [Google Scholar]
 Kordzadeh, N.; Ghasemaghaei, M. Algorithmic bias: Review, synthesis, and future research directions. Eur. J. Inf. Syst. 2022, 31, 388–409. [Google Scholar] [CrossRef]
 Boratto, L.; Fenu, G.; Marras, M. Connecting user and item perspectives in popularity debiasing for collaborative recommendation. Inf. Process. Manag. 2021, 58, 102387. [Google Scholar] [CrossRef]
 Gao, C.; Lei, W.; Chen, J.; Wang, S.; He, X.; Li, S.; Li, B.; Zhang, Y.; Jiang, P. CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender System. arXiv 2022, arXiv:2204.01266. [Google Scholar]
 Wang, Y.; Ma, W.; Zhang, M.; Liu, Y.; Ma, S. A survey on the fairness of recommender systems. ACM Trans. Inf. Syst. 2022, 41, 52. [Google Scholar] [CrossRef]
 Yalcin, E.; Bilge, A. Evaluating unfairness of popularity bias in recommender systems: A comprehensive usercentric analysis. Inf. Process. Manag. 2022, 59, 103100. [Google Scholar] [CrossRef]
 Ahanger, A.B.; Aalam, S.W.; Bhat, M.R.; Assad, A. Popularity Bias in Recommender SystemsA Review. In Proceedings of the International Conference on Emerging Technologies in Computer Engineering, Jaipur, India, 4–5 February 2022; pp. 431–444. [Google Scholar]
 Tran, T.N.T.; Felfernig, A.; Tintarev, N. Humanized recommender systems: Stateoftheart and research issues. ACM Trans. Interact. Intell. Syst. 2021, 11, 9. [Google Scholar] [CrossRef]
 Steck, H.; Baltrunas, L.; Elahi, E.; Liang, D.; Raimond, Y.; Basilico, J. Deep learning for recommender systems: A Netflix case study. AI Mag. 2021, 42, 7–18. [Google Scholar] [CrossRef]
 Khan, Z.Y.; Niu, Z.; Sandiwarno, S.; Prince, R. Deep learning techniques for rating prediction: A survey of the stateoftheart. Artif. Intell. Rev. 2021, 54, 95–135. [Google Scholar] [CrossRef]
 Mu, R. A survey of recommender systems based on deep learning. IEEE Access 2018, 6, 69009–69022. [Google Scholar] [CrossRef]
 Chizari, N.; Shoeibi, N.; MorenoGarcía, M.N. A Comparative Analysis of Bias Amplification in Graph Neural Network Approaches for Recommender Systems. Electronics 2022, 11, 3301. [Google Scholar] [CrossRef]
 Dai, E.; Wang, S. Say no to the discrimination: Learning fair graph neural networks with limited sensitive attribute information. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, Virtual Event, Israel, 8–12 March 2021; pp. 680–688. [Google Scholar]
 Wu, S.; Sun, F.; Zhang, W.; Xie, X.; Cui, B. Graph neural networks in recommender systems: A survey. ACM Comput. Surv. (CSUR) 2022, 55, 97. [Google Scholar] [CrossRef]
 Zhou, J.; Cui, G.; Hu, S.; Zhang, Z.; Yang, C.; Liu, Z.; Wang, L.; Li, C.; Sun, M. Graph neural networks: A review of methods and applications. AI Open 2020, 1, 57–81. [Google Scholar] [CrossRef]
 Zhang, Q.; Wipf, D.; Gan, Q.; Song, L. A biased graph neural network sampler with nearoptimal regret. Adv. Neural Inf. Process. Syst. 2021, 34, 8833–8844. [Google Scholar]
 Alelyani, S. Detection and Evaluation of Machine Learning Bias. Appl. Sci. 2021, 11, 6271. [Google Scholar] [CrossRef]
 Zeng, Z.; Islam, R.; Keya, K.N.; Foulds, J.; Song, Y.; Pan, S. Fair representation learning for heterogeneous information networks. In Proceedings of the International AAAI Conference on Weblogs and Social Media, Atlanta, GA, USA, 6–9 June 2022; Volume 15. [Google Scholar]
 Bruce, P.; Bruce, A.; Gedeck, P. Practical Statistics for Data Scientists, 2nd ed.; O’Reilly Media, Inc.: Sebastopol, CA, USA, 2020; pp. 50–51. [Google Scholar]
 Mehrabi, N.; Morstatter, F.; Saxena, N.; Lerman, K.; Galstyan, A. A survey on bias and fairness in machine learning. ACM Comput. Surv. (CSUR) 2021. [Google Scholar] [CrossRef]
 Bernhardt, M.; Jones, C.; Glocker, B. Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms. Nat. Med. 2022, 28, 1157–1158. [Google Scholar] [CrossRef] [PubMed]
 Hall, M.; van der Maaten, L.; Gustafson, L.; Adcock, A. A Systematic Study of Bias Amplification. arXiv 2022, arXiv:2201.11706. [Google Scholar]
 Gu, J.; Oelke, D. Understanding bias in machine learning. arXiv 2019, arXiv:1909.01866. [Google Scholar]
 Blanzeisky, W.; Cunningham, P. Algorithmic factors influencing bias in machine learning. In Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Bilbao, Spain, 13–17 September 2021; pp. 559–574. [Google Scholar]
 Akter, S.; Dwivedi, Y.K.; Sajib, S.; Biswas, K.; Bandara, R.J.; Michael, K. Algorithmic bias in machine learningbased marketing models. J. Bus. Res. 2022, 144, 201–216. [Google Scholar] [CrossRef]
 Caton, S.; Haas, C. Fairness in machine learning: A survey. arXiv 2020, arXiv:2010.04053. [Google Scholar]
 Verma, S.; Rubin, J. Fairness definitions explained. In Proceedings of the 2018 IEEE/ACM International Workshop on Software Fairness (Fairware), Gothenburg, Sweden, 29 May 2018; pp. 1–7. [Google Scholar]
 Feldman, M.; Friedler, S.A.; Moeller, J.; Scheidegger, C.; Venkatasubramanian, S. Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, Australia, 10–13 August 2015; pp. 259–268. [Google Scholar]
 Hardt, M.; Price, E.; Srebro, N. Equality of opportunity in supervised learning. arXiv 2016, arXiv:1610.02413. [Google Scholar]
 Seymour, W. Detecting Bias: Does an Algorithm Have to Be Transparent in Order to Be Fair? BIAS 2018 2018, 4, 543–555. [Google Scholar]
 Ashokan, A.; Haas, C. Fairness metrics and bias mitigation strategies for rating predictions. Inf. Process. Manag. 2021, 58, 102646. [Google Scholar] [CrossRef]
 Dong, Y.; Wang, S.; Wang, Y.; Derr, T.; Li, J. On Structural Explanation of Bias in Graph Neural Networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, 14–18 August 2022; pp. 316–326. [Google Scholar]
 Wu, Z.; Pan, S.; Chen, F.; Long, G.; Zhang, C.; Philip, S.Y. A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 2020, 32, 4–24. [Google Scholar] [CrossRef] [Green Version]
 Dong, Y.; Liu, N.; Jalaian, B.; Li, J. Edits: Modeling and mitigating data bias for graph neural networks. In Proceedings of the ACM Web Conference 2022, Lyon, France, 25–29 April 2022; pp. 1259–1269. [Google Scholar]
 Chen, Z.; Xiao, T.; Kuang, K. BAGNN: On Learning BiasAware Graph Neural Network. In Proceedings of the 2022 IEEE 38th International Conference on Data Engineering (ICDE), Virtual Event. 9–12 May 2022; pp. 3012–3024. [Google Scholar]
 Xu, B.; Shen, H.; Sun, B.; An, R.; Cao, Q.; Cheng, X. Towards consumer loan fraud detection: Graph neural networks with roleconstrained conditional random field. In Proceedings of the AAAI Conference on Artificial Intelligence, Virtual Event. 2–9 February 2021; Volume 35, pp. 4537–4545. [Google Scholar]
 Tang, X.; Yao, H.; Sun, Y.; Wang, Y.; Tang, J.; Aggarwal, C.; Mitra, P.; Wang, S. Graph convolutional networks against degreerelated biases. arXiv 2020, arXiv:2006.15643. [Google Scholar]
 Li, P.; Wang, Y.; Zhao, H.; Hong, P.; Liu, H. On dyadic fairness: Exploring and mitigating bias in graph connections. In Proceedings of the International Conference on Learning Representations, Virtual Event, Austria. 3–7 May 2021. [Google Scholar]
 Loveland, D.; Pan, J.; Bhathena, A.F.; Lu, Y. FairEdit: Preserving Fairness in Graph Neural Networks through Greedy Graph Editing. arXiv 2022, arXiv:2201.03681. [Google Scholar]
 Kose, O.D.; Shen, Y. FairNorm: Fair and Fast Graph Neural Network Training. arXiv 2022, arXiv:2205.09977. [Google Scholar]
 BaezaYates, R. Data and algorithmic bias in the web. In Proceedings of the 8th ACM Conference on Web Science, Hannover, Germany, 22–25 May 2016. [Google Scholar]
 Sun, W.; Khenissi, S.; Nasraoui, O.; Shafto, P. Debiasing the humanrecommender system feedback loop in collaborative filtering. In Proceedings of the 2019 World Wide Web Conference, San Francisco, CA, USA, 13–17 May 2019; pp. 645–651. [Google Scholar]
 Fabbri, F.; Croci, M.L.; Bonchi, F.; Castillo, C. Exposure Inequality in People Recommender Systems: The LongTerm Effects. In Proceedings of the International AAAI Conference on Web and Social Media, Atlanta, GA, USA, 6–9 June 2022; Volume 16, pp. 194–204. [Google Scholar]
 Mansoury, M.; Abdollahpouri, H.; Pechenizkiy, M.; Mobasher, B.; Burke, R. A graphbased approach for mitigating multisided exposure bias in recommender systems. ACM Trans. Inf. Syst. (TOIS) 2021, 40. [Google Scholar] [CrossRef]
 Ovaisi, Z.; Heinecke, S.; Li, J.; Zhang, Y.; Zheleva, E.; Xiong, C. RGRecSys: A Toolkit for Robustness Evaluation of Recommender Systems. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, Tempe, AZ, USA, 21–25 February 2022; pp. 1597–1600. [Google Scholar]
 Abdollahpouri, H.; Burke, R.; Mobasher, B. Managing popularity bias in recommender systems with personalized reranking. In Proceedings of the ThirtySecond International Flairs Conference, Sarasota, FL, USA, 19–22 May 2019. [Google Scholar]
 Wang, S.; Hu, L.; Wang, Y.; He, X.; Sheng, Q.Z.; Orgun, M.A.; Cao, L.; Ricci, F.; Yu, P.S. Graph learning based recommender systems: A review. arXiv 2021, arXiv:2105.06339. [Google Scholar]
 Boratto, L.; Fenu, G.; Marras, M.; Medda, G. Consumer fairness in recommender systems: Contextualizing definitions and mitigations. In Proceedings of the European Conference on Information Retrieval, Stavanger, Norway, 10–14 April 2022; pp. 552–566. [Google Scholar]
 Liu, H.; Wang, Y.; Lin, H.; Xu, B.; Zhao, N. Mitigating sensitive data exposure with adversarial learning for fairness recommendation systems. Neural Comput. Appl. 2022, 34, 18097–18111. [Google Scholar] [CrossRef]
 Shakespeare, D.; Porcaro, L.; Gómez, E.; Castillo, C. Exploring artist gender bias in music recommendation. arXiv 2020, arXiv:2009.01715. [Google Scholar]
 Saxena, S.; Jain, S. Exploring and Mitigating Gender Bias in Recommender Systems with Explicit Feedback. arXiv 2021, arXiv:2112.02530. [Google Scholar]
 Neophytou, N.; Mitra, B.; Stinson, C. Revisiting popularity and demographic biases in recommender evaluation and effectiveness. In Proceedings of the European Conference on Information Retrieval, Stavanger, Norway, 10–14 April 2022; pp. 641–654. [Google Scholar]
 Gómez, E.; Boratto, L.; Salamó, M. Provider fairness across continents in collaborative recommender systems. Inf. Process. Manag. 2022, 59, 102719. [Google Scholar] [CrossRef]
 Rahmani, H.A.; Naghiaei, M.; Dehghan, M.; Aliannejadi, M. Experiments on Generalizability of UserOriented Fairness in Recommender Systems. arXiv 2022, arXiv:2205.08289. [Google Scholar]
 Fang, M.; Liu, J.; Momma, M.; Sun, Y. FairRoad: Achieving Fairness for Recommender Systems with Optimized Antidote Data. In Proceedings of the 27th ACM on Symposium on Access Control Models and Technologies, New York, NY, USA, 8–10 June 2022; pp. 173–184. [Google Scholar]
 Naghiaei, M.; Rahmani, H.A.; Deldjoo, Y. Cpfair: Personalized consumer and producer fairness reranking for recommender systems. arXiv 2022, arXiv:2204.08085. [Google Scholar]
 Li, Y.; Hedia, M.L.; Ma, W.; Lu, H.; Zhang, M.; Liu, Y.; Ma, S. Contextualized Fairness for Recommender Systems in Premium Scenarios. Big Data Res. 2022, 27, 100300. [Google Scholar] [CrossRef]
 Rahman, T.; Surma, B.; Backes, M.; Zhang, Y. Fairwalk: Towards fair graph embedding. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, Honolulu, HI, USA, 1–7 January 2019. [Google Scholar]
 Wu, L.; Chen, L.; Shao, P.; Hong, R.; Wang, X.; Wang, M. Learning fair representations for recommendation: A graphbased perspective. In Proceedings of the Web Conference 2021, Virtual. 19–23 April 2021; 2021; pp. 2198–2208. [Google Scholar]
 Chen, J.; Wu, W.; Shi, L.; Zheng, W.; He, L. Longtail sessionbased recommendation from calibration. Appl. Intell. 2022, 53, 4685–4702. [Google Scholar] [CrossRef]
 Kim, M.; Oh, J.; Do, J.; Lee, S. Debiasing Neighbor Aggregation for Graph Neural Network in Recommender Systems. arXiv 2022, arXiv:2208.08847. [Google Scholar]
 Zhao, M.; Wu, L.; Liang, Y.; Chen, L.; Zhang, J.; Deng, Q.; Wang, K.; Shen, X.; Lv, T.; Wu, R. Investigating AccuracyNovelty Performance for Graphbased Collaborative Filtering. arXiv 2022, arXiv:2204.12326. [Google Scholar]
 Yang, L.; Liu, Z.; Dou, Y.; Ma, J.; Yu, P.S. Consisrec: Enhancing gnn for social recommendation via consistent neighbor aggregation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Online, 11–15 July 2021; pp. 2141–2145. [Google Scholar]
 Liu, Z.; Wan, M.; Guo, S.; Achan, K.; Yu, P.S. Basconv: Aggregating heterogeneous interactions for basket recommendation with graph convolutional neural network. In Proceedings of the 2020 SIAM International Conference on Data Mining, Cincinnati, OH, USA, 7–9 May 2020; pp. 64–72. [Google Scholar]
 Harper, M.; Konstan, J.A. The MovieLens Datasets: Distributed by GroupLens at the University of Minnesota. 2021. Available online: https://grouplens.org/datasets/movielens/ (accessed on 14 February 2023).
 Floridi, L.; Holweg, M.; Taddeo, M.; Amaya Silva, J.; Mökander, J.; Wen, Y. capAIA Procedure for Conducting Conformity Assessment of AI Systems in Line with the EU Artificial Intelligence Act. University of Oxford. 2022. Available online: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4064091 (accessed on 14 February 2023).
 Celma, O. Music Recommendation and Discovery in the Long Tail; Springer: Berlin, Germany, 2010. [Google Scholar]
 Mobius, A. Book Recommendation Dataset. 2020. Available online: https://www.kaggle.com/datasets/arashnic/bookrecommendationdataset (accessed on 14 February 2023).
 AlGhamdi, M.; Elazhary, H.; Mojahed, A. Evaluation of Collaborative Filtering for Recommender Systems. Int. J. Adv. Comput. Sci. Appl. 2021, 12, 5–53. [Google Scholar] [CrossRef]
 Airen, S.; Agrawal, J. Movie recommender system using knearest neighbors variants. Natl. Acad. Sci. Lett. 2022, 45, 75–82. [Google Scholar] [CrossRef]
 Deshpande, M.; Karypis, G. Itembased topn recommendation algorithms. ACM Trans. Inf. Syst. (TOIS) 2004, 22, 143–177. [Google Scholar] [CrossRef]
 Bahadorpour, M.; Neysiani, B.S.; Shahraki, M.N. Determining optimal number of neighbors in itembased kNN collaborative filtering algorithm for learning preferences of new users. J. Telecommun. Electron. Comput. Eng. (JTEC) 2017, 9, 163–167. [Google Scholar]
 Sang, L.; Xu, M.; Qian, S.; Wu, X. Knowledge graph enhanced neural collaborative filtering with residual recurrent network. Neurocomputing 2021, 454, 417–429. [Google Scholar] [CrossRef]
 Girsang, A.S.; Wibowo, A.; Jason; Roslynlia. Neural collaborative for music recommendation system. In IOP Conference Series: Materials Science and Engineering; IOP Publishing: Bristol, UK, 2021; Volume 1071, p. 012021. [Google Scholar]
 Bai, T.; Wen, J.R.; Zhang, J.; Zhao, W.X. A neural collaborative filtering model with interactionbased neighborhood. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, Singapore, 6–10 November 2017; pp. 1979–1982. [Google Scholar]
 Kuang, H.; Xia, W.; Ma, X.; Liu, X. Deep matrix factorization for crossdomain recommendation. In Proceedings of the 2021 IEEE 5th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chongqing, China, 13–14 March 2021; Volume 5, pp. 2171–2175. [Google Scholar]
 Himabindu, T.V.; Padmanabhan, V.; Pujari, A.K. Conformal matrix factorization based recommender system. Inf. Sci. 2018, 467, 685–707. [Google Scholar] [CrossRef]
 Xue, H.J.; Dai, X.; Zhang, J.; Huang, S.; Chen, J. Deep matrix factorization models for recommender systems. In Proceedings of the IJCAI, Melbourne, Australia, 19–25 August 2017; Volume 17, pp. 3203–3209. [Google Scholar]
 Yi, B.; Shen, X.; Liu, H.; Zhang, Z.; Zhang, W.; Liu, S.; Xiong, N. Deep matrix factorization with implicit feedback embedding for recommendation system. IEEE Trans. Ind. Inform. 2019, 15, 4591–4601. [Google Scholar] [CrossRef]
 Liang, G.; Sun, C.; Zhou, J.; Luo, F.; Wen, J.; Li, X. A General Matrix Factorization Framework for Recommender Systems in Multiaccess Edge Computing Network. Mobile Netw. Appl. 2022, 27, 1629–1641. [Google Scholar] [CrossRef]
 Zhang, Z.; Liu, Y.; Xu, G.; Luo, G. Recommendation using DMFbased fine tuning method. J. Intell. Inf. Syst. 2016, 47, 233–246. [Google Scholar] [CrossRef]
 Schafer, J.B.; Frankowski, D.; Herlocker, J.; Sen, S. Collaborative filtering recommender systems. In The Adaptive Web; Springer: Berlin, Germany, 2007; pp. 291–324. [Google Scholar]
 He, X.; Deng, K.; Wang, X.; Li, Y.; Zhang, Y.; Wang, M. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event. 25–30 July 2020; pp. 639–648. [Google Scholar]
 Broman, N. Comparasion of Recommender Systems for Stock Inspiration. Bachelor’s Thesis, Stockholm School of Economics, Stockholm, Sweden, 2021. [Google Scholar]
 Ding, S.; Feng, F.; He, X.; Liao, Y.; Shi, J.; Zhang, Y. Causal incremental graph convolution for recommender system retraining. arXiv 2022, arXiv:2108.06889. [Google Scholar] [CrossRef] [PubMed]
 Sun, W.; Chang, K.; Zhang, L.; Meng, K. INGCF: An Improved Recommendation Algorithm Based on NGCF. In Proceedings of the International Conference on Algorithms and Architectures for Parallel Processing, Xiamen, China, 3–5 December 2021; pp. 116–129. [Google Scholar]
 Wang, X.; He, X.; Wang, M.; Feng, F.; Chua, T.S. Neural graph collaborative filtering. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France, 21–25 July 2019; pp. 165–174. [Google Scholar]
 Wu, J.; Wang, X.; Feng, F.; He, X.; Chen, L.; Lian, J.; Xie, X. Selfsupervised graph learning for recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Online. 11–15 July 2021; pp. 726–735. [Google Scholar]
 Yang, C. Supervised Contrastive Learning for Recommendation. arXiv 2022, arXiv:2201.03144. [Google Scholar] [CrossRef]
 Tang, H.; Zhao, G.; Wu, Y.; Qian, X. Multisamplebased Contrastive Loss for Topk Recommendation. IEEE Trans. Multimed. 2021, 25, 339–351. [Google Scholar] [CrossRef]
 Wang, X.; Jin, H.; Zhang, A.; He, X.; Xu, T.; Chua, T.S. Disentangled graph collaborative filtering. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, 25–30 July 2020; pp. 1001–1010. [Google Scholar]
 Bourhim, S.; Benhiba, L.; Idrissi, M.J. A CommunityDriven Deep Collaborative Approach for Recommender systems. In Proceedings of the 2019 IEEE International Conference on Web Services (ICWS), Milan, Italy, 8–13 July 2019. [Google Scholar]
 Sha, X.; Sun, Z.; Zhang, J. Disentangling MultiFacet Social Relations for Recommendation. IEEE Trans. Comput. Soc. Syst. 2021, 9, 867–878. [Google Scholar] [CrossRef]
 Foulds, J.R.; Islam, R.; Keya, K.N.; Pan, S. Differential Fairness. NeurIPS 2019 Workshop on Machine Learning with Guarantees, Vancouver, Canada. UMBC Faculty Collection. 2019. Available online: https://www.semanticscholar.org/paper/DifferentialFairnessFouldsIslam/cf3081d5fa83750a89898ae1adcef7925ed8af81 (accessed on 14 February 2023).
 Foulds, J.R.; Islam, R.; Keya, K.N.; Pan, S. An intersectional definition of fairness. In Proceedings of the 2020 IEEE 36th International Conference on Data Engineering (ICDE), Dallas, TX, USA, 20–24 April 2020; pp. 1918–1921. [Google Scholar]
 Yao, S.; Huang, B. Beyond parity: Fairness objectives for collaborative filtering. arXiv 2017, arXiv:1705.08804. [Google Scholar]
 Farnadi, G.; Kouki, P.; Thompson, S.K.; Srinivasan, S.; Getoor, L. A fairnessaware hybrid recommender system. arXiv 2018, arXiv:1809.09030. [Google Scholar]
 Naghiaei, M.; Rahmani, H.A.; Dehghan, M. The Unfairness of Popularity Bias in the Book Recommendation. arXiv 2022, arXiv:2202.13446. [Google Scholar]
 Lazovich, T.; Belli, L.; Gonzales, A.; Bower, A.; Tantipongpipat, U.; Lum, K.; Huszar, F.; Chowdhury, R. Measuring disparate outcomes of content recommendation algorithms with distributional inequality metrics. arXiv 2022, arXiv:2202.01615. [Google Scholar] [CrossRef]
 Wang, X.; Wang, W.H. Providing Itemside Individual Fairness for Deep Recommender Systems. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, Seoul, Republic of Korea, 21–24 June 2022; pp. 117–127. [Google Scholar]
 Islam, R.; Keya, K.N.; Zeng, Z.; Pan, S.; Foulds, J. Debiasing career recommendations with neural fair collaborative filtering. In Proceedings of the Web Conference 2021, Ljubljana, Slovenia, 19–23 April 2021; pp. 3779–3790. [Google Scholar]
 Aalam, S.W.; Ahanger, A.B.; Bhat, M.R.; Assad, A. Evaluation of Fairness in Recommender Systems: A Review. In Proceedings of the International Conference on Emerging Technologies in Computer Engineering, Jaipur, India, 4–5 February 2022; pp. 456–465. [Google Scholar]
Area of Research  Focus  Publications 

Bias and fairness in ML  Understanding, detection, and evaluation of bias and/or fairness in ML  [26,29,30,31,32,33,34,35,38,40] 
Fairness in information networks  [27]  
Bias management in ML  [10]  
Bias and fairness in GNNs  Understanding, detection, and evaluation of bias and/or fairness in GNNs  [25,41,43,45] 
Bias and/or fairness mitigation in GNNs  [46,47,48,49]  
Bias and fairness in RS  Understanding, detection, and evaluation of bias and/or fairness in RS  [5,16,50,52] 
Bias and/or fairness mitigation in RS  [4,7,51,54,55,60,61,62,63,64,65,66,69]  
Bias and fairness in GNNbased RS  Understanding, detection, and evaluation of bias and/or fairness in GNNbased RS  [18,20,21,23,56,57] 
Bias and/or fairness mitigation in GNNbased RS  [22,53,58,67,68,70,71,72,73] 
Dataset  Features  Description  Data Type 

Age  Age of users  int  
Rating  Rating on movies provided by users  float  
User id  IDs of the users  int  
MovieLens  Movie id  IDds of the movies  int 
Gender  Gender of users  String  
Occupation  Users’ job  String  
Movie title  The title of rated movies  String  
Weight  Listening count per artist and user  float  
Age  Age of users  float  
User id  IDs of the users  int  
LastFM  Item id  IDs of the artists  int 
Gender  Gender of users  String  
Country  Users’ territory of living  String  
Name  Name of the artists  String  
Rating  User score on books  float  
Age  Age of users  float  
Book Recommendation  User id  IDs of the users  int 
Item id  IDs of the books  int  
Location  Users’ location  String 
Data Class  Data Subclass  Representing Graph 

General interaction  Explicit interaction, implicit interaction  Weighted bipartite graph unweighted bipartite graph 
Sequential interaction  Singletype interactions Multitype interactions  Directed homogeneous graph Directed heterogeneous graph 
Side information  Attribute information, social information, external knowledge  Heterogeneous graph homogeneous graph tree or heterogeneous graph 
Notation  Definition 

U  A set of users 
I  A set of items 
u  A user 
i, j  An item 
$R(u)$  A groundtruth set of items that user u interacted with 
$\widehat{R}(u)$  A ranked list of items that a model produces 
K  The length of the recommendation list 
$M(x)$  Algorithmic mechanism for RS with input x and output y 
$\theta $  Distribution which generates x 
$\Theta $  A set of distributions of $\theta $ which generate each instance x 
${g}_{i}$  a variable indicating which group the ith user belongs to 
${E}_{g}{\left[y\right]}_{j}$  The average predicted score for the jth item from disadvantaged users 
${E}_{\neg g}{\left[y\right]}_{j}$  The average predicted score for the jth item from advantaged users 
${E}_{g}{\left[r\right]}_{j}$  The average ratings for the disadvantaged users 
${E}_{\neg g}{\left[r\right]}_{j}$  The average ratings for the advantaged users 
Metric Name  Description 

Mean Reciprocal Rank (MRR)  It is calculated for the first relevant element found in the topK item list. Let $Ran{k}_{u}^{*}$ be the position of that element in the list provided by a given algorithm for the user u. 
$$MRR@K=\frac{1}{U}\sum _{u\in U}\frac{1}{Ran{k}_{u}^{*}}$$
 
Normalized Discounted Cumulative Gain (NDCG)  Discounted cumulative gain (DCG) is a metric applicable to relevant items in the list, where the graded relevance of the items is penalized logarithmically as their position descends in the list. The cumulative gain is calculated up to a rank K. NDCG is the ratio between DCG and the maximum possible DCG. $\delta (0)$ is an indicator function. 
$$NDCG@K=\frac{1}{U}\sum _{u\in U}\frac{1}{{\sum}_{i=1}^{min(R(u),K)}\frac{1}{lo{g}_{2}(i+1)}}\sum _{i=1}^{K}\delta (i\in R(u))\frac{1}{{log}_{2}(i+1)}$$
 
Precision  A wellknown measure for ranked lists, which represents the fraction of relevant items out of all the recommended items. Usually expressed as the average of the metric values for each user. $\widehat{R}(u)$ represents the item count of $\widehat{R}(u)$ 
$$Precision@K=\frac{1}{U}\sum _{u\in U}\frac{\widehat{R}(u)\cap R(u)}{\widehat{R}(u)}$$
 
Recall  It is a measure similar to precision, but in this case, its value is the ratio between relevant items in the topK recommendation list and all relevant items. $R(u)$ represents the item count of $R(u)$ 
$$Recall@K=\frac{1}{U}\sum _{u\in U}\frac{\widehat{R}(u)\cap R(u)}{R(u)}$$
 
Hit Ratio (HR)  This metric evaluates how many ‘hits’ were included in a topK item list. A hit is an item that appears in the groundtruth set. $\delta (0)$ is an indicator function. $\delta (b)=1$ if b is true, otherwise it would be 0. ∅ denotes the empty set. 
$$HR@K=\frac{1}{U}\sum _{u\in U}\delta (\widehat{R}(u)\cap R(u)\ne \varnothing )$$

Approach  Method  Top K  Recall  Precision  MRR  NDCG  HIT  Item Coverage  Gini Index  Average Popularity 

CF  ItemKNN  K = 5  0.15  0.23  0.44  0.28  0.63  0.19  0.93  231.96 
CF  ItemKNN  K = 10  0.22  0.18  0.46  0.27  0.75  0.24  0.93  249.74 
CF  ItemKNN  K = 15  0.31  0.16  0.46  0.29  0.84  0.29  0.89  208.12 
CF  NNCF  K = 5  0.15  0.24  0.47  0.29  0.64  0.17  0.95  284.47 
CF  NNCF  K = 10  0.24  0.19  0.46  0.22  0.78  0.25  0.91  217.70 
CF  NNCF  K = 15  0.28  0.15  0.47  0.27  0.81  0.30  0.91  231.28 
MF  DMF  K = 5  0.14  0.22  0.43  0.26  0.62  0.18  0.94  256.29 
MF  DMF  K = 10  0.21  0.17  0.42  0.25  0.73  0.20  0.93  252.25 
MF  DMF  K = 15  0.29  0.16  0.45  0.28  0.83  0.28  0.90  219.49 
MF  NeuMF  K = 5  0.15  0.23  0.45  0.27  0.65  0.25  0.91  228.52 
MF  NeuMF  K = 10  0.23  0.18  0.46  0.27  0.78  0.36  0.89  212.41 
MF  NeuMF  K = 15  0.30  0.16  0.46  0.28  0.83  0.40  0.86  196.89 
GNN  NGCF  K = 5  0.15  0.24  0.48  0.29  0.66  0.15  0.95  277.85 
GNN  NGCF  K = 10  0.25  0.20  0.49  0.30  0.77  0.25  0.93  255.49 
GNN  NGCF  K = 15  0.32  0.17  0.49  0.31  0.86  0.32  0.89  219.13 
GNN  LightGCN  K = 5  0.11  0.17  0.36  0.21  0.55  0.05  0.98  245.13 
GNN  LightGCN  K = 10  0.18  0.14  0.37  0.21  0.67  0.07  0.97  312.47 
GNN  LightGCN  K = 15  0.23  0.12  0.38  0.21  0.76  0.10  0.96  292.8 
GNN  SGL  K = 5  0.15  0.25  0.47  0.29  0.66  0.24  0.91  229.24 
GNN  SGL  K = 10  0.25  0.20  0.49  0.29  0.80  0.31  0.89  209.39 
GNN  SGL  K = 15  0.31  0.17  0.49  0.30  0.85  0.34  0.88  200.63 
GNN  DGCF  K = 5  0.12  0.15  0.30  0.18  0.55  0.27  0.94  269.46 
GNN  DGCF  K = 10  0.22  0.14  0.40  0.23  0.74  0.17  0.96  278.14 
GNN  DGCF  K = 15  0.26  0.11  0.36  0.22  0.82  0.46  0.90  232.08 
Approach  Method  Top K  Recall  Precision  MRR  NDCG  HIT  Item Coverage  Gini Index  Average Popularity 

CF  ItemKNN  K = 5  0.10  0.06  0.13  0.17  0.30  0.10  0.94  25.90 
CF  ItemKNN  K = 10  0.14  0.04  0.12  0.19  0.45  0.17  0.90  19.19 
CF  ItemKNN  K = 15  0.21  0.03  0.46  0.29  0.84  0.23  0.85  15.33 
CF  NNCF  K = 5  0.10  0.09  0.29  0.33  0.45  0.10  0.94  39.79 
CF  NNCF  K = 10  0.15  0.03  0.30  0.37  0.57  0.17  0.90  25.46 
CF  NNCF  K = 15  0.17  0.03  0.47  0.38  0.64  0.23  0.85  18.91 
MF  DMF  K = 5  0.12  0.09  0.29  0.33  0.46  0.10  0.94  40.21 
MF  DMF  K = 10  0.16  0.05  0.31  0.37  0.58  0.17  0.90  20.09 
MF  DMF  K = 15  0.17  0.04  0.45  0.39  0.65  0.23  0.85  18.10 
MF  NeuMF  K = 5  0.11  0.09  0.29  0.33  0.46  0.10  0.94  39.89 
MF  NeuMF  K = 10  0.17  0.05  0.30  0.37  0.58  0.17  0.90  25.49 
MF  NeuMF  K = 15  0.20  0.04  0.46  0.39  0.64  0.23  0.85  18.77 
GNN  NGCF  K = 5  0.08  0.05  0.16  0.19  0.28  0.77  0.66  22.23 
GNN  NGCF  K = 10  0.11  0.03  0.17  0.22  0.37  0.95  0.56  14.87 
GNN  NGCF  K = 15  0.14  0.02  0.49  0.23  0.42  0.99  0.48  11.44 
GNN  LightGCN  K = 5  0.09  0.05  0.15  0.18  0.25  0.33  0.85  22.18 
GNN  LightGCN  K = 10  0.12  0.03  0.16  0.19  0.30  0.70  0.97  13.66 
GNN  LightGCN  K = 15  0.17  0.02  0.16  0.20  0.35  0.89  0.53  10.04 
GNN  SGL  K = 5  0.07  0.03  0.09  0.11  0.16  0.83  0.49  13.16 
GNN  SGL  K = 10  0.09  0.02  0.49  0.13  0.28  0.31  0.88  11.46 
GNN  SGL  K = 15  0.13  0.03  0.49  0.15  0.34  0.91  0.40  9.56 
GNN  DGCF  K = 5  0.07  0.03  0.13  0.14  0.19  0.22  0.83  18.64 
GNN  DGCF  K = 10  0.11  0.01  0.08  0.12  0.20  0.45  0.65  9.16 
GNN  DGCF  K = 15  0.13  0.01  0.13  0.16  0.27  0.59  0.57  8.12 
Approach  Method  Top K  Recall  Precision  MRR  NDCG  HIT  Item Coverage  Gini Index  Average Popularity 

CF  ItemKNN  K =5  0.19  0.01  0.05  0.08  0.19  0.13  0.93  2.44 
CF  ItemKNN  K = 10  0.19  0.01  0.05  0.08  0.2  0.21  0.89  2.23 
CF  ItemKNN  K = 15  0.17  0.03  0.45  0.39  0.65  0.23  0.85  18.1 
CF  NNCF  K = 5  0.15  0.03  0.08  0.1  0.16  0.08  0.95  5.08 
CF  NNCF  K = 10  0.22  0.02  0.09  0.12  0.24  0.15  0.9  4.04 
CF  NNCF  K = 15  0.26  0.04  0.1  0.13  0.29  0.21  0.87  3.4 
MF  DMF  K = 5  0.17  0.03  0.12  0.12  0.18  0.08  0.95  5.85 
MF  DMF  K = 10  0.19  0.02  0.13  0.13  0.22  0.17  0.91  4.12 
MF  DMF  K = 15  0.21  0.03  0.13  0.14  0.24  0.19  0.87  3.26 
MF  NeuMF  K = 5  0.18  0.03  0.13  0.14  0.2  0.08  0.95  6.13 
MF  NeuMF  K = 10  0.24  0.02  0.13  0.15  0.26  0.14  0.9  4.35 
MF  NeuMF  K = 15  0.27  0.04  0.14  0.16  0.29  0.2  0.86  3.49 
GNN  NGCF  K = 5  0.04  0.01  0.02  0.02  0.04  0.34  0.71  1.28 
GNN  NGCF  K =10  0.08  0.01  0.02  0.03  0.09  0.57  0.56  1.28 
GNN  NGCF  K = 15  0.13  0.02  0.03  0.05  0.12  0.72  0.48  1.27 
GNN  LightGCN  K = 5  0.04  0.01  0.02  0.02  0.04  0.27  0.79  2.08 
GNN  LightGCN  K = 10  0.05  0.01  0.02  0.02  0.05  0.49  0.64  1.62 
GNN  LightGCN  K = 15  0.08  0.02  0.03  0.04  0.08  0.63  0.54  1.54 
GNN  SGL  K = 5  0.03  0.01  0.01  0.02  0.03  0.27  0.83  1.62 
GNN  SGL  K = 10  0.07  0.02  0.02  0.04  0.11  0.39  0.74  1.63 
GNN  SGL  K = 15  0.1  0.03  0.02  0.11  0.34  0.5  0.68  1.59 
GNN  DGCF  K = 5  0.03  0.01  0.01  0.02  0.04  0.29  0.78  1.57 
GNN  DGCF  K = 10  0.01  0.01  0.02  0.03  0.08  0.51  0.63  1.4 
GNN  DGCF  K = 15  0.09  0.01  0.02  0.04  0.11  0.67  0.53  1.36 
Approach  Method  Avg. DF  VU  AU  NonParity Unfairness 

CF  ItemKNN  3.2702  2.0264  2.0195  1.8335 
CF  NNCF  1.5023  0.4967  0.4523  0.0613 
MF  DMF  2.3341  0.233  0.1685  0.0211 
MF  NeuMF  2.188  0.2113  0.1413  0.0072 
GNN  NGCF  1.5023  0.4967  0.4523  0.0613 
GNN  LightGCN  1.4138  0.2159  0.1851  0.0233 
GNN  SGL  0.6419  0.2152  0.2148  0.0002 
GNN  DGCF  0.6855  0.2116  0.2103  0.0015 
Approach  Method  Avg. DF  VU  AU  NonParity Unfairness 

CF  ItemKNN  0.7052  0.3101  0.3101  0 
CF  NNCF  5.9771  0.1507  0.1491  0.0016 
MF  DMF  6.0327  0.1486  0.1476  0.0032 
MF  NeuMF  5.0174  0.2228  0.222  0.0003 
GNN  NGCF  2.4747  0.351  0.3422  0.002 
GNN  LightGCN  0.6764  0.3094  0.3094  0.0001 
GNN  SGL  0.7263  0.3032  0.3032  0 
GNN  DGCF  0.7287  0.4249  0.4249  0.0004 
Approach  Method  Avg. DF  VU  AU  NonParity Unfairness 

CF  ItemKNN  2.7436  1.6859  1.6808  0.5607 
CF  NNCF  1.9234  0.1935  0.1233  0.5607 
MF  DMF  2.1169  0.2116  0.1434  0.0022 
MF  NeuMF  2.188  0.2113  0.1413  0.0049 
GNN  NGCF  1.2998  0.3895  0.3456  0.032 
GNN  LightGCN  1.0907  0.2051  0.1752  0.003 
GNN  SGL  0.6419  0.2152  0.2148  0.0002 
GNN  DGCF  0.6509  0.2011  0.2008  0 
Approach  Method  Avg. DF  VU  AU  NonParity Unfairness 

CF  ItemKNN  0.8071  0.2865  0.2865  0 
CF  NNCF  6.4780  0.1405  0.1397  0.0138 
MF  DMF  0.7182  0.4294  0.4294  0.0164 
MF  NeuMF  5.4643  0.2033  0.2029  0.0185 
GNN  NGCF  2.8171  0.3309  0.3262  0.0597 
GNN  LightGCN  0.7237  0.2943  0.2943  0.0003 
GNN  SGL  0.7504  0.2903  0.2903  0 
GNN  DGCF  0.6960  0.431  0.4323  0.0004 
Approach  Method  Avg. DF  VU  AU  NonParity Unfairness 

CF  ItemKNN  0.7875  0.4928  0.4927  0.0034 
CF  NNCF  7.2301  0.4794  0.4417  0.0001 
MF  DMF  7.3435  0.4814  0.4426  0.0058 
MF  NeuMF  4.3972  0.5739  0.565  0.0093 
GNN  NGCF  5.1748  0.5609  0.541  0.0004 
GNN  LightGCN  0.8551  0.4965  0.4964  0.0002 
GNN  SGL  0.5654  0.4931  0.4931  0.00015 
GNN  DGCF  0.6408  0.494  0.494  0.0004 
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. 
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Chizari, N.; Tajfar, K.; MorenoGarcía, M.N. Bias Assessment Approaches for Addressing UserCentered Fairness in GNNBased Recommender Systems. Information 2023, 14, 131. https://doi.org/10.3390/info14020131
Chizari N, Tajfar K, MorenoGarcía MN. Bias Assessment Approaches for Addressing UserCentered Fairness in GNNBased Recommender Systems. Information. 2023; 14(2):131. https://doi.org/10.3390/info14020131
Chicago/Turabian StyleChizari, Nikzad, Keywan Tajfar, and María N. MorenoGarcía. 2023. "Bias Assessment Approaches for Addressing UserCentered Fairness in GNNBased Recommender Systems" Information 14, no. 2: 131. https://doi.org/10.3390/info14020131