# Dynamic Asynchronous Anti Poisoning Federated Deep Learning with Blockchain-Based Reputation-Aware Solutions

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

- Another limitation is vulnerability to poisoning attacks. A poisoning attack means that malicious clients can generate anomalous parameters to mislead the model decision. Particularly, recent research works have illustrated that poisoned parameters can mislead the federated learning model on the attacker-chosen poison subtask while working well on the main task [2]. The limitations including inefficiency and vulnerability to poisoning attacks of federated learning significantly reduce federated learning performance, which motivates us to solve these problems. This paper proposes a novel local reliability mutual evaluation mechanism to enhance the security of poisoning attacks, where each parameter is evaluated over the local data of other parties. According to the evaluation scores, the server can adjust the weight proportion of model aggregation. The local reliability mutual evaluation mechanism uses the local reliability to detect poisoned parameters instead of statistical difference analysis enabling the approach to work well in the case of small data samples. The main contributions of this paper are as the following: We propose a dynamic asynchronous anti poisoning federated deep learning framework to pursue both the efficiency and security of defending against poisoning attacks. In particular, the dynamic asynchronous algorithm considering the averaging frequency control and parameter selection for federated learning is proposed to speed up model averaging. The proposed algorithm enables federated learning to adaptively remove the stragglers with low computing power, bad channel conditions.
- A novel local reliability mutual evaluation mechanism is presented to enhance the security of poisoning attacks. The proposed mechanism enables federated learning to detect the anomalous parameter of poisoning attacks and adjust the weight proportion of model aggregation based on the evaluation result.
- The experiment results on three datasets illustrate that our design can reduce the training time by 30% and is robust to representative poisoning attacks significantly compared with other state-of-art methods, confirming the applicability of our scheme.

## 2. Related Work

#### 2.1. Efficient Federated Learning

#### 2.2. Defenses against Poisoning Attack on Federated Learning

## 3. Background

#### 3.1. Federated Deep Learning

#### 3.2. Blockchain Technology

## 4. The RAPFDL Framework

- Reliability evaluation: One of our key insights is delegating anomaly detection tasks to clients which is able to detect anomalous parameters via evaluating the local reliability of model parameter updates with its private dataset. Put simply, the central server sends model parameters updates to the selected clients which evaluate the accuracy performance of the parameter updates with local data according to the predefined rule. The server can adjust the weight proportion of model aggregation based on the matrix of received evaluation scores.
- Blockchain decentralized architecture: Most existing FL frameworks rely on a central server to aggregate model parameter updates of parties involved in FL. Compared with centralized architectures, RAPFDL inherits the blockchain architecture, which enables every party to remain modular when interacting with other parties. Rather than ceding control to central servers, every party maintains full control of private data. In addition, blockchain enables federated learning with the native ability to coordinate the entry and exit of parties automatically, further guaranteeing the auditability of the training process of the FL. Robustness can also benefit from the blockchain because of no single point of failure.
- Local model training: Every party performs local model training independently. After completing the training process, the party generates a contract to trade its local model parameter updates by attaching its local model parameters to the contract.
- Federated model aggregation: Parties of a cooperative group train a deep learning model collaboratively. The model is trained in an iterative manner after deciding on the same deep learning model and parameter initialization. All parties trade their parameter updates, and workers download the contracts to process the parameter updates in each iteration. The processed parameter updates are then sent out via smart contract namely the processing contract of blockchain. The correctly processed parameter updates are used to update the federated global model by the leader selected from workers. Each party downloads the federated model to update its local model accordingly. After that, the next iteration of federated learning begins.

- Upload Process: Once receives download request for local parameter updates, the party can determine the number of local parameter updates to send back according to the download request and its own sharing level.
- Download Process: Since the contribution of each party is different for various parties, the party reliability may be various from the view of different parties. Thus, every party maintains a local reliability list for other parties to record their reliability. The higher the reliability of the party $k$ in the private reliability list of a party, the more likely it is the party will download model parameter updates from it, and thus, more points will be awarded to the party $k$.

## 5. Implementation of RAPFDL

#### 5.1. System Initialization

#### 5.1.1. Initial Evaluation Algorithm

- To gain prior information about the performance of models before federated learning. If the training data of the party is not enough to generate an excellent model, the party will perform poorly in the evaluation algorithm. Thus, other parties shall be more cautious when sharing parameters updates with it, taking into account the evaluation results.
- To gain the preliminary estimate of the data distribution. Only when the data distribution is different but there is some overlap, can two parties mutually benefit. Assume that two parties ${P}_{1}$ and ${P}_{2}$ have released artificial samples with high similarity, which means its local data distributions are nearly the same. Under the circumstances, parameter updates from ${P}_{2}$ are less likely to improve the model accuracy performance of ${P}_{1}$. Thus, ${P}_{1}$ and ${P}_{2}$ shall avoid sharing parameter updates from each other during the subsequent model training. And other parties should not download the model parameter updates from both ${P}_{1}$ and ${P}_{2}$, but from either. Contrarily, assume that two parties have published completely different data distribution artificial samples. Hence, the parameter updates from ${P}_{2}$ have a negligible enhancement effect on the accuracy of the model of ${P}_{1}$. Furthermore, suppose that the data distribution of party ${P}_{1}$ differs from that of all the other parties. Therefore, it is reasonable for other parties to assign low reliability to ${P}_{1}$ and avoid downloading parameter updates from ${P}_{1}$ based on previous assumptions. Algorithm 1 presents the detailed procedures of initial evaluation, including local reliability evaluation, sharing level, and reward points initialization and differentially private data samples generation.

Algorithm 1 Anomalous Model Parameter Detection |

01. Server Executes: |

02. the global model parameter initialization: ${w}_{0}$ |

03. for iteration round $t=1$ to $I$ do |

04. ${C}_{t}\leftarrow $ Randomly choose $N$ clients |

05. for client $n$ in ${C}_{t}$ do |

06. ${L}_{t+1}^{n}\leftarrow ClientUpdate(n,{w}_{t})$ |

07. end for |

08. Client Detection |

09. for each client performing the detection task do |

10. ${r}_{k}\leftarrow $ Return the evaluation results matrix |

11. end for |

12. for $i=1$ to $s$ do |

13. Calculate the penalty coefficient ${f}_{t+1}^{i}$ |

14. end for |

15. ${w}_{t+1}\leftarrow \frac{1}{s}{\displaystyle {\sum}_{i=0}^{s}{f}_{t+1}^{i}{w}_{t+1}^{i}}$ |

16. end for |

#### 5.1.2. Local Reliability Initialization

#### 5.1.3. Sharing Level and Reward Points Initialization

#### 5.2. Differentially Private Data Samples Generation

#### 5.3. Anti Poisoning Privacy-Preserving Federated Learning

Algorithm 2 Anti Poisoning Privacy-Perserving Federated Learning |

01. Input: $C,{r}_{i},{r}_{j},{s}_{i},{\delta}_{j},{w}_{i},\Delta {w}_{i}$. |

02. Output: updated reward points ${r}_{j}^{\prime},{r}_{i}^{\prime}$, parameters ${w}_{i}^{\prime}$, and local reliability ${f}_{i}^{{j}^{\prime}}$. |

03. 1: Trade gradients via sharing level, reward points and local reliability: At every |

04. round, the goal of party $i$ is to download ${s}_{i}={r}_{i}$ model parameter updates from |

05. the other parties, while party $j\in C$ is able to provide about ${\delta}_{j}\ast \left|{w}_{j}\right|$ model |

06. parameter updates, one reward point is spent for every download and rewarded for |

07. every upload. Parties update their model according to the model parameter updates |

08. of party $j\in C$ as the following: |

09. for $j\in C$ do |

10. ${s}_{i}^{j}=\mathrm{min}\left({f}_{i}^{j}\ast {s}_{i},{\delta}_{j}\ast \left|\u25b3{w}_{j}\right|\right),{r}_{j}^{\prime}={r}_{j}+{s}_{i}^{j},{r}_{i}^{\prime}={r}_{i}-{s}_{i}^{j},\Delta {w}_{j}^{i}=\Delta {w}_{j}$, party $j$ |

11. first choose ${s}_{i}^{j}$ meaningful gradients from $\Delta {w}_{j}^{i}$ according to largest values |

12. criterion: sort gradients in $\Delta {w}_{j}^{i}$ and choose top ${s}_{i}^{j}$ of them, and mask the |

13. remaining $\left|\u25b3{w}_{j}^{i}\right|-{s}_{i}^{j}$ model parameter updates with 0 as $\u25b3{\tilde{w}}_{j}^{i}$ |

14. end for |

15. 2: Model parameter update: party $i$ utilizes the secret key $s{k}_{i}$ to decrypt received |

16. encrypted symmetric key as ${f}_{s}k$, and utilizes it to decrypt the encrypted parameter |

17. updates as $c=Enc\left({\tilde{w}}_{j}^{i},{k}_{j}\right)$ at the end decrypts the sum of model paramter updates |

18. via homomorphic encryption and thus local model can be updated via integrating all |

19. the plain paramter updates ${\tilde{w}}_{i}$ as ${w}_{i}^{\prime}={w}_{i}+\Delta {w}_{i}+Dec\left({\sum}_{j\in C\backslash i}Enc\left(\u25b3{\tilde{w}}_{j}^{i},{k}_{j}\right),-{k}_{i}\right)=$ |

20. ${w}_{i}+\u25b3{w}_{i}+{\sum}_{j\in C\backslash i}\u25b3{\tilde{w}}_{j}^{i}$. |

21. 3: Local reliability update: party $i$ publishes ${s}_{i}$ artificial private data samples to |

22. other party $j$ for labeling. Mutual evaluation is utilized to compute the local |

23. reliability of the party $j$ as ${f}_{i}^{{j}^{\prime}}$ at current round. Thus party $i$ updates party $j$ |

24. local reliability via integrating the historical reliability as ${f}_{i}^{{j}^{\prime}}=0.3\ast {f}_{i}^{j}+0.7\ast {f}_{i}^{{j}^{\prime}}$. |

25. 4: Local reliability normalization: ${f}_{i}^{{j}^{\prime}}=\frac{{f}_{i}^{{j}^{\prime}}}{{\displaystyle {\sum}_{j\in C}{f}_{i}^{{j}^{\prime}}}}$ |

26. if ${f}_{i}^{{j}^{\prime}}<{f}_{th}$ then |

27. party $i$ will report party $j$ as the party with low contribution. |

28. end if |

29. 5: Set of reliable party: The reliable party set in blockchain will be reconstructed in |

30. form of removing the low-contribution party reported by the majority of parties. |

#### 5.3.1. Federated Learning Model Training with Homomorphic Encryption

#### 5.3.2. Local Reliability Update

#### 5.4. Dynamic Asynchronous Federated Learning

Algorithm 3 Dynamic Asynchronous Federated Averaging (FedDasync) |

01. Server Process: |

02. Input: $\alpha \in (0,1)$ |

03. Initialize the global model: ${w}_{0},{\alpha}_{t}\leftarrow \alpha ,{\beta}_{k}\leftarrow \frac{{D}_{k}}{D}$ |

04. Scheduler Thread: |

05. Scheduler periodically triggers some training tasks on some clients, and sents them |

06. from the latest global model with time stamp. |

07. Updater Thread: |

08. for each round $t=1,2\dots $ do |

09. $\theta =KMeans(\mathrm{timeList},K)$ |

10. loop for $\theta $ dynamic seconds after receiving update |

11. Receive the pair $\left({x}_{new},\tau \right)$ from any client |

12. timeList.append $(t-\tau )$ |

13. ${\gamma}_{t}\leftarrow \alpha \times S(t-\tau )\times {\beta}_{k},S(\cdot )$ is function of stateness |

14. ${x}_{t}=\left(1-{\gamma}_{t}\right){x}_{t-1}+{\gamma}_{t}{x}_{new}$ |

15. end loop |

16. ${x}_{t}\leftarrow {\displaystyle \sum _{k=1}^{K}\frac{n}{{n}_{k}}}{x}_{t}^{k}$ |

17. end for |

#### 5.5. Quantification of Federated Learning Fairness

## 6. Experimental Evaluation

#### 6.1. Datasets

#### 6.2. Experiment Setup

#### 6.3. Experimental Results

## 7. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Lueth, K. State of the Iot 2020: Number of Iot Devices Now at 7b-Market Accelerating. Available online: https://iot-analytics.com/state-of-the-iot-2020-12-billion-iot-connections-surpassing-non-iot-for-the-first-time/ (accessed on 1 December 2021).
- Chen, M.; Yang, Z.; Saad, W.; Yin, C.; Poor, H.V.; Cui, S. A Joint Learning and Communications Framework for Federated Learning over Wireless Networks. IEEE Trans. Wirel. Commun.
**2021**, 20, 269–283. [Google Scholar] [CrossRef] - Wang, S.; Tuor, T.; Salonidis, T.; Leung, K.K.; Makaya, C.; He, T.; Chan, K. Adaptive federated learning in resource constrained edge computing systems. IEEE J. Sel. Areas Commun.
**2020**, 37, 1205–1221. [Google Scholar] [CrossRef] [Green Version] - Nishio, T.; Yonetani, R. Client selection for federated learning with heterogeneous resources in mobile edge. In Proceedings of the ICC 2019-2019 IEEE International Conference on Communications (ICC), Shanghai, China, 20–24 May 2019; pp. 1–7. [Google Scholar]
- Yang, Y.; Hong, Y.; Park, J. Efficient gradient updating strategies with adaptive power allocation for federated learning over wireless backhaul. Sensors
**2021**, 21, 6791. [Google Scholar] [CrossRef] [PubMed] - Lu, Y.; Huang, X.; Dai, Y.; Maharjan, S.; Zhang, Y. Differentially private asynchronous federated learning for mobile edge computing in urban informatics. IEEE Trans. Ind. Inform.
**2019**, 16, 2134–2143. [Google Scholar] [CrossRef] - Bagdasaryan, E.; Veit, A.; Hua, Y.; Estrin, D.; Shmatikov, V. How to backdoor federated learning. In Proceedings of the International Conference on Artificial Intelligence and Statistics, Palermo, Italy, 3–5 June 2020; pp. 2938–2948. [Google Scholar]
- Shen, S.; Tople, S.; Saxena, P. Auror: Defending against poisoning attacks in collaborative deep learning systems. In Proceedings of the 32nd Annual Conference on Computer Security Applications, Los Angeles, CA, USA, 5–8 December 2016; pp. 508–519. [Google Scholar]
- Fung, C.; Yoon, C.J.; Beschastnikh, I. Mitigating sybils in federated learning poisoning. arXiv
**2020**, arXiv:1808.04866. [Google Scholar] - Blanchard, P.; el Mhamdi, E.M.; Guerraoui, R.; Stainer, J. Machine learning with adversaries: Byzantine tolerant gradient descent. In Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; pp. 118–128. [Google Scholar]
- Li, L.; Fan, Y.; Tse, M.; Lin, K.-Y. A review of applications in federated learning. Comput. Ind. Eng.
**2020**, 149, 106854. [Google Scholar] [CrossRef] - Wang, X.; Han, Y.; Wang, C.; Zhao, Q.; Chen, X.; Chen, M. In-edge ai: Intelligentizing mobile edge computing, caching and communication by federated learning. IEEE Netw.
**2019**, 33, 156–165. [Google Scholar] [CrossRef] [Green Version] - Jeong, E.; Oh, S.; Kim, H.; Park, J.; Bennis, M.; Kim, S.-L. Communication-efficient on-device machine learning: Federated distillation and augmentation under non-iid private data. arXiv
**2019**, arXiv:1811.11479. [Google Scholar] - Caldas, S.; Konečny, J.; McMahan, H.B.; Talwalkar, A. Expanding the reach of federated learning by reducing client resource requirements. arXiv
**2019**, arXiv:1812.07210. [Google Scholar] - Yang, Q.; Liu, Y.; Chen, T.; Tong, Y. Federated machine learning: Concept and applications. ACM Trans. Intell. Syst. Technol. (TIST)
**2020**, 10, 1–19. [Google Scholar] [CrossRef] - Li, T.; Sahu, A.K.; Zaheer, M.; Sanjabi, M.; Talwalkar, A.; Smith, V. Federated optimization in heterogeneous networks. arXiv
**2021**, arXiv:1812.06127. [Google Scholar] - Wang, H.; Yurochkin, M.; Sun, Y.; Papailiopoulos, D.; Khazaeni, Y. Federated learning with matched averaging. arXiv
**2020**, arXiv:2002.06440. [Google Scholar] - Lian, X.; Zhang, W.; Zhang, C.; Liu, J. Asynchronous decentralized parallel stochastic gradient descent. In Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, 10–15 July 2018; pp. 3043–3052. [Google Scholar]
- Zheng, S.; Meng, Q.; Wang, T.; Chen, W.; Yu, N.; Ma, Z.-M.; Liu, T.-Y. Asynchronous stochastic gradient descent with delay compensation. In Proceedings of the 34th International Conference on Machine Learning, Sydney, NSW, Australia, 6–11 August 2017; pp. 4120–4129. [Google Scholar]
- Xie, C.; Koyejo, S.; Gupta, I. Asynchronous federated optimization. arXiv
**2020**, arXiv:1903.03934. [Google Scholar] - Xu, G.; Li, H.; Ren, H.; Yang, K.; Deng, R.H. Data security issues in deep learning: Attacks, countermeasures, and opportunities. IEEE Commun. Mag.
**2021**, 57, 116–122. [Google Scholar] [CrossRef] - Mohassel, P.; Zhang, Y. Secureml: A system for scalable privacy-preserving machine learning. In Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP), San Jose, CA, USA, 22–26 May 2017; pp. 19–38. [Google Scholar]
- Aono, Y.; Hayashi, T.; Wang, L.; Moriai, S. Privacy-preserving deep learning via additively homomorphic encryption. IEEE Trans. Inf. Forensics Secur.
**2020**, 13, 1333–1345. [Google Scholar] - Jayaraman, B.; Wang, L. Distributed learning without distress: Privacy-preserving empirical risk minimization. Adv. Neural Inf. Process. Syst.
**2021**, 7, 33–49. [Google Scholar] - Zhao, L.; Wang, Q.; Zou, Q.; Zhang, Y.; Chen, Y. Privacy-preserving collaborative deep learning with unreliable participants. IEEE Trans. Inf. Forensics Secur.
**2019**, 15, 1486–1500. [Google Scholar] [CrossRef] [Green Version] - Bonawitz, K.; Ivanov, V.; Kreuter, B.; Marcedone, A. Practical secure aggregation for privacy-preserving machine learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, Dallas, TX, USA, 30 October–3 November 2017; pp. 1175–1191. [Google Scholar]
- Yu, D.; Zhang, H.; Chen, W.; Liu, T.-Y.; Yin, J. Gradient perturbation is underrated for differentially private convex optimization. arXiv
**2020**, arXiv:1911.11363. [Google Scholar] - Paillier, P. Public-key cryptosystems based on composite degree residuosity classes. In Proceedings of the International Conference on the Theory and Applications of Cryptographic Techniques, Prague, Czech Republic, 2–6 May 1999; Springer: Berlin/Heidelberg, Germany, 2019; pp. 223–238. [Google Scholar]
- Acs, G.; Castelluccia, C. I have a dream (differentially private smart metering). In International Workshop on Information Hiding; Springer: Berlin/Heidelberg, Germany, 2018; pp. 118–132. [Google Scholar]
- Goryczka, S.; Xiong, L. A comprehensive comparison of multiparty secure additions with differential privacy. IEEE Trans. Dependable Secur. Comput.
**2019**, 14, 463–477. [Google Scholar] [CrossRef] [PubMed] [Green Version]

Property | Public Blockchain | Consortium Blockchain | Private Blockchain |
---|---|---|---|

Read Permission | Public | Restricted | Restricted |

Immutability | Nearly impossible | Could be tampered | Could be tampered |

Efficiency | Low | High | High |

Centralized | No | Partial | Yes |

Dataset | Input Size | Training Samples | Testing Samples | Structure |
---|---|---|---|---|

MNIST | 28 × 28 × 1 | 60,000 | 10,000 | CNN |

F-MNIST | 28 × 28 × 1 | 60,000 | 10,000 | CNN |

CIFAR-10 | 32 × 32 × 1 | 50,000 | 10,000 | ResNet18 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Chen, Z.; Cui, H.; Wu, E.; Yu, X.
Dynamic Asynchronous Anti Poisoning Federated Deep Learning with Blockchain-Based Reputation-Aware Solutions. *Sensors* **2022**, *22*, 684.
https://doi.org/10.3390/s22020684

**AMA Style**

Chen Z, Cui H, Wu E, Yu X.
Dynamic Asynchronous Anti Poisoning Federated Deep Learning with Blockchain-Based Reputation-Aware Solutions. *Sensors*. 2022; 22(2):684.
https://doi.org/10.3390/s22020684

**Chicago/Turabian Style**

Chen, Zunming, Hongyan Cui, Ensen Wu, and Xi Yu.
2022. "Dynamic Asynchronous Anti Poisoning Federated Deep Learning with Blockchain-Based Reputation-Aware Solutions" *Sensors* 22, no. 2: 684.
https://doi.org/10.3390/s22020684