# A Simple Neural-Network-Based Decoder for Short Binary Linear Block Codes

^{1}

^{2}

^{*}

## Abstract

**:**

^{−4}. For the (23, 12) quadratic residue (QR) code, the SNR is improved by 2dB when the BER is 10

^{−3}. The developed NN-based decoder is quite general and applicable to various short linear block codes with good BER performance.

## 1. Introduction

- (1)
- Multi-class NN-based decoding framework is presented, where only the received signal sequence acts as the input and one fully connected layer is required.
- (2)
- (3)
- The decoding performance of the proposed decoder is close to the well-known SOD algorithm [12], which requires plenty of candidate codewords for evaluation and is involved in complicated Gaussian elimination.
- (4)
- The presented multi-class NN-based decoder is general and applicable to different short linear block codes with no additional HDD required as compared to the Chase-type algorithm [11].

## 2. Preliminaries of Conventional Soft Decision Decoder

#### 2.1. Message Encoding Process

_{0}, m

_{1}, …, m

_{k}

_{−1}], where m ∈ {0, 1}

^{k}and k is the number of bits in the message. The polynomial form is expressed as (1):

**C**is

**G**is

**P**is the parity sub-matrix (p. 7, [15]).

_{0}, c

_{1}, …, c

_{n}

_{−1}] and c

_{i}∈ {0, 1}. Here, n is the codeword length. The codeword c can be expressed as (4) in a polynomial form.

**H**is the parity check matrix, which is defined as follows:

#### 2.2. Data Transmission over Channels

_{i}in the codeword c, the received signal y

_{i}by BPSK over the AWGN channel is

_{i}represents the white Gaussian noise with variance N

_{0}/2.

_{0}, y

_{1}, …, y

_{n}

_{−1}] is the received signal sequence and r = [r

_{0}, r

_{1}, …, r

_{n}

_{−1}] be the received word. Because of BPSK signaling, the received word can be determined by

_{i}is derived as

#### 2.3. Chase-II Decoding Algorithm

- Step 1: Obtain the received signal sequence y and demodulate it as the received word r.
- Step 2: Find the t least reliable bits and generate 2
^{t}test error patterns. - Step 3: Decode 2
^{t}words and check their syndromes. The decoded word with zero syndrome will be viewed as the candidate codeword dr. - Step 4: Evaluate ${D}_{1}\left(dr\right)=\left\{i\right|{r}_{i}={dr}_{i},0\le i\le n-1\}$ and compute the maximum likelihood metric ${\sum}_{i\in {D}_{1}\left(dr\right)}\left|{y}_{i}\right|$ of the codeword dr.
- Step 5: Output the best codeword as dc with the minimum value of ${\sum}_{i\in {D}_{1}\left(dr\right)}\left|{y}_{i}\right|$.

#### 2.4. Ordered Statistics Decoding (OSD)

- Step 1: Obtain the signal sequence y and obtain the ordered index set of reliability idx. $idx=\{{\eta}_{0},{\eta}_{1},\dots ,{\eta}_{n-1}\}$ where $\left|{r}_{{\eta}_{k}}\right|\ge \left|{r}_{{\eta}_{l}}\right|$
**,**for $k>l$. - Step 2: Permute
**G**according to idx and perform Gaussian elimination as a systematic form ${\mathbf{G}}^{\mathit{\prime}}=\left[{\mathbf{I}}_{k}\right|\stackrel{~}{\mathbf{P}}]$. If the form is unavailable, reorder idx based on the reliability. - Step 3: Permute y and r as y’ and r’ according idx. Let
**m**’ be the first k bits of r’. - Step 4: Generate the test error patterns $ep=[{ep}_{0},{ep}_{1},\dots ,{ep}_{k-1}]$ for
**m**’, where the Hamming weight of ep is less than or equal to w. As a result, there are possible $\sum _{i=0}^{w}\left(\begin{array}{c}k\\ i\end{array}\right)$ candidate messages. The candidate codewords dr will be obtained by**G’.** - Step 5: Evaluate ${D}_{1}\left(dr\right)=\left\{i\right|{r}_{i}={dr}_{i},0\le i\le n-1\}$ and compute the maximum likelihood metric ${\sum}_{i\in {D}_{1}\left(dr\right)}\left|{y}_{i}\right|$ of the codeword dr
- Step 6: Output the best codeword as dc with the minimum value of ${\sum}_{i\in {D}_{1}\left(dr\right)}\left|{y}_{i}\right|$.

## 3. Deep Learning-Based Decoder

#### 3.1. Soft Decision Decoding Based on DNN-Based Binary Classification

**s**and the reliability of the received word |y|, so as to improve the decoding performance. The output is the n-bit error pattern ${e}^{\mathrm{*}}=[{e}_{0}^{*},{e}_{1}^{*},\dots ,{e}_{n-1}^{*}]$

**,**where ${e}_{i}^{*}\in \left\{\mathrm{0,1}\right\},0\le i\le n-1$. Each bit ${e}_{i}^{*}$ is determined by the binary classification of the DNN. As ${\mathrm{e}}^{\mathrm{*}}$ is obtained, the decoded codeword is $dc=r\u2a01{e}^{*}$

**,**where $\u2a01$ is exclusive OR (XOR) operation.

#### 3.2. DNN-Based Denoiser for Soft Decision Decoding

**dy**is obtained by the DNN and the received word

**r**can be determined based on

**dy**. Finally, the HDD produces the codeword based on

**r**. This architecture combines the DNN-based denoiser and the traditional HDD, resulting in better decoding performance as compared to the HDD only. The DNN architecture [14] in Figure 7 consists of three fully connected layers. The total numbers of nodes in these three layers are 256, 128 and 64, respectively. The activation function used in each layer is the ReLU function.

## 4. Proposed Multi-Class Neural Network-Based Decoder

**y**to estimate occurrence probabilities of all possible message ${\mathbf{m}}_{i}$, called ${\delta}_{i}$, where $i=\mathrm{0,1},\dots .,{2}^{k}-1$ and k is the message length. Then, the sorting operation is performed based on the occurrence probability ${\delta}_{i}$. Let the ordered set of the occurrence probability be $\mathsf{\Delta}\equiv \{{\nu}_{0},{\nu}_{1},\dots ,{\nu}_{{2}^{k}-1}\}$, where ${\delta}_{{\nu}_{l}}\ge {\delta}_{{\nu}_{\kappa}}$, for $l\ge \kappa $. ${\mathbf{m}}_{\left(i\right)}$ is denoted as the message with the occurrence probability ${\delta}_{{\nu}_{i}}$. The messages of l highest occurrence probabilities, called ${\mathbf{m}}_{\left(1\right)},{\mathbf{m}}_{\left(2\right)},\dots ,{\mathbf{m}}_{\left(l\right)}$, are thus selected. These messages are encoded as the codewords ${\stackrel{~}{\mathbf{c}}}_{1},{\stackrel{~}{\mathbf{c}}}_{2},\dots ,{\stackrel{~}{\mathbf{c}}}_{l}$ by the generator matrix

**G**. We evaluate ${D}_{1}\left({\stackrel{~}{\mathbf{c}}}_{j}\right)=\left\{i\right|{r}_{i}={\stackrel{~}{c}}_{j,i},0\le i\le n-1\}$ and compute the maximum likelihood metric ${\sum}_{i\in {D}_{1}\left({\stackrel{~}{\mathbf{c}}}_{j}\right)}\left|{y}_{i}\right|$ of the codeword ${\stackrel{~}{\mathbf{c}}}_{j}$

**.**Finally, the best codeword is selected as

**dc**with the minimum value of ${\sum}_{i\in {D}_{1}\left({\stackrel{~}{\mathbf{c}}}_{j}\right)}\left|{y}_{i}\right|,j=\mathrm{1,2},\dots ,l$

**.**

## 5. Dataset Generation and Neural Network Training

- Randomly generate a message
**m.** - Encode
**m**to produce the codeword**c**. - Use BPSK modulation and AWGN to generate the received signal sequence
**y**. - Repeat steps 1–4 one million times to produce the dataset.
- Divide the dataset into the training and validation sets with a ratio of 9:1.

#### 5.1. Generation of Received Signal by BPSK over AWGN Channel

Algorithm 1 Generative Algorithm of Additive White Gaussian Noise |

Input: codeword c, SNR γ |

Output: received signal sequence y |

1: Compute N_{0} by (14) |

2: for i = 0, 1, …, n−1 do |

3: Generate Gaussian noise e_{i} N(0, N0/2) |

4: if c_{i} = 1 then |

5: y_{i}= −1 + e_{i} |

6: else |

7: y_{i}= 1 + e_{i} |

8: end if |

9: end for |

10: return y |

#### 5.2. Training of Neural Network

^{−3}. The loss function is determined based on the output of the NN. Since the expected output of each node in the output layer is either 0 or 1, the binary cross entropy (BCE) loss function is utilized as shown in (16):

## 6. Experimental Results of Decoding by Proposed Multi-class NN-Based Decoder

#### 6.1. Simulation Environment and Procedure

#### 6.2. Experimental Results

#### 6.2.1. Results for BCH (15, 7) Code

#### 6.2.2. Results for BCH (15, 5)

#### 6.2.3. Results for QR (23, 12) Code

#### 6.2.4. Results for BCH (31, 11) Code

## 7. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Esposito, A.; Rampone, S.; Tagliaferri, R. A neural network for error correcting decoding of binary linear codes. Neural Netw.
**1994**, 7, 195–202. [Google Scholar] [CrossRef] - Tallini, L.G.; Cull, P. Neural nets for decoding error-correcting codes. In Proceedings of the IEEE Technical applications Conference and Workshops Northcon/95 Conference Record, Portland, OR, USA, 10–12 October 1995; p. 89. [Google Scholar] [CrossRef]
- Bruck, J.; Blaum, M. Neural networks, error-correcting codes, and polynomials over the binary n-cube. IEEE Trans. Inf. Theory
**1989**, 35, 976–987. [Google Scholar] [CrossRef][Green Version] - Xu, C.; Van Luong, T.; Xiang, L.; Sugiura, S.; Maunder, R.G.; Yang, L.-L.; Hanzo, L. Turbo Detection Aided Autoencoder for Multicarrier Wireless Systems: Integrating Deep Learning Into Channel Coded Systems. IEEE Trans. Cogn. Commun. Netw.
**2022**, 8, 600–614. [Google Scholar] [CrossRef] - Lee, Y.; Lee, U.; Fisseha, H.H.; Sunwoo, M.H. Deep Learning aided BP-Flip Decoding of Polar Codes. In Proceedings of the 2022 IEEE 4th International Conference on Artificial Intelligence Circuits and Systems (AICAS), Incheon, Republic of Korea, 13–15 June 2022; pp. 114–117. [Google Scholar] [CrossRef]
- Guo, X.; Chang, T.-H.; Wang, Y. Model-Driven Deep Learning ADMM Decoder for Irregular Binary LDPC Codes. IEEE Commun. Lett.
**2022**, 27, 571–575. [Google Scholar] [CrossRef] - Li, W.; Tian, Q.; Zhang, Y.; Tian, F.; Li, Z.; Zhang, Q.; Wang, Y. A rate-compatible punctured Polar code decoding scheme based on deep learning. In Proceedings of the 2022 20th International Conference on Optical Communications and Networks (ICOCN), Shenzhen, China, 12–15 August 2022; pp. 1–3. [Google Scholar] [CrossRef]
- Lu, Y.; Zhao, M.; Lei, M.; Wang, C.; Zhao, M. Deep learning aided SCL decoding of polar codes with shifted-pruning. China Commun.
**2023**, 20, 153–170. [Google Scholar] [CrossRef] - Dong, R.; Lu, F.; Dong, Y.; Yan, H. The Negative BER Loss Function for Deep Learning Decoders. IEEE Commun. Lett.
**2022**, 26, 1824–1828. [Google Scholar] [CrossRef] - Pearl, J. Reverend Bayes on Inference Engines: A Distributed Hierarchical Approach. In Probabilistic and Causal Inference: The Works of Judea Pearl; Association for Computing Machinery: New York, NY, USA, 2022; pp. 129–138. [Google Scholar] [CrossRef]
- Chase, D. Class of algorithms for decoding block codes with channel measurement information. IEEE Trans. Inf. Theory
**1972**, 18, 170–182. [Google Scholar] [CrossRef][Green Version] - Fossorier, M.; Lin, S. Soft-decision decoding of linear block codes based on ordered statistics. IEEE Trans. Inf. Theory
**1995**, 41, 1379–1396. [Google Scholar] [CrossRef][Green Version] - Bennatan, A.; Choukroun, Y.; Kisilev, P. Deep Learning for Decoding of Linear Codes—A Syndrome-Based Approach. In Proceedings of the 2018 IEEE International Symposium on Information Theory (ISIT), Vail, CO, USA, 17–22 June 2018; pp. 1595–1599. [Google Scholar] [CrossRef][Green Version]
- Zhu, H.; Cao, Z.; Zhao, Y.; Li, D. A Novel Neural Network Denoiser for BCH Codes. In Proceedings of the 2020 IEEE/CIC International Conference on Communications in China (ICCC), Chongqing, China, 9–11 August 2020; pp. 272–276. [Google Scholar] [CrossRef]
- Morelos-Zaragoza, R.H. The Art of Error Correcting Coding; John Wiley & Sons: Hoboken, NJ, USA, 2002. [Google Scholar]
- Nachmani, E.; Marciano, E.; Lugosch, L.; Gross, W.J.; Burshtein, D.; Be'Ery, Y. Deep Learning Methods for Improved Decoding of Linear Codes. IEEE J. Sel. Top. Signal Process.
**2018**, 12, 119–131. [Google Scholar] [CrossRef][Green Version] - Kavvousanos, E.; Paliouras, V. Hardware Implementation Aspects of a Syndrome-based Neural Network Decoder for BCH Codes. In Proceedings of the 2019 IEEE Nordic Circuits and Systems Conference (NORCAS): NORCHIP and International Symposium of System-on-Chip (SoC), Helsinki, Finland, 29–30 October 2019. [Google Scholar] [CrossRef]
- Benammar, M.; Piantanida, P. Optimal Training Channel Statistics for Neural-based Decoders. In Proceedings of the 2018 52nd Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA, 28–31 October 2018; pp. 2157–2161. [Google Scholar] [CrossRef]
- Tian, C.; Fei, L.; Zheng, W.; Xu, Y.; Zuo, W.; Lin, C.-W. Deep learning on image denoising: An overview. Neural Netw.
**2020**, 131, 251–275. [Google Scholar] [CrossRef] [PubMed] - Kingma, D.P.; Ba, J. ADAM: A method for stochastic optimization. arXiv
**2014**, arXiv:1412.6980. [Google Scholar]

**Figure 5.**DNN Architecture in [13].

Numbers of Nodes | Activation Function | |
---|---|---|

Input layer | n | |

Fully connected layer | 2^{k} | ReLU |

Output layer | 2^{k} | Softmax |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Hsieh, K.; Lin, Y.-W.; Chu, S.-I.; Chang, H.-C.; Cho, M.-Y. A Simple Neural-Network-Based Decoder for Short Binary Linear Block Codes. *Appl. Sci.* **2023**, *13*, 4371.
https://doi.org/10.3390/app13074371

**AMA Style**

Hsieh K, Lin Y-W, Chu S-I, Chang H-C, Cho M-Y. A Simple Neural-Network-Based Decoder for Short Binary Linear Block Codes. *Applied Sciences*. 2023; 13(7):4371.
https://doi.org/10.3390/app13074371

**Chicago/Turabian Style**

Hsieh, Kunta, Yan-Wei Lin, Shao-I Chu, Hsin-Chiu Chang, and Ming-Yuan Cho. 2023. "A Simple Neural-Network-Based Decoder for Short Binary Linear Block Codes" *Applied Sciences* 13, no. 7: 4371.
https://doi.org/10.3390/app13074371