Fully Parallel Proposal of Naive Bayes on FPGA

Abstract

This work proposes a fully parallel hardware architecture of the Naive Bayes classifier to obtain high-speed processing and low energy consumption. The details of the proposed architecture are described throughout this work. Besides, a fixed-point implementation on a Stratix V Field Programmable Gate Array (FPGA) is presented and evaluated regarding the hardware area occupation, processing time (throughput), and dynamic power consumption. In addition, a comparative design analysis was carried out with state-of-the-art works, showing that the proposed implementation achieved a speedup of up to ${10}^4\times$ and power savings of up to ${10}^7\times$-times while also reducing the hardware occupancy by up to ${10}^2\times$-times fewer logic cells.

1. Introduction

Machine learning (ML) is a set of techniques widely used to analyze large volumes of data, requiring intensive processing power [1]. Usually, the use of ML techniques results in high energy consumption. Therefore, energy consumption can become a critical problem for large technology companies such as Microsoft, IBM, Amazon, and Google. Given that, the servers and databases are improving at an architectural level to adopt low-power alternatives also capable of high-speed processing [2].

Applications in emerging fields such as 5G Communication, Advanced Driver Assistance Systems, and bioinformatics often require high-speed processing and low energy consumption. Given the complexity of ML techniques, parallelization of the algorithms has often been adopted to overcome the need for high computational performance when processing large volumes of data. Besides, the use of hardware solutions implemented in parallel has proven to be very efficient in reaching the mentioned requirements. For instance, Field Gate Programmable Arrays (FPGAs) allow high performance and reduced energy consumption compared to conventional software implementations due to the high parallelization degree and the optimization at the level of logic gates, thus increasing the data processing speed of ML techniques up to a thousand times [3,4,5,6,7,8,9,10].

Naive Bayes (NB) is a widely used ML technique for solving classification problems, such as facial recognition and network packet classification [11,12,13,14,15,16,17,18,19]. Several of these applications demand real-time processing, but reaching this performance in software becomes difficult due to the large volume of data usually handled. Thus, many works in the literature have proposed implementing the Naive Bayes method in hardware seeking to achieve high throughputs and, in addition, low power consumption [20,21,22,23,24,25].

Some works in the literature propose a hardware design for both stages of the NB method, training and inference [20,21]. However, they present a serial implementation with floating-point representation, which reduces the architecture’s performance. Moreover, these works present an implementation designed for a fixed amount of attributes and classes. In contrast, this paper proposes a fully parallel architecture with fixed-point representation for the training and classification steps of the NB method, easily adjustable to handle different numbers of attributes and classes. This architecture aims to increase the data processing speed while maintaining low energy consumption. The design was developed on a Stratix V 5SGXMBBR3H43C3 FPGA and evaluated for a different quantity of attributes in the inference and training steps.

The remainder of this paper is organized as follows: Section 2 presents the related works in the literature; Section 3 addresses the theoretical foundation of the Naive Bayes method; Section 4 gives a detailed description of the architectures proposed in this paper; Section 5 provides validation data for the hardware implementation by comparing its results to the software implementation method, while Section 6 presents and analyzes the synthesis results obtained from the described implementation, including a comparison to other works; Section 7 presents the final considerations.

2. Related Works

Hardware implementations of the NB method have been developed to increase the processing speed of different applications that rely on massive datasets.

An NB classifier was implemented by [20] on a Virtex-4 XC4VLX160 FPGA to perform multiple class object classification by analyzing vectors with binary attributes. The training and inference steps were serially implemented. The total area occupied by their architecture was 784 slices, 730 Flip-Flops (FFs), 819 Look-Up Tables (LUTs), and 3 Block RAMs (BRAMs), being less than the area occupation of a bSOM also implemented on an FPGA.

In [26], the NB was developed on the Raspberry Pi 2 development board. The board has a Quad-Core ARMv7 900 MHz Central Processing Unit (CPU) and 1 GB of RAM. The NB method was validated with the Iris, Diabetes, Hepatitis, and other datasets, achieving accuracy levels between $73\%$ and $99\%$, according to the dataset used.

Meanwhile, in [21], a Gaussian Naive Bayes was proposed to optimize the algorithm’s learning process. The implementation was developed using High-Level Synthesis (HLS) on a ZedBoard Zynq-7020 SoC Development Board with an Artix-7 FPGA to obtain a heterogeneous system. Regarding the area occupation, 197 DSPs, 56 BRAMs, 37,929 LUTs, and 29,271 FFs were used for the training step, while the inference step required 93 DSPs, 112 BRAMs, 34,977 LUTs, and 31,779 FFs. Meanwhile, the processing time for the training and inference steps were $1.1$ s and $2.7$ s, respectively, for a total of 784 attributes and 10 labels, using 2000 samples from the MNIST dataset.

The work presented by [22] implements three classifiers: Decision Tree, NB, and K-Nearest Neighbors, to classify packets sent within a network and identify possible malicious packets that can infect the target machine. They were developed on a Cyclone IV GX FPGA. Regarding NB, six implementations are proposed (three combinational and three sequential) using floating-point notation for 32, 16, and 10 bits in each type of implementation. The work presented data on area occupancy, throughput, and power consumption, for all designs.

In [23], a hardware architecture of an NB classifier is implemented on Altera’s Stratix EP1S10F780C5ES for email spam classification, using a logarithmic numerical system to reduce the computational complexity. The proposal was designed using floating-point notation, but can receive fixed-point inputs with 8, 12, and 16 bits. The floating-point implementation was able to classify 8889.375 emails per second using 15 attributes. The hardware was validated with SpamAssassin’s public corpus dataset and presented each architecture’s data processing time and area occupancy.

In [24], another hardware architecture of an NB classifier is presented for real-time classification of seven facial expressions: happy, surprised, sad, disgusted, afraid, angry, and neutral. The proposed design was developed in the fixed-point notation to reduce the area occupation and achieved an operating frequency of 241.55 MHz, with a classification accuracy of $81.94\%$. The proposed architecture was designed using the Xilinx System Generator, and its target FPGA was Xilinx Virtex-II.

Meanwhile, in [25], a hardware system is proposed to increase the processing speed and reduce the power consumption for a malware detection application. The system was developed using HLS on a ZedBoard Zynq-7020 SoC Development Board with an Artix-7 FPGA, and it is ready to be integrated into mobile devices’ CPUs. Three classification algorithms are commonly used to detect malware: Logistic Regression, Naive Bayes, and Support Vector Machines. These classifiers were optimized regarding power consumption while maintaining an accuracy similar to software implementations.

Therefore, it is clear from the works mentioned that FPGA implementations of NB could optimize applications’ processing speed and power consumption with these requirements. Unlike the works mentioned above, this work proposes implementing both steps of Naive Bayes, training and inference, with a fully parallel architecture using fixed-point representation. In addition, the architecture was designed to be easily adapted to handle different numbers of attributes and classes. Through this architecture, the aim is to obtain better throughput and power savings results.

Table 1. Summary of the proposal’s architectural characteristics and other works in the literature.

References	Training	Inference	FPGA	Arithmetic	Design
[20]	Yes	Yes	Virtex-4	Floating-point	Serial
[21]	Yes	Yes	Artix-7	Floating-point	Serial
[23]	No	Yes	Stratix	Floating-point	Serial
[25]	No	Yes	Artix-7	Floating-point	Serial
[22]	No	Yes	Cyclone IV	Floating-point	Serial/Parallel
[24]	No	Yes	Virtex-II	Fixed-point	Parallel
This work	Yes	Yes	Stratix V	Fixed-point	Parallel

summarizes the characteristics of the proposal’s architecture and works in the literature. The first column shows the architecture analyzed. Meanwhile, the second and third columns indicate whether the work has implemented the training and inference phases of the NB algorithm in hardware. The fourth column displays the FPGA for which the implementation was synthesized. The fifth column indicates the arithmetic representation of the values in the hardware. Finally, the sixth column shows how the architecture implementation was accomplished.

3. Naive Bayes Method

The Naive Bayes method is based on Bayes’ Theorem, in which a “naive” premise of total independence between the attributes is adopted for a given sample. This premise is often unrealistic for most classification problems in which the technique is used; however, it is still possible to achieve high accuracy values compared to other ML techniques [1].

According to [27,28], the classification process calculates the probability of an input vector $\mathbf{x}$ belonging to a k-th class $c_k$ of a set of K classes. Therefore, the probability value $P(c_k\mid \mathbf{x})$ is obtained, which can be expressed as

$$P(c_k\mid \mathbf{x})=\frac{P(c_k)P(\mathbf{x}\mid c_k)}{P(\mathbf{x})}$$

(1)

where $P(c_k)$ and $P(\mathbf{x})$ are the probabilities of the k-th class $c_k$ and an $\mathbf{x}$ occurring, respectively, while $P(\mathbf{x}\mid c_k)$ is the probability of $\mathbf{x}$ happening given that $c_k$ occurred. The input vector $\mathbf{x}$ is expressed as

$$\mathbf{x}=\left[x_1,\cdots ,x_i,\cdots ,x_N\right]$$

(2)

where each i-th $x_i$ represents an independent attribute.

Given the premise of independence between attributes, that $P(\mathbf{x})$ does not depend on $c_k$, and that each i-th input, $x_i$, is already presented, then $P(\mathbf{x})$ can be ignored in (1). Hence, it can be rewritten as

$$P(c_k\mid \mathbf{x})\propto P(c_k)\prod _{i=1}^{N}P(x_i\mid c_k).$$

(3)

Thus, after obtaining the NB probabilistic model, the classifier is performed. For this purpose, the classification process requires a decision mechanism, which can be defined by the highest class probability for a given input sample, i.e.,

$$\widehat{k}=arg\underset{k\in 1,..,K}{max}P(c_k)\prod _{i=1}^{N}P(x_i\mid c_k)$$

(4)

where $\widehat{k}$ is the estimate of a k-th class $c_k$.

Figure 1 shows a flowchart illustrating the operation of the Naive Bayes method. Initially, the dataset is split into training and testing sets. Each sample in the training data contains the attribute vector, $\mathbf{x}$, and its classification, $c_k$, which are provided for the training stage. In this step, the prior probability of each k-th $c_k$, $P(c_k)$, and the likelihood of each i-th $x_i$ and k-th $c_k$, $P(x_i\mid c_k)$, are calculated. The values of $P(c_k)$ and $P(x_i\mid c_k)$ are then provided to the inference stage, which uses these values to compute the posterior probabilities, $P(c_k\mid \mathbf{x})$, of the samples from the test set. Finally, the highest posterior probability value determines the analyzed sample’s classification.

4. Hardware Description

The architecture was designed in fixed-point notation using 15 bits for the fractional part and from 3 to 17 bits in the integer part, depending on the pipeline stage, to reduce the hardware area occupation and increase the throughput compared to floating-point implementations. Two modules were implemented: training and inference, based on the NB method described in Section 3.

4.1. Training Module

Figure 2 presents the Training Module (TM) used to obtain the probabilities $P(\mathbf{x}|c_k)$ and $P(c_k)$, according to Expression (1). Therefore, at each n-th time sample, $t_s$, the TM receives as input the vector of attributes $\mathbf{x}(n)$, described in (2), and the k-th class, $c_k(n)$, for the respective vector of attributes.

Figure 3 details the TM modules. As can be seen, it is constituted by two: the Class Probability Module (CPM) and the Attribute Probability Module (APM). Note that the APM is replicated N times for parallel processing. Hence, at each n-th time sample, the CPM receives one $c_k(n)$ value, which is used for calculating the probabilities, $P(c_k)$, and the occurrence number of each k-th class, $n{c}_k$. Meanwhile, each i-th ${\mathrm{APM}}_i$ receives as input the i-th attribute, $x_i^j(n)$, the k-th class, $c_k(n)$, and its occurrence number, $n{c}_k$, to obtain the probabilities $P(x_i^j|c_k)$. Besides, it is important to mention that each i-th attribute is represented by a j-th possible value, in which $J_i$ is the maximum number of distinct values assumed by its attribute.

Figure 4 shows the internal structure of the CPM. Firstly, at each n-th time sample, the k-th class, $c_k(n)$, is compared with the remaining classes of the dataset ($c_1,\cdots ,c_K$), through comparator circuits. Besides, note that the classes are represented in the architecture as constant values. Secondly, the comparators output the bit 1 if the classes are equal; otherwise, they output the bit 0. Thirdly, the counter increments its value by 1 every time sample the comparator outputs the bit 1. Thereby, the counter defines the occurrence of each class, $n{c}_k$, and propagates these values to every $AP{M}_i$. Lastly, each k-th $n{c}_k$ is multiplied by $\frac{1}{K}$ for obtaining the probability of occurrence of each k-th class, $P(c_K)$, according to Expression (3). The calculated probabilities are stored in Registers (R) and provided as output from the CPM.

Meanwhile, each i-th $AP{M}_i$ is performed in parallel, and its internal structure is shown in Figure 5. As can be observed, each i-th $AP{M}_i$ receives as input the i-th attribute, $x_i(n)$, the k-th class, $c_k(n)$, and its occurrence, $n{c}_k$. Initially, $x_i(n)$ is concatenated to $c_k(n)$ by the $CXU$ block, generating a $z_k^{j_i}$ value. This value represents the occurrence of a k-th class for an i-th attribute with a j-th value.

Secondly, for each possible $z_k^{j_i}$ value, there is a comparator block. Similar to the submodules in the CPM, the comparator outputs the bit 1 according to

$$\mathrm{Comparator} \mathrm{block}=\left\{\begin{array}{c}1,\mathrm{if}{z}_k^{j_i}=\mathrm{CXU} \mathrm{output}\mathrm{value}\hfill \\ 0,\mathrm{otherwise}.\hfill \end{array}\right.$$

Subsequently, the counter submodule ($Counter$) increases its value by 1 every n-th time sample the comparator output is equal to 1. Hence, the counter defines the total occurrence of each $z_k^{j_i}$, which is stored in a Register (R).

Thirdly, the output of each counter has to be divided by the k-th $n{c}_k$ related to $z_k^{j_i}$. Due to the hardware complexity of performing division circuits that provide the decimal values of the result, a multiplier was added before and after the division block. Therefore, the counter output is multiplied by ${\alpha }_1=$ 1000, divided by $n{c}_k$, and finally, multiplied by ${\alpha }_2=0.001$.

Lastly, to avoid the Naive Bayes zero-frequency problem, when $P(x_i^j|c_k)=0$, logic OR gates were added after the last multiplier blocks. The OR gates receive as input the multiplier value and a constant $\beta$, which in turn has the smallest possible value that can be represented for the data precision defined, e.g., $\beta \approx 0.00031$. Thus, the probabilities $P(x_i^j|c_k)$ of having a value equal to zero are replaced by the value of $\beta$. The outputs of the logic gates, i.e., the probabilities $P(x_i^j|c_k)$, are stored in Registers (R) and provided as the outputs of the module. At the end of this procedure, the NB training step is complete, and all the values of $P(x_i^j|c_k)$ and $P(C_k)$ are available.

4.2. Inference Module

The NB inference process consists of classifying a sample $x(n)$ according to (4). For this purpose, it is necessary to calculate the probability $P(c_k|\mathbf{x})$, defined on Expression (3). Figure 6 shows the Inference Module (IM) developed to perform this process. As can be seen, there is one submodule for each k-th $P(c_k|\mathbf{x})$, and it receives as inputs the probabilities $P(c_k)$ from the TM and the N samples, $x(n)$.

Figure 7 shows the internal structure of each k-th $P(c_k|\mathbf{x})$ submodule. As illustrated, there are N 15 bit-wide LUTs, of depth $J_i$, where each stores all $P(c_k|\mathbf{x})$ values. The sample input value, $x_i^j(n)$, is used as the LUT address to select its corresponding probability, $P(x_i^j|c_k)$. Subsequently, the probabilities $P(x_i^j|c_k)$ addressed in each i-th LUT are multiplied through a tree of multipliers, and in the last multiplier of the tree, the result is multiplied by the k-th probability $P(c_k)$, thus obtaining the value of $P(c_k|X)$, according to (3).

According to (4), the decision process of the class corresponding to the sample under evaluation consists of selecting the highest probability $P(c_k|\mathbf{x})$. This process is performed by a relational circuit that compares the $P(c_k|x)$ values from all K submodules of the IM and outputs the class $c_k$ related to the highest probability value calculated, therefore completing the inference process.

5. Hardware Validation

The software implementation was used to validate the proposed NB hardware architecture. The software was developed using floating-point representation, according to the IEEE 754 standard, and both hardware and software implementations were applied to the divorce predictors dataset [29]. This dataset contains 170 samples, representing couples’ responses to a survey to predict divorce basis on Gottman couples therapy. Each sample has 54 attributes corresponding to a question in the survey, which can have five different answers represented numerically by: $0=\mathrm{Never}$; $1=\mathrm{Rarely}$; $2=\mathrm{Averagely}$; $3=\mathrm{Frequently}$; and $4=\mathrm{Always}$. The target of this dataset is to predict, based on the answers given, whether the couple will divorce, Class 1, or not, Class 2.

The dataset was divided into two groups: one with $80\%$ of the data for the training step and another for the classification test (inference step) with the remaining $20\%$. For each data point, four discrete attributes were selected, each of which can assume five distinct values. Thus, the hardware parameters to perform the tests were set as $N=4$, $J_1=J_2=J_3=J_4=5$, and $K=2$.

5.1. Training Step Validation

The absolute error between the hardware and software implementations was obtained for each k-th probability $P(x_i^j|c_K)$, as shown in Figure 8 and Figure 9, and it can be defined as

$$e_{i,k}^j=|P_{\mathrm{H}}(x_i^j|c_k)-P_{\mathrm{S}}(x_i^j|c_k)|$$

(5)

where $P_{\mathrm{H}}(x_i^j|c_k)$ and $P_{\mathrm{S}}(x_i^j|c_k)$ are the probability values obtained in hardware and software, respectively.

In addition, the absolute errors were also calculated for each k-th probability $P(c_K)$ as follows:

$$e{c}_k=|P_{\mathrm{H}}(c_k)-P_{\mathrm{S}}(c_k)|$$

(6)

where $P_{\mathrm{H}}(c_k)$ and $P_{\mathrm{S}}(c_k)$ correspond, respectively, to the probability values obtained in hardware and software. The absolute error calculated for each class was $e{c}_1=0.00303$ and $e{c}_2=0.00379$.

5.2. Inference Step Validation

As previously mentioned, $20\%$ of the dataset was used for the classification test, that is on the inference step. The classification result of each sample was compared to the expected class, which is present in the dataset. The accuracy achieved by the proposed implementation was $97.06\%$. Figure 10 presents the confusion matrix obtained after performing the inference on the IM.

6. Results

6.1. Hardware Proposal Results

6.1.1. Area Occupation

Synthesis results were analyzed for training and inference modules and presented in

Table 2. Area occupation for the TM.

${\mathit{N}}_{\mathit{A}}$	${\mathit{N}}_{\mathbf{LC}}$	${\mathit{N}}_{\mathbf{Bits}}$	${\mathit{N}}_{\mathbf{MULT}}$
4	$\mathrm{13,950}$	2550	0
8	$\mathrm{27,825}$	4590	0
16	$\mathrm{55,459}$	8670	0

and

Table 3. Area occupation for the IM.

${\mathit{N}}_{\mathit{A}}$	${\mathit{N}}_{\mathbf{LC}}$	${\mathit{N}}_{\mathbf{Bits}}$	${\mathit{N}}_{\mathbf{MULT}}$
4	24	1024	8
8	27	2048	16
16	37	4096	32
32	59	8192	64
64	12	16,384	128

, respectively. In both tables, the first column indicates the number of attribute, $N_A$, used for the implementation. Meanwhile, from the second to fourth columns are shown the area occupation in the target FPGA, that is the Number of Logical Cells, $N_{\mathrm{LC}}$, the Number of Bits of memory blocks, $N_{\mathrm{Bits}}$, and the Number of Multipliers implemented using DSP blocks, $N_{\mathrm{MULT}}$.

6.1.2. Processing Speed (Throughput)

The processing speeds for the TM and IM are presented in

Table 4. Processing speed of the TM.

${\mathit{N}}_{\mathit{A}}$	THPT	CLK
4	40 Msps	40 MHz
8	40 Msps	40 MHz
16	40 Msps	40 MHz

and

Table 5. Processing speed of the IM.

${\mathit{N}}_{\mathit{A}}$	THPT	CLK
4	72.52 Msps	75.50 MHz
8	59.00 Msps	59.00 MHz
16	50.00 Msps	50.00 MHz
32	43.01 Msps	43.00 MHz
64	35.00 Msps	35.00 MHz

, respectively. In both tables, the first column indicates the number of attributes in the analysis, $N_A$. Meanwhile, the second column presents the Throughput (THPT) in Mega samples per second (Msps), while the third column shows the maximum Clock (CLK) frequency.

As is evident from Table 4, the throughput values remain constant with the increase of $N_A$ due to the high parallelism adopted for the NB training step by replicating the modules. In contrast, the throughput reduces as $N_A$ increases due to the rise in the number of multipliers in the IM for the inference step, as shown in Table 5.

6.2. Comparison with State-of-the-Art Works

Regarding the area occupation, the use of logical cells and the number of bits in the memory were analyzed. For a fair comparison of the proposed architecture with state-of-the-art works, it was necessary to conduct some conversions to have similar metrics for the hardware area occupancy. Usually, this is required due to the use of different FPGAs in each work, which present the data based on the optimization process. The conversions were realized considering the relationship between ALMs, LUTs, and Slices with their equivalents in the number of logical cells, as stated in the documentation provided by the manufacturer of the FPGA devices.

Based on [30], the dynamic power, $E_d$, can be expressed as

$$E_d\propto N_g\times \mathrm{CLK}\times V_{DD}^2$$

(7)

where $N_g$ is the number of elements occupied in hardware, CLK is the clock frequency, and $V_{DD}$ is the supply voltage. The frequency at which a CMOS circuit can operate is approximately proportional to the voltage [31], so Equation (7) can be expressed as

$$E_d\propto N_g\times {\mathrm{CLK}}^3.$$

(8)

For all comparisons, $N_g$ was calculated as

$$N_g=N_{\mathrm{Bits}}+N_{\mathrm{Mult}}$$

(9)

Besides, the inference module was only compared due to the scarcity of data about the training step in the state-of-the-art works. Table 3 shows the synthesis results for the inference module, which were used to derive equations for comparing the area occupation. For this purpose, linear regression was used, which resulted in the following:

$$N_{\mathrm{LC}}=1.6333\times N_A+13.0833,$$

(10)

$$N_{\mathrm{Bits}}=256\times N_A$$

(11)

and

$$N_{\mathrm{Mult}}=2\times N_A.$$

(12)

Likewise, the data presented in Table 5 and a logarithmic adjustment were used to derive the following throughput metric:

$$\mathrm{THPT}=\frac{1}{3.5869\times {log}_2{N}_A+6.1680},$$

(13)

the behavior of which is observed in Figure 11.

The derived equations allow estimating the hardware area occupation, throughput, and dynamic power consumption for any number of attributes and classes, to compare the performance with state-of-the-art works.

6.2.1. Area Occupation Comparison

Table 6. Hardware area occupation comparison.

Reference	${\mathit{N}}_{\mathit{A}}$	${\mathit{N}}_{\mathbf{LC}}^{\mathbf{Ref}}$	${\mathit{N}}_{\mathbf{LC}}^{\mathbf{Author}}$	${\mathit{N}}_{\mathbf{LC}}^{\mathbf{Gain}}$	${\mathit{N}}_{\mathbf{Bits}}^{\mathbf{Ref}}$	${\mathit{N}}_{\mathbf{Bits}}^{\mathbf{Author}}$	${\mathit{N}}_{\mathbf{Bits}}^{\mathbf{Gain}}$
[20]	762	81	1258	≈$0.65\times$	48	$\mathrm{19,507}$	≈$0.00\times$
[22] I	10	$\mathrm{11,488}$	$29.40$	≈$390.75\times$	0	2560	–
[22] II	10	5480	$29.40$	≈$186.40\times$	0	2560	–
[22] III	10	2867	$29.40$	≈$97.52\times$	0	2560	–
[22] IV	10	1733	$29.40$	≈$58.95\times$	$\mathrm{10,112}$	2560	≈$3.95\times$
[22] V	10	1000	$29.40$	≈$34.01\times$	6656	2560	≈$2.60\times$
[22] VI	10	705	$29.40$	≈$23.98\times$	5120	2560	≈$2.00\times$
[23] I	15	146	$37.60$	≈$3.88\times$	$\mathrm{38,656}$	3840	≈$10.07\times$
[23] II	15	278	$37.60$	≈$7.39\times$	$\mathrm{81,920}$	3840	≈$21.33\times$
[23] III	15	313	$37.60$	≈$8.32\times$	$\mathrm{135,168}$	3840	≈$35.20\times$
[23] IV	15	144	$37.60$	≈$3.83\times$	$\mathrm{135,168}$	3840	≈$35.20\times$

presents the area occupation for the hardware proposals of [20,22,23] and compares them to the proposed implementation. The second column gives the number of attributes, $N_A$, used on the referenced proposal and this work. Meanwhile, from the third to fifth columns are shown the number of logical cells for the referenced work ($N_{\mathrm{LC}}^{\mathrm{Ref}}$), for this work ($N_{\mathrm{LC}}^{\mathrm{Author}}$), and the gain ($N_{\mathrm{LC}}^{\mathrm{Gain}}$), respectively. Likewise, the last three columns present the number of memory block bits.

As can be seen, except for the architecture presented by [20], the implementation proposed achieved gains both in the number of logical cells ($N_{\mathrm{LC}}^{\mathrm{Gain}}$) and the number of bits in the memory block ($N_{\mathrm{Bits}}^{\mathrm{Gain}}$). This gain can be attributed to the use of a fixed-point representation, while [22] used a floating-point and [23] deployed a selection scheme between fixed and floating-point. Concerning the number of bits in the memory block, it was not possible to compare the gains against Designs I, II, and III of [22] as this did not apply to the work developed, and therefore, these values were not included for the compared reference.

6.2.2. Throughput Comparison

Table 7. Throughput comparison.

Reference	${\mathit{N}}_{\mathit{A}}$	THPTRef.	THPTAuthor	Speedup
[21]	784	$7.41\times {10}^{-4}$ Msps	$24.59$ Msps	$\approx \mathrm{33,200}\times$
[22] I	10	$2.80$ Msps	$54.67$ Msps	≈$19.53\times$
[22] II	10	$4.16$ Msps	$54.67$ Msps	≈$13.14\times$
[22] III	10	$5.51$ Msps	$54.67$ Msps	≈$9.92\times$
[22] IV	10	$0.60$ Msps	$54.67$ Msps	≈$118.85\times$
[22] V	10	$0.77$ Msps	$54.67$ Msps	≈$71.00\times$
[22] VI	10	$0.98$ Msps	$54.67$ Msps	≈$55.79\times$
[23] I	15	$9.76$ Msps	$49.55$ Msps	≈$5.08\times$
[23] II	15	$8.05$ Msps	$49.55$ Msps	≈$6.16\times$
[23] III	15	$7.33$ Msps	$49.55$ Msps	≈$6.67\times$
[23] IV	15	$8.89$ Msps	$49.55$ Msps	≈$5.57\times$
[25] I	7	$4.18\times {10}^2$ Msps	$61.58$ Msps	≈$0.15\times$
[25] II	7	$2.11\times {10}^3$ Msps	$61.58$ Msps	≈$0.03\times$

presents the processing time synthesis results. The second column shows the number of attributes ($N_A$), while the third and fourth columns show the throughput for the reference (THPT^Ref) and this work (THPT^Author), respectively. Meanwhile, the fifth column presents the speedup obtained by this work regarding the reference.

As is evident from Table 7, the proposed architecture obtained satisfactory performance, reaching a speedup from $5\times$ to ${10}^3\times$ regarding the other works. This speedup results from the high degree of parallelism adopted in the proposed architecture, which provides a greater processing speed for the technique. The throughput achieved was inferior only to the architecture proposed in [25].

6.2.3. Dynamic Power Consumption Comparison

Table 8. Dynamic power consumption comparison.

Reference	${\mathit{N}}_{\mathit{A}}$	CLKRef.	${\mathit{N}}_{\mathit{g}}^{\mathbf{Ref}}$	${\mathit{E}}_{\mathit{d}}^{\mathbf{Ref}}$	CLKAuthor	${\mathit{N}}_{\mathit{g}}^{\mathbf{Author}}$	${\mathit{E}}_{\mathit{d}}^{\mathbf{Author}}$	PWRSave
[22] IV	10	$33.46$ MHz	1747	$6.54\times {10}^7$	$0.46$ MHz	$49.40$	$0.48\times {10}^1$	≈$1.36\times {10}^7\times$
[22] V	10	$56.12$ MHz	1004	$1.77\times {10}^8$	$0.77$ MHz	$49.40$	$2.25\times {10}^1$	≈$7.87\times {10}^6\times$
[22] VI	10	$71.70$ MHz	707	$2.61\times {10}^8$	$0.98$ MHz	$49.40$	$4.65\times {10}^1$	≈$5.60\times {10}^6\times$
[23] I	15	$156.18$ MHz	146	$5.56\times {10}^8$	$9.76$ MHz	$67.60$	$6.28\times {10}^4$	≈$8.86\times {10}^3\times$
[23] II	15	$128.78$ MHz	278	$5.94\times {10}^8$	$8.05$ MHz	$67.60$	$3.53\times {10}^3$	≈$1.68\times {10}^5\times$
[23] III	15	$117.26$ MHz	313	$5.05\times {10}^8$	$7.33$ MHz	$67.60$	$2.66\times {10}^4$	≈$1.90\times {10}^4\times$
[23] IV	15	$142.23$ MHz	144	$4.14\times {10}^8$	$8.89$ MHz	$67.60$	$4.75\times {10}^4$	≈$8.72\times {10}^3\times$

presents the synthesis results regarding dynamic power consumption. The second column shows the number of attributes of the reference, $N_A$, while from the third to the fifth columns are shown the clock frequency, CLK^Ref, the $N_g^{\mathrm{Ref}}$, and dynamic power, $E_d^{\mathrm{Ref}}$, respectively, for the reference. Likewise, from the sixth to eighth columns are shown the clock frequency, CLK^Author, the $N_g^{\mathrm{Author}}$, and dynamic power, $E_d^{\mathrm{Author}}$, respectively, for this work. The last column shows the dynamic power saved by this work compared to the reference (PWRSave). To achieve the reference works throughput, the CLK^Author was defined as THPT^Ref. Besides, it is important to mention that the dynamic power consumption values were estimated using Expression (8).

The work [23] did not use multipliers implemented with DSP blocks, and [22] presented multipliers of size 18 × 18 bits. Size equals the multipliers presented in this work, allowing direct comparison of these quantities with both works. As can be observed in Table 8, this proposal presents a lower dynamic power compared to all designs analyzed, reducing the dynamic power between ${10}^3\times$ and ${10}^7\times$. This result was achieved due to the high throughput, low clock frequency, and reduced hardware area occupation.

7. Conclusions

This work presented a hardware implementation of the Naive Bayes technique, developed with a fully parallel architecture and fixed-point representation. Different from the works presented in the literature, the training and inference steps were implemented in parallel and fixed points. The validation was carried out for the training and inference steps using a floating-point software implementation, and the synthesis results were obtained for a Stratix V 5SGXMBBR3H43C3 FPGA. The occupancy, throughput, and power consumption results were compared with other results found in the literature. Compared with other works in the literature, the proposed architecture achieved a speedup of up to ${10}^3\times$, a lower hardware occupancy of up to $35\times$, and reduced dynamic power consumption of up to ${10}^7\times$.