Intuitionistic Fuzzy Deep Neural Network

Abstract

The concept of an intuitionistic fuzzy deep neural network (IFDNN) is introduced here as a demonstration of a combined use of artificial neural networks and intuitionistic fuzzy sets, aiming to benefit from the advantages of both methods. The investigation presents in a methodological way the whole process of IFDNN development, starting with the simplest form—an intuitionistic fuzzy neural network (IFNN) with one layer with single-input neuron, passing through IFNN with one layer with one multi-input neuron, further subsequent complication—an IFNN with one layer with many multi-input neurons, and finally—the true IFDNN with many layers with many multi-input neurons. The formulas for strongly optimistic, optimistic, average, pessimistic and strongly pessimistic formulas for NN parameters estimation, represented in the form of intuitionistic fuzzy pairs, are given here for the first time for each one of the presented IFNNs. To demonstrate its workability, an example of an IFDNN application to biomedical data is here presented.

1. Introduction

The idea for combined use of Artificial Neural Networks (ANNs, see, e.g., [1,2,3,4,5,6,7,8]) and Intuitionistic Fuzzy Sets (IFSs, see, e.g., [9,10]) is to benefit from the advantages of both concepts. ANN [4,5], often called just as NN, is a mathematical or computational model based on biological neural networks. It consists of an interconnected group of artificial neurons that process information using the connectionism approach to computation. IFSs were defined as extensions of Lotfi Zadeh’s fuzzy sets [11].

In [12], fuzzy NN (FNN) is represented as a combination of an ANN and fuzzy logic. FNN are successfully applied for control (see, e.g., [13,14]), pattern recognition (see, e.g., [15]), identification (see, e.g., [16]), mineral exploration (see, e.g., [17]), etc. Up to now, there are a few papers [18,19,20] that combine ANN and IFSs. During the last several years, IFSs have different applications, and in particular, for the introduction of intuitionistic fuzzy NNs (IFNN, see [9,18]).

In [21], a nonlinear network structure known as fuzzy cellular NN of type II is reported by integrating fuzzy operations with the classical cellular NN structure, which is an extension of the cellular non-linear network from classical to fuzzy form. In [22], a multi-layer perceptron NN is proposed consisting of fuzzy flip-flop neurons based on various fuzzy operations applied to approximate a real-life application. In [23,24], a new adaptive fuzzy inference NN has been described. In [25], a way for aggregation of multi-layer NNs is considered.

An extension of Kohonen’s self-organizing map for structure identification in linguistic (fuzzy) system modeling applications is presented in [26]. The authors have shown in particular the granular self-organizing map neural model to induce a distribution of non-parametric fuzzy interval numbers from the data.

In [27,28], the supervised learning process of multi-layer feed-forward NNs is considered as a class of multi-objective, multi-stage optimal control problems. An iterative parametric mini-max method is proposed in which the original optimization problem is embedded into a weighted mini-max formulation. In [29], the supervised learning process is successfully applied to the dynamic programming of multi-layer neural networks learning.

In [30], a max-min intuitionistic fuzzy Hopfield NN is proposed by combining IFSs with Hopfield NNs. The authors have shown that, for any given weight matrix and any given initial intuitionistic fuzzy pattern, the iteration process of the proposed NN converges to a limit cycle.

In [31], intuitionistic fuzzy NNs (IFNNs) are designed to adapt the antecedent and consequent parameters of intuitionistic fuzzy inference systems. A mean of maximum defuzzification method is proposed for a class of Takagi–Sugeno intuitionistic fuzzy inference systems, and the method is compared to the center of area and basic defuzzification distribution operator. On credit scoring data, it is shown that the IFNN trained with gradient descent and Kalman filter algorithms outperforms the traditional adaptive network based fuzzy inference system method.

An IFNN with Gaussian membership function is proposed in [32]. Since the intuitionistic fuzzy pairs (IFPs)—elements of intuitionistic fuzzy logic (IFL, see [33])—consider membership and non-membership values simultaneously, the incorporation of the concept of IFL into a FNN can enhance its performance. A back-propagation learning algorithm is developed to optimize the IFNN parameters and weights. The proposed IFNN is applied to ten tasks, including nonlinear control and prediction problems. The computational results indicate that the proposed IFNN is more efficient than conventional algorithms, such as ANN, FNN, and a support vector regression.

The diversity of medical factors makes the analysis and judgment of uncertainty one of the challenges of medical diagnosis. A well-designed classification and judgment system for medical uncertainty can increase the rate of correct medical diagnosis. A new multi-dimensional classifier is proposed in [34] applying an intelligent algorithm, which is the general fuzzy cerebellar model NN. To obtain more information about uncertainty, an intuitionistic fuzzy linguistic term is employed to describe medical features. The solution of classification is obtained by a similarity measurement. The advantages of the novel classifier proposed are drawn out by comparing the same medical example under the methods of IFSs and intuitionistic fuzzy cross-entropy with different score functions.

Convolutional NNs have been applied on the raw ECG data for detection of life threatening cardiac arrhythmias [35,36]. Dense NNs have been applied on engineering (hand-crafted) features for detection of life threatening cardiac arrhythmias [37,38].

The present paper has been provoked by [39], where an intuitionistic fuzzy feed-forward NN (IFFFNN) was constructed by combining feed-forward NN and IFL. Some operations, as well as two types of transferring functions involved in the IFFFNN working process, were introduced. Here, we intend to define a NN that uses intuitionistic fuzzy information, called an intuitionistic fuzzy deep NN (IFDNN).

The paper presents an application of IFSs tools sequentially on NN with one layer with single-input neuron (Section 3), on NN with one layer with one with multi-input neuron (Section 4), on NN with one layer with many multi-input neurons (Section 5), as well as on NN with many layers with many multi-input neurons in each layer (Section 6). The equations for strongly optimistic, optimistic, average, pessimistic and strongly pessimistic formula are given for each type of NNs. The application of developed here IFDNN is illustrated on biomedical data (Section 7).

2. Definitions of an IFDNN and of an IFP

The formal definition of an IFDNN is as follows:

$$\langle \langle p_1,p_2,\dots ,p_{m_0}\rangle ,\left\{\langle a_1^i,a_2^i,\dots ,a_{m_i}^i\rangle \right|1\le i\le k\},$$

$$\left\{\langle w_{j,1}^i,w_{j,2}^i,\dots ,w_{j,m_{i+1}}^i\rangle \right|1\le j\le m_i,1\le i\le k\},\left\{\langle b_1^i,b_2^i,\dots ,b_{m_i}^i\rangle \right|1\le i\le k\}\rangle ,$$

where

• k is the number of layers;
• $m_i$—number of the neurons on the i-th layer, $1\le i\le k$ ($m_0$ is the number of the neurons in the zeroth (input) layer);
• $p_1,p_2,\dots ,p_{m_0}$—the values for the input neuron; p – for $k=1,m_k=1$;
• $a_1^i,a_2^i,\dots ,a_{m_i}^i$—the output values of the neurons on the i-th layer;
• $w_{j,l}^i$—the weight coefficient from the j-th input neuron to the l-th neuron in layer i ($1\le j\le m_i,1\le l\le m_{i+1},0\le i\le k-1)$;
• $b_1^i,b_2^i,\dots ,b_{m_i}^i$—bias coefficients for the neurons on the i-th layer; b—for $k=1,m_k=1$;
• $F_1^i,F_2^i,\dots ,F_{m_i}^i$—transfer functions for the neurons on the i-th layer; F—for $k=1$, $m_k=1$.

The ordered pair $\langle a,b\rangle$, where $a,b,a+b\in [0,1]$ is called an IFP. In it, a and b represent the degrees of validity (membership, etc.) and of non-validity (non-membership, etc.), respectively. Over two IFPs’ different operations, relations and operators are defined (see, e.g., [33]). The operations, which are bases of the above ones, are the following:

$$\begin{array}{c}\langle a,b\rangle +\langle c,d\rangle =\langle a+c-ac,bd\rangle ,\hfill \\ \langle a,b\rangle \vee \langle c,d\rangle =\langle max(a,c),min(b,d)\rangle ,\hfill \\ \langle a,b\rangle @\langle c,d\rangle =\langle \frac{a+c}{2},\frac{b+d}{2}\rangle ,\hfill \\ \langle a,b\rangle \wedge \langle c,d\rangle =\langle min(a,c),max(b,d)\rangle ,\hfill \\ \langle a,b\rangle .\langle c,d\rangle =\langle ac,b+d-bd\rangle .\hfill \end{array}$$

3. An IFNN with One Layer with Single-Input Neuron

We start with the simplest case—an IFNN with one layer with single-input neuron, as shown in Figure 1. The input value p for the IFNN is represented by IFP.

The intuitionistic fuzzy weight w of the neuron that has the value $\langle {\mu }_w,{\nu }_w\rangle$ is multiplied by the input value $p=\langle {\mu }_p,{\nu }_p\rangle$ by one of the following formulas:

• Strongly optimistic formula:

  $$wp=\langle {\mu }_{wp},{\nu }_{wp}\rangle =\langle 1-(1-{\mu }_w)(1-{\mu }_p),{\nu }_w{\nu }_p\rangle ;$$
• Optimistic formula:

  $$wp=\langle {\mu }_{wp},{\nu }_{wp}\rangle =\langle max({\mu }_w,{\mu }_p),min({\nu }_w,{\nu }_p)\rangle ;$$
• Average formula:

  $$wp=\langle {\mu }_{wp},{\nu }_{wp}\rangle =\langle \frac{{\mu }_w+{\mu }_p}{2},\frac{{\nu }_w+{\nu }_p}{2}\rangle ;$$
• Pessimistic formula:

  $$wp=\langle {\mu }_{wp},{\nu }_{wp}\rangle =\langle min({\mu }_w,{\mu }_p),max({\nu }_w,{\nu }_p)\rangle ;$$
• Strongly pessimistic formula:

  $$wp=\langle {\mu }_{wp},{\nu }_{wp}\rangle =\langle {\mu }_w{\mu }_p,1-(1-{\nu }_w)(1-{\nu }_p)\rangle .$$

The reason for these names we find in the relations

$$1-(1-{\mu }_w)(1-{\mu }_p)\ge max({\mu }_w,{\mu }_p)\ge \frac{{\mu }_w+{\mu }_p}{2}\ge min({\mu }_w,{\mu }_p)\ge {\mu }_w{\mu }_p$$

and

$${\nu }_w{\nu }_p\le min({\nu }_w,{\nu }_p)\le \frac{{\nu }_w+{\nu }_p}{2}\le max({\nu }_w,{\nu }_p)\le 1-(1-{\nu }_w)(1-{\nu }_p).$$

For example, let

$$\langle {\mu }_w,{\nu }_w\rangle =\langle 0.3,0.5\rangle ,$$

$$\langle {\mu }_p,{\nu }_p\rangle =\langle 0.6,0.2\rangle ,$$

$$\langle {\mu }_b,{\nu }_b\rangle =\langle 0.4,0.3\rangle .$$

Then, the values for strongly optimistic, optimistic, average, pessimistic and strongly pessimistic formulas are:

$$wp=\langle 0.832,0.03\rangle ,$$

$$wp=\langle 0.7,0,2\rangle ,$$

$$wp=\langle 0.56\left(6\right),0.3\left(3\right)\rangle ,$$

$$wp=\langle 0.4,0.5\rangle ,$$

$$wp=\langle 0.168,0.72\rangle ,$$

respectively.

The next groups of formula might be illustrated in the same manner.

In general, if $\phi ,\psi :[0,1]\times [0,1]\to [0,1]$ are two fixed functions, so that, for every $x,y\in [0,1]$

$$0\le \phi (x,y)+\psi (x,y)\le 1,$$

then n may have the form

$$n=\langle \phi ({\mu }_w,{\mu }_p),\psi ({\nu }_w,{\nu }_p)\rangle .$$

The same is valid for the next formulas.

The result achieved by one of the above formulas enters the summator (summation function, weighted sum) $\Sigma$. The other element that is passed to the summator is the IF-bias b represented by the IFP $\langle {\mu }_b,{\nu }_b\rangle$, where ${\mu }_b$ and ${\nu }_b$ are real numbers from the interval $[0,1]$. The obtained result is presented again in one of the aforementioned five forms, namely:

• Strongly optimistic formula:

  $$n=\langle {\mu }_n,{\nu }_n\rangle =\langle 1-(1-{\mu }_{wp})(1-{\mu }_b),{\nu }_{wp}{\nu }_b\rangle$$

  $$=\langle 1-(1-{\mu }_w)(1-{\mu }_p)(1-{\mu }_b),{\nu }_w{\nu }_p{\nu }_b\rangle ;$$
• Optimistic formula:

  $$n=\langle {\mu }_n,{\nu }_n\rangle =\langle max({\mu }_{wp},{\mu }_b),min({\nu }_{wp},{\nu }_b)\rangle$$

  $$=\langle max({\mu }_w,{\mu }_p,{\mu }_b),min({\nu }_w,{\nu }_p,{\nu }_b)\rangle ;$$
• Average formula:

  $$n=\langle {\mu }_n,{\nu }_n\rangle =\langle \frac{{\mu }_{wp}+{\mu }_b}{2},\frac{{\nu }_{wp}+{\nu }_b}{2}\rangle$$

  $$=\langle \frac{{\mu }_w+{\mu }_p+2{\mu }_b}{4},\frac{{\nu }_w+{\nu }_p+2{\nu }_b}{4}\rangle ;$$
• Pessimistic formula:

  $$n=\langle {\mu }_n,{\nu }_n\rangle =\langle min({\mu }_{wp},{\mu }_b),max({\nu }_{wp},{\nu }_b)\rangle$$

  $$=\langle min({\mu }_w,{\mu }_p,{\mu }_b),max({\nu }_w,{\nu }_p,{\nu }_b)\rangle ;$$
• Strongly pessimistic formula:

  $$n=\langle {\mu }_n,{\nu }_n\rangle =\langle {\mu }_{wp}{\mu }_b,1-(1-{\nu }_{wp})(1-{\nu }_b)\rangle$$

  $$=\langle {\mu }_w{\mu }_p{\mu }_b,1-(1-{\nu }_w)(1-{\mu }_p)(1-{\nu }_b)\rangle .$$

It is worth noting that it is possible for the last IFP to be calculated by one formula, while the IFP $\langle {\mu }_p,{\nu }_p\rangle$ by another. If the IFP $\langle {\mu }_p,{\nu }_p\rangle$ is obtained by the first average formula, then the last average formula may have the form

$$n=〈{\mu }_n,{\nu }_n〉=〈\frac{{\mu }_w+{\mu }_p+{\mu }_b}{3},\frac{{\nu }_w+{\nu }_p+{\nu }_b}{3}〉.$$

The summator output n acts as an input for the transfer function F, which produces the neuron’s output a, calculated by:

$$a=F(wp+b).$$

In the classical feed-forward NN, two types of transfer functions with the form $a=F\left(n\right)$ can be used: linear and logical sigmoidal ones.

The output value of the linear transfer function is equal to

$$a=F\left(n\right)=\langle {\mu }_n,{\nu }_n\rangle .$$

The output value of the logical sigmoidal transfer function is equal to the output in the interval $[0,1]$ according to the expression:

$$a=\frac{1}{1+e^{-n}}.$$

Now, we construct the pair

$$F_{sigm}=〈\frac{2}{3(1+\frac{1}{1+e^{{\mu }_n}})},\frac{2}{3(1+\frac{1}{1+e^{{\nu }_n}})}〉.$$

Proposition 1.

$F_{sigm}$ is an IFP.

Proof. 

We check directly that

$$\frac{2}{3(1+\frac{1}{1+e^{{\mu }_n}})},\frac{2}{3(1+\frac{1}{1+e^{{\nu }_n}})}\in [0,1],$$

and

$$\frac{2}{3(1+\frac{1}{1+e^{{\mu }_n}})}+\frac{2}{3(1+\frac{1}{1+e^{{\nu }_n}})}=\frac{2}{3}\left(\frac{1}{1+\frac{1}{1+e^{{\mu }_n}}}+\frac{1}{1+\frac{1}{1+e^{{\nu }_n}}}\right)$$

$$=\frac{2}{3}\left(\frac{1+e^{{\mu }_n}}{2+e^{{\mu }_n}}+\frac{1+e^{{\nu }_n}}{2+e^{{\nu }_n}}\right)=\frac{2}{3}\left(\frac{4+3e^{{\mu }_n}+3e^{{\nu }_n}+2e^{{\mu }_n}{e}^{{\nu }_n}}{4+2e^{{\mu }_n}+2e^{{\nu }_n}+e^{{\mu }_n}{e}^{{\nu }_n}}\right).$$

From

$$12+6e^{{\mu }_n}+6e^{{\nu }_n}+3e^{{\mu }_n}{e}^{{\nu }_n}-8-6e^{{\mu }_n}-6e^{{\nu }_n}-4e^{{\mu }_n}{e}^{{\nu }_n}=4-e^{{\mu }_n+{\nu }_n}>0$$

the validity of the assertion follows. □

4. An IFNN with One Layer with One Multi-Input Neuron

Let us have a neuron (see Figure 2) that have as input values a vector of IFPs $\langle {\mu }_{p_1},{\nu }_{p_1}\rangle$, $\langle {\mu }_{p_2},{\nu }_{p_2}\rangle$, …, $\langle {\mu }_{p_{m_0}},{\nu }_{p_{m_0}}\rangle$, a vector of weight coefficients with IFPs $\langle {\mu }_{w_{1,1}},{\nu }_{w_{1,1}}\rangle$, $\langle {\mu }_{w_{1,2}},{\nu }_{w_{1,2}}\rangle$, …, $\langle {\mu }_{w_{1,m_0}},{\nu }_{w_{1,m_0}}\rangle$, and IFP $\langle {\mu }_{b_1},{\nu }_{b_1}\rangle$ for the bias coefficient $b_1$.

Then, by analogy with Section 3, we can use, e.g., one of the following five formulas:

• Strongly optimistic formula:

  $$n_1=〈1-\left(\prod _{j=1}^{m_0}(1-{\mu }_{w_{1,j}})(1-{\mu }_{p_j})\right)(1-{\mu }_{b_1}),\left(\prod _{j=1}^{m_0}{\nu }_{w_{1,j}}{\nu }_{p_j}\right){\nu }_{b_1}〉;$$
• Optimistic formula:

  $$n_1=〈max\left(\underset{1\le j\le m_0}{max}({\mu }_{w_{1,j}},{\mu }_{p_j}),{\mu }_{b_1}\right),min\left(\underset{1\le j\le m_0}{min}({\nu }_{w_{1,j}},{\nu }_{p_j}),{\nu }_{b_1}\right)〉;$$
• Average formula:

  $$n_1=〈\frac{1}{2m_0}\left(m_0{\mu }_{b_1}+\sum _{j=1}^{m_0}({\mu }_{w_{1,j}}+{\mu }_{p_j})\right),\frac{1}{2m_0}\left(m_0{\nu }_{b_1}+\sum _{j=1}^{m_0}({\nu }_{w_{1,j}}+{\nu }_{p_j})\right)〉$$

  or

  $$n_1=〈\frac{1}{m_0+1}\left({\mu }_{b_1}+\sum _{j=1}^{m_0}({\mu }_{w_{1,j}}+{\mu }_{p_j})\right),\frac{1}{m_0+1}\left({\nu }_{b_1}+\sum _{j=1}^{m_0}({\nu }_{w_{1,j}}+{\nu }_{p_j})\right)〉;$$
• Pessimistic formula:

  $$n_1=〈min\left(\underset{1\le j\le m_0}{min}({\mu }_{w_{1,j}},{\mu }_{p_j}),{\mu }_{b_1}\right),max\left(\underset{1\le j\le m_0}{max}({\nu }_{w_{1,j}},{\nu }_{p_j}),{\nu }_{b_1}\right)〉;$$
• Strongly pessimistic formula:

  $$n_1=〈\left(\prod _{j=1}^{m_0}{\mu }_{w_{1,j}}{\mu }_{p_j}\right){\mu }_{b_1},1-\left(\prod _{j=1}^{m_0}(1-{\nu }_{w_{1,j}})(1-{\nu }_{p_j})\right)(1-{\nu }_{b_1})〉.$$

5. An IFNN with One Layer with Many Multi-Input Neurons

Let us have $m_0$ neurons in the input layer (see Figure 3) that have as input values the IFPs $\langle {\mu }_{p_1},{\nu }_{p_1}\rangle$, $\langle {\mu }_{p_2},{\nu }_{p_2}\rangle$, …, $\langle {\mu }_{p_{m_0}},{\nu }_{p_{m_0}}\rangle$, weight coefficients with values the IFPs $\langle {\mu }_{w_{1,1}},{\nu }_{w_{1,1}}\rangle$, $\langle {\mu }_{w_{1,2}},{\nu }_{w_{1,2}}\rangle$, …, $\langle {\mu }_{w_{1,m_1}},{\nu }_{w_{1,m_1}}\rangle$, $\langle {\mu }_{w_{2,1}},{\nu }_{w_{2,1}}\rangle$…, $\langle {\mu }_{w_{m_0,1}},{\nu }_{w_{m_0,1}}\rangle$…, $\langle {\mu }_{w_{m_0,m_1}},$${\nu }_{w_{m_0,m_1}}\rangle$, and bias coefficients $\langle {\mu }_{b_1},{\nu }_{b_1}\rangle$, $\langle {\mu }_{b_2},{\nu }_{b_2}\rangle$, …, $\langle {\mu }_{b_{m_1}},{\nu }_{b_{m_1}}\rangle$.

When distinct neurons are operating, one cannot expect a high success rate in recognizing individual images, patterns, and behaviors. Therefore, when distinct neurons are used in NNs, they are grouped together with a uniform connectivity between neurons called a layer. Typically, the same transfer functions are used for all neurons in a layer, but this is not a dogma. Using the same function is a prerequisite for the layer behavior to be uniform and for computations to be more easily performed.

By analogy with Section 3 and Section 4, we can use, e.g., one of the following five formulas for the calculation of the value of $n_l$ for $1\le l\le m_1$:

• Strongly optimistic formula:

  $$n_l=〈1-\left(\prod _{j=1}^{m_0}(1-{\mu }_{w_{j,l}})(1-{\mu }_{p_j})\right)(1-{\mu }_{b_l}),\left(\prod _{j=1}^{m_0}{\nu }_{w_{j,l}}{\nu }_{p_j}\right){\nu }_{b_l}〉;$$
• Optimistic formula:

  $$n_l=〈max\left(\underset{1\le j\le m_0}{max}({\mu }_{w_{j,l}},{\mu }_{p_j}),{\mu }_{b_l}\right),min\left(\underset{1\le j\le m_0}{min}({\nu }_{w_{j,l}},{\nu }_{p_j}),{\nu }_{b_l}\right)〉;$$
• Average formula:

  $$n_l=〈\frac{1}{2m_0}\left(m_0{\mu }_{b_l}+\sum _{j=1}^{m_0}({\mu }_{w_{j,l}}+{\mu }_{p_j})\right),\frac{1}{2m_0}\left(m_0{\nu }_{b_l}+\sum _{j=1}^{m_0}({\nu }_{w_{j,l}}+{\nu }_{p_j})\right)〉$$

  or

  $$n_l=〈\frac{1}{m_0+1}\left({\mu }_{b_l}+\sum _{j=1}^{m_0}({\mu }_{w_{j,l}}+{\mu }_{p_j})\right),\frac{1}{m_0+1}\left({\nu }_{b_l}+\sum _{j=1}^{m_0}({\nu }_{w_{j,l}}+{\nu }_{p_j})\right)〉;$$
• Pessimistic formula:

  $$n_l=〈min\left(\underset{1\le j\le m_0}{min}({\mu }_{w_{j,l}},{\mu }_{p_j}),{\mu }_{b_l}\right),max\left(\underset{1\le j\le m_0}{max}({\nu }_{w_{j,l}},{\nu }_{p_j}),{\nu }_{b_l}\right)〉;$$
• Strongly pessimistic formula:

  $$n_l=〈\left(\prod _{j=1}^{m_0}{\mu }_{w_{j,l}}{\mu }_{p_j}\right){\mu }_{b_l},1-\left(\prod _{j=1}^{m_0}(1-{\nu }_{w_{j,l}})(1-{\nu }_{p_j})\right)(1-{\nu }_{b_l})〉.$$

6. An IFDNN with Many Layers with Many Multi-Input Neurons

Finally, we discuss the most general case, in which we have k layers, so that the net becomes an IFDNN.

Let the first layer coincide with the layer, described in Section 5, and let the next layers have similar forms as the first one, so that each component of these layers to be numbered by a upper index i ($1\le i\le k$) is as it is shown in Figure 4.

Let us have $m_0$ neurons in the input layer (see Figure 3) that have as input values the IFPs $\langle {\mu }_{p_1},{\nu }_{p_1}\rangle$, $\langle {\mu }_{p_2},{\nu }_{p_2}\rangle$, …, $\langle {\mu }_{p_{m_0}},{\nu }_{p_{m_0}}\rangle$, weight coefficients with values the IFPs $\langle {\mu }_{w_{1,1}},{\nu }_{w_{1,1}}\rangle$, $\langle {\mu }_{w_{1,2}},{\nu }_{w_{1,2}}\rangle$, …, $\langle {\mu }_{w_{1,m_1}},{\nu }_{w_{1,m_1}}\rangle$, $\langle {\mu }_{w_{2,1}},{\nu }_{w_{2,1}}\rangle$, …, $\langle {\mu }_{w_{m_0,1}},{\nu }_{w_{m_0,1}}\rangle$…, $\langle {\mu }_{w_{m_0,m_1}},{\nu }_{w_{m_0,m_1}}\rangle$, and bias coefficients $\langle {\mu }_{b_1},{\nu }_{b_1}\rangle$, $\langle {\mu }_{b_2},{\nu }_{b_2}\rangle$, …, $\langle {\mu }_{b_{m_0}},{\nu }_{b_{m_0}}\rangle$.

To achieve better results, the connection of several neural layers into a common NN is used, where the outputs of each neural layer are connected as inputs of the next layer. In this structure, there are three clear parts: an input neural layer that serves to perceive external signals; an output layer which is adapted to be able to obtain the exact output values; and a hidden layer (or layers), which is the “real“ part of the NN that defines the behavior of whole system. If the number of hidden neural layers is greater than 3, it is called to be a deep NN and to use deep learning.

By analogy with Section 3, Section 4 and Section 5, we can use, e.g., one of the following five formulas for the calculation of value of $n_l^i$ for $1\le l\le m_i$, $1\le l\le k$:

• Strongly optimistic formula:

  $$n_l^i=〈1-\left(\prod _{j=1}^{m_i}(1-{\mu }_{w_{j,l}^i})(1-{\mu }_{p_j^i})\right)(1-{\mu }_{b_l^i}),\left(\prod _{j=1}^{m_i}{\nu }_{w_{j,l}^i}{\nu }_{p_j^i}\right){\nu }_{b_l^i}〉;$$
• Optimistic formula:

  $$n_l^i=〈max\left(\underset{1\le j\le m_i}{max}({\mu }_{w_{j,l}^i},{\mu }_{p_j^i}),{\mu }_{b_l^i}\right),min\left(\underset{1\le j\le m_i}{min}({\nu }_{w_{j,l}^i},{\nu }_{p_j^i}),{\nu }_{b_l^i}\right)〉;$$
• Average formula:

  $$n_l^i=〈\frac{1}{2m_i}\left(m_i{\mu }_{b_l^i}+\sum _{j=1}^{m_i}({\mu }_{w_{j,l}^i}+{\mu }_{p_j^i})\right),\frac{1}{2m_i}\left(m_i{\nu }_{b_l^i}+\sum _{j=1}^{m_i}({\nu }_{w_{j,l}^i}+{\nu }_{p_j^i})\right)〉$$

  or

  $$n_l^i=〈\frac{1}{m_i+1}\left({\mu }_{b_l^i}+\sum _{j=1}^{m_i}({\mu }_{w_{j,l}^i}+{\mu }_{p_j^i})\right),\frac{1}{m_i+1}\left({\nu }_{b_l^i}+\sum _{j=1}^{m_i}({\nu }_{w_{j,l}^i}+{\nu }_{p_j^i})\right)〉;$$
• Pessimistic formula:

  $$n_l^i=〈min\left(\underset{1\le j\le m_i}{min}({\mu }_{w_{j,l}^i},{\mu }_{p_j^i}),{\mu }_{b_l^i}\right),min\left(\underset{1\le j\le m_i}{max}({\nu }_{w_{j,l}^i},{\nu }_{p_j^i}),{\nu }_{b_l^i}\right)〉;$$
• Strongly pessimistic formula:

  $$n_l^i=〈\left(\prod _{j=1}^{m_i}{\mu }_{w_{j,l}^i}{\mu }_{p_j^i}\right){\mu }_{b_l^i},1-\left(\prod _{j=1}^{m_i}(1-{\nu }_{w_{j,l}^i})(1-{\nu }_{p_j^i})\right)(1-{\nu }_{b_l^i})〉.$$

The training of all presented here different types of IFNNs is performed by a standard NN algorithm. It is based on the root mean square error obtained at the NN output. The error back-propagation algorithm distributes the individual sensitivity coefficients among the layers of NN.

As shown above, the investigation describes different types of assessments that correspond to the strength of signals in an abstract NN. Insofar as it is a model of a real set of neurons, these formulas can also be used to estimate boundaries (upper and lower—optimistic and pessimistic) in which the signal strength falls when flowing through the real neurons. When a neuron sends to the next one a signal but with lower strength, it might not continue its propagation, i.e., it might die out. If the (strong) optimistic assessment of the signal strength is below any preset constant, this will correspond to the fading case. On the other hand, if the (strong) pessimistic assessment for the signal is above any preset constant, this could be interpreted in one of the following two ways:

• The signal is strong enough to reach its final goal;
• The signal is so strong that it results in the blocking of neurons system (e.g., shock from severe pain).

7. An Example for an IFDNN Application to Biomedical Data

To illustrate the IFDNN work, we will use data from one of our experiments to control a robotic arm with IFPs. Applying them, the movement of a stepper motor is controlled to determine the position of the hand based on signals about the density of the muscle that should ensure movement.

In the considered case, a signal from MyoWare muscle sensor (AT-04-001) is fed to the NN input. The MyoWare sensor signal is of the Envelope Output (ENV) type. As the NN output, a voltage is obtained to control a selected motor of a robotic manipulator. Based on these signals, IFPs $\langle \mu ,\nu \rangle$ are calculated [40].

The structure of the NN with one intuitionistic fuzzy input, with eight neurons in the first, second and third hidden layers, respectively, and one intuitionistic fuzzy output is presented in Figure 5.

The training set contains 1428 values, which is divided into three parts. The first part is the largest and represents 70% of all data. The second part is the testing data set, containing 20% of all data. The last part is the validation data set. This is the so-called external data and do not participate in the NN training. They are 10% of all data. It is important to say that the distribution of training/testing/validation data is randomized by the system.

The training process of the neural network and the error distribution is shown in Figure 6.

Figure 7 presents the correlation coefficient (R) for the training, testing and validation data sets, as well as for the whole data set (the right bottom subfigure).

In order to validate the training, the target values from the training sets of the neural network are compared with all values $\mu$ and $\nu$ obtained at the outputs. During the testing all 1263 intuitionistic fuzzy pairs, we obtained the following result: arithmetic mean deviation of 0.002067 for $\mu$ and 0.00214 for $\nu$.

In the future work, the IFDNN developed here is going to be applied to studies performed in [41], where data from three groups of proximal humerus fracture (PHF) patients were analyzed: group 1 with 63 patients without augmentation, group 2 with 28 patients with bone graft augmentation, and group 3 with 29 patients with cement augmentation. IFDNN is expected to predict the recovery process from PHFx to the respective patient. They reflect the degree of a successful range of motion $\mu$, or a non-successful range of motion $\nu$ for each patient. The degree of uncertainty $\pi$ = 1 −$\mu$−$\nu$ will represent the cases in which the treatment procedure has not been completed or the information for the current patient is not complete.

The IFDNN developed here can also be used to predict with a better possibility the choice of the most suitable surgical approach and implants, to reduce the operative time, to obtain less blood loss and lower X-ray radiation. In [42], patients with complex acetabulum fractures were divided into two groups: (1) conventional group (n = 12); and (2) 3D printed group (n = 10). Both groups included participants with a posterior column fracture plus a posterior wall fracture, a transverse fracture plus a posterior wall fracture, or a fracture of both acetabular columns. The scan datasets are segmented and converted to stereo lithography (STL) format, with bones and fragments separated for 3D printing in different colors. The comparison between the two groups was performed in terms of quality of fracture reduction—equal to or less than 2 mm displacement and greater than 2 mm displacement, functional assessment, operative time, blood loss and number of intraoperative X-rays.

8. Conclusions

The investigation presented here methodologically demonstrates the whole process of IFDNN elaboration, starting with IFNN one layer with a single-input neuron, passing through the IFNN with one layer with multi-input neurons, IFNN with one layer with many multi-input neurons, and reaching the idea of a set of layers with multi-input neurons in neural networks—the so-called deep learning. For the first time and for each type of the IFNN presented here, strongly optimistic, optimistic, average, pessimistic and strongly pessimistic formulas of the output n of the summator are presented. The IFDNN developed here has been shown as successfully applied to biomedical data. IFDNN might be useful to other scientific applications, for which the intuitionistic fuzzy evaluations of the considered data should exist.