# Energy and Entropy Measures of Fuzzy Relations for Data Analysis

^{1}

^{2}

^{*}

## Abstract

**:**

_{1}with greatest energy provides information about the greatest strength of the input-output, and the fuzzy relation R

_{2}with the smallest entropy provides information about uncertainty of the input-output relationship. We consider a new index of the fuzziness of the input-output based on R

_{1}and R

_{2}. In our method, this index is calculated for each pair of input and output fuzzy sets in a fuzzy rule. A threshold value is set in order to choose the most relevant fuzzy rules with respect to the data.

## 1. Introduction

_{1}, …, x

_{m}} be a finite set and A be a fuzzy set of X. In [1,2] two categories of fuzziness, measures are defined as energy and entropy (see, e.g., also [3]). The energy measure of the fuzziness of A is given by:

_{1}, …, y

_{n}}, and a fuzzy relation R defined by X × Y:

_{1}is defined as ${\mathrm{R}}_{1}(\mathrm{x},\mathrm{y})=\mathrm{A}(\mathrm{x})\mathsf{\tau}\mathrm{B}(\mathrm{y})$, where $\mathsf{\tau}:[0,1]\times [0,1]\to [0,1]$ is given:

_{1}is the fuzzy relation having the maximum energy E. Furthermore, in [4,5] the authors propose an algorithm for finding the relation R

_{2}, solution of (5) not unique, having the minimum entropy H.

_{1}and R

_{2}of the Equation (5) with the Yager t-norm. In Section 3, our algorithm is presented for evaluating the strength of fuzzy rules with respect to the data. In Section 4, we present the results of two experiments in which we apply our algorithm. Final considerations are shown in Section 5.

## 2. Algorithm for Calculating Fuzzy Relations Having the Greatest Energy and Smallest Entropy

_{1}, …, x

_{m}}, Y = {y

_{1}, …, y

_{n}}, A (resp., B) be a fuzzy set on X (resp., Y). In [4,5] it is proven that R

_{1}is the solution of the Equation (5) with maximum energy. For the calculus of R

_{2}, the following algorithm is developed in [4,5]. Let h be defined as in Section 1. For each y

_{j}∈ Y, we consider Γ(y

_{j}) = {x

_{i}∈ X: A(x

_{i}) ≥ B(y

_{j})}. If B(y

_{j}) > 0, the algorithm finds some x

_{c}∈ Γ(y

_{j}) (generally not unique), such that A(x

_{c})τB(y

_{j}) is not zero and h(A(x

_{c})τB(y

_{j})) assumes the minimum value. Then, R

_{2}(x

_{i},y

_{j}) = A(x

_{i})τB(y

_{j}) if x

_{i}= x

_{c}and R

_{2}(x

_{i}, y

_{j}) = 0 if x

_{i}≠ x

_{c}. If B(y

_{j}) = 0, R

_{2}(x

_{i}, y

_{j}) = 0 for each i = 1, …, m. Below, we show the pseudocodes for calculating R

_{1}(Algorithm 1) and R

_{2}(Algorithm 2).

Algorithm 1 Calculate R_{1} | ||

Description: | Calculate the matrix R_{1} | |

Input: | X, Y, A, B | |

Output: | R_{1} | |

1 | FOR j = 1 TO n | |

2 | { | |

3 | FOR i = 1 TO m | |

4 | { | |

5 | R_{1}(x_{i},y_{j}): = A(x_{i}) τB(y_{j}); | |

6 | } | |

7 | } | |

8 | END |

Algorithm 2 Calculate R_{2} | ||

Description: | Calculate the matrix R_{2} | |

Input: | X, Y, A, B | |

Output: | R_{2} | |

1 | FOR j = 1 TO n | |

2 | { | |

3 | IF B(y_{j})>0 | |

4 | { | |

5 | xc: = 0: | |

6 | hmin: = 1; | |

7 | FOR each x in Γ(y_{j}) | |

8 | { | |

9 | IF h(A(x), B(y_{j})) < hmin THEN | |

10 | { | |

11 | hmin: = h(A(x), B(yj)); | |

12 | xc: = x; | |

13 | } | |

14 | } | |

15 | FOR i = 1 TO m | |

16 | { | |

17 | IF (x_{i} = xc) | |

18 | R_{2}(x_{i},y_{j}): = A(x_{i}) τB(_{yj}) ; | |

19 | ELSE | |

20 | R_{2}(x_{i},y_{j}):= 0; | |

21 | } | |

22 | } | |

23 | ELSE | |

24 | { | |

25 | FOR i = 1 TO m | |

26 | R_{2}(x_{i},y_{j}):= 0; | |

27 | } | |

28 | } | |

29 | END |

_{1}, x

_{2}, x

_{3}, x

_{4}}, Y = {y

_{1}, y

_{2}, y

_{3}, y

_{4}}, A = (0.2, 0.3, 0.5, 0.8) and B = (0.4, 0.0, 0.6, 0.7). For p = 2 in Formula (6), we obtain that

_{2}, we have Γ(y

_{1}) = {x

_{3}, x

_{4}}, Γ(y

_{3}) = {x

_{4}}, Γ(y

_{4}) = {x

_{4}} and hence R

_{2}(x

_{3}, y

_{1}) = 0.67, R

_{2}(x

_{4}, y

_{3}) = 0.65 and R

_{2}(x

_{4}, y

_{4}) = 0.78. For B(y

_{2}) = 0, we have that R

_{2}(x

_{i}, y

_{2}) = 0 for each i = 1, …, 4. Then, the fuzzy relation with minimum entropy is given by:

## 3. Evaluating the Strength of the Fuzzy Rules with Respect to the Data

_{1}, …, A

_{q}} of the universe of the discourse U

_{x}of the input variable x, and a fuzzy partition of s fuzzy sets {B

_{1}, …, B

_{s}} of the universe of the discourse U

_{y}of the output variable y. Subsequently, he defines a set of fuzzy rules relating the input and the output variables in the following form:

_{w}Then y is B

_{z}, w = 1, …, q, z = 1, …, s

_{1}, …, x

_{m}}, and a dataset composed by n measures of the output variable y, Y = {y

_{1}, …, y

_{n}}. For each rule we extract the pair (A

_{w},B

_{z}) formed by the input and the output fuzzy sets in (7), and we calculate a normalized index based on the maximum energy and minimum entropy. The index represents the strength of the kth fuzzy rule with respect to the data. Let R be the fuzzy automaton (relation) connecting A

_{w}and B

_{z}by means of Equation (5) with the Yager t-norm. Let R

_{1wz}and R

_{2wz}serve as the solutions of (5), with maximum energy and minimum entropy calculated using the algorithms of Section 2. The index of strength for the pair (A

_{w},B

_{z}) is defined [4] as:

_{wz}= 1, we obtain E(R

_{1wz}) = n·m and H(R

_{2wz}) = 0. If I

_{wz}is greater or equal to a pre-defined threshold, then the fuzzy rule is confirmed by the data. In Figure 1, this process is schematized.

_{x}and U

_{y}and creates the fuzzy rule set. Then, the expert analyzes each fuzzy rule with respect to a set of data. For the input-output pair (A

_{w},B

_{z}), A

_{w}(x

_{1}), …, A

_{w}(x

_{m}), B

_{z}(y

_{1}), …, B

_{z}(y

_{n}), the fuzzy relations R

_{1}and R

_{2}, the Energy E, the Entropy H, and the index I are calculated. If the index I is greater or equal to a prefixed threshold, then the rule is considered to be significant to the fuzzy rule set with respect to the input/output data. We can generalize this model to the case in which two or more input variables are considered. The generalized form of a fuzzy rule is given by the form:

Algorithm 3 Energy-Entropy fuzzy rules evaluation | ||

Description: | Calculate the matrix R_{2} | |

Input: | X, Y, A, B | |

Output: | R_{2} | |

1 | SET I_{th} // set the threshold value | |

2 | FOR k = 1 TO D // for all the D fuzzy rules in the dataset | |

2 | { | |

3 | Imin: = 2; // Imin is initialized to a value greater than 1 | |

4 | Create the fuzzy subsets B_{z}(y_{1}),…, B_{z}(y_{n}); | |

5 | FOR l = 1 to v | |

6 | { | |

7 | Create the fuzzy subsets A^{(l)}_{wl}(x_{1}),…, A^{(l)}_{wl}(x_{m}); | |

8 | Calculate R_{1} and R_{2}; | |

9 | Calculate E and H; | |

10 | Calculate I; | |

11 | IF (I < Imin) | |

12 | Imin = I; | |

13 | } | |

14 | IF (Imin ≥ I_{th}) | |

15 | Annotate the k-th fuzzy rule as significant; | |

16 | } | |

17 | END |

_{th}can be settled by the expert by using an opportune calibration. This calibration can be obtained by testing the algorithm applied on a sample dataset for which the expert can evaluate the strength of fuzzy rules with respect to the data. In Section 4, we present some results obtained by using various datasets. The first experiment is used for calibrating the threshold value I

_{th}. Obviously the computational time is polynomial, being given by O(n·m·v).

## 4. Test Results

_{1},a

_{2},a

_{3}) and B = (b

_{1},b

_{2},b

_{3}). In Table 3 we show the four fuzzy sets forming the fuzzy partition of the domain U

_{x}.

_{y}.

_{x}and U

_{y}, respectively.

- Rule 1 → IF A= low THEN B = very low
- Rule 2 → IF A = adequate THEN B = mean
- Rule 3 → IF A = fair THEN B = high

_{th}= 0.7 in all the experiments.

_{1}= Percentage of families in residential properties with respect to the total resident families and x

_{2}= Percentage of graduates with respect to the total workforce. The output is y = Unemployment rate.

_{x1}, U

_{x2}, U

_{y}, respectively.

_{x1}, U

_{x2}, U

_{y}, respectively.

- Rule 1 →IF A
_{1}= very low AND A_{2}= low THEN B = very high - Rule 2 → IF A
_{1}= low AND A_{2}= low THEN B = high - Rule 3 → IF A
_{1}= mean AND A_{2}= adequate THEN B = mean - Rule 4 → IF A
_{1}= mean AND A_{2}= fair THEN B = mean - Rule 5 → IF A
_{1}= mean AND A_{2}= high THEN B = low - Rule 6 → IF A
_{1}= high AND A_{2}= fair THEN B = low - Rule 7 → IF A
_{1}= high AND A_{2}= high THEN B = very low - Rule 8 → IF A
_{1}= very high AND A_{2}= high THEN B = very low

_{th}= 0.7, except for the fuzzy rules 1 and 2.

## 5. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- De Luca, A.; Termini, S. Entropy and energy measures of fuzzy sets. In Advances in Fuzzy Set Theory and Applications; Gupta, M.M., Ragade, R.K., Yager, R.R., Eds.; North-Holland: Amsterdam, The Netherlands, 1979; pp. 321–338. [Google Scholar]
- De Luca, A.; Termini, S. A definition of non-probabilistic entropy in the setting of fuzzy sets theory. Inf. Control
**1972**, 20, 301–312. [Google Scholar] [CrossRef] - Wang, W.-J.; Chiu, C.-H. Entropy and information energy for fuzzy sets. Fuzzy Sets Syst.
**1999**, 108, 333–339. [Google Scholar] [CrossRef] - Di Nola, A.; Pedrycz, W.; Sessa, S. On measures of fuzziness of solutions of fuzzy relation equations with generalized connectives. J. Math. Anal. Appl.
**1985**, 106, 443–453. [Google Scholar] [CrossRef] - Di Nola, A.; Pedrycz, W.; Sessa, S. Fuzzy relation equations with LSC and USC T-norms and their Boolean solutions. Stochastica
**1987**, 11, 151–183. [Google Scholar] - Pedrycz, W. Fuzzy relational equations with generalized connectives and their applications. Fuzzy Sets Syst.
**1983**, 10, 185–201. [Google Scholar] [CrossRef] - Sanchez, E. Solutions in composite fuzzy relation equations: Application to medical diagnosis in Brouwerian logic. In Fuzzy Automata and Decision Processes; Gupta, M.M., Saridis, G.N., Gaines, B.R., Eds.; North-Holland: Amsterdam, The Netherlands, 1973; pp. 221–234. [Google Scholar]
- Yager, R.R. On general class of fuzzy connectives. Fuzzy Sets Syst.
**1980**, 4, 235–242. [Google Scholar] [CrossRef] - Das, S.; Ghosh, S.; Kar, S.; Pal, M.T. An algorithmic approach for predicting unknown information in incomplete fuzzy soft set. Arab. J. Sci. Eng.
**2017**, 42, 3563–3571. [Google Scholar] [CrossRef] - Lee, H.-M.; Chen, C.-M.; Chen, J.-M.; Jou, Y.-L. An efficient fuzzy classifier with feature selection based on fuzzy entropy. IEEE Trans. Syst. Man Cybern. Part B Cybern.
**2001**, 3, 426–432. [Google Scholar] - Matiaško, K.; Boháčik, J.; Levashenko, V.; Kovalík, S. Learning fuzzy rules from fuzzy decision tree. J. Inf. Control Manag. Syst.
**2006**, 4, 143–154. [Google Scholar] - Zeng, W.; Li, H. Relationship between similarity measure and entropy of interval valued fuzzy sets. Fuzzy Sets Syst.
**2006**, 157, 1477–1484. [Google Scholar] [CrossRef] - Zhang, H.; Zhang, W.; Mei, C. Entropy of interval-valued fuzzy sets based on distance and its relationship with similarity measure. Knowl. Based Syst.
**2009**, 22, 449–454. [Google Scholar] [CrossRef] - Markechová, D.; Riečan, B. Entropy of fuzzy partitions and entropy of fuzzy dynamical systems. Entropy
**2016**, 18, 19. [Google Scholar] [CrossRef] - Markechová, D.; Riečan, B. Logical entropy of fuzzy dynamical systems. Entropy
**2016**, 18, 157. [Google Scholar] [CrossRef] - Barchielli, A.; Gregoratti, M.; Toigo, A. measurement uncertainty relations for position and momentum: Relative entropy formulation. Entropy
**2017**, 19, 301. [Google Scholar] [CrossRef] - Xiao, F. A novel evidence theory and fuzzy preference approach-based multi-sensor data fusion technique for fault diagnosis. Sensors
**2017**, 17, 2504. [Google Scholar] [CrossRef] [PubMed] - Sarwar, M.; Akram, M. Certain algorithms for computing strength of competition in bipolar fuzzy graphs. Int. J. Uncertain. Fuzzy Knowl. Based Syst.
**2017**, 25, 877–896. [Google Scholar] [CrossRef] - Tang, H.-C. Decomposition and intersection of two fuzzy numbers for fuzzy preference relations. Symmetry
**2017**, 9, 228. [Google Scholar] [CrossRef]

Municipality Number | Districts |
---|---|

1 | Chiaia, Posillipo, S.Ferdinando |

2 | Avvocata, Montecalvario, Porto, S.Giuseppe, Pendino, Mercato |

3 | Stella, S.Carlo all’Arena |

4 | Vicaria, S.Lorenzo, Poggioreale |

5 | Vomero, Arenella |

6 | Ponticelli, Barra, S.Giovanni aTeduccio |

7 | Miano, Secondigliano, S.Pietro a Patierno |

8 | Chiaiano, Piscinola-Marianella, Scampia |

9 | Pianura, Soccavo |

10 | Bagnoli, Fuorigrotta |

Municipality | x | y |
---|---|---|

1 | 4.26% | 5 |

2 | 4.77% | 6 |

3 | 5.05% | 6 |

4 | 4.93% | 3 |

5 | 3.80% | 3 |

6 | 5.61% | 9 |

7 | 5.40% | 5 |

8 | 5.35% | 8 |

9 | 5.29% | 6 |

10 | 4.11% | 5 |

Label | a_{1} | a_{2} | a_{3} |
---|---|---|---|

low | 0 | 2 | 4 |

adequate | 2 | 4 | 5 |

fair | 4 | 5 | 6 |

high | 5 | 6 | 8 |

Label | a_{1} | a_{2} | a_{3} |
---|---|---|---|

very low | 0 | 1 | 3 |

low | 1 | 3 | 4 |

mean | 3 | 4 | 7 |

high | 4 | 7 | 10 |

very high | 7 | 10 | 12 |

Rule | p = 1 | ||
---|---|---|---|

E | H | I | |

Rule 1 | 99.00 | 0.00 | 0.99 |

Rule 2 | 82.50 | 3.68 | 0.79 |

Rule 3 | 75.78 | 5.76 | 0.70 |

Rule | p = 1 | ||
---|---|---|---|

E | H | I | |

Rule 1 | 95.60 | 0.00 | 0.95 |

Rule 2 | 75.85 | 4.36 | 0.71 |

Rule 3 | 64.66 | 6.87 | 0.58 |

Municipality | x_{1} | x_{2} | y |
---|---|---|---|

1 | 30.86% | 60.86% | 13.46 |

2 | 13.62% | 52.52% | 26.77 |

3 | 11.58% | 53.47% | 26.53 |

4 | 8.330% | 48.41% | 30.34 |

5 | 29.94% | 69.54% | 13.53 |

6 | 4.410% | 43.85% | 36.51 |

7 | 4.280% | 36.34% | 41.52 |

8 | 5.640% | 36.21% | 40.69 |

9 | 6.880% | 54.69% | 31.42 |

10 | 12.84% | 62.39% | 22.76 |

Label | a_{1} | a_{2} | a_{3} |
---|---|---|---|

very low | 0 | 1 | 3 |

low | 1 | 3 | 4 |

mean | 3 | 4 | 7 |

high | 4 | 7 | 10 |

very high | 7 | 10 | 12 |

Label | a_{1} | a_{2} | a_{3} |
---|---|---|---|

low | 0 | 30 | 40 |

adequate | 30 | 40 | 60 |

fair | 40 | 60 | 80 |

high | 60 | 80 | 100 |

Label | a_{1} | a_{2} | a_{3} |
---|---|---|---|

very low | 0 | 10 | 15 |

low | 10 | 15 | 30 |

mean | 15 | 30 | 50 |

high | 30 | 50 | 60 |

very high | 50 | 60 | 100 |

Rule | Pair | p = 2 | |||
---|---|---|---|---|---|

E | H | I | I Rule | ||

Rule 1 | (A_{1} = very low, B = very high) | 32.00 | 0.00 | 0.32 | 0.32 |

(A_{2} = low, B = very high) | 84.50 | 0.00 | 0.84 | ||

Rule 2 | (A_{1} = low, B = high) | 64.24 | 2.67 | 0.61 | 0.61 |

(A_{2} = low, B = high) | 88.88 | 0.00 | 0.89 | ||

Rule 3 | (A_{1} = mean, B = mean) | 84.65 | 1.20 | 0.83 | 0.80 |

(A_{2} = adequate, B = mean) | 82.92 | 2.67 | 0.80 | ||

Rule 4 | (A_{1} = mean, B = mean) | 95.30 | 0.00 | 0.95 | 0.72 |

(A_{2} = fair, B = mean) | 76.58 | 5.68 | 0.72 | ||

Rule 5 | (A_{1} = mean, B = low) | 88.59 | 2.00 | 0.87 | 0.87 |

(A_{2} = high, B = low) | 90.81 | 0.00 | 0.91 | ||

Rule 6 | (A_{1} = high, B = low) | 90.60 | 2.00 | 0.89 | 0.89 |

(A_{2} = high, B = low) | 90.81 | 0.00 | 0.91 | ||

Rule 7 | (A_{1} = high, B = very low) | 86.68 | 1.85 | 0.85 | 0.85 |

(A_{2} = high, B = very low) | 86.20 | 0.00 | 0.86 | ||

Rule 8 | (A_{1} = very high, B = very low) | 100.00 | 0.00 | 1.00 | 0.91 |

(A_{2} = high, B = very low) | 90.81 | 0.00 | 0.91 |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Di Martino, F.; Sessa, S.
Energy and Entropy Measures of Fuzzy Relations for Data Analysis. *Entropy* **2018**, *20*, 424.
https://doi.org/10.3390/e20060424

**AMA Style**

Di Martino F, Sessa S.
Energy and Entropy Measures of Fuzzy Relations for Data Analysis. *Entropy*. 2018; 20(6):424.
https://doi.org/10.3390/e20060424

**Chicago/Turabian Style**

Di Martino, Ferdinando, and Salvatore Sessa.
2018. "Energy and Entropy Measures of Fuzzy Relations for Data Analysis" *Entropy* 20, no. 6: 424.
https://doi.org/10.3390/e20060424