# Information Entropy-Based Metrics for Measuring Emergences in Artificial Societies

^{*}

## Abstract

**:**

**PACS Codes:**05.10.−a; 05.65.+b; 02.70.−c; 07.05.Tp; 89.75.−k; 89.65.−s; 07.05.Pj

## 1. Introduction

## 2. Method for Measuring Emergences

#### 2.1. Prerequisite Knowledge

_{in}represents the input information and I

_{out}denotes the output information. It is well known, however, that an artificial society is different from a communication system. A communication system is considered as a whole, but an artificial society comprises multiple agents. An emergence in a communication system is generally considered as the whole behavior of the system and caused by the complexity of the communication system, while the emergence in an artificial society is created by individuals and their nonlinear interactions or other behaviors, as shown in Figure 1. Therefore, the metric of emergences in the communication system cannot be directly used to measure emergences in artificial societies.

#### 2.2. Metrics of Various Emergences

#### 2.2.1. Emergence of Attribute and Behavior

_{i}(t) means the statistical probability of agents owning the i

^{th}value of an attribute or performing the i

^{th}behavior at t moment. Then, based on the information entropy, we could give the entropy of the artificial society in Equation (2):

_{i}(t) are the same at t moment, i.e., any p

_{i}(t) = 1/m. If the entropy of an artificial society reaches the maximum value, we could conclude that this artificial society is in an absolute disorder:

_{Max}, the value of metric E(t) is 0, which means the artificial society absolutely has no emergences of attribute or behavior. When the value of an attribute of all agents is the same or all agents perform a same behavior, H(t) is equal to 0 and then E(t) is 1, which means the emergence of an attribute or behavior has appeared in the artificial society. Metric E(t) denotes the levels of an emergence, and E(t) increases with the growing levels of the emergence. As shown in Figure 3, there are four scenes (different moments) of an artificial society, and this artificial society contains 12 agents.

_{2}1/3 ≈ 1.585. As the same, H(b) = −(5/6log

_{2}5/6 + 1/12 log

_{2}1/12 + 1/12log

_{2}1/12) ≈ 0.817, H(c) = 0, H(d) = −∑1/12log

_{2}1/12 ≈ 3.585, and H

_{Max}= −∑1/12log

_{2}1/12 ≈ 3.585. Then, we could acquire the metric values of metric E(t) for measuring emergences in these four scenes: E(a) ≈ 0.558, E(b) ≈ 0.772, E(c) = 1, and E(d) = 0. These results have clearly and correctly reflected the levels of emergences of attribute and behavior.

#### 2.2.2. Emergence of Structure

_{i}indicates the probability (in the objective distribution) that the value of this item is in the i

^{th}value interval, and p

_{i}(t) represents the estimated probability that the value of this item is during the i

^{th}value interval at moment t. In addition, $\frac{\mid {r}_{i}-{p}_{i}(t)\mid}{{\displaystyle \sum _{i=1}^{M}}{r}_{i}}$ means the i

^{th}relative probability redundancy. As presented in Equation (5), the entropy H

_{S}(t) means the disorder between the estimated distribution and the objective distribution at moment t. In addition, the entropy H

_{S}could get the maximum value (signed as H

_{S_Max}), when the values of all $\frac{\mid {r}_{i}-{p}_{i}(t)\mid}{{\displaystyle \sum _{i=1}^{M}}{r}_{i}}$ are the same and equal to $1-\frac{{\displaystyle \sum _{i=1}^{M}}\mid {r}_{i}-{p}_{i}(t)\mid}{{\displaystyle \sum _{i=1}^{M}}{r}_{i}}$. Equation (6) is the definition of H

_{S_Max}. Above all, we could calculate the value of metric E

_{S}(t) of distribution emergence with E

_{S}(t) and H

_{S_Max}, as shown in Equation (7). $1-\frac{{\displaystyle \sum _{i=1}^{M}}\mid {r}_{i}-{p}_{i}(t)\mid}{{\displaystyle \sum _{i=1}^{M}}{r}_{i}}$ means the whole probability with the same distribution. When two distributions are the same, the value of $1-\frac{{\displaystyle \sum _{i=1}^{M}}\mid {r}_{i}-{p}_{i}(t)\mid}{{\displaystyle \sum _{i=1}^{M}}{r}_{i}}$ is 1, thus the value of H

_{S}(t) is 0 and E

_{S}(t) is equal to 1.

_{S}(b) of the artificial society at moment b is about 0.569. In the same way, H

_{S}(a) = 0, H

_{S}(c) ≈ 2.446, and H

_{S_Max}= −∑1/6log

_{2}1/6 ≈ 2.585. Then, we could calculate all these metrics of the artificial society in these three scenes, i.e., E

_{S}(a) = 1, E

_{S}(b) ≈ 0.780, and E

_{S}(c) ≈ 0.054. The metric values clearly and correctly reflect the real levels of emergences in these three scenes.

_{1}, C

_{2}, …, C

_{M}}, where C

_{i}denotes a cluster, and each agent should belong to one of these clusters. These clusters may be based-on the physical environment or social network. |C

_{i}| means the number of agents that belong to the cluster C

_{i}. In addition, N represents the number of agents in the artificial society. Then, |C

_{i}|/N means the probability that agents belong to the cluster C

_{i}. The entropy H

_{C}(t) could be defined in Equation (8), and Equations (9) and (10) present the maximum entropy and metric of emergence, respectively. E

_{C}(t) could measure the levels of the clustering emergence at moment t:

_{C}(a) and H

_{C}(b)) in these two scenes are 0, and then the metric value of clustering emergence are 1, i.e., E

_{C}(a) = 1 and E

_{C}(b) = 1. In the scene c, there are two clusters, and one cluster has eight agents and another one contains five agents. Then, H

_{C}(c) = −(5/13log

_{2}5/13 + 8/13log

_{2}8/13) ≈ 0.961, and H

_{C}_

_{Max}= −∑1/13 log

_{2}1/13 ≈ 3.7. Thus, E

_{C}(c) ≈ 0.74. In the same way, E

_{C}(d) ≈ 0.099. The value of E

_{C}(t) increases with the decreasing number of clusters.

## 3. Experiments

#### Case 1: The Spread of an Infectious Influenza

_{i}(t) denotes the percent of the i

^{th}role at moment t, m is 4 in this case, and then H

_{Max}equals 2. E(t) changes with the spread status of the infectious influenza. As presented in Figure 6b, the metric E(t) reaches the peak point, while almost agents are in the infected or exposed states. Therefore, this metric has clearly and correctly reflected the degree of emergence of role attribute.

#### Case 2: A Dynamic Microblog Network

_{p}) of the network is N(N − 1), and the existing total out-degree number is ${n}_{e}=\frac{1}{2}\sum {z}_{i\to}$ (z

_{i}

_{→}represents the out-degree of node i). Meanwhile, ownz

^{*}

_{i}

_{→}denotes the limited out-degree of node i (i.e., maximum out-degree of node i), and maximum out-degree of all nodes follows a power-law as the exponential distribution ( $p(k)\propto {e}^{-\frac{k}{\mu}}$):

- (1)
- In each time, randomly select n
_{p}r_{0}pairs of nodes. For each pair of nodes, randomly choose one of these two nodes signed as node i. If out-degree of node i is smaller than ownz*_{i}_{→}, then node i will connect to the other one. - (2)
- In each time, randomly choose n
_{p}r_{1}pairs of nodes. For each pair of nodes, if one of the chose nodes (node j) connects to the other (node i), and node i does not connect to node j and out-degree of the node i is less than ownz*_{i}_{→}, then node i will connect to node j. - (3)
- In each time, randomly select n
_{p}r_{2}pairs of nodes. For each pair of nodes, if one of the selected nodes (node i) with the smaller in-degree does not connect to the other node and out-degree of node i is less than ownz*_{i}_{→}, then node i will connect to the other node. - (4)
- In each time, randomly choose n
_{m}r_{3}nodes ( ${n}_{m}=\frac{1}{2}\sum {z}_{i\to}({z}_{i\to}-1)$). For each node, randomly select one of nodes from its in-neighbor nodes (called node i), and randomly choose one of nodes from its out-neighbor nodes (signed as node j). If node i does not connect to node j and out-degree of node i is smaller than ownz*_{i}_{→}, and then node i will connect to node j. - (5)
- In each time, randomly choose n
_{e}γ nodes (γ is a constant). For each node, randomly select one of its out-links and cancel this link.

_{0}= 0.0015, r

_{1}= 0.1, r

_{2}= 0.0015, r

_{3}= 2, and γ = 0.001. Figure 7 shows a snapshot of the dynamic microblog network, and we could observe that this network have the feature of clustering. In this case, we mainly focus on whether the out-degree distribution of this microblog follows the distribution of power-law ( $p(k)\propto {e}^{-\frac{k}{\mu}}$), which is an emergence of structure. With Equations (5), (6), and (7), we could dynamically compute the metric E

_{S}(t) with time. As shown in Figure 8a, the value of E

_{S}quickly grows to nearly 0.98, and then this value is stable with few fluctuations. Because with a smaller γ few relations or links will be deleted in each time. If we set γ from 0.001 to 0.1, in each time more relations or links will be deleted. As illustrated in Figure 8b, the metric E

_{S}(t) fluctuates around 0.8. The metric E

_{S}(t) has correctly and truly reflected the levels of this structure emergence in a quantitative way.

#### Case 3: Flock of Birds (Flocking Birds)

- (1)
- Cohesion: If an agent is far away from its nearest neighbor, and then this agent will turn towards its nearest neighbor.
- (2)
- Separation: If an agent is too close to the nearest neighbor, and then this agent will turn away from the nearest neighbor.
- (3)
- Alignment: All agents keep the average direction of all agents.

_{i}| in Equation (8) indicates the number of agents whose cluster_id equals i. The entropy H

_{C}(t) could be calculated with Equation (8), and H

_{C_Max}= −log

_{2}1/300 ≈ 5.044. Real-time metric E

_{C}(t) of flocking emergence could be shown in Figure 11, and Figure 9 also presents the corresponding E

_{C}(t) according to different flocking processes at various simulation ticks. In the Figure 11, we could see that metric E

_{C}(t) is increasing with some fluctuations. For example, as shown in Figure 9, while the tick is 489, there are five clusters and the E

_{C}(t) is about 0.832. When the tick is 538, there are six clusters and then the E

_{C}(t) is about 0.754. However, the whole trend of the metric is rising. From these real data, we could conclude that this metric E

_{C}(t) has accurately reflected the levels of the clustering emergence.

## 4. Discussion

_{C}(t) could also be used to measure the levels of the clustering or community emergence in a social network. However, there are still some limitations for these three metrics. Metric E(t) for measuring emergences of attributes or behaviors could estimate the levels of one attribute or behavior emergence, but it cannot point out which attribute or behavior is emerging. In some applications, people may not know the number (m) of the possible values of attribute and m < n, which may be another limitation of the metric E(t). In such scenario, we may assume that the value of m is equal to n. For metric E

_{S}(t), when the number of value intervals in objective distribution is very different from the estimated distribution, we should make previous processing to make these two numbers of value intervals as the same. Meanwhile, if people want to acquire an emergence that agents cluster into two or more similar clusters, and E

_{C}(t) may have a limitation to directly measure this kind of emergence, because the meaning of metric E

_{C}(t) is the level of structure emergence that agents should cluster into one cluster and community but not two or more. Another limitation of these metrics is that these metrics do not capture the time-dependent emergences, which may change from time to time [19]. Moreover, there may be some attributes with continuous values in practical applications and even some applications may have several continuous attributes [20]. These relative entropies for measuring emergences do not consider the continuous attributes or behaviors. However, if we measure emergences with these metrics by quantizing the continuous variables, then this may result in bad results. In the future work, we should consider the continuous attributes and behaviors, and even multiple continuous attribute and behaviors, which could be solved by borrowing and referencing the idea of differential entropy and joint entropy, even though a continuous entropy may result in negative values.

_{Emergence}. At each moment t, an artificial society is in a scenario signed as S

_{t}. If the scenario of the artificial society satisfies the scenario of the emergence at moment t, it could be depicted as S

_{t}∝ S

_{Emergence}. Figure 12 presents how to abstract dynamic scenarios of the artificial society and sense these scenarios by the observer with metrics for measuring emergences, and whether an emergence appears could conduct the real society.

_{Emergence}as the thresholds of metrics of emergences such as E(t), E

_{C}(t), and E

_{S}(t), and S

_{t}is seen as the real-time value of the metric of emergence. It is assumed that if S

_{t}≥ S

_{Emergence}then S

_{t}∝ S

_{Emergence}. As we know, an emergency may give rise to an emergence, which may result in the loss of health and property, and even endanger social stability and safety. Emergency management policies should be made to control and manage this emergency, and the metrics proposed in the paper could be used to evaluate the effectiveness of a policy in an artificial society. When a policy makes the corresponding metric with a smaller value, it means that this policy is more effective. Therefore, these metrics could be used to conduct the emergency management in the real society.

## 5. Conclusions

_{S}(t), and E

_{C}(t)) for measuring various emergences, and study three complex cases to verify the correctness of these metrics; meanwhile, in the third case study, a spatial clustering algorithm has been proposed to calculate clusters in an artificial society.

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Goldstein, J. Emergence as a construct: History and issues. Emergence
**1999**, 1, 49–72. [Google Scholar] - Wolf, T.D.; Holvoet, T. Emergence Versus Self-Organisation: Different Concepts but Promising When Combined. In Engineering Self-Organising Systems; Proceedings of Third International Workshop, ESOA 2005, Utrecht, The Netherlands, 25 July 2005, Brueckner, S.A., Serugendo, G.D.M., Karageorgos, A., Nagpal, R., Eds.; Lecture Notes in Computer Science, Volume 3464; Springer: Berlin/Heidelberg, Germany, 2005; pp. 1–15. [Google Scholar]
- Prokopenko, M.; Boschetti, F.; Ryan, A.J. An information-theoretic primer on complexity, self-organization, and emergence. Complexity
**2009**, 15, 11–28. [Google Scholar] - Waugh, W.L.; Streib, G. Collaboration and leadership for effective emergency management. Public Adm. Rev
**2006**, 66, 131–140. [Google Scholar] - Wang, F.Y. Toward a Paradigm Shift in Social Computing: The ACP Approach. IEEE Intell. Syst
**2007**, 22, 65–67. [Google Scholar] - Denbigh, K.G. Note on entropy, disorder and disorganization. British J. Philos. Sci
**1989**, 40, 323–332. [Google Scholar] - Kostic, M.M. The Elusive Nature of Entropy and Its Physical Meaning. Entropy
**2014**, 16, 953–967. [Google Scholar] - Cengel, Y.A.; Boles, M.A.; Kanoglu, M. Thermodynamics: An engineering approach; McGraw-Hill: New York, NY, USA, 2011. [Google Scholar]
- Goldstein, S.; Lebowitz, J.L. On the (Boltzmann) entropy of non-equilibrium systems. Physica D
**2004**, 193, 53–66. [Google Scholar] - Baldassarre, G. Self-organization as phase transition in decentralized groups of robots: A study based on Boltzmann entropy. In Advances in Applied Self-organizing Systems; Springer: London, UK, 2013; pp. 157–177. [Google Scholar]
- Swanson, G.A.; Bailey, K. Social Entropy Theory, Macro Accounting, and Entropy Related Measures. Proceedings of the Annual Meeting of the International Society for the Systems Sciences, Rohnert Park, CA, USA, 9–14 July 2006; Curran Associates Inc: New York, NY, USA, 2006; pp. 159–174. [Google Scholar]
- Balch, T. Hierarchic social entropy: An information theoretic measure of robot group diversity. Auton. Robots
**2000**, 8, 209–238. [Google Scholar] - Vedral, V. The role of relative entropy in quantum information theory. Rev. Mod. Phys
**2002**, 74, 197–234. [Google Scholar] - Machado, J.T. Fractional Order Generalized Information. Entropy
**2014**, 16, 2350–2361. [Google Scholar] - Van Dyke Parunak, H.; Brueckner, S. Entropy and self-organization in multi-agent systems. Proceedings of the Fifth International Conference on Autonomous Agents, Montreal, Canada, 28 May–1 June 2001; ACM: New York, NY, USA, 2001; pp. 124–130. [Google Scholar]
- Mnif, M.; Müller-Schloer, C. Quantitative emergence. In Organic Computing—A Paradigm Shift for Complex Systems; Springer: Basel, Switzerland, 2011; pp. 39–52. [Google Scholar]
- Bedau, M.A.; Norman, H.P. Measurement of Evolutionary Activity, Teleology, and Life. In Artificial Life II; Christopher, G., Taylor, C.L., Farmer, D., Rasmussen, S., Eds.; SFI Studies in the Sciences of Complexity; Addison Wesley Longman: Boston, MA, USA, 1992; pp. 431–461. [Google Scholar]
- Hovda, P. Quantifying weak emergence. Minds Mach
**2008**, 18, 461–473. [Google Scholar] - Chan, W.K.V. Interaction metric of emergent behaviors in agent-based simulation. Proceedings of the 2011 Winter Simulation Conference, Phoenix, AZ, USA, 11–14 December 2011; pp. 357–368.
- Fisch, D.; Janicke, M.; Sick, B.; Muller-Schloer, C. Quantitative Emergence-A Refined Approach Based on Divergence Measures. Proceedings of 4th IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO), Budapest, Hungary, 27 September–1 October 2010; pp. 94–103.
- Bar-Yam, Y. A mathematical theory of strong emergence using multiscale variety. Complexity
**2004**, 9, 15–24. [Google Scholar] - Laughlin, R.B. A Different Universe: Reinventing Physics from the Bottom Down; Basic Books: New York, NY, USA, 2006. [Google Scholar]
- Bedau, M.A. Weak emergence. Noûs
**1997**, 31, 375–399. [Google Scholar] - Chalmers, D.J. Strong and weak emergence. In The Re-emergence of Emergence; Oxford University Press: Oxford, UK, 2006; pp. 244–256. [Google Scholar]
- Seth, A.K. Measuring emergence via nonlinear Granger causality. Proceedings of Artificial Life XI: Eleventh International Conference on the Simulation and Synthesis of Living Systems, Winchester, UK, 5–8 August 2008; pp. 545–553.
- Bedau, M. Downward causation and the autonomy of weak emergence. Principia
**2003**, 6, 5–50. [Google Scholar] - Tang, M.; Mao, X.; Zhou, H. Zombie-city: A new artificial society model. J. Comput. Inf. Sys
**2013**, 9, 4989–4996. [Google Scholar] - Gershenson, C.; Fernández, N. Complexity and information: Measuring emergence, self-organization, and homeostasis at multiple scales. Complexity
**2012**, 18, 29–44. [Google Scholar] - Grabowski, A.; Kosiñski, R.A. Epidemic spreading in a hierarchical social network. Phys. Rev. E
**2004**, 70. [Google Scholar] [CrossRef] - Tang, M.; Mao, X.; Yang, S.; Zhou, H. A Dynamic Microblog Network and Information Dissemination in “@” Mode. Math. Probl. Eng
**2014**, 2014. [Google Scholar] [CrossRef] - Usherwood, J.R.; Stavrou, M.; Lowe, J.C.; Roskilly, K.; Wilson, A.M. Flying in a flock comes at a cost in pigeons. Nature
**2011**, 474, 494–497. [Google Scholar] - Tang, M.; Mao, X.; Guessoum, Z. Research on an Infectious Disease Transmission by Flocking Birds. Sci. World J
**2013**, 2013. [Google Scholar] [CrossRef] - Zhu, H.; Wang, F.; Wang, S. On the Convergence of Autonomous Agent Communities. J. Multiagent Grid Syst
**2010**, 6, 315–352. [Google Scholar] - Randles, M.; Zhu, H.; Taleb-Bendiab, A. A Formal Approach to the Engineering of Emergence and its Recurrence. Proceedings of 2nd International Workshop on Engineering Emergence in Decentralised Autonomic Systems, Jacksonville, FL, USA, 11 June 2007.

**Figure 3.**

**(a)**Agents play three roles at moment a;

**(b)**Agents play three roles at moment b;

**(c)**All agents play the same role at moment c;

**(d)**All agents play different roles at moment d.

**Figure 4.**

**(a)**The distribution at moment a (Scene a) and also the objective distribution;

**(b**) The distribution at moment b (Scene b);

**(c)**The distribution at moment c (Scene c).

**Figure 5.**(

**a**) Agents are in one cluster at a moment (Scene a); (

**b**) Agents are in one cluster at b moment (Scene b); (

**c**) Agents are in two clusters at c moment (Scene c); (

**d**) Agents are in eleven clusters at d moment (Scene d).

**Figure 6.**

**(a)**The spread status of the infectious influenza in the artificial school;

**(b)**The metric E(t) of role attribute emergence changes with time.

**Figure 7.**A snapshot of the dynamic microblog network, where N = 250, μ = 5, r

_{0}= 0.0015, r

_{1}= 0.1, r

_{2}=0.0015, r

_{3}= 2, and γ = 0.1.

**Figure 8.**

**(a)**The metric E

_{S}(t) of structure emergence changes with time, where γ = 0.001;

**(b)**The metric E

_{S}(t) changes with time, where γ=0.1.

**Figure 12.**Dynamic scenarios of the artificial society could be sensed by the observer with metrics to conduct the real society.

Classifications | Applications | Metrics | ||
---|---|---|---|---|

Weak emergence | Emergence of attribute | Outbreak of infectious disease | Relative entropy, e.g., E(t), E_{S}(t), E_{C}(t). | |

Emergence of behavior | Emergence of interactions | |||

Emergence of structure | Emergence of distribution | Matthew effect in the wealth distribution, power-law distribution | ||

Emergence of cluster | Flocking birds, fish school | |||

Strong emergence | emergence of consciousness like qualia from the neurobiological processes | Multi-scale variety [21] |

© 2014 by the authors; licensee MDPI, Basel, Switzerland This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

## Share and Cite

**MDPI and ACS Style**

Tang, M.; Mao, X.
Information Entropy-Based Metrics for Measuring Emergences in Artificial Societies. *Entropy* **2014**, *16*, 4583-4602.
https://doi.org/10.3390/e16084583

**AMA Style**

Tang M, Mao X.
Information Entropy-Based Metrics for Measuring Emergences in Artificial Societies. *Entropy*. 2014; 16(8):4583-4602.
https://doi.org/10.3390/e16084583

**Chicago/Turabian Style**

Tang, Mingsheng, and Xinjun Mao.
2014. "Information Entropy-Based Metrics for Measuring Emergences in Artificial Societies" *Entropy* 16, no. 8: 4583-4602.
https://doi.org/10.3390/e16084583