# Semantic Networks: Structure and Dynamics

^{*}

## Abstract

**:**

## 1. Introduction to complex networks

#### 1.1. Terminology in complex networks

#### 1.2. Complex network descriptors

**Degree and Degree Distribution**The simplest and the most intensively studied one vertex characteristic is degree. Degree, k, of a vertex is the total number of its connections. If we are dealing with a directed graph, in-degree, ${k}_{i}$, is the number of incoming arcs of a vertex. Out-degree, ${k}_{o}$ is the number of its outgoing arcs. Degree is actually the number of nearest neighbors of a vertex. Total distributions of vertex degrees of an entire network, $p\left(k\right)$, ${p}_{i}\left({k}_{i}\right)$ (the in-degree distribution), and ${p}_{o}\left({k}_{o}\right)$ (the out-degree distribution) are its basic statistical characteristics. We define $p\left(k\right)$ to be the fraction of vertices in the network that have degree k. Equivalently, $p\left(k\right)$ is the probability that a vertex chosen uniformly at random has degree k. Most of the work in network theory deals with cumulative degree distributions, $P\left(k\right)$. A plot of $P\left(k\right)$ for any given network is built through a cumulative histogram of the degrees of vertices, and this is the type of plot used throughout this article (and often referred to just as “degree distribution”). Although the degree of a vertex is a local quantity, we shall see that a cumulative degree distribution often determines some important global characteristics of networks. Yet another important parameter measured from local data and affecting the global characterization of the network is average degree $\langle k\rangle $. This quantity is measured by the equation:

**Strength Distribution**In weighted networks the concept of degree of a node i (${k}_{i}$) is not as important as the notion of strength of that node, ${\omega}_{i}={\sum}_{j\in {\mathsf{\Gamma}}_{i}}{\omega}_{ij}$, i.e., the sum over the nodes j in the of i, of weights from node i towards each of the nodes j in its neighborhood ${\mathsf{\Gamma}}_{i}$. In this type of network it is possible to measure the average strength $\langle k\rangle $ with a slight modification of eq.1. On the other hand, it is also possible to plot the cumulative strength distribution $P\left(s\right)$, but it is important to make a good choice in the number of bins of the histogram (this depends on the particular distribution of weights for each network).

**Shortest Path and Diameter**For each pair of vertices i and j connected by at least one path, one can introduce the shortest path length, the so-called intervertex distance ${d}_{ij}$, the corresponding number of edges in the shortest path. Then one can define the distribution of the shortest-path lengths between pairs of vertices of a network and the average shortest-path length L of a network. The average here is over all pairs of vertices between which a path exists and over all realizations of a network. It determines the effective “linear size” of a network, the average separation of pairs of vertices. In a fully connected network, $d=1$. Recall that shortest paths can also be measured in weighted networks, then the path’s cost equals the sum of the weights. One can also introduce the maximal intervertex distance over all the pairs of vertices between which a path exists. This descriptor determines the maximal extent of a network; the maximal shortest path is also referred to as the diameter (D) of the network.

**Clustering Coefficient**The presence of connections between the nearest neighbors of a vertex i is described by its clustering coefficient. Suppose that a node (or vertex) i in the network has ${k}_{i}$ edges and they connect this node to ${k}_{i}$ other nodes. These nodes are all neighbors of node i. Clearly, at most

**Centrality Measures**Centrality measures are some of the most fundamental and frequently used measures of network structure. Centrality measures address the question, “Which is the most important or central node in this network?”, that is, the question whether nodes should all be considered equal in significance or not (whether exists some kind of hierarchy or not in the system). The existence of such hierarchy would then imply that certain vertices in the network are more central than others. There are many answers to this question, depending on what we mean by important. In this Section we briefly explore two centrality indexes (betweenness and eigenvector centrality) that are widely used in the network literature. Note however that betweenness or eigenvector centrality are not the only method to classify nodes’ importance. Within graph theory and network analysis, there are various measures of the centrality of a vertex within a graph that determine the relative importance of a vertex within the graph. For instance, besides betweenness, there are two other main centrality measures that are widely used in network analysis: degree centrality and closeness. The first, and simplest, is degree centrality, which assumes that the larger is the degree of a node, the more central it is. The closeness centrality of a vertex measures how easily other vertices can be reached from it (or the other way: how easily it can be reached from the other vertices). It is defined as the number of vertices minus one divided by the sum of the lengths of all geodesics from/to the given vertex.

**a. Betweenness**One of the first significant attempts to solve the question of node centrality is Freeman’s proposal (originally posed from a social point of view): betweenness as a centrality measure [28]. As Freeman points out, a node in a network is central to the extent that it falls on the shortest path between pairs of other nodes. In his own words, “suppose that in order for node i to contact node j, node k must be used as an intermediate station. Node k in such a context has a certain “responsibility” to nodes i and j. If we count all the minimum paths that pass through node k, then we have a measure of the “stress” which node k must undergo during the activity of the network. A vector giving this number for each node of the network would give us a good idea of stress conditions throughout the system” [28]. Computationally, betweenness is measured according to the next equation:

**b. Eigenvector centrality**A more sophisticated version of the degree centrality is the so-called eigenvector centrality [30]. Where degree centrality gives a simple count of the number of connections a vertex has, eigenvector centrality acknowledges that not all connections are equal. In general, connections to people who are themselves influential will lend a person more influence than connections to less influential people. If we denote the centrality of vertex i by ${x}_{i}$ , then we can allow for this effect by making ${x}_{i}$ proportional to the average of the centralities of is network neighbors:

**Degree-Degree correlation: assortativity**It is often interesting to check for correlations between the degrees of different vertices, which have been found to play an important role in many structural and dynamical network properties. The most natural approach is to consider the correlations between two vertices connected by an edge. A way to determine the degree correlation is by considering the Pearson correlation coefficient of the degrees at both ends of the edges [33,34]

#### 1.3. Network models

**Regular Graphs**Although regular graphs do not fall under the definition of complex networks (they are actually quite far from being complex, thus their name), they play an important role in the understanding of the concept of “small world”, see below. For this reason we offer a brief comment on them.

**Random Graphs**Before the burst of attention on complex networks in the decade of 1990s, a particularly rich source of ideas has been the study of random graphs, graphs in which the edges are distributed randomly. Networks with a complex topology and unknown organizing principles often appear random; thus random-graph theory is regularly used in the study of complex networks. The theory of random graphs was introduced by Paul Erdös and Alfréd Rényi [15,37,38] after Erdös discovered that probabilistic methods were often useful in tackling problems in graph theory. A detailed review of the field is available in the classic book of Bollobás [39]. Here we briefly describe the most important results of random graph theory, focusing on the aspects that are of direct relevance to complex networks.

**a. The Erdös–Rényi Model**In their classic first article on random graphs, Erdös and Rényi define a random graph as N labeled nodes connected by n edges, which are chosen randomly from the $N(N-1)/2$ possible edges [15].

**b. Watts–Strogatz small-world network**In simple terms, the small-world concept describes the fact that despite their often large size, in most networks there is a relatively short path between any two nodes. The distance between two nodes is defined as the number of edges along the shortest path connecting them. The most popular manifestation of small worlds is the “six degrees of separation” concept, uncovered by the social psychologist Stanley Milgram [4,5], who concluded that there was a path of acquaintances with a typical length of about six between most pairs of people in the United States. This feature (short path lengths) is also present in random graphs. However, in a random graph, since the edges are distributed randomly, the clustering coefficient is considerably small. Instead, in most, if not all, real networks the clustering coefficient is typically much larger than it is in a comparable random network (i.e., same number of nodes and edges as the real network). Beyond Milgram’s experiment, it was not until 1998 that Watts and Strogatz’ work [6] stimulated the study of such phenomena. Their main discovery was the distinctive combination of high clustering with short characteristic path length, which is typical in real-world networks (either social, biological or technological) that cannot be captured by traditional approximations such as those based on regular lattices or random graphs. From a computational point of view, Watts and Strogatz proposed a one-parameter model that interpolates between an ordered finite dimensional lattice and a random graph. The algorithm behind the model is the following [6]:

- Start with order: Start with a ring lattice with N nodes in which every node is connected to its first k neighbors ($k/2$ on either side). In order to have a sparse but connected network at all times, consider $N\gg k\gg ln\left(N\right)\gg 1$.
- Randomize: Randomly rewire each edge of the lattice with probability p such that self-connections and duplicate edges are excluded. This process introduces $pNK/2$ long-range edges which connect nodes that otherwise would be part of different neighborhoods. By varying p one can closely monitor the transition between order (p=0) and randomness (p=1).

**Scale-Free Networks**Certainly, the SW model initiated a revival of network modeling in the past few years. However, there are some real-world phenomena that small-world networks can’t capture, the most relevant one being evolution. In 1999, Barabási and Albert presented some data and formal work that has led to the construction of various scale-free models that, by focusing on the network dynamics, aim to offer a universal theory of network evolution [16].

**a. The Barabási–Albert model**These two ingredients, growth and preferential attachment, inspired the introduction of the Barabási–Albert model (BA), which led for the first time to a network with a power-law degree distribution. The algorithm of the BA model is the following:

- Growth: Starting with a small number (${m}_{0}$) of nodes, at every time step, we add a new node with m($\le {m}_{0}$) edges that link the new node to m different nodes already present in the system.
- Preferential attachment: When choosing the nodes to which the new node connects, we assume that the probability ∏ that a new node will be connected to node i depends on the degree ${k}_{i}$ of node i, such that$$\prod =\frac{{k}_{i}}{{\displaystyle \sum _{j}{k}_{j}}}$$

**b. Other SF models**The BA model has attracted an exceptional amount of attention in the literature. In addition to analytic and numerical studies of the model itself, many authors have proposed modifications and generalizations to make the model a more realistic representation of real networks. Various generalizations, such as models with nonlinear preferential attachment, with dynamic edge rewiring, fitness models and hierarchically and deterministically growing models, can be found in the literature. Such models yield a more flexible value of the exponent γ which is restricted to $\gamma =3$ in the original BA construction. Furthermore, modifications to reinforce the clustering property, which the BA model lacks, have also been considered.

#### 1.4. The mesoscale level

- The study at the micro level attempts to understand the behavior of single nodes. Such level includes degree, clustering coefficient or betweenness and other parameters.
- Meso level points at group or community structure. At this level, it is interesting to focus on the interaction between nodes at short distances, or classification of nodes, as we shall see.
- Finally, macro level clarifies the general structure of a network. At this level, relevant parameters are average degree $\langle k\rangle $, degree distribution $P\left(k\right)$, average path length L, average clustering coefficient C, etc.

## 2. Building language networks

#### 2.1. Text analysis: co-occurrence graphs

#### 2.2. Dictionaries and Thesauri

#### 2.3. Semantic features

#### 2.4. Associative networks

## 3. Language networks: topology, function, evolution

## 4. The cognitive pole I: Language and conceptual development

## 5. The cognitive pole II: Cognitive-linguistic processes on language networks

#### 5.1. Google and the mind

#### 5.2. Clustering and switching dynamics

#### 5.3. Encoding semantic similarity

## 6. Conclusions and perspectives

## References

- Miller, G.A.; Beckwith, R.; Fellbaum, C.; Gross, D.; Miller, K.J. Introduction to wordnet: an on-line lexical database. Int. J. Lexico.
**1990**, 3, 235–244. [Google Scholar] [CrossRef] - Roget, P. Roget’s Thesaurus of English Words and Phrases; TY Crowell co.: New York, NY, USA, 1911. [Google Scholar]
- WordNet: An electronic lexical database; Fellbaum, C. (Ed.) MIT press: Cambridge, MA, USA, 1998.
- Milgram, S. The small world problem. Psychol. Today
**1967**, 2, 61–67. [Google Scholar] - Travers, J.; Milgram, S. An experimental study of the small world problem. Sociometry
**1969**, 32, 425–443. [Google Scholar] [CrossRef] - Watts, D.; Strogatz, S. Collective dynamics of ’small-world’ networks. Nature
**1998**, 393, 440. [Google Scholar] [CrossRef] [PubMed] - Ferrer i Cancho, R.; Solé, R.; Köhler, R. Patterns in syntactic dependency networks. Phys. Rev. E
**2004**, 69, 051915. [Google Scholar] [CrossRef] [PubMed] - Ferrer i Cancho, R. Euclidean distance between syntactically linked words. Phys. Rev. E
**2004**, 70, 56135. [Google Scholar] [CrossRef] [PubMed] - Solé, R.V.; Murtra, B.; Valverde, S.; Steels, L. Language Networks: their structure, function and evolution. Trends Cogn. Sci.
**2006**. cplx.20305. [Google Scholar] - Ferrer i Cancho, R. Why do syntactic links not cross? Europhys. Lett.
**2006**, 76, 1228–1234. [Google Scholar] [CrossRef] - Corominas-Murtra, B.; Valverde, S.; Solé, R.V. The ontogeny of scale-free syntax networks through language acquisition. Adv. Compl. Syst.
**2009**, 12, 371–392. [Google Scholar] [CrossRef] - Corominas-Murtra, B.; Valverde, S.; Solé, R.V. Evolution of Communication and Language in Embodied Agents. Evol. Commun. Lang. Embodied Agent.
**2010**, 83–101. [Google Scholar] - Kello, C.; Beltz, B. Scale-free networks in phonological and orthographic wordform lexicons. In Approaches to Phonological Complexity; Chitoran, I., Coup, C., Marsico, E., Pellegrino, F., Eds.; Mouton De Gruyter: Berlin, Germany, 2009; p. 171. [Google Scholar]
- Vitevitch, M. The clustering coefficient of phonological neighborhoods influences spoken word recognition. J. Acoust. Soc. Am.
**2006**, 120, 3252. [Google Scholar] [CrossRef] - Erdös, P.; Rényi, A. On Random Graphs. Publ. Math. (Debrecen)
**1959**, 6, 290–297. [Google Scholar] - Barabási, A.; Albert, R. Emergence of Scaling in Random Networks. Science
**1999**, 286, 509. [Google Scholar] [PubMed] - Albert, R.; Barabási, A. Statistical mechanics of complex networks. Rev. Mod. Phys.
**2002**, 74, 47–97. [Google Scholar] [CrossRef] - Newman, M.E.J. The structure and function of complex networks. SIAM Rev
**2003**, 45, 167–256. [Google Scholar] [CrossRef] - Boccaletti, S.; Latora, V.; Moreno, Y.; Chavez, M.; Hwang, D. Complex networks: structure and dynamics. Phys. Rep.
**2006**, 424, 175–308. [Google Scholar] [CrossRef] - Chung, F.; Lu, L. Complex Graphs and Networks; American Mathematical Society: Providence, RI, USA, 2006. [Google Scholar]
- Newman, M.; Barabási, A.; Watts, D. The Structure and Dynamics of Networks; Princeton University Press: Princeton, NJ, USA, 2006. [Google Scholar]
- Caldarelli, G. Scale-free Networks: Complex Webs in Nature and Technology; Oxford University Press: New York, NY, USA, 2007. [Google Scholar]
- Barrat, A.; Barthélemy, M.; Vespignani, A. Dynamical Processes on Complex Networks; Cambridge University Press: New York, NY, USA, 2008. [Google Scholar]
- Dorogovtsev, S.; Goltsev, A.; Mendes, J. Critical phenomena in complex networks. Rev. Mod. Phys.
**2008**, 80, 1275–1335. [Google Scholar] [CrossRef] - Cohen, R.; Havlin, S. Complex Networks: Structure, Robustness and Function; Cambridge University Press: New York, NY, USA, 2010. [Google Scholar]
- Rozenfeld, H. Structure and Properties of Complex Networks: Models, Dynamics, Applications; VDM Verlag: Saarbrcken, Germany, 2010. [Google Scholar]
- Wasserman, S.; Faust, K. Social Network Analysis: Methods and Applications; Cambridge University Press: Cambridge, UK, 1994. [Google Scholar]
- Freeman, L. A set of measures of centrality based upon betweenness. Sociometry
**1977**, 40, 35–41. [Google Scholar] [CrossRef] - Brandes, U. A Faster Algorithm for Betweenness Centrality. J. Math. Soc.
**2001**, 25, 163–177. [Google Scholar] [CrossRef] - Bonacich, P. Factoring and weighting approaches to status scores and clique identification. J. Math. Sociol.
**1972**, 2, 113–120. [Google Scholar] [CrossRef] - Adamic, L. The small world web. Res. Adv. Tech. Digit. Libr.
**1999**, 852–852. [Google Scholar] - Page, L.; Brin, S.; Motwani, R.; Winograd, T. The PageRank citation ranking: Bringing order to the web. Technical report. Stanford Digital Library Technologies Project: Standford, CA, USA, 1998. [Google Scholar]
- Newman, M. Assortative mixing in networks. Phys. Rev. Lett.
**2002**, 89, 208701. [Google Scholar] [CrossRef] [PubMed] - Newman, M. Mixing patterns in networks. Phys. Rev. E
**2003**, 67, 26126. [Google Scholar] [CrossRef] [PubMed] - Solé, R.V.; Valverde, S. Chapter: Information Theory of Complex Networks: On Evolution and Architectural Constraints. In Networks: Structure, Dynamics and Function; Lecture Notes in Physics; Springer-Verlag: Berlin, Germany, 2004; pp. 169–190. [Google Scholar]
- Read, R.; Wilson, R. An atlas of graphs; Oxford University Press: Oxford, UK, 1998. [Google Scholar]
- Erdös, P.; Rényi, A. On the Evolution of Random Graphs. Publ. Math. Inst. Hung. Acad. Sci.
**1960**, 5, 17. [Google Scholar] - Erdös, P.; Rényi, A. On the Evolution of Random Graphs. Bull. Inst. Int. Stat.
**1961**, 38, 343. [Google Scholar] - Bollobás, B. Random Graphs, 2nd ed.; Academic Press: New York, NY, USA, 2001. [Google Scholar]
- Rapoport, A. Spread of Information through a Population with Socio- Structural Bias I. Assumption of Transitivity. Bull. Math. Biophys.
**1953**, 15, 523–533. [Google Scholar] [CrossRef] - Rapoport, A. Spread of Information through a Population with Socio- Structural Bias II. Various Models with Partial Transitivity. Bull. Math. Biophys.
**1953**, 15, 533–546. [Google Scholar] [CrossRef] - Rapoport, A. Spread of Information through a Population with Socio- Structural Bias III. Suggested Experimental Procedures. Bull. Math. Biophys.
**1954**, 16, 75–81. [Google Scholar] [CrossRef] - Rapoport, A.; Horvath, W. A Study of a Large Sociogram. Behav. Sci.
**1961**, 6, 79–91. [Google Scholar] [CrossRef] - Watts, D.J. Small Worlds: The Dynamics of Networks between Order and Randomness; Princeton University: Princeton, NJ, USA, 1999. [Google Scholar]
- Klemm, K.; Eguiluz, V. Highly clustered scale-free networks. Phys. Rev. E
**2002**, 65, 36123. [Google Scholar] [CrossRef] [PubMed] - Girvan, M.; Newman, M.E.J. Community structure in social and biological networks. Proc. Natl. Acad. Sci. U.S.A.
**2002**, 99, 7821–7826. [Google Scholar] [CrossRef] [PubMed] - Guimerà, R.; Danon, L.; Díaz-Guilera, A.; Giralt, F.; Arenas, A. Self-similar community structure in a network of human interactions. Phys. Rev. E
**2003**, 68, 065103. [Google Scholar] [CrossRef] [PubMed] - Gleiser, P.; Danon, L. Community structure in jazz. Adv. Compl. Syst.
**2003**, 6, 565–573. [Google Scholar] [CrossRef] - Arenas, A.; Fernandez, A.; Gomez, S. Analysis of the structure of complex networks at different resolution levels. New J. Phys.
**2008**, 10, 053039. [Google Scholar] [CrossRef] - Newman, M.E.J.; Girvan, M. Finding and evaluating community structure in networks. Phys. Rev. E
**2004**, 69, 026113. [Google Scholar] [CrossRef] [PubMed] - Gómez, S.; Jensen, P.; Arenas, A. Analsys of community structure in networks of correlated data. Phys. Rev. E
**2009**, 80, 016114. [Google Scholar] [CrossRef] [PubMed] - Duch, J.; Arenas, A. Community detection in complex networks using extremal optimization. Phys. Rev. E
**2005**, 72, 027104. [Google Scholar] [CrossRef] [PubMed] - Newman, M.E.J. Fast algorithm for detecting community structure in networks. Phys. Rev. E
**2004**, 69, 321–330. [Google Scholar] [CrossRef] [PubMed] - Arenas, A.; Duch, J.; Fernández, A.; Gómez, S. Size reduction of complex networks preserving modularity. New J. Phys.
**2007**, 9, 176. [Google Scholar] [CrossRef] - Danon, L.; Duch, J.; Arenas, A.; Díaz-Guilera, A. Comparing community structure identification. J. Stat. Mech.
**2005**, P09008. [Google Scholar] [CrossRef] - Sergio Gómez homepage. Available online: http://deim.urv.cat/ sgomez/radatools.php (accessed on 15 April 2010).
- Arenas, A.; Díaz-Guilera, A.; Pérez-Vicente, C. Synchronization reveals topological scales in complex networks. Phys. Rev. Lett.
**2006**, 96, 114102. [Google Scholar] [CrossRef] [PubMed] - Arenas, A.; Fernández, A.; Gómez, S. A complex network approach to the determination of functional groups in the neural system of C. Elegans. Lect. Notes Comput. Sci.
**2008**, 5151, 9–18. [Google Scholar] - Mehler, A. Chapter: Large Text Networks as an Object of Corpus Linguistic Studies. In Corpus Linguistics. An International Handbook; de Gruyter: Berlin, Germany/New York, NY, USA, 2007. [Google Scholar]
- Sinclair, J. Corpus, Concordance, Collocation; Oxford University Press: Oxford, UK, 1991. [Google Scholar]
- Stubbs, M. Words and Phrases: Corpus Studies of Lexical Semantics; Blackwell Pub.: Cambridge, MA, USA, 2001. [Google Scholar]
- Stubbs, M. Text and Corpus Analysis; Blackwell Pub.: Cambridge, MA, USA, 1996. [Google Scholar]
- Halliday, M. Lexis as a linguistic level. In memory of JR Firth; 1966; pp. 148–162. [Google Scholar]
- Antiqueira, L.; Nunes, M.; Oliveira, O.; Costa, L. Strong correlations between text quality and complex networks features. Phys. A
**2006**, 373, 811–820. [Google Scholar] [CrossRef] - Ferrer i Cancho, R.; Solé, R. The small-world of human language. Proc. R. Soc. Lond., B, Biol. Sci.
**2001**, 268, 2261–2265. [Google Scholar] [CrossRef] [PubMed] - Collins, P.; Peters, P. The Australian corpus project. In Corpus linguistics, hard and soft, Proceedings of the Eighth International Conference on English Language Research on Computerized Corpora; Editions Rodopi: Atlanta, GA, USA, 1988; p. 103. [Google Scholar]
- Sigman, M.; Cecchi, G. Global organization of the WordNet lexicon. Proc. Natl. Acad. Sci. U.S.A.
**2002**, 99, 1742–1747. [Google Scholar] [CrossRef] [PubMed] - McRae, K.; Cree, G.; Seidenberg, M.; McNorgan, C. Semantic feature production norms for a large set of living and nonliving things. Behav. Res. Meth.
**2005**, 37, 547–559. [Google Scholar] [CrossRef] - Nelson, D.L.; McEvoy, C.L.; Schreiber, T.A. The University of South Florida word association, rhyme, and word fragment norms. Behav. Res. Meth.
**1998**, 36, 402–407. [Google Scholar] [CrossRef] - Callejas, A.; Correa, A.; Lupiáñez, J.; Tudela, P. Normas asociativas intracategoriales para 612 palabras de seis categorías semánticas en español. Psicológica
**2003**, 24, 185–214. [Google Scholar] - Fernández, A.; Díez, E.; Alonso, M.; Beato, M. Free-association norms for the Spanish names of the Snodgrass and Vanderwart pictures. Behav. Res. Meth.
**2004**, 36, 577–583. [Google Scholar] [CrossRef] - Melinger, A.; Weber, A. Database of Noun Associations for German. Available online: http://www.coli.uni-saarland.de/projects/nag/ (accessed on 10 April 2010).
- Ferrand, L.; Alario, F.X. Normes d’associations verbales pour 366 noms d’objets concrets. L’Année Psychol.
**1998**, 98, 531–552. [Google Scholar] [CrossRef] - Motter, A.E.; de Moura, A.P.S.; Lai, Y.C.; Dasgupta, P. Topology of the conceptual network of language. Phys. Rev. E
**2002**, 65, e065102. [Google Scholar] [CrossRef] [PubMed] - ftp://ibiblio.org/pub/docs/books/gutenberg/etext02/mthes10.zip (accessed on 27 December 2009).
- Dorogovtsev, S.; Mendes, J. Language as an evolving word web. Proc. R. Soc. Lond., B, Biol. Sci.
**2001**, 268, 2603–2606. [Google Scholar] [CrossRef] [PubMed] - Snodgrass, J.; Vanderwart, M. A standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. J. Exp. Psychol. Hum. Learn.
**1980**, 6, 174–215. [Google Scholar] [CrossRef] [PubMed] - Ferrer i Cancho, R. Zipf’s law from a communicative phase transition. Eur. Phys. J. B
**2005**, 47, 449–457. [Google Scholar] [CrossRef] - Ferrer i Cancho, R. When language breaks into pieces. A conflict between communication through isolated signals and language. BioSystems
**2006**, 84, 242–253. [Google Scholar] [CrossRef] [PubMed] - Zipf, H. Human behaviour and the Principle of Least Effort: An Introduction to Human Ecology; Addison Wesley: Cambridge, MA, USA, 1949. [Google Scholar]
- Shannon, C.; Weaver, W. A mathematical theory of communication. Bell Syst. Tech. J.
**1948**, 27, 623. [Google Scholar] [CrossRef] - Ferrer i Cancho, R.; Solé, R. Least effort and the origins of scaling in human language. Proc. Natl. Acad. Sci. U.S.A.
**2003**, 100, 788. [Google Scholar] [CrossRef] [PubMed] - Steyvers, M.; Tenenbaum, J.B. The largescale structure of semantic networks: statistical analyses and a model of semantic growth. Cog. Sci.
**2005**, 29, 41–78. [Google Scholar] [CrossRef] [PubMed] - Collins, A.M.; Quillian, M.R. Retrieval time from semantic memory. J. Verbal Learn. Verbal Beh.
**1969**, 8, 240–247. [Google Scholar] [CrossRef] - Collins, A.; Loftus, E. A spreading activation theory of semantic memory. Psychol. Rev.
**1975**, 82, 407–428. [Google Scholar] [CrossRef] - Griffiths, T.; Steyvers, M.; Firl, A. Google and the mind: Predicting fluency with PageRank. Psychol. Sci.
**2007**, 18, 1069–1076. [Google Scholar] [CrossRef] [PubMed] - Arenas, A.; Cabrales, A.; Danon, L.; Díaz-Guilera, A.; Guimerà, R.; Vega-Redondo, F. Optimal information transmission in organizations: search and congestion. Rev. Econ. Des.
**2010**, 14, 75–93. [Google Scholar] [CrossRef] - Danon, L.; Arenas, A.; Díaz-Guilera, A. Impact of community structure on information transfer. Phys. Rev. E
**2008**, 77, 36103. [Google Scholar] [CrossRef] [PubMed] - Goñi, J.; Martincorena, I.; Corominas-Murtra, B.; Arrondo, G.; Ardanza-Trevijano, S.; Villoslada, P. Switcher-random-walks: a cognitive inspired mechanism for network exploration. Arxiv
**2009**, 0903.4132. [Google Scholar] [CrossRef] - Troyer, A.; Moscovitch, M.; Winocur, G. Clustering and switching as two components of verbal fluency: Evidence from younger and older healthy adults. Neuropsycholgy
**1997**, 11, 138–146. [Google Scholar] [CrossRef] - Bousfield, W.; Sedgewick, C. An analysis of sequences of restricted associative responses. J. Gen. Psychol.
**1944**, 30, 149–165. [Google Scholar] [CrossRef] - Borge-Holthoefer, J.; Arenas, A. Navigating word association norms to extract semantic information. In Proceedings of the 31st Annual Conference of the Cognitive Science Society, Amsterdam, The Netherlands; 2009. [Google Scholar]
- Borge-Holthoefer, J.; Arenas, A. Categorizing words through semantic memory navigation. Eur. Phys. J. B
**2010**, 74, 265–270. [Google Scholar] [CrossRef] - Landauer, T.; Dumais, S. A solution to Plato’s problem: the Latent Semantic Analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev.
**1997**, 104, 211–240. [Google Scholar] [CrossRef] - Landauer, T.; McNamara, D.; Dennis, S.; Kintsch, W. Handbook of Latent Semantic Analysis; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 2007. [Google Scholar]
- Steyvers, M.; Shiffrin, R.M.; Nelson, D.L. Word association spaces for predicting semantic similarity effects in episodic memory. In Experimental cognitive psychology and its applications; Healy, A.F., Ed.; American Psychological Association: Washington, DC, USA, 2004; pp. 237–249. [Google Scholar]
- Steels, L. Language as a complex adaptive system. Lect. Notes Comput. Sci.
**2000**, 1917, 17–28. [Google Scholar] - Steels, L. The emergence and evolution of linguistic structure: from lexical to grammatical communication systems. Conn. Sci.
**2005**, 17, 213–230. [Google Scholar] [CrossRef] - Baronchelli, A.; Felici, M.; Loreto, V.; Caglioti, E.; Steels, L. Sharp transition towards shared vocabularies in multi-agent systems. J. Stat. Mech.
**2006**, 6, P06014. [Google Scholar] [CrossRef] - Sporns, O.; Kötter, R. Motifs in brain networks. PLoS Biol.
**2004**, 2, e369. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Sporns, O.; Chialvo, D.; Kaiser, M.; Hilgetag, C. Organization, development and function of complex brain networks. Trends Cogn. Sci.
**2004**, 8, 418–425. [Google Scholar] [CrossRef] [PubMed] - Zhou, C.; Zemanová, L.; Zamora-López, G.; Hilgetag, C.; Kurths, J. Hierarchical Organization Unveiled by Functional Connectivity in Complex Brain Networks. Phys. Rev. Lett.
**2006**, 97, 238103. [Google Scholar] [CrossRef] [PubMed] - Zhou, C.; Zemanová, L.; Zamora-López, G.G.; Hilgetag, C.; Kurths, J. Structure–function relationship in complex brain networks expressed by hierarchical synchronization. New J. Phys.
**2007**, 9, 178. [Google Scholar] [CrossRef] - Friederici, A. Pathways to language: fiber tracts in the human brain. Trends Cogn. Sci.
**2009**, 13, 175–181. [Google Scholar] [CrossRef] [PubMed] - Hillis, A.; Caramazza, A. Category-specific naming and comprehension impairment: A double dissociation. Brain
**1991**, 114, 2081–2094. [Google Scholar] [CrossRef] [PubMed] - Martin, A.; Wiggs, C.; Ungerleider, L.; Haxby, J. Neural correlates of category-specific knowledge. Nature
**1996**, 379, 649–652. [Google Scholar] [CrossRef] [PubMed] - Polyn, S.; Natu, V.; Cohen, J.; Norman, K. Category-specific cortical activity precedes retrieval during memory search. Science
**2005**, 310, 1963. [Google Scholar] [CrossRef] [PubMed] - Mitchell, T.; Shinkareva, S.; Carlson, A.; Chang, K.; Malave, V.; Mason, R.; Just, M. Predicting human brain activity associated with the meanings of nouns. Science
**2008**, 320, 1191. [Google Scholar] [CrossRef] [PubMed] - Binder, J.; Desai, R.; Graves, W.; Conant, L. Where is the semantic system? Cereb Cortex
**2009**, 120, 2767–2796. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**An illustration of the concept of clustering C, calculated on the gray node. In the left figure, every neighbor of the mentioned node is connected to each other; therefore, clustering coefficient is 1. In the middle picture, only two of the gray node neighbors’ are connected, yielding a clustering coefficient of 1/3; finally, in the last illustration none of the gray node’s neighbors are linked to each other, which yields a clustering coefficient of 0. From Wikipedia Commons.

**Figure 2.**From regularity to randomness: note the changes in average path length and clustering coefficient as a function of the rewiring probability $L\left(p\right)$, $C\left(p\right)$ for the family of randomly rewired graphs. For low rewiring probabilities the clustering is still close to its initial value, whereas the average path length has already decreased significantly. For high probabilities, the clustering has dropped to an order of ${10}^{-2}$. This figure illustrates the fact that small-world is not a network, but a family of networks.

**Figure 3.**Cumulative degree distribution for a SF with $N=10000$, constructed according to the BA model. For each node entering the network, 3 new edges are placed. The horizontal axis is vertex degree k and the vertical axis is the cumulative probability distribution of degrees, i.e., the fraction of vertices that have degree greater than or equal to k.

**Figure 4.**The RB model yields a hierarchical network, that combines the scale-free property with a high degree of clustering. The starting point is a small cluster of five densely linked nodes; next, four replicas of this hypothetical module are generated. The four external nodes of the replicated clusters are connected to the central node of the old cluster, obtaining a large 25-node module. This replication and connection can be repeated recursively, thus obtaining networks of size 25, 125, etc.

**Figure 5.**A network structure out of semantic features data. Left: each subject assigns semantic features to given nouns, and features build up a semantic vector. In the example, features are is alive, has tail, is wild, can fly, is underwear, is long, is warm and has buttons. The number in each cell reflects the number of participants who assigned that feature to the corresponding item. Right: cosine overlapping between each pair of vectors from the left matrix. This new similarity matrix can be suitably interpreted as a semantic network. Note that values in both matrices do not represent actual results, and have been put merely for illustrative purposes.

**Figure 6.**Dorogovstev and Mendes’ scheme of the language network growth [76]: a new word is connected to some old one i with the probability proportional to its degree ${k}_{i}$ (Barabási and Albert’s preferential attachment); in addition, at each increment of time, $ct$ new edges emerge between old words, where c is a constant coefficient that characterizes a particular network.

**Figure 7.**Plots of the cumulative degree distribution in four networks. All of them have been converted to unweighted and undirected. (a) WordNet, hypernymy relationships; (b) Co-occurrence networks for variable window size, from the ACE corpus; (c) English Free Association Norms (USF-FA); (d) Roget’s thesaurus. Note that the plots are drawn in log-log scale. Only (a) and (b) display a power-law decay, whereas (c) and (d) do not follow a scale-free distribution. All of them, nonetheless, fit in the small-world definition.

**Figure 8.**Directions and weights matter. Left: log-log plots of the cumulative degree distributions for psycholinguistic data in four languages (from top to bottom: USF, SFA-SV, SFA and GFA). Directions are symmetrized and weights are not taken into account. Right: log-log plots of the cumulative in-strength distribution for the same data without manipulation. Note that there exist striking differences between degree and strength distributions of psycholinguistic data. These differences are also evident in other descriptors, which suggests that comprehension about cognitive-linguistic processes demand attention to such details.

**Figure 9.**The Collins and Quillians tree data structure provides a particularly economical system for representing knowledge about categories. The cognitive economy principle prevents the structure from having redundant information, thus features which belong to one level do not appear in any other. Despite some positive experimental results with humans, the structure is far too rigid to accommodate actual semantic knowledge.

**Figure 10.**An illustration of the output of the PageRank algorithm. A link from an important web page is a better indicator of importance than a link from an unimportant web page. Under such a view, an important web page is one that receives many links from other important web pages. From Wikipedia Commons.

**Table 1.**Results for the conceptual network defined by the Thesaurus dictionary, and a comparison with a corresponding random network with the same parameters. N is the total number of nodes, $\langle k\rangle $ is the average number of links per node, C is the clustering coefficient, and L is the average shortest path. After [74].

N | $\langle k\rangle $ | C | L | |

Moby Thesaurus | 30244 | 59.9 | 0.53 | 3.16 |

Randomized MT | 30244 | 59.9 | 0.002 | 2.5 |

**Table 2.**Some parameters obtained from four different data-sets: the University of South Florida word association (USF-FA, [69]), Free-association norms for the Spanish names of the Snodgrass and Vanderwart pictures (SFA-SV, [71]), association norms in Spanish (SFA, [70]) and association norms for the German names of the Snodgrass and Vanderwart pictures (GFA-SV, [72]). As the ones reported on Table 1, they all conform sparse structures with very low L (if compared to the size of the network). However, only USF-FA and SFA clearly fit in the small-world definition. Low C in the data sets based on the drawings from Snodgrass and Vanderwart [77] can be explained by the specific experimental setup with this material. N is the total number of nodes, $\langle k\rangle $ is the average number of links per node, C is the clustering coefficient, L is the average shortest path, and D is the diameter. The latter descriptors (L and D) have been measured from the undirected, unweighted networks of the data sets.

N | $\langle k\rangle $ | C | L | D | |

USF-FA | 5018 | 22 | 0.1928 | 3.04 | 5 |

SFA-SV | 7759 | 3.05 | 0.0867 | 3.71 | 5 |

SFA | 2901 | 4.9 | 0.1487 | 4.50 | 8 |

GFA-SV | 3632 | 2.05 | 0.034 | 4.57 | 8 |

Graph | Source Network | Vertex | Edge | Orient. | N | $\langle k\rangle $ | L | C | Reference |
---|---|---|---|---|---|---|---|---|---|

thesaurus graph | Moby’s thesaurus | word | sense relation | undir. | 30,244 | 59.9 | 3.16 | 0.53 | [74] |

collocation graph | BNC corpus | word | collocation | undir. | 460,902 | 70.13 | 2.67 | 0.44 | [65] |

co-occurrence graph | BNC corpus | word | co-occurrence | undir. | 478,773 | 74.2 | 2.63 | 0,69 | [65] |

thesaurus graph | Roget’s thesaurus | word | sense relation | undir. | 29,381 | S. (3.3) | 5.60 | 0.87 | [83] |

concept graph | WordNet | word | sense relation | undir. | 122,005 | 3.3 | 10.56 | 0.03 | [83] |

association graph | free assoc. data | word | association | undir. | 5,018 | 22.0 | 3.04 | 0.19 | [83] |

association graph | free assoc. data | word | association | dir. | 5,018 | 12.7 | 4.27 | 0.19 | [83] |

**Table 4.**Results of model simulations (undirected version). γ is the exponent of the power-law that describes $P\left(k\right)$. Standard deviations of 50 simulations given between parentheses.

N | $\langle k\rangle $ | C | L | D | γ | |

USF-FA | 5018 | 23.5 | 0.1928 | 3.04 | 5 | 3.01 |

Synthetic USF | 5018 | 22 | 0.174 | 3.00(.012) | 5(.000) | 2.95(.054) |

**Table 5.**Statistical parameters for Free Association norms FA (substrate of the dynamic process), Feature Production norms FP (empirical target) , and the synthetic networks obtained using Latent Semantic Analysis LSA, Word Association Space WAS and Random Inheritance Model RIM.

Descriptor | FA | FP | LSA | WAS | RIM |

N | 376 | 376 | 376 | 376 | 376 |

$\langle s\rangle $ | 0.26 | 13.43 | 39.60 | 10.29 | 15.62 |

L | 4.41 | 1.68 | 0.02 | 2.00 | 1.77 |

D | 9 | 3 | 2 | 4 | 3 |

C | 0.192 | 0.625 | 0.961 | 0.492 | 0.584 |

r | 0.325 | 0.295 | 0.125 | 0.303 | 0.305 |

**Table 6.**Some illustrative examples of LSA, WAS and RIM’s predictive capacity, when comparing closest neighbors to McRae’s FP.

TUBA | |||

FP | LSA | WAS | RIM |

trombone | clarinet | bathtub | trombone |

trumpet | violin | faucet | saxophone |

drum | flute | sink | trumpet |

cello | guitar | bucket | flute |

clarinet | trombone | bridge | clarinet |

saxophone | fork | submarine | cello |

flute | trumpet | drain | violin |

harp | cake | raft | harp |

banjo | drum | tap | banjo |

piano | piano | dishwasher | stereo |

ERROR | 4.83 | 11 | 2.5 |

ROOSTER | |||

chicken | cat | chicken | chicken |

goose | gate | crow | turkey |

pigeon | donkey | skillet | crow |

sparrow | barn | rice | robin |

penguin | turnip | spinach | sparrow |

pelican | owl | bowl | bluejay |

bluejay | pig | beans | pigeon |

dove | fence | robin | pelican |

hawk | lion | tomato | goose |

turkey | strawberry | sparrow | hawk |

ERROR | 11 | 8.5 | 2.87 |

© 2010 by the authors; licensee MDPI, Basel, Switzerland. This article is an Open Access article distributed under the terms and conditions of the Creative Commons Attribution license http://creativecommons.org/licenses/by/3.0/.

## Share and Cite

**MDPI and ACS Style**

Borge-Holthoefer, J.; Arenas, A.
Semantic Networks: Structure and Dynamics. *Entropy* **2010**, *12*, 1264-1302.
https://doi.org/10.3390/e12051264

**AMA Style**

Borge-Holthoefer J, Arenas A.
Semantic Networks: Structure and Dynamics. *Entropy*. 2010; 12(5):1264-1302.
https://doi.org/10.3390/e12051264

**Chicago/Turabian Style**

Borge-Holthoefer, Javier, and Alex Arenas.
2010. "Semantic Networks: Structure and Dynamics" *Entropy* 12, no. 5: 1264-1302.
https://doi.org/10.3390/e12051264