# On the Interplay between Entropy and Robustness of Gene Regulatory Networks

^{*}

## Abstract

**:**

_{∞}stabilization and filtering perspective. In this review, we will also discuss their balancing roles in evolution and potential applications in systems and synthetic biology.

## 1. Introduction

_{∞}filtering [27].

_{∞}stabilization perspective to tolerate intrinsic random molecular noises and extrinsic random molecular noises [35,36].

_{∞}filtering perspectives, simultaneously [10,40,41,42,43,44,45].

## 2. On the Entropy Measure of Gene Regulatory Networks

_{0(}t) = [X

_{01}(t)…X

_{0}

_{n}(t)]

^{T}, ${S}_{0}=\left[\begin{array}{ccc}{S}_{0,11}& \cdots & {S}_{0,1n}\\ \vdots & \ddots & \vdots \\ {S}_{0,n1}& \cdots & {S}_{0,nn}\end{array}\right]$. X

_{0}(t) denotes the nominal gene expressions of n genes and S

_{0}denotes the stoichiometric matrix, which collects all the kinetic parameters and decay rates of gene regulatory network. In vivo, the kinetic parameters and decay rates in S

_{0}suffer from the random parameter fluctuations due to gene expression fluctuations in binding, transcription, splicing and translation processes:

_{0,ij}→S

_{0,ij}+ΔS

_{l}

_{,ij}n

_{l}(t)

_{0,ij}denotes the nominal parameters; ΔS

_{l}

_{,ij}denotes the amplitude of random kinetic parameter fluctuations; n

_{l}(t) is a white noise with zero mean and unit variance, i.e., ΔS

_{l}

_{,ij}denotes the deterministic parts of the corresponding parameter variations, and n

_{l}(t) absorbs the stochastic property of intrinsic parameter fluctuations. n

_{1}(t), n

_{2}(t),…, n

_{L}(t) are independent stochastic sources of random parameter fluctuations, where covariances are given as:

_{l}

_{,ij}n

_{l}(t),ΔS

_{l}

_{,ij}n

_{l}(τ)] = ΔS

_{l}

_{,ij}

^{2}δ(t,τ)

_{l}

_{,ij}denotes the corresponding standard deviation of the stochastic parameter fluctuationsΔS

_{l}

_{,ij}n

_{l}(t) from the random source n

_{l}(t). The reason why n

_{l}(t) in (2) are assumed to be zero mean with unit variance is that the mean of n

_{l}(t) could be merged in the nominal S

_{0,ij}and the variance can be absorbed in ΔS

_{l}

_{,ij}in (2). Therefore, without loss of generality, n

_{l}(t) are assumed white noises with zero mean and unit variance.

_{1}(t) …v

_{i}(t)…v

_{n}(t)]

^{T}.

_{l}denotes the parameter fluctuation matrix infected by the random source, n

_{l}(t), for example, the alternative splicing in gene expression process. If the ij-th component of S

_{0}is free of parameter fluctuation due to random source, n

_{l}(t), then ΔS

_{l}

_{,ij}in S is zero. In (4), there are L random fluctuation sources. G denotes the coupling matrix from environmental disturbances to the gene network. v

_{i}(t) denotes the environmental molecular disturbance on gene i due to upstream genes’ regulations or interactions with cellular context.

_{l}(t) is a standard Wiener process or Brownian motion with dw

_{l}(t) = n

_{l}(t)dt. The term ${\sum}_{l=1}^{L}{S}_{l}X(t)d{w}_{l}(t)$ denotes the stochastic parameter fluctuations due to L random sources. Obviously, in vivo, a linear gene regulatory network under random parameter fluctuations and external disturbance can be represented by the stochastic system in (5).

_{v}denotes the covariance of external noise.

_{v}of the covariance matrix of external random noise and $\sum}_{l=1}^{L}{S}_{l}^{T}{S}_{l}{R}_{X$ due to the intrinsic random parameter fluctuations.

_{0}) denotes the nonlinear regulation function vector.

## 3. Measurement of Robust Stability and Filtering Ability for Stochastic Gene Regulatory Networks

_{e}is of interest. For the convenience of analysis, the origin of nonlinear stochastic gene network in (11) is shifted to x

_{e}. In such a situation, if the shifted nonlinear stochastic system is robustly stable at the origin, then the equilibrium point of interest is also equivalently robustly stable. This will simplify the analysis procedure of robust stability and filtering of gene regulatory networks. Let us denote $\tilde{X}(t)=X(t)-{x}_{e}$, then we get the following shifted stochastic system [26]:

_{e}of the original stochastic system in (11).

_{e}. The attenuation level of external disturbance v(t) by the gene regulatory network can be measured by the following H

_{∞}filtering ability [35,40,55,56,57,58]:

^{2}from the energy point of view or the attenuation level ρ is the upper bound of the filtering ability of the gene network.

_{e}in probability in the case v(t) = 0, or the stochastically asymptotical stability is achieved.

_{∞}disturbance attenuation level in (17) can be measured because the asymptotical stability in probability (or stochastically asymptotical stability) can not be achieved due to the continuous interference of stochastic external disturbances, i.e., $\tilde{X}(t)\to 0$ or X(t)→x

_{e}, can not be achieved in probability as t→∞ and the stochastic deviation from x

_{e}(i.e., $\tilde{X}(t)$) due to stochastic external disturbances can only be attenuated below a level ρ, which is the upper bound of the filtering ability of stochastic gene network of (16).

_{e}in probability.

_{0}of the stochastic gene regulatory network in (11) or (16) could be found by solving the following constrained optimization problem [27,32]:

_{0}is a stochastic disturbance attenuation measure or the filtering ability of a stochastic gene network to attenuate the stochastic disturbance v(t) in (16) under robust stabilization to tolerate intrinsic parameter fluctuation at x

_{e}. If ρ

_{0}<1, then the stochastic external disturbance is attenuated by the gene regulatory network. If ρ

_{0}>1, then the stochastic external disturbance is amplified by the gene regulatory network. In some cases, extrinsic noise is enhanced by some gene networks [9]. (iii) If the Wiener noises w

_{l}(t) in (16) are replaced by the mutually independent Poisson noises p

_{l}(t) with mean E[p

_{l}(t)] = λ

_{l}t and variance var[p

_{l}(t)] = λ

_{l}t as follows:

_{0}denotes the convex hull of the polytope with M vertices defined in (25), then the state trajectories $\tilde{X}(t)$ of the shifted stochastic gene network in (16) will belong to the convex combination of the state trajectories of the following M linearized stochastic gene networks derived from the vertices of the polytope [42]:

_{0}, then the original nonlinear stochastic gene network in (16) will have the same robust stabilization and disturbance attenuation property. The convex combination of M linearized gene networks in (26) can be written as [42,62]:

_{e}under the parameter fluctuations.

_{e}is to be achieved.

_{e}is achieved.

_{i}, F

_{li}and G

_{i}are of different values by the interpolation of the fuzzy bases ${\beta}_{i}(\tilde{X})$ [35]. By the global linearization in (27) or fuzzy interpolation in (29) the optimal disturbance attenuation level or the filtering ability ρ

_{0}of the stochastic gene network in (22) can be measured by solving the following LMI constrained optimization [27,41]:

_{0}of stochastic gene network can be achieved by decreasing ρ in (33) until there doesn’t exist a positive solution P>0 via the software algorithm of LMI toolbox in Matlab [62].

_{0}F

_{i}are more negative, i.e., in the far left-hand side of s-complex domain, then the stochastic gene network has more robust stability to tolerate random parameter fluctuations. From (32), if ρ is small, i.e., to attenuate more environmental disturbances, the eigenvalues of S

_{0}F

_{i}of stochastic gene network should be in more far the left-hand s-complex domain in order to attenuate environmental random noises to a small level ρ. (ii) Recently, the filtering ability and enhancing ability of random noises in neural firing system are also discussed in [63]. It is found that the activation random noise is filtered but the Na

^{+}and K

^{+}channel noises are enhanced.

_{1}(t), …, x

_{n}

_{+m}(t) are metabolites, such as substrates, enzymes, factors or products of a biochemical network in which x

_{1}(t), …, x

_{n}(t) denote the n-dependent variables (intermediate metabolites and products), and x

_{n}

_{+1}(t), …, x

_{n+m}(t) denote the independent variables (initial reactants and enzymes), α

_{i}and β

_{j}denote the rate constants, and g

_{ij}and h

_{ij}represent the kinetic parameters of the biochemical network. These parameters could be estimated by experimental data or microarray data. Suppose transient time is neglected and for simplicity we shall focus on the randomness and robustness of biochemical network at the steady state case. Consider the steady state of biochemical network in (35), we get [61]:

_{j}(t) = ln[x

_{j}(t)], a

_{ij}= g

_{ij}-h

_{ij}, b

_{i}= ln(β

_{j}/α

_{i}) and after some rearrangement, we get [61]:

_{D}Y

_{D}(t) = b-A

_{I}Y

_{I}(t)

_{D}(t) = [y

_{1}(t)…y

_{n}(t)]

^{T}, b = [b

_{1}…b

_{n}]

^{T}, Y

_{I}(t) = [y

_{n}

_{+1}(t)…y

_{n}

_{+}

_{m}(t)]

^{T}, ${A}_{D}=\left[\begin{array}{ccc}{a}_{11}& \cdots & {a}_{1n}\\ \vdots & \ddots & \vdots \\ {a}_{n1}& \cdots & {a}_{nn}\end{array}\right]$ and ${A}_{I}=\left[\begin{array}{ccc}{a}_{1,n+1}& \cdots & {a}_{1,n+m}\\ \vdots & \ddots & \vdots \\ {a}_{n,n+1}& \cdots & {a}_{n,n+m}\end{array}\right]$ in which A

_{D}denotes the system matrix of the catalytic interactions among the dependent variables A

_{D}and A

_{I}indicates the catalytic interactions between the dependent variables Y

_{D}(t) and the independent variables Y

_{I}(t) (i.e., the environmental medium of the metabolic system). From the simple algebraic steady state equation in (38), obviously, the S-system in (35) is an useful model for describing the phenotype of biochemical network [61]. If the inverse of A

_{D}exists, the steady state (or phenotype) of the biochemical network is solved by [61]:

_{D}(t) = A

_{D}

^{-1}(b-A

_{I}Y

_{I}(t))

_{D}(t) in (39) is one of the equilibrium points of the nonlinear biochemical network in (35). Actually, there are many equilibrium points for (35), which represent different phenotypes. Only the equilibrium point (or phenotype) in (39) is favored by natural selection in evolution. Actually, a real biochemical network in (35) suffers from random parameter fluctuations and environmental disturbance. i.e.:

_{i}→α

_{i}+Δα

_{i}(t), β

_{i}→β

_{i+}Δβ

_{i}(t), h

_{ij}→h

_{ij+}Δh

_{ij}(t), g

_{ij}→g

_{ij}+Δg

_{ij}(t), Y

_{I}(t)→Y

_{I}(t)+ΔY

_{I}(t)

_{D}+ΔA

_{D}(t))(Y

_{D}(t)+ΔY

_{D}(t)) = (b+Δb(t))-(A

_{I}+ΔA

_{I}(t)) (Y

_{I}(t)+ΔY

_{I}(t))

_{D}(t)+ΔY

_{D}(t) = (A

_{D}+ΔA

_{D}(t))

^{-1}[(b+Δb(t))-(A

_{I}+ΔA

_{I}(t)) (Y

_{I}(t)+ΔY

_{I}(t))]

_{D}(t) of system matrix is less than A

_{D}A

_{D}

^{T}, then the random parameter fluctuations will be tolerated and the phenotype of biochemical network is only with a small perturbation as (43).

_{I}(t) and ΔA

_{I}(t) will influence the phenotypic variation ΔY

_{D}(t) in (43). Their effects on the phenotype have been discussed by the following sensitivity analysis of biochemical network:

_{I}(t) and ΔA

_{I}(t) to preserve the phenotype of a biochemical network, the sensitivity in (44) should be below some values as follows [65]:

_{1}, s

_{2}and s

_{3}are some small sensitivity values so that the phenotype Y

_{D}(t)+ΔY

_{D}(t) of stochastically perturbed biochemical networks would not change too much in (43) in comparison with the nominal values in (39), i.e., ΔY

_{D}(t) in (43) should be small enough. The sensitivity criteria in (45) or (46) determine the ranges of sensitivities of phenotype change ΔY

_{D}(t) to random parameter and environmental fluctuations. For a functional biochemical network, it should satisfy the sensitivity criteria to prevent metabolic concentration from being changed too much by parametric and environmental random changes.

## 4. Interplay Between Entropy and Robustness in the Evolutionary Process of Gene Networks

_{e}denotes the trait of gene network. As seen in Figure 1, the genetic network lies at the lowest level of the biological network evolution. In order to maintain the proper function of the favored biochemical network at high-level, the lower-level genetic networks have to function properly. Therefore, in evolution, robust stability is the natural selection force specified by higher-level networks. A stochastic gene network in evolutionary process can be represented as (11) but the random parameter fluctuations are mainly due to genetic mutations of genes in sex-chromosome in the evolutionary process. Further, the time-scale of gene network is much larger in the evolutionary process. Therefore, the robust stability condition of a gene network at the equilibrium point x

_{e}(trait) is the same as (20) except the parameter fluctuations S

_{l}mainly due to the random gene mutations of sex-chromosome in evolutionary process, i.e., there should exist a positive solution $V(\tilde{X})>0$ for HJI in (20) in the disturbance-free case [52]. If the gene network can tolerate the intrinsic random fluctuation and can attenuate the environmental disturbance to a level ρ simultaneously in the evolutionary process, then there should exist a positive solution $V(\tilde{X})>0$ in (21) [52]. Similarly, by the global linearization technique, the robust stability of gene network to tolerate intrinsic genetic mutations in the evolutionary process is guaranteed for a trait if there exists a positive solution P>0 for LMIs in (31). In this situation, the trait of gene network is kept under gene mutations in evolution. The disturbance filtering ability ρ

_{0}of a gene network in the evolutionary process could be measured by solving the constrained optimization problem in (34). This could measure the influence of environment on the gene network in the evolution process. Obviously, as remarks 6 and 7, if the eigenvalues of linearized gene networks are all in the far left-hand side of s-complex domain, then a gene network has enough robustness to tolerate intrinsic random genetic mutation and filter environmental random disturbances to keep its trait in evolution to avoid extinction by natural selection. The genetic mutations, which could lead to feedback circuits to improve robustness and noise filtering ability, are favored by natural selection in the evolutionary process. However, some gene mutations, which could make the corresponding parameter fluctuations S

_{l}and coupling function g(X) small, such as those mutations to redundancy, duplicating and self-regulation so that it is not easy to violate robust stability conditions (20)-(21) or (31)-(32), due to these buffers, are also favored by natural selection [68,69]. If filtering ability ρ

_{0}<1 in (22) or (34), the eigenvalues of linearized gene networks should be in the far left-hand of s-complex domain side to attenuate the environmental random noises. The trait of this kind of gene networks is more robust in evolution. If ρ

_{0}>1, it is easier for the gene network to satisfy this amplification ability. In this situation, this gene network is much influenced by environmental random noises to easily move toward another equilibrium points (traits) so that the gene network is more adaptive to the environmental changes in the evolutionary process, especially for ρ

_{0}»1, to generate a new trait (or phenotype) to a new environment. This is a tradeoff between the robust stability and the adaptability of gene networks in the evolutionary process [52,53].

_{i}(t), Δβ

_{i}(t), Δh

_{ij}(t), Δg

_{ij}(t) due to random genetic mutations and ΔY

_{I}(t) due to environmental random disturbances in the evolutionary process. These random parameter variations due to genetic mutations in the sex-chromosome could be considered as design parameters in the evolutionary process. Because the biochemical networks are the backbone of physiological system of organisms, a biochemical network should be sufficiently robust to tolerate the random parameter variations and environmental changes due to genetic mutations to maintain its function properly in the evolutionary process (see Figure 1). However, if the robustness condition in (42) is violated, the steady state phenotype of the perturbed biochemical network in (43) may not exist or move to another equilibrium point with a change of the trait. The random variations Δb(t), ΔY

_{I}(t) and ΔA

_{I}(t) will influence the trait variation ΔY

_{D}(t) in (34), and their effects on the trait can be obtained from the sensitivities in (44). In general, the perturbed biochemical networks with random parameter variations that violate the robustness criterion in (42) will be eliminated by natural selection. Therefore, the perturbed biochemical network should satisfy the robustness criterion in order to guarantee not to be perturbed too much from its equilibrium point (the normal physiological function) in the evolutionary process. Because the violation of (42) means a lethal genetic mutation, the robustness criterion in (42) is the necessary condition for the survival of biochemical networks under natural selection [52,53]. From the robustness criterion in (42), natural selection favors the perturbed biochemical networks with small-variance random genetic mutation, i.e., with small E[ΔA

_{D}(t)ΔA

_{D}

^{T}(t)], so that the robustness criterion is not violated. A biochemical network with redundancy and self-regulation can attenuate random fluctuation ΔA

_{D}(t). Furthermore, a biochemical network with adequate negative feedbacks can increase A

_{D}A

_{D}

^{T}in (42) to tolerate random parameter fluctuations with large variance E[ΔA

_{D}ΔA

_{D}

^{T}] in the evolutionary process. These robust adaptive designs with feedbacks are also favored by natural selection in the evolutionary process of biochemical networks. This is why there are so many redundancies, duplicated genes, modularities, self-regulations and feedback pathways in the biochemical networks in organisms [52,53,68,69,70]. A scale-free structure could reduce the effect of E[ΔA

_{D}(t)ΔA

_{D}

^{T}(t)] on the stability of bio-networks and is also favored by natural selection in evolution [36].

**Figure 1.**[42] The natural selection process on the interplaying of hierarchical biological networks with random parameter fluctuations and environmental disturbance. The high-level biological system selection will become the selection force on low level biological systems. The natural selection on organisms selects its favored organisms. However, the random parameter fluctuations and stochastic environmental disturbance will influence the natural selection process of gene networks in evolution. Once the favored organisms are selected, the low-level biological networks have to maintain the favored physiological systems of the selected organisms. Hence, these favored organisms become the selection force to select their favored physiological systems. The favored physiological systems will lead to the selection force on biochemical networks. The favored biochemical networks by natural selection will become the selection force on genetic networks. On the other hand, the lower-level selected networks will feedback to influence the higher-level networks in evolution. Therefore, the natural selection actually acts on the interplaying of the multiple bionetworks.

_{D}(t) to random parameter variations due to genetic mutations and random environmental changes by natural selection in the evolutionary process. For a functional biochemical network, it should satisfy the sensitivity criteria to prevent the metabolic concentration from being changed too much by random genetic mutations and environmental changes. Hence, the steady state (trait) of a biochemical network can be preserved while exposing the random genetic mutations and environmental changes to natural selection in the evolutionary process. The assumption that the three sensitivity criteria in (45) or (46) all hold for natural selection is derived from the fact that biochemical networks are the backbone of physiological systems and cannot be too sensitive to random genetic mutations and environmental changes, especially for some core (conserved) biochemical networks [52]. Actually, the sensitivity values s

_{i}of sensitivity criteria in (45) or (46) are dependent on the biochemical network. If some sensitivity criteria in (45) or (46) are relaxed, i.e., some of the inequalities in (45) or (46) are with much larger value s

_{i}, the phenotype of the biochemical network will be changed quickly with some random genetic mutations and environmental changes and will be more favored by natural selection. In this situation, the traits of biochemical networks are much influenced by random genetic mutation and environmental variations and are more easily toward another equilibrium points (traits) so that they may be more adaptive to environmental changes through random genetic mutations in the evolutionary process. In this case, new traits are more easily generated in order to be more adaptive to the natural selection force on biochemical networks under random genetic mutations and environmental variations in the evolutionary process [52,53]. The robustness criterion in (42) and the sensitivity criteria in (45) are called adaptive rules of natural selection on biochemical network via genetic mutations of sex-chromosome in the evolutionary process. There are many perturbed biochemical networks that can satisfy the adaptive rules of natural selection in the evolutionary process. If they are selected by natural selection, there are some differences in trait among these selected biochemical networks with random genetic mutations. After several generations in the evolutionary process, due to co-option of existing biochemical networks, diversities of the biochemical networks with conserved physiological function but with different structures will be developed [66]. This is the origin of the diversities of biochemical networks within organisms in evolution. However, if the requirements on the robustness in (42) and the sensitivities in (45) are stricter (or more conservative), i.e., with small s

_{i}, only a few solutions (or structures) can be selected by natural selection to meet these requirements. This is the reason why a conserved core biochemical network has less diversity [53].

## 5. The Balancing Roles of Entropy and Robustness in the Design of Systems Biology

_{D}(t), Δb(t) and enzyme change ΔY

_{I}(t) due to genetic mutation, environmental changes and diseases. A robust design method can be developed for biochemical networks to improve their robustness to compensate the effect of random parameter fluctuations and to attenuate the effect of random disturbances.

_{k}(t) for regulating the production of x

_{i}(t) with a kinetic parameter f

_{ik}and ${x}_{k}^{{e}_{ik}}(t)$ denotes a biochemical control circuit via x

_{k}(t) for regulating the degradation of x

_{i}(t) with the kinetic parameter e

_{ik}. The choice between regulating objects, x

_{k}(t) and x

_{i}(t) and the specification of kinetic parameters, f

_{ik}and e

_{ik}, are to be made according to the feasibility of biochemical circuit linkages to achieve both the robust stability to tolerate random parameter fluctuations ΔA

_{D}(t) and the desired sensitivities to attenuate the other random parameter fluctuations Δb(t), ΔA

_{I}(t) and the random disturbance ΔY

_{I}(t). Since f

_{ik}and e

_{ik}are the elasticities of the corresponding enzymes in the designed gene control circuits, the implementation of gene control circuits is highly dependent on the specification of elasticities of these enzymes [65]. Based on the robustness analysis from (35) to (46), a biochemical circuit design scheme for robust control of biochemical networks can be developed in the following. Consider the robust control of the biochemical network in (51). Using a similar procedure from (35) to (46), we obtain:

_{D}+F+ΔA

_{D}(t))(Y

_{D}(t)+ΔY

_{D}(t)) = (b+Δb(t))-(A

_{I}+ΔA

_{I}(t)) (Y

_{I}(t)+ΔY

_{I}(t))

_{ik}and e

_{ik}are the kinetic parameters of the biochemical control circuits to be specified in (51).

_{D}(t) for a biochemical network.

_{1}, s

_{2}and s

_{3}so that the phenotypes of the perturbed biological network will not change too much. According to the above analysis, if we can specify the entries of f

_{ik}and e

_{ik}of F in (53) for the biochemical network in (51) to satisfy the requirement of robustness in (54) and the sensitivities in (55), then the biochemical network could tolerate the random parameter variations and stochastic disturbances. These results are potential for application to medical technology and robust design of synthetic biology in future.

## 6. The Balancing Roles of Entropy and Robustness in Synthetic Biology

_{a}and x

_{b}denote the concentrations of proteins A and B. k’s and γ’s are the kinetic parameters and decay rates, respectively, and r

_{i}(x

_{i}) are the regulation functions, which capture the regulator effect of an effector protein on gene expression and are smooth sigmoidal function (e.g., Hill functions). The simple cross-inhibition synthetic network in (56) can be represented by the following stoichiometic matrix equation:

_{a}→k

_{a}+Δk

_{a}n

_{a}(t), γ

_{a}→γ

_{a}+Δγ

_{a}n

_{a}(t),

k

_{b}→k

_{b}+Δk

_{b}n

_{b}(t), γ

_{b}→γ

_{b}+Δγ

_{b}n

_{b}(t)

_{i}and Δγ

_{i}denote the amplitude of fluctuations of stochastic kinetic parameters and decay rate; and n

_{i}(t) is white noises to denote different random sources. Then, the stochastic synthetic gene network under random parameter fluctuations and stochastic disturbances in the host cell can be represented by:

_{2}(t) = [x

_{a}(t) x

_{b}(t)]

^{T}and υ

_{2}(t) = [v

_{a}(t) v

_{b}(t)]

^{T}denote the state vector and external random disturbance vector of the synthetic gene network in the host cell, respectively. These random intrinsic parameter fluctuations and stochastic external disturbances may cause the engineered gene network to be dysfunction in the host cell. The robust synthetic biology design is to choose two kinetic parameters k

_{a}and k

_{b}and two decay rates γ

_{a}and γ

_{b}in N

_{2}so that the desired steady state x

_{ad}and x

_{bd}can be achieved under random intrinsic parameter fluctuations and stochastic disturbances.

_{1}(t)… x

_{n}(t)]

^{T}denotes the concentrations of n proteins in the synthetic gene network. N denotes the corresponding stoichiometric matrix of n-gene network. M

_{i}, i = 1,…,m, denote fluctuation matrices due to independent random noise sources n

_{i}, i = 1,…,m, and the elements of M

_{i}denote standard deviations of the corresponding random parameter fluctuations. υ(t) = [υ

_{1}(t)…υ

_{n}(t)]

^{T}denotes the vector of random environmental disturbances. The stochastic system in (60) is used to mimic the realistic stochastic behavior of a synthetic gene network of n genes in the host cell. The synthetic network, however, suffers from the random parameter fluctuations and environmental disturbances in the context of the host cell. In [49], a robust synthetic gene network design has been proposed with the ability not only to tolerate these random parameter fluctuations and to attenuate the stochastic disturbances but also to achieve the desired steady state behaviors via the following four design specifications [49]:

_{i}in the state dependent terms ${\sum}_{i=1}^{m}{M}_{i}{g}_{i}(x)d{w}_{i}(t)$ to be tolerated by the synthetic gene network should be specified.

_{d}should be achieved, where x

_{d}is the desired steady state specified by the designer for some design purposes of the synthetic gene network inserted in the host cell.

_{∞}filtering) should be achieved:

_{d}= [x

_{1d}… x

_{nd}]

^{T}denotes the desired steady states of n proteins.

_{i}(t) = n

_{i}(t)dt, i.e., the origin $\tilde{x}(t)=0$ of the stochastic system in (62) is at the desired steady state x

_{d}of the original stochastic system in (60).

_{d}in probability, or equivalently, the original stochastic synthetic gene network in (60) will asymptotically converge to the desired steady state x

_{d}in probability, i.e., the design specification (iii) is achieved.

_{d}in probability, i.e., the design specification (iii) is achieved.

## 7. Discussion and Conclusions

## Acknowledgements

## References and Notes

- Ozbudak, E.M.; Thattai, M.; Kurtser, I.; Grossman, A.D.; van Oudenaarden, A. Regulation of noise in the expression of a single gene. Nature Genetics
**2002**, 31, 69–73. [Google Scholar] [CrossRef] [PubMed] - Thattai, M.; van Oudenaarden, A. Intrinsic noise in gene regulatory networks. Proc. Natl. Acad. Sci. USA
**2001**, 98, 8614–8619. [Google Scholar] [CrossRef] [PubMed] - Thattai, M.; van Oudenaarden, A. Attenuation of noise in ultrasensitive signaling cascades. Biophys. J.
**2002**, 82, 2943–2950. [Google Scholar] [CrossRef] - Pedraza, J.M.; van Oudenaarden, A. Noise propagation in gene networks. Science
**2005**, 307, 1965–1969. [Google Scholar] [CrossRef] [PubMed] - Mettetal, J.T.; Muzzey, D.; Pedraza, J.M.; Ozbudak, E.M.; van Oudenaarden, A. Predicting stochastic gene expression dynamics in single cells. Proc. Natl. Acad. Sci. USA
**2006**, 103, 7304–7309. [Google Scholar] [CrossRef] [PubMed] - Mettetal, J.T.; van Oudenaarden, A. Necessary noise. Science
**2007**, 317, 463–464. [Google Scholar] [CrossRef] [PubMed] - Goentoro, L.; Shoval, O.; Kirschner, M.W.; Alon, U. The Incoherent Feedforward Loop Can Provide Fold-Change Detection in Gene Regulation. Mol. Cell.
**2009**, 36, 894–899. [Google Scholar] [CrossRef] [PubMed] - Mangan, S.; Zaslaver, A.; Alon, U. The coherent feedforward loop serves as a sign-sensitive delay element in transcription networks. J. Mol. Biol.
**2003**, 334, 197–204. [Google Scholar] [CrossRef] [PubMed] - McAdams, H.H.; Arkin, A. It's a noisy business! Genetic regulation at the nanomolar scale. Trends. Genet.
**1999**, 15, 65–69. [Google Scholar] [CrossRef] - Diaz, J.; Alvarez-Buylla, E.R. Information flow during gene activation by signaling molecules: ethylene transduction in Arabidopsis cells as a study system. BMC Syst. Biol.
**2009**, 3, 48. [Google Scholar] [CrossRef] [PubMed] - Manke, T.; Demetrius, L.; Vingron, M. An entropic characterization of protein interaction networks and cellular robustness. J. R. Soc. Interface
**2006**, 3, 843–850. [Google Scholar] [CrossRef] [PubMed] - Ji, S.; Krishnapuram, B.; Carin, L. Variational Bayes for continuous hidden Markov models and its application to active learning. IEEE Trans. Pattern. Anal. Mach. Intell.
**2006**, 28, 522–532. [Google Scholar] [PubMed] - Wang, X.L.; Yuan, Z.F.; Guo, M.C.; Song, S.D.; Zhang, Q.Q.; Bao, Z.M. Maximum entropy principle and population genetic equilibrium. Acta. Genet. Sin.
**2002**, 29, 562–564. [Google Scholar] [PubMed] - Demetrius, L. Thermodynamics and evolution. J. Theor. Biol.
**2000**, 206, 1–16. [Google Scholar] [CrossRef] [PubMed] - Krawitz, P.; Shmulevich, I. Basin entropy in Boolean network ensembles. Phys. Rev. Lett.
**2007**, 98, 158701. [Google Scholar] [CrossRef] [PubMed] - Brown, K.S.; Sethna, J.P. Statistical mechanical approaches to models with many poorly known parameters. Phys. Rev. E Stat. Nonlin. Soft. Matter Phys.
**2003**, 68, 021904. [Google Scholar] [CrossRef] [PubMed] - Krawitz, P.; Shmulevich, I. Entropy of complex relevant components of Boolean networks. Phys. Rev. E
**2007**, 76, 036115. [Google Scholar] [CrossRef] - Sayyed-Ahmad, A.; Tuncay, K.; Ortoleva, P.J. Transcriptional regulatory network refinement and quantification through kinetic modeling, gene expression microarray data and information theory. BMC Bioinform.
**2007**, 8, 20. [Google Scholar] [CrossRef] [PubMed] - Lezon, T.R.; Banavar, J.R.; Cieplak, M.; Maritan, A.; Fedoroff, N.V. Using the principle of entropy maximization to infer genetic interaction networks from gene expression patterns. Proc. Natl. Acad. Sci. USA
**2006**, 103, 19033–19038. [Google Scholar] [CrossRef] [PubMed] - Stoll, G.; Rougemont, J.; Naef, F. Representing perturbed dynamics in biological network models. Phys. Rev. E
**2007**, 76, 011917. [Google Scholar] [CrossRef] - Ribeiro, A.S. Dynamics of a two-dimensional model of cell tissues with coupled stochastic gene networks. Phys. Rev. E
**2007**, 76, 051915. [Google Scholar] [CrossRef] - Fujita, A.; Gomes, L.R.; Sato, J.R.; Yamaguchi, R.; Thomaz, C.E.; Sogayar, M.C.; Miyano, S. Multivariate gene expression analysis reveals functional connectivity changes between normal/tumoral prostates. BMC Syst. Biol.
**2009**, 2, 106. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Farhy, L.S.; Straume, M.; Johnson, M.L.; Kovatchev, B.; Veldhuis, J.D. Unequal autonegative feedback by GH models the sexual dimorphism in GH secretory dynamics. Amer. J. Physiol.-Regul. Integr. C.
**2002**, 282, R753–R764. [Google Scholar] [CrossRef] [PubMed] - Yildirim, N.; Mackey, M.C. Feedback regulation in the lactose operon: A mathematical modeling study and comparison with experimental data. Biophys. J.
**2003**, 84, 2841–2851. [Google Scholar] [CrossRef] - Wlaschin, A.P.; Trinh, C.T.; Carlson, R.; Srienc, F. The fractional contributions of elementary modes to the metabolism of Escherichia coli and their estimation from reaction entropies. Metab. Eng.
**2006**, 8, 338–352. [Google Scholar] [CrossRef] [PubMed] - Zhao, W.T.; Serpedin, E.; Dougherty, E.R. Inferring gene regulatory networks from time series data using the minimum description length principle. Bioinformatics
**2006**, 22, 2129–2135. [Google Scholar] [CrossRef] [PubMed] - Chen, B.S.; Wang, Y.C. On the attenuation and amplification of molecular noise in genetic regulatory networks. BMC Bioinform.
**2006**, 7, 52. [Google Scholar] [CrossRef] [PubMed] - Farcot, E. Geometric properties of a class of piecewise affine biological network models. J. Math. Biol.
**2006**, 52, 373–418. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Michoel, T.; Maere, S.; Bonnet, E.; Joshi, A.; Saeys, Y.; Van den Bulcke, T.; Van Leemput, K.; van Remortel, P.; Kuiper, M.; Marchal, K.; de Peer, Y.V. Validating module network learning algorithms using simulated data. BMC Bioinform.
**2007**, 8, S5. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Meyer, P.E.; Lafitte, F.; Bontempi, G. minet: A R/Bioconductor package for inferring large transcriptional networks using mutual information. BMC Bioinform.
**2008**, 9, 461. [Google Scholar] [CrossRef] [PubMed] - Chen, B.S.; Wang, Y.C.; Wu, W.S.; Li, W.H. A new measure of the robustness of biochemical networks. Bioinformatics
**2005**, 21, 2698–2705. [Google Scholar] [CrossRef] [PubMed] - Chen, B.S.; Wu, W.S. Robust filtering circuit design for stochastic gene networks under intrinsic and extrinsic molecular noises. Math. Biosci.
**2008**, 211, 342–355. [Google Scholar] [CrossRef] [PubMed] - Caretta-Cartozo, C.; De Los Rios, P.; Piazza, F.; Lio, P. Bottleneck genes and community structure in the cell cycle network of S-pombe. Plos. Comput. Biol.
**2007**, 3, 968–976. [Google Scholar] [CrossRef] [PubMed] - Nagarajan, R.; Aubin, J.E.; Peterson, C.A. Robust dependencies and structures in stem cell differentiation. Int. J. Bifurcation Chaos
**2005**, 15, 1503–1514. [Google Scholar] [CrossRef] - Chen, B.S.; Chang, Y.T.; Wang, Y.C. Robust H-infinity-stabilization design in gene networks under Stochastic molecular noises: Fuzzy-interpolation approach. IEEE Trans. Syst. Man. Cybern. B Part B-Cybernetics
**2008**, 38, 25–42. [Google Scholar] [CrossRef] [PubMed] - Kitano, H. Biological robustness. Nat. Rev. Genet.
**2004**, 5, 826–837. [Google Scholar] [CrossRef] [PubMed] - Wang, J.; Xu, L.; Wang, E.K. Potential landscape and flux framework of nonequilibrium networks: Robustness, dissipation, and coherence of biochemical oscillations. Proc. Natl. Acad. Sci. USA
**2008**, 105, 12271–12276. [Google Scholar] [CrossRef] [PubMed] - Edwards, J.S.; Palsson, B.O. Robustness analysis of the Escherichia coli metabolic network. Biotech. Progr.
**2000**, 16, 927–939. [Google Scholar] [CrossRef] [PubMed] - Lenz, P.; Swain, P.S. An entropic mechanism to generate highly cooperative and specific binding from protein phosphorylations. Curr. Biol.
**2006**, 16, 2150–2155. [Google Scholar] [CrossRef] [PubMed] - Chen, B.S.; Chang, Y.T. A systematic molecular circuit design method for gene networks under biochemical time delays and molecular noises. BMC Syst. Biol.
**2008**, 2, 103. [Google Scholar] [CrossRef] [PubMed] - Chen, B.S.; Chen, P.W. Robust engineered circuit design principles for stochastic biochemical networks with parameter uncertainties and disturbances. IEEE Trans. Biomed. Circ. Syst.
**2008**, 2, 114–132. [Google Scholar] [CrossRef] [PubMed] - Chen, B.S.; Chen, P.W. On the estimation of robustness and filtering ability of dynamic biochemical networks under process delays, internal parametric perturbations and external disturbances. Math. Biosci.
**2009**, 222, 92–108. [Google Scholar] [CrossRef] [PubMed] - Denic, S. Z.; Vasic, B.; Charalambous, C. D.; Palanivelu, R. Robust control of uncertain context-sensitive probabilistic Boolean networks. IET Syst. Biol.
**2009**, 3, 279–295. [Google Scholar] [CrossRef] [PubMed] - Hasty, J.; McMillen, D.; Collins, J.J. Engineered gene circuits. Nature
**2002**, 420, 224–230. [Google Scholar] [CrossRef] [PubMed] - Lapidus, S.; Han, B.; Wang, J. Intrinsic noise, dissipation cost, and robustness of cellular networks: The underlying energy landscape of MAPK signal transduction. Proc. Natl. Acad. Sci. USA
**2008**, 105, 6039–6044. [Google Scholar] [CrossRef] [PubMed] - Elowitz, M.B.; Leibler, S. A synthetic oscillatory network of transcriptional regulators. Nature
**2000**, 403, 335–338. [Google Scholar] [CrossRef] [PubMed] - Batt, G.; Yordanov, B.; Weiss, R.; Belta, C. Robustness analysis and tuning of synthetic gene networks. Bioinformatics
**2007**, 23, 2415–2422. [Google Scholar] [CrossRef] [PubMed] - Chen, B.S.; Chang, C.H.; Lee, H.C. Robust synthetic biology design: stochastic game theory approach. Bioinformatics
**2009**, 25, 1822–1830. [Google Scholar] [CrossRef] [PubMed] - Chen, B.S.; Wu, C.H. A systematic design method for robust synthetic biology to satisfy design specifications. BMC Syst. Biol.
**2009**, 3, 66. [Google Scholar] [CrossRef] [PubMed] - Kaern, M.; Elston, T.C.; Blake, W.J.; Collins, J.J. Stochasticity in gene expression: From theories to phenotypes. Nat. Rev. Genet.
**2005**, 6, 451–464. [Google Scholar] [CrossRef] [PubMed] - Li, C.W.; Chen, B.S. Stochastic spatio-temporal dynamic model for gene/protein interaction network in early Drosophila development. Gene Regul. Syst. Biol.
**2009**, 3, 191. [Google Scholar] - Chen, B.S.; Wu, W.S. Underlying principles of natural selection in network evolution: systems biology approach. Evol. Bioinform.
**2007**, 3, 245–262. [Google Scholar] - Chen, B.S.; Wu, W.S.; Wu, W.S.; Li, W.H. On the adaptive design rules of biochemical networks in evolution. Evol. Bioinform.
**2007**, 3, 27–39. [Google Scholar] - Chen, B.S.; Tseng, C.S.; Uang, H.J. Robustness design of nonlinear dynamic systems via fuzzy linear control. IEEE Trans. Fuzzy Syst.
**1999**, 7, 571–585. [Google Scholar] [CrossRef] - Chen, B.S.; Tseng, C.S.; Uang, H.J. Mixed H-2/H-infinity fuzzy output feedback control design for nonlinear dynamic systems: An LMI approach. IEEE Trans. Fuzzy Syst.
**2000**, 8, 249–265. [Google Scholar] [CrossRef] - Chen, B.S.; Zhang, W.H. Stochastic H-2/H-infinity control with state-dependent noise. IEEE Trans. Auto. Cont.
**2004**, 49, 45–57. [Google Scholar] [CrossRef] - Zhang, W.H.; Chen, B.S. State feedback H-infinity control for a class of nonlinear stochastic systems. SIAM J. Cont. Optim.
**2006**, 44, 1973–1991. [Google Scholar] [CrossRef] - Zhang, W.H.; Chen, B.S.; Tseng, C.S. Robust H-infinity filtering for nonlinear stochastic systems. IEEE Trans. Sig. Process.
**2005**, 53, 589–598. [Google Scholar] [CrossRef] - Blake, W.J.; Kaern, M.; Cantor, C.R.; Collins, J.J. Noise in eukaryotic gene expression. Nature
**2003**, 422, 633–637. [Google Scholar] [CrossRef] [PubMed] - McAdams, H.H.; Arkin, A. Stochastic mechanisms in gene expression. Proc. Natl. Acad. Sci. USA
**1997**, 94, 814–819. [Google Scholar] [CrossRef] [PubMed] - Voit, E.O.; Ferreira, A.E.N. Computational analysis of biochemical systems: a practical guide for biochemists and molecular biologists; Cambridge University Press: Cambridge ; New York, 2000; pp. xii, 531. [Google Scholar]
- Boyd, S.P. Linear matrix inequalities in system and control theory; Society for Industrial and Applied Mathematics: Philadelphia, PA, USA, 1994; pp. ix, 193. [Google Scholar]
- Li, C.W.; Chen, B.S. On the noise enhancing ability of stochastic Hodgkin-Huxley neuron systems. Neural Comput
**2010**, in press. [Google Scholar] - Savageau, M.A. Biochemical systems analysis: a study of function and design in molecular biology; Addison-Wesley Pub. Co., Advanced Book Program: Reading, MA, USA, 1976; pp. xvii, 379. [Google Scholar]
- Chen, B.S.; Wu, W.S.; Wang, Y.C.; Li, W.H. On the robust circuit design schemes of biochemical networks: steady-state approach. IEEE Trans. Biomed. Circ. Syst.
**2007**, 1, 91–104. [Google Scholar] [CrossRef] [PubMed] - Freeman, S.; Herron, J.C. Evolutionary analysis, 3rd ed.; Pearson/Prentice Hall: Upper Saddle River, NJ, USA, 2004; pp. xiv, 802. [Google Scholar]
- Lynch, M.; Walsh, B. Genetics and analysis of quantitative traits; Sinauer: Sunderland, MA, USA, 1998; pp. xvi, 980. [Google Scholar]
- Isaacs, F.J.; Hasty, J.; Cantor, C.R.; Collins, J.J. Prediction and measurement of an autoregulatory genetic module. Proc. Natl. Acad. Sci. USA
**2003**, 100, 7714–7719. [Google Scholar] [CrossRef] [PubMed] - Teichmann, S.A.; Babu, M.M. Gene regulatory network growth by duplication. Nat. Gen.
**2004**, 36, 492–496. [Google Scholar] [CrossRef] [PubMed] - Freeman, M. Feedback control of intercellular signalling in development. Nature
**2000**, 408, 313–319. [Google Scholar] [CrossRef] [PubMed] - Andrianantoandro, E.; Basu, S.; Karig, D.K.; Weiss, R. Synthetic biology: new engineering rules for an emerging discipline. Mol. Syst. Biol.
**2006**, 2006.0028. [Google Scholar] [CrossRef] [PubMed] - Endy, D. Foundations for engineering biology. Nature
**2005**, 438, 449–453. [Google Scholar] [CrossRef] [PubMed] - Chen, B.S.; Wu, C.H. Robust optimal reference tracking design method for stochastic synthetic biology systems: T-S fuzzy approach. BIEEE Trans. Fuzzy Syst
**2010**, (in press). [Google Scholar] [CrossRef] - Chen, B.S.; Chen, P.W. GA-based design algorithms for the robust synthetic genetic oscillators with prescribed amplitude, period and phase. Gene Reg. Syst. Biol
**2010**, (in press). [Google Scholar] - Bickel, P.J.; Li, B. Regularization in statistics. Test
**2006**, 15, 271–303. [Google Scholar] [CrossRef] - Cucker, F.; Smale, S. Best choices for regularization parameters in learning theory: On the bias-variance problem. Found. Comput. Math.
**2002**, 2, 413–428. [Google Scholar] [CrossRef]

© 2010 by the authors; licensee MDPI, Basel, Switzerland. This article is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

## Share and Cite

**MDPI and ACS Style**

Chen, B.-S.; Li, C.-W.
On the Interplay between Entropy and Robustness of Gene Regulatory Networks. *Entropy* **2010**, *12*, 1071-1101.
https://doi.org/10.3390/e12051071

**AMA Style**

Chen B-S, Li C-W.
On the Interplay between Entropy and Robustness of Gene Regulatory Networks. *Entropy*. 2010; 12(5):1071-1101.
https://doi.org/10.3390/e12051071

**Chicago/Turabian Style**

Chen, Bor-Sen, and Cheng-Wei Li.
2010. "On the Interplay between Entropy and Robustness of Gene Regulatory Networks" *Entropy* 12, no. 5: 1071-1101.
https://doi.org/10.3390/e12051071