Cell Decision Making through the Lens of Bayesian Learning

Abstract

Cell decision making refers to the process by which cells gather information from their local microenvironment and regulate their internal states to create appropriate responses. Microenvironmental cell sensing plays a key role in this process. Our hypothesis is that cell decision-making regulation is dictated by Bayesian learning. In this article, we explore the implications of this hypothesis for internal state temporal evolution. By using a timescale separation between internal and external variables on the mesoscopic scale, we derive a hierarchical Fokker–Planck equation for cell-microenvironment dynamics. By combining this with the Bayesian learning hypothesis, we find that changes in microenvironmental entropy dominate the cell state probability distribution. Finally, we use these ideas to understand how cell sensing impacts cell decision making. Notably, our formalism allows us to understand cell state dynamics even without exact biochemical information about cell sensing processes by considering a few key parameters.

1. Introduction

Decision making is the process of choosing different actions based on certain goals [1]. Similarly, cells make decisions as a response to microenvironmental signals [2]. When external cues, such as signaling molecules, are received by the cell, a series of chemical reactions is triggered inside the cell [3]. This decision-making process is influenced by intrinsic signal transduction pathways [4], the genetic cell network [5], extrinsic cues [6], and molecular noise [7]. In turn, such intracellular regulation produces an appropriately diverse range of decisions, in the context of differentiation, phenotypic plasticity, proliferation, migration, and apoptosis. Understanding the underlying principles of cellular decision making is essential to comprehend the behavior of complex biological systems.

Cell sensing is a fundamental process that enables cells to respond to their environment and make decisions. Typically, receptors on the cell membrane can detect various stimuli, such as changes in temperature [8], pH [9] or the presence of specific molecules. The specificity of the receptors and the signaling pathways that are activated are critical in determining the response of the cell. However, receptors are not the sole sensing unit of the cell. Recent studies have also revealed that cells use mechanical cues to make decisions about their behavior [10]. For example, cells can sense the stiffness of the substrate they are growing on [11]. In turn, cells make decisions about changing their shape, migration, proliferation or gene expression, in the context of a phenomenon called mechanotransduction [12]. Errors in cell sensing can lead to possible pathologies, such as cancer [13], autoimmunity [14], diabetes [15], etc.

Bayesian inference or updating has been the main toolbox for general-purpose decision making [16]. In the context of cell decision making, this mathematical framework assumes that cells integrate new information and update their internal state based on the likelihood of different outcomes [17]. Although static Bayesian inference was the main tool for understanding cell decisions, recently, Bayesian forecasting has been additionally employed to understand the dynamics of decisions [18]. In particular, Mayer et al. [19] used dynamic Bayesian prediction to model the estimation of the future pathogen distribution by adaptive immune cells. A dynamic Bayesian prediction model was also used for bacterial chemotaxis [20]. Finally, the authors developed the least microenvironmental uncertainty principle (LEUP) that employs Bayesian-based dynamic theory for cell decision making [21,22,23,24].

To understand the stochastic dynamics of the cell-microenvironment system, we focus on the mesoscopic scale and we derive a Fokker–Planck equation. Fokker–Planck formalism was developed to study the time-dependent probability distribution function for the Brownian motion under the influence of a drift force [25]. We can see nowadays a huge number of applications of Fokker–Planck equations (linear and non-linear) across disciplines [26,27]. Here, we will additionally assume a timescale separation between internal and external variables [28]. Timescale separation has been studied rigorously [29] from the microscopic point of view using Langevin equations. In the case of cell decision making, microscopic dynamics have been studied, specifically in the context of active Brownian motion and cell migration using Langevin equations [22,30,31]. Understanding dynamics induced by a timescale separation at the mesoscopic scale, using Fokker–Planck equations, was studied only recently by S. Abe [32].

We will assume a timescale separation, where cell decision time, when internal states evolve, is slower than the characteristic time of the variables that belong to the cellular microenvironment. This assumption is particularly valid for cell decision making at the timescale of a cell cycle, such as differentiation. The underlying molecular regulation underlying these decisions may evolve over many cell cycles [33,34]. When these molecular expressions cross a threshold, the cell decision emerges.

The structure of our paper is as follows: In Section 2 we present the Bayesian learning dynamics for cell decision making. In turn, we derive a fluctuation–dissipation relation and the corresponding continuous-time dynamics of cellular internal states. After that, in Section 3, we elaborate on the concept of the hierarchical Fokker–Planck equation in relation to cellular decision making and the underlying Bayesian learning process. In Section 4, we demonstrate the use of a simple example of coarse-grained dynamics for cell sensing to analyze the steady-state distribution of cellular states in two scenarios: (i) in the absence and (ii) presence of cell sensing. Then, in Section 5, we connect this idea with the least microenvironmental uncertainty principle (LEUP) as a special case of Bayesian learning. Finally, in Section 6, we conclude and discuss our results and findings.

2. Cell Decision Making as Bayesian Learning

Cell decisions, here interpreted as changes in the cellular internal states $\mathbf{X}$ within a decision time $\tau$, are realized via (i) sensing their microenvironment $\mathbf{Y}$ and combining this information with (ii) an existing predisposition about their internal state. In a Bayesian language, the former can be interpreted as the empirical likelihood $P\left(\mathbf{Y}\mid \mathbf{X}\right)$ and the latter as the prior distribution $P\left(\mathbf{X}\right)$. Interestingly, the previously mentioned distributions are time dependent since we assumed that the cell tries to build increasingly informative priors over time to minimize the cost of energy associated with sampling the cellular microenvironment. For instance, assuming that cell fate decisions follow such Bayesian learning dynamics, during tissue differentiation, we observe the microenvironment evolving into a more organized state (e.g., pattern formation). Therefore, one can observe a reduction in microenvironmental entropy over time, which is further associated with the microenvironmental probability distribution or likelihood in Bayesian inference. Here, we postulate that the cells evolve the distribution of their internal states in the form of Bayesian learning.

2.1. A Fluctuation–Dissipation Relation

Formalizing the above, let us assume that after a decision time $\tau$, the cell updates its state from $\mathbf{X}$ to ${\mathbf{X}}^{\prime }$ both belonging to ${\mathbb{R}}^n$. Moreover, we assume that the microenvironmental variables $\mathbf{Y}\in {\mathbb{R}}^m$. According to Bayesian learning, the posterior of the previous time $P\left(\mathbf{X}\mid \mathbf{Y}\right)$ becomes prior to the next time step, i.e., $P\left({\mathbf{X}}^{\prime }\right)=P\left(\mathbf{X}\mid \mathbf{Y}\right)$. Therefore, the Bayesian learning dynamics read:

$$\begin{array}{cc}\hfill & P\left({\mathbf{X}}^{\prime }\right)=\frac{P\left(\mathbf{Y}\mid \mathbf{X}\right)P\left(\mathbf{X}\right)}{P\left(\mathbf{Y}\right)},\hfill \\ \hfill & \Rightarrow ln\frac{P\left({\mathbf{X}}^{{}^{\prime }}\right)}{P\left(\mathbf{X}\right)}=ln\frac{P\left(\mathbf{Y}\mid \mathbf{X}\right)}{P\left(\mathbf{Y}\right)}.\hfill \\ \hfill & \Rightarrow \int P\left({\mathbf{X}}^{{}^{\prime }},\mathbf{X},\mathbf{Y}\right)ln\left(\frac{P\left({\mathbf{X}}^{{}^{\prime }}\right)}{P\left(\mathbf{X}\right)}\right)d{\mathbf{X}}^{{}^{\prime }}d\mathbf{X}d\mathbf{Y}\hfill \\ \hfill & =\int P\left({\mathbf{X}}^{{}^{\prime }},\mathbf{X},\mathbf{Y}\right)ln\left(\frac{P\left(\mathbf{Y}\mid \mathbf{X}\right)}{P\left(\mathbf{Y}\right)}\right)d{\mathbf{X}}^{{}^{\prime }}d\mathbf{X}d\mathbf{Y}\hfill \\ \hfill & \Rightarrow {〈D\left({\mathbf{X}}^{{}^{\prime }}\mid \mid \mathbf{X}\right)〉}_{P({\mathbf{X}}^{{}^{\prime }}\mid \mathbf{X})}=\tilde{\beta }I\left(\mathbf{Y},\mathbf{X}\right),\hfill \end{array}$$

(1)

where $\tilde{\beta }=\frac{\int P\left({\mathbf{X}}^{{}^{\prime }}\mid \mathbf{X},\mathbf{Y}\right)d{\mathbf{X}}^{{}^{\prime }}}{\int P\left(\mathbf{Y}\mid \mathbf{X},{\mathbf{X}}^{{}^{\prime }}\right)d\mathbf{X}d\mathbf{Y}}$, which is different from one if the corresponding conditional distributions require different finite support for their normalization. In the above relation, the Kullback–Leibler divergence $D\left({\mathbf{X}}^{{}^{\prime }}\mid \mid \mathbf{X}\right)=\int P\left({\mathbf{X}}^{{}^{\prime }}\right)ln\left(\frac{P\left({\mathbf{X}}^{{}^{\prime }}\right)}{P\left(\mathbf{X}\right)}\right)d{\mathbf{X}}^{{}^{\prime }}$ that quantifies the convergence to the equilibrium distribution of the internal value of $\mathbf{X}$ is connected to the amount of available information $I\left(\mathbf{Y},\mathbf{X}\right)=\int P\left(\mathbf{X},\mathbf{Y}\right)ln\left(\frac{P\left(\mathbf{Y}\mid \mathbf{X}\right)}{P\left(\mathbf{Y}\right)}\right)d\mathbf{X}d\mathbf{Y}$ between the cell and its microenvironment. From Equation (1), the Kullbeck–Leibler divergence can be further elaborated in terms of Fisher information as

$$\begin{array}{cc}\hfill & D\left({\mathbf{X}}^{{}^{\prime }}\mid \mid \mathbf{X}\right)=\int P\left({\mathbf{X}}^{{}^{\prime }}\right)ln\left(\frac{P\left({\mathbf{X}}^{{}^{\prime }}\right)}{P\left(\mathbf{X}\right)}\right)d{\mathbf{X}}^{{}^{\prime }}\hfill \\ \hfill & =\int P\left({\mathbf{X}}^{{}^{\prime }}\right)ln\left(P\left({\mathbf{X}}^{{}^{\prime }}\right)\right)d{\mathbf{X}}^{{}^{\prime }}-\int P\left({\mathbf{X}}^{{}^{\prime }}\right)ln\left(P\left(\mathbf{X}\right)\right)d{\mathbf{X}}^{{}^{\prime }}\hfill \\ \hfill & =\int P\left({\mathbf{X}}^{{}^{\prime }}\right)ln\left(P\left({\mathbf{X}}^{{}^{\prime }}\right)\right)d{\mathbf{X}}^{{}^{\prime }}-\int P\left({\mathbf{X}}^{{}^{\prime }}\right)ln\left(P\left({\mathbf{X}}^{{}^{\prime }}-\mathrm{\Delta }{\mathbf{X}}^{{}^{\prime }}\right)\right)d{\mathbf{X}}^{{}^{\prime }}\hfill \\ \hfill & \approx \frac{1}{2}\mathrm{\Delta }{{\mathbf{X}}^{{}^{\prime }}}^T\int d{\mathbf{X}}^{{}^{\prime }}P\left({\mathbf{X}}^{{}^{\prime }}\right)\left({\nabla }_{{\mathbf{X}}^{{}^{\prime }}}^{2}ln\left(P\left({\mathbf{X}}^{{}^{\prime }}\right)\right)\right)\mathrm{\Delta }{\mathbf{X}}^{{}^{\prime }}\hfill \\ \hfill & =\frac{1}{2}\mathrm{\Delta }{{\mathbf{X}}^{{}^{\prime }}}^T\mathcal{F}\left({\mathbf{X}}^{{}^{\prime }}\right)\mathrm{\Delta }{\mathbf{X}}^{{}^{\prime }},\hfill \end{array}$$

(2)

where ${\nabla }_{{\mathbf{X}}^{{}^{\prime }}}^2$ denotes the corresponding Hessian matrix. Please note that we have assumed very small changes in the internal variable vector $\mathbf{X}$. Here, $\mathcal{F}\left(·\right)$ is noted as the Fisher information metric. Since the last formula does not depend on $\mathbf{X}$, then the averaging in Equation (1) becomes obsolete. Using the relations Equations (1) and (2) provides a connection between the Fisher information of the cell internal state and the mutual information with the cellular microenvironment:

$$I\left(\mathbf{Y},\mathbf{X}\right)=\frac{1}{2\tilde{\beta }}\mathrm{\Delta }{{\mathbf{X}}^{{}^{\prime }}}^T\mathcal{F}\left({\mathbf{X}}^{{}^{\prime }}\right)\mathrm{\Delta }{\mathbf{X}}^{{}^{\prime }}$$

(3)

The latter formula implies that the fidelity of the future cell’s internal state is related to the available information in the microenvironment. The above quadratic form makes us view mutual information as a kind of energy functional.

2.2. Continuous Time Dynamics

Now, we further assume a very short decision time for the internal variable evolution $\tau \ll 1$. Along with the Bayesian learning, we assume that the microenvironmental distribution is a quasi-steady state, and therefore we focus only on the dynamics of the internal variable pdf $P\left({\mathbf{X}}^{\prime })=P(\mathbf{X}+\mathrm{\Delta }\mathbf{X},t+\tau \right)$, where the increment $\mathrm{\Delta }\mathbf{X}\in {\mathbb{R}}^n$. Using the multivariate Taylor series expansion, we write

$$\begin{array}{cc}\hfill & P\left(\mathbf{X}+\mathrm{\Delta }\mathbf{X},t+\tau \right)=\frac{P\left(\mathbf{Y}\mid \mathbf{X},t\right)P\left(\mathbf{X},t\right)}{P\left(\mathbf{Y}\right)},\hfill \\ \hfill & \Rightarrow P\left(\mathbf{X},t\right)+\mathrm{\Delta }\mathbf{X}·{\nabla }_{\mathbf{X}}P\left(\mathbf{X},t\right)+\tau \frac{\partial P\left(\mathbf{X},t\right)}{\partial t}+\mathcal{O}({\tau }^2,\mathrm{\Delta }{\mathbf{X}}^2)=\frac{P\left(\mathbf{Y}\mid \mathbf{X},t\right)P\left(\mathbf{X},t\right)}{P\left(\mathbf{Y}\right)},\hfill \\ \hfill & \Rightarrow \frac{\partial P\left(\mathbf{X},t\right)}{\partial t}\approx -\frac{\mathrm{\Delta }\mathbf{X}}{\tau }·{\nabla }_{\mathbf{X}}P\left(\mathbf{X},t\right)-\frac{1}{\tau }\left(1-\frac{P\left(\mathbf{Y}\mid \mathbf{X},t\right)}{P\left(\mathbf{Y}\right)}\right)P\left(\mathbf{X},t\right)\hfill \end{array}$$

(4)

The term $\frac{P\left(\mathbf{Y}\mid \mathbf{X},t\right)}{P\left(\mathbf{Y}\right)}$ is the information flow due to cell sensing (empirical likelihood). Now, Equation (4) reaches a steady state only when the cell senses perfectly the microenvironment, i.e., $P\left(\mathbf{Y}\mid \mathbf{X},t\right)$ is equal as $P\left(\mathbf{Y}\right)$. The steady solution of the evolution of probability distribution helps us to understand how it evolved over a long time, which can tell us how the internal variables of cells settle. So, close to the steady state (i.e., $\frac{\partial P\left(\mathbf{X},t\right)}{\partial t}=0$), the Equation (4) further reads as

$$\begin{array}{cc}\hfill & \mathrm{\Delta }\mathbf{X}·{\nabla }_{\mathbf{X}}P\left(\mathbf{X}\right)=-\left(1-\frac{P\left(\mathbf{Y}\mid \mathbf{X}\right)}{P\left(\mathbf{Y}\right)}\right)P\left(\mathbf{X},t\right)\approx \mathbf{i}\left(\mathbf{X}:\mathbf{Y}\right)P\left(\mathbf{X}\right)\hfill \\ \hfill & \Rightarrow \sum _{i=1}^n\mathrm{\Delta }{X}_i\frac{\partial }{\partial X_i}P\left(\mathbf{X}\right)=\mathbf{i}\left(\mathbf{X}:\mathbf{Y}\right)P\left(\mathbf{X}\right)\hfill \\ \hfill & \Rightarrow \sum _{i=1}^n\mathrm{\Delta }{X}_i\frac{\partial }{\partial X_i}\left(lnP\left(\mathbf{X}\right)\right)=\mathbf{i}\left(\mathbf{X}:\mathbf{Y}\right)\hfill \end{array}$$

(5)

Above, we used the identity $ln\left(x\right)\approx 1-x$ for small x and the definition of the point-wise mutual information as $\mathbf{i}\left(\mathbf{X}:\mathbf{Y}\right)=ln\frac{P\left(\mathbf{Y}\mid \mathbf{X}\right)}{P\left(\mathbf{Y}\right)}$.

Deriving an analytical solution for Equation (5) is a daunting task. Therefore, we use a Gibbs ansatz, which additionally assumes that mutual independence of the random variable $X_i\perp X_j$ for $i\ne j$:

$$\begin{array}{cc}\hfill & P\left(\mathbf{X}\right)\approx \prod _{i=1}^{n}P\left(X_i\right)=\frac{e^{-{\sum }_{i=1}^n{\alpha }_i{U}_i}}{Z}\hfill \\ \hfill & \Rightarrow P\left(X_i\right)=\frac{e^{-{\alpha }_i{U}_i}}{Z_i}\hfill \end{array}$$

(6)

Combining the above ansatz with the Equation (5), we obtain

$$\mathbf{i}\left(\mathbf{X}:\mathbf{Y}\right)=-\sum _{i=1}^n\mathrm{\Delta }{X}_i{\alpha }_i\frac{\partial U_i\left(X_i\right)}{\partial X_i}$$

(7)

Using our results in the Appendix A and in particular Equation (A4), we can write

$$\mathbf{i}\left(\mathbf{X}:\mathbf{Y}\right)=lnP\left(\mathbf{Y}\mid \mathbf{X}\right)-lnP\left(\mathbf{Y}\right)=\sum _{i=1}^{n}lnP\left(\mathbf{Y}\mid X_i\right)-nlnP\left(\mathbf{Y}\right)=\sum _{i=1}^n\mathbf{i}\left(X_i:\mathbf{Y}\right)$$

(8)

Now combining the above equations and integrating for the variable $X_i$, we can obtain an explicit formula for the potential $U_i$:

$$U_i\left(X_i\right)=-\frac{1}{{\alpha }_i\mathrm{\Delta }{X}_i}{\int }^{X_i}\mathbf{i}\left({\tilde{X}}_i:\mathbf{Y}\right)d{\tilde{X}}_i.$$

(9)

Therefore, the probability distribution for the internal variable $X_i$ reads

$$P\left(X_i\right)=\frac{e^{{\beta }_i{\int }^{X_i}\mathbf{i}\left({\tilde{X}}_i:\mathbf{Y}\right)d{\tilde{X}}_i}}{Z_i},$$

(10)

where we introduce the sensitivity parameter ${\beta }_i\propto \mathrm{\Delta }{X}_i^{-1}$. Working out further the above equation, we obtain

$$\begin{array}{cc}\hfill & P\left(X_i\right)=\frac{e^{{\beta }_i{\int }^{X_i}i\left(\mathbf{Y}:\widetilde{X_i}\right)d{\tilde{X}}_i}}{Z_i}\hfill \\ \hfill & =\frac{e^{{\beta }_i{\int }^{X_i}d{\tilde{X}}_{i}i\left(\mathbf{Y}:\widetilde{X_i}\right){\int }_{{\mathbb{R}}^m}P\left(\mathbf{Y}\right|{\tilde{X}}_i)d\mathbf{Y}}}{\int d{X}_i{e}^{{\beta }_i{\int }^{X_i}d{\tilde{X}}_{i}i\left(\mathbf{Y}:\widetilde{X_i}\right){\int }_{{\mathbb{R}}^m}P\left(\mathbf{Y}\right|{\tilde{X}}_i)d\mathbf{Y}}}\hfill \\ \hfill & =\frac{e^{-{\beta }_i{\int }^{X_i}S\left(\mathbf{Y}\mid X={\tilde{X}}_i\right)d{\tilde{X}}_i-{\beta }_i{\int }^{X_i}d{\tilde{X}}_i{\int }_{{\mathbb{R}}^m}d\mathbf{Y}P\left(\mathbf{Y}\right|{\tilde{X}}_i)lnp\left(\mathbf{Y}\right)}}{\int d{X}_i{e}^{-{\beta }_i{\int }^{X_i}S\left(\mathbf{Y}\mid X={\tilde{X}}_i\right)d{\tilde{X}}_i-{\beta }_i{\int }^{X_i}d{\tilde{X}}_i{\int }_{{\mathbb{R}}^m}d\mathbf{Y}P\left(\mathbf{Y}\right|{\tilde{X}}_i)lnp\left(\mathbf{Y}\right)}}\hfill \\ \hfill & =\frac{e^{-{\beta }_i{\int }^{X_i}d{\tilde{X}}_{i}S\left(\mathbf{Y}\right|X={\tilde{X}}_i)-{\beta }_i^{\prime }{X}_i}}{Z_i}.\hfill \end{array}$$

(11)

where we used the fact that the ${\int }_{{\mathbb{R}}^m}P\left(\mathbf{Y}\right|{\tilde{X}}_i)d\mathbf{Y}=1$ and the definition of the conditional entropy $S(\mathbf{Y}\mid X={\tilde{X}}_i)=-\int d\mathbf{Y}P\left(\mathbf{Y}\right|X={\tilde{X}}_i)lnP\left(\mathbf{Y}\right|X={\tilde{X}}_i)$. The parameter ${\beta }_i^{\prime }={\beta }_{i}d{\tilde{X}}_i{\int }_{{\mathbb{R}}^m}d\mathbf{Y}P\left(\mathbf{Y}\right|{\tilde{X}}_i)lnp\left(\mathbf{Y}\right)$ is a real constant.

3. Connection between Hierarchical Fokker–Planck Equation and Bayesian Learning Process

In this section, we shall discuss the connection between dissipative dynamics and Bayesian learning regarding the cell decision-making process. Since cell decision making is a stochastic process of the continuous internal variable $\mathbf{X}$, we can assume the existence of the Fokker–Planck description. When there exists a timescale separation between two dynamical variables, a hierarchical Fokker–Planck equation [32] can be derived. In this section, we shall show how this formalism can be applied in cell decision making and also will show how it helps us to study the origin of biophysical forces in terms of the information-theoretic quantities as shown in Figure 1.

Let us consider $\mathbf{X}$ and $\mathbf{Y}$ to be the internal variables which evolve in a slow timescale and external variables that are fast, and the corresponding 2-tuple random variables (which evolve over time) as

$$\mathit{M}=\left(\begin{array}{c}{\mathbf{M}}_{\mathbf{1}}\\ {\mathbf{M}}_{\mathbf{2}}\end{array}\right)=\left(\begin{array}{c}\mathbf{X}\\ \mathbf{Y}\end{array}\right)$$

(12)

Now for a random variable $\mathit{M}$, one can write in the Ito-sense the generalized stochastic differential equation for multiplicative noise processes as

$$d\mathit{M}=\mathbf{K}\left(\mathit{M},t\right)dt+\mathrm{\Sigma }\left(\mathit{M},t\right)d\mathit{W}$$

(13)

In this above Equation (13), we define the drift term $\mathbf{K}$, the $\mathrm{\Sigma }$ that is a $2\times 2$ covariance matrix and $d\mathit{W}$ as the Wiener process [35], which satisfies the mutual independence condition below

$$d{\mathit{W}}_{\mathbf{i}}d{\mathit{W}}_{\mathbf{j}}={\delta }_{ij}dt$$

(14)

The realization of $\mathbf{X}={\mathbf{M}}_{\mathbf{1}}$ and $\mathbf{Y}={\mathbf{M}}_{\mathbf{2}}$, obeys the time-dependent joint probability. $P\left(\mathbf{X},\mathbf{Y},t\right)$ which satisfies the generalized Fokker–Planck equation. Now, the generalized Fokker–Planck equation [35,36,37] corresponding to the Langevin Equation (13) for two-variable homogeneous processes can be written as

$$\frac{\partial P}{\partial t}=-\sum _{p=1}^2\frac{\partial }{\partial M_p}\left(K_{p}P\right)+\sum _{p,q=1}^2\frac{{\partial }^2}{\partial M_p\partial M_q}\left({\sigma }_{pq}P\right)$$

(15)

where drift coefficients $K_p=K_p\left(\mathbf{X},\mathbf{Y},t\right)$ and diffusion coefficients ${\sigma }_{pq}={\sigma }_{qp}={\sigma }_{pq}\left(\mathbf{X},\mathbf{Y},t\right)$.

The Fokker–Planck equations represent the mesoscopic scale of a dynamical system [38]. Interestingly, in a large timescale separation at the mesoscopic level, the degrees of freedom associated with the fast variables depend on slow variables but not vice versa. Since we assumed that the microenvironmental variables $\mathbf{Y}$ evolve at the fastest timescale, it follows that $K_1\equiv K_1\left(\mathbf{X},\mathbf{Y}\right)$, $K_2\equiv K_2\left(\mathbf{X}\right)$ and ${\sigma }_{22}\left(\mathbf{Y},\mathbf{X},t\right)\equiv {\sigma }_{22}\left(\mathbf{X}\right)$. To use the separation method adiabatically, we shall substitute

$$P\left(\mathbf{X},\mathbf{Y},t\right)=P\left(\mathbf{Y},t\mid \mathbf{X}\right)P\left(\mathbf{X}\right),$$

(16)

where the $P\left(\mathbf{X}\right)$ is time invariant relative to the evolution of the microenvironmental variables. Thus, the dynamics of the joint probability reduces to the dynamics of the fast variable $\mathbf{Y}$ and using Equation (15), we have

$$\begin{array}{cc}\hfill & \frac{\partial P\left(\mathbf{X},\mathbf{Y},t\right)}{\partial t}=P\left(\mathbf{X}\right)\frac{\partial P\left(\mathbf{Y},t\mid \mathbf{X}\right)}{\partial t}=-P\left(\mathbf{X}\right){\nabla }_{\mathbf{Y}}·\left(K_1\left(\mathbf{Y},\mathbf{X},t\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)\right)\hfill \\ \hfill & -{\nabla }_{\mathbf{X}}·\left(K_2\left(\mathbf{X}\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)P\left(\mathbf{X}\right)\right)\hfill \\ \hfill & +P\left(\mathbf{X}\right){\nabla }_{\mathbf{Y}}^2\left({\sigma }_{11}\left(\mathbf{Y},\mathbf{X},t\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)\right)\hfill \\ \hfill & +2{\nabla }_{\mathbf{X}}·\left[P\left(\mathbf{X}\right){\nabla }_{\mathbf{Y}}\left({\sigma }_{12}\left(\mathbf{Y},\mathbf{X},t\right)\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)\right]\hfill \\ \hfill & +{\nabla }_{\mathbf{X}}^2\left({\sigma }_{22}\left(\mathbf{X}\right)P\left(\mathbf{X}\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)\right)\hfill \end{array}$$

(17)

From this point, the equations for the fast degree of freedom and the others (slow degree of freedom and coupling between them) are derived, respectively, as follows:

$$\frac{\partial P\left(\mathbf{Y},t\mid \mathbf{X}\right)}{\partial t}=-{\nabla }_{\mathbf{Y}}·\left(K_1\left(\mathbf{Y},\mathbf{X},t\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)\right)+{\nabla }_{\mathbf{Y}}^2\left({\sigma }_{11}\left(\mathbf{Y},\mathbf{X},t\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)\right),$$

(18)

$$\begin{array}{cc}\hfill & {\nabla }_{\mathbf{X}}·\left(K_2\left(\mathbf{X}\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)P\left(\mathbf{X}\right)\right)+2{\nabla }_{\mathbf{X}}·\left[P\left(\mathbf{X}\right){\nabla }_{\mathbf{Y}}·\left({\sigma }_{12}\left(\mathbf{Y},\mathbf{X},t\right)\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)\right]\hfill \\ \hfill & +{\nabla }_{\mathbf{X}}^2\left({\sigma }_{22}\left(\mathbf{X}\right)P\left(\mathbf{X}\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)\right)=0\hfill \end{array}$$

(19)

From Equation (19), if we integrate once over $\mathbf{X}$, it follows

$$\begin{array}{cc}\hfill & -K_2\left(\mathbf{X}\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)P\left(\mathbf{X}\right)+2P\left(\mathbf{X}\right){\nabla }_{\mathbf{Y}}·\left({\sigma }_{12}\left(\mathbf{Y},\mathbf{X},t\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)\right)\hfill \\ \hfill & +{\nabla }_{\mathbf{X}}·\left({\sigma }_{22}\left(\mathbf{X}\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)P\left(\mathbf{X}\right)\right)=0,\hfill \end{array}$$

(20)

and working further on the equations

$$\begin{array}{cc}\hfill & -K_2\left(\mathbf{X}\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)P\left(\mathbf{X}\right)\hfill \\ \hfill & +2P\left(\mathbf{X}\right){\nabla }_{\mathbf{Y}}·\left({\sigma }_{12}\left(\mathbf{Y},\mathbf{X},t\right)\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)+2P\left(\mathbf{X}\right){\sigma }_{12}\left(\mathbf{Y},\mathbf{X},t\right){\nabla }_{\mathbf{Y}}\left(P\left(\mathbf{Y},t\mid \mathbf{X}\right)\right)\hfill \\ \hfill & +{\nabla }_{\mathbf{X}}·\left({\sigma }_{22}\left(\mathbf{X}\right)\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)P\left(\mathbf{X}\right)+{\sigma }_{22}\left(\mathbf{X}\right)P\left(\mathbf{X}\right){\nabla }_{\mathbf{X}}\left(P\left(\mathbf{Y},t\mid \mathbf{X}\right)\right)\hfill \\ \hfill & +{\sigma }_{22}\left(\mathbf{X}\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right){\nabla }_{\mathbf{X}}\left(P\left(\mathbf{X}\right)\right)=0.\hfill \end{array}$$

(21)

To isolate the slow degree of freedom, we further separate Equation (21) as follows:

$$-\left(K_2\left(\mathbf{X}\right)-{\nabla }_{\mathbf{X}}·\left({\sigma }_{22}\left(\mathbf{X}\right)\right)\right)P\left(\mathbf{X}\right)+{\sigma }_{22}\left(\mathbf{X}\right){\nabla }_{\mathbf{X}}\left(P\left(\mathbf{X}\right)\right)=0,$$

(22)

$$2{\nabla }_{\mathbf{Y}}·\left({\sigma }_{12}\left(\mathbf{Y},\mathbf{X},t\right)P\left(\mathbf{Y},t\mid \mathbf{X}\right)\right)+{\sigma }_{22}\left(\mathbf{X}\right){\nabla }_{\mathbf{Y}}\left(P\left(\mathbf{Y},t\mid \mathbf{X}\right)\right)=0,$$

(23)

which are the equations for the slow degree of freedom and the coupling, respectively. Thus, Equations (18), (22) and (23) are the ones to be analyzed. Now, we try to establish the connection between hierarchical Fokker–Planck equations and steady-state Bayesian learning when the internal variable is one-dimensional. The general solution of Equation (22) in one dimension can be written as

$$P\left(X_i\right)=f_{0}exp\left({\int }^{X_i}d{\tilde{X}}_i\frac{K_2\left({\tilde{X}}_i\right)}{{\sigma }_{22}\left({\tilde{X}}_i\right)}-ln{\sigma }_{22}\left(X_i\right)\right).$$

(24)

where $f_0$ is a positive constant. If we have information about the drift term $K_2\left({\tilde{X}}_i\right)$ and diffusion coefficient ${\sigma }_{22}\left(X_i\right)$, we can easily calculate the probability distribution of the internal variables from Equation (24), which is independent of the fast variable. So, comparing Equations (11) and (24), one can obtain

$$\begin{array}{cc}\hfill & P\left(X=X_i\right)=\frac{e^{-{\beta }_i{\int }^{X_i}d{\tilde{X}}_{i}S\left(\mathbf{Y}\right|X={\tilde{X}}_i)-{\beta }_i^{\prime }{X}_i}}{Z_i}=f_{0}exp\left({\int }^{X_i}d{\tilde{X}}_i\frac{K_2\left({\tilde{X}}_i\right)}{{\sigma }_{22}\left({\tilde{X}}_i\right)}-ln{\sigma }_{22}\left(X_i\right)\right),\hfill \\ \hfill & \Rightarrow \frac{e^{-{\beta }_i{\int }^{X_i}d{\tilde{X}}_{i}S\left(\mathbf{Y}\right|X={\tilde{X}}_i)-{\beta }_i^{\prime }{X}_i}}{Z_i}=\tilde{f}exp\left(\frac{1}{{\sigma }_{22}}{\int }^{X_i}{K}_2\left(\widetilde{X_i}\right)d\widetilde{X_i}\right),\hfill \\ \hfill & \Rightarrow -{\beta }_i{\int }^{X_i}d{\tilde{X}}_{i}S\left(\mathbf{Y}\right|X={\tilde{X}}_i)-{\beta }_i^{\prime }{X}_i=ln\left[\tilde{f}Z\right]+\frac{1}{{\sigma }_{22}}{\int }^{X_i}{K}_2\left(\widetilde{X_i}\right)d\widetilde{X_i},\hfill \\ \hfill & \Rightarrow K_2\left(X_i\right)=-{\beta }_i{\sigma }_{22}S\left(\mathbf{Y}\right|X=X_i)-\frac{{\beta }_i^{\prime }}{2}{X}_i^2.\hfill \end{array}$$

(25)

In the above Equation (25), $\tilde{f}$ is defined as $\frac{f_0}{{\sigma }_{22}}$ and the diffusion coefficient ${\sigma }_{22}\left(X_i\right)$ in Equation (25) is considered constant, i.e., ${\sigma }_{22}\left(X_i\right)={\sigma }_{22}$. Therefore, we can directly see how the microenvironmental entropy and the drift force have a unique relation.

4. Implications of Cell Sensing Activity

Cell sensing is usually defined as a process where cells communicate with the external environment based on their internal regulatory network of signaling molecules. In the context of Bayesian learning cells, the cell sensing distribution $P\left(\mathbf{Y}\right|\mathbf{X})$ plays a central role. The problem is that the regulation between a particular sensing molecule and the set of microenvironmental variables can be complex [39]. For simplicity, we constrain ourselves to one-dimensional internal and external variables. Let us consider that the microenvironment Y is sensed by the internal state X as

$$Y_X=Y\mid X=F\left(X,\langle Y^n\rangle \right).$$

(26)

Here, we assume that the cell sensing function $F(·)$ also depends on moments of the microenvironmental variable and, consequently, we assume their existence. Now, if we perform a Taylor series expansion around the mean value of the internal state $\overline{X}$ in Equation (26),

$$\begin{array}{cc}\hfill & Y_X=F\left(\overline{X}\right)+\left|\frac{\partial }{\partial X}F\left(\overline{X}\right)\right|\left(X-\overline{X}\right)\hfill \\ \hfill & Y_X-\overline{Y}=F\left(\overline{X}\right)-\overline{Y}+\left|\frac{\partial }{\partial X}F\left(\overline{X}\right)\right|\left(X-\overline{X}\right)\hfill \\ \hfill & {\sigma }_{Y\mid X}^2\left(x\right)={〈{\left(b+g(x-\overline{X})\right)}^2〉}_{P\left(Y\right)}={\left(b+g(x-\overline{X})\right)}^2\hfill \end{array}$$

(27)

Here, we define the bias term $b=F\left(\overline{X}\right)-\overline{Y}$ and the linear sensing response to microenvironmental changes Y defined by $g=\mid \frac{\partial }{\partial X}F\left(\overline{X}\right)\mid$. Please note that both b and g depend only on the moments of Y. The biological relevance of this linear sensing function can be found in the classical receptor–ligand models [40]. In particular, let us assume that the sensed environment variable $Y|X$ is the ligand–receptor complex and the variable X corresponds to the receptor density. If g is a first-order Hill function for the first moment of Y, which in this context is the ligand concentration, and if $F\left(\overline{X}\right)=0$, then first, Equation (26) corresponds to the textbook steady state of the complex formation [40].

Moreover, we consider the microenvironmental distribution as Gaussian, where the entropy of the microenvironment, conditioned by the corresponding internal states, can be written as

$$S(Y\mid X=x)=\frac{1}{2}ln\left(2\pi e{\sigma }_{Y\mid X}^2\left(x\right)\right).$$

(28)

Now, using the above expression of microenvironmental conditional entropy, one can calculate the steady state of cellular internal variables from Bayesian learning using Equation (11). In turn, it can be written as

$$\begin{array}{cc}\hfill & P\left(X\right)\propto e^{-\beta {\int }^{X}S\left(Y\mid X=\tilde{X}\right)d\tilde{X}-{\beta }^{{}^{\prime }}X}\hfill \\ \hfill & =e^{-\beta {\int }^{X}ln\left(b+g(\tilde{X}-\overline{X})\right)d\tilde{X}-{\beta }^{{}^{\prime }}X}\hfill \\ \hfill \end{array}$$

(29)

Interestingly, we have two cases to study the steady-state distribution of the cellular internal states: $\left(\mathbf{I}\right)$ when the response of X to microenvironmental changes is negligible and $\left(\mathbf{II}\right)$ when there exists a finite correlation value between internal cellular state and microenvironmental state, which follows as

$$\begin{array}{cc}\hfill & P\left(X\right)=C_1{e}^{-\overline{\beta }X},g\ll 1\hfill \\ \hfill & P\left(X\right)=C_2{\left(b+g\left(X-\overline{X}\right)\right)}^{\beta \left(X-\overline{X}+\frac{b}{g}\right)}{e}^{-(\beta +{\beta }^{{}^{\prime }})X},g=\mathcal{O}\left(1\right)\hfill \end{array}$$

(30)

Here, $C_0$ and $C_1$ are normalization constants of corresponding probability distributions, and $\overline{\beta }$ is defined as $(\beta lnb+{\beta }^{{}^{\prime }})$. In case $\left(\mathbf{I}\right)$, i.e., when g is equal to 0, the steady-state distribution of internal variables converges to an exponential distribution. Please note that the sensor OFF probability distribution makes sense only for $\overline{\beta }>0$. In the ON case, when the linear response g is finite and $\beta <0$, the expression of the steady state is unimodal. Interestingly, for $\beta >0$ and for a finite range of X values, the distribution is bimodal with the highest probability density around the boundaries of the domain. Please note that for very large ${\beta }^{{}^{\prime }}$ values, the exponential decay term dominates. In a nutshell, the above expression of the internal state shows how an ON-OFF switching case can happen when the environment correlates with the cell and as a response the cell senses the microenvironment changing its phenotype, which confirms the existence of the monostable–bistable regime as shown in Figure 2.

5. Bayesian Learning Minimizes the Microenvironmental Entropy in Time

Recently, we postulated the least environmental uncertainty principle (LEUP) for the decision making of cells in their multicellular context [21,22]. The main premise of LEUP is that the microenvironmental entropy/uncertainty decreases over time. Here, we hypothesized that cells use Bayesian learning to infer their internal states from microenvironmental information. In particular, we previously showed that $\frac{dS\left(\mathbf{Y}\right|\mathbf{X})}{dt}\le 0$ [22], which is the case in the Bayesian learning case. To illustrate this, let us focus on the Gaussian 1D case of the previous section. Averaging Equation (27) for the distribution $p(X,Y)$, we can obtain the following:

$${\sigma }_{Y\mid X}^2=b^2+g^2{\sigma }_x^2.$$

(31)

One can show that the linear response term is proportional to the covariance of the internal and external variables, i.e., $g\propto cov(X,Y)$ as a result of the Gaussian conditional variable. As the Bayesian learning is reaching equilibrium, according to Equation (1), the covariance approaches zero and consequently, ${\sigma }_{Y\mid X}^2\stackrel{t\to \infty }{\to }{b}^2.$

Please note that we still assume that the microenvironmental pdf is in a quasi-steady state due to the time scale separation [22]. The latter implies that the variance of $Y|X$ is monotonically decreasing and therefore $S\left(Y\right|X)$ is also a decaying function in time. Therefore, we can postulate that Bayesian learning is compatible with the LEUP idea.

Mathematically speaking, the original LEUP formulation was employing an entropy maximization principle, where one can calculate the distribution of cell internal states using as a constraint the mutual information between local microenvironment variables and internal variables. Adding as a constraint the expected value of internal states, the corresponding variational formulation reads:

$$\begin{array}{cc}\hfill \frac{\delta }{\delta P\left(X_i\right)}& \{S\left(X_i\right)+{\beta }_i\left[\int d{X}_{i}P\left(X_i\right)\int d\mathbf{Y}P(\mathbf{Y}\mid X_i)\mathfrak{i}\left(\mathbf{Y}:X_i\right)-\overline{I}\left(\mathbf{Y}:X_i\right)\right]\hfill \\ \hfill & -{\beta }_i^{{}^{\prime }}\left[\int P\left(X_i\right)X_{i}d{X}_i-\overline{X_i}\right]-{\lambda }_i\left[\int P\left(X_i\right)d{X}_i-1\right]\}=0,\hfill \end{array}$$

(32)

Here, $\delta /\delta P\left(X_i\right)$ is the functional derivative with respect to the internal states. Three Lagrange multipliers in Equation (32), i.e., ${\beta }_i$, ${\beta }_i^{{}^{\prime }}$ and ${\lambda }_i$, are associated with the steady-state value of the mutual information $\overline{I}\left(\mathbf{Y},X_i\right)$, mean value of the internal variables and the normalization constant of the probability distribution. The constraint or the partial information about the internal and external variables is written in terms of the statistical observable. Solving Equation (32), we can find a Gibbs-like probability distribution:

$$P\left(X_i\right)=\frac{e^{{\beta }_{i}D\left(\mathbf{Y}\mid X={\tilde{X}}_i\left|\right|\mathbf{Y}\right)-{\beta }_i^{{}^{\prime }}{X}_i}}{Z_i}=\frac{e^{-{\beta }_{i}S\left(\mathbf{Y}\mid X_i={\tilde{X}}_i\right)-{\beta }_i^{\prime }{X}_i}}{Z_i^{{}^{\prime }}}.$$

(33)

Here, $Z_i^{{}^{\prime }}=\int e^{-{\beta }_{i}S\left(\mathbf{Y}\mid X_i={\tilde{X}}_i\right)-{\beta }_i^{{}^{\prime }}{\tilde{X}}_i}d{\tilde{X}}_i$ is the normalization constants. Please note that we used the fact that $D\left(\mathbf{Y}\mid X=X_i\left|\right|\mathbf{Y}\right)=-S\left(\mathbf{Y}\right|X=X_i))-\int d\mathbf{Y}p(\mathbf{Y}\mid X)lnp\left(\mathbf{Y}\right)$, where the second term gets simplified since it is independent of $X_i$. Interestingly, it can coincide with the Bayesian learning context as a special case, where the $i\left(\mathbf{Y}:X_i\right)\to 0$. Using Equation (11) in a finite domain $X_i\in \mathrm{\Omega }$ and the mean value theorem for integration, there exists a value ${\widehat{X}}_i$ such that

$$P\left(X_i\right)=\frac{e^{-{\beta }_{i}S\left(\mathbf{Y}\right|X_i={\widehat{X}}_i)-{\beta }_i^{\prime }{X}_i}}{Z_i^{\prime }}.$$

(34)

Therefore, the form of the maximum entropy distribution (33) and the Bayesian learning steady-state distribution (11) coincide when the random variable $X_i$ takes values in the vicinity of ${\widehat{X}}_i$.

6. Discussion

In this paper, we elaborate on the idea of cellular decision-making based on Bayesian learning, assuming a time-scale separation between environmental and internal variables. We derive a stochastic description of the temporal evolution of the corresponding dynamics, studying the impact of cell sensing on the internal state distribution and the corresponding microenvironmental entropy evolution.

An interesting finding is the steady-state distributions of the internal state depending on the state of the cell sensor activity. When the cell weakly senses its microenvironment, the internal state follows an exponential distribution (see Equation (29)). In terms of the receptor–ligand sensing mechanism, this implies that no specific amount of receptors is expressed by the cell. When the sensor is in the ON state, then a unimodal distribution occurs, which implies that the cell expresses a precise number of, for example, receptors as a response to a certain stimulus. The former can be viewed as the physiological modus operandi of the cell. However, when the sensitivity $\beta$ changes sign, then the probability mass is distributed to the extreme values of the internal state space. This can be potentially mediated by a bistability regulation mechanism, e.g., for the receptor production. Such bimodality is relevant in the context of cancer, where it is considered a malignancy prognostic biomarker [41,42]. However, it can also occur in physiological cases such as in healthy immune cells [43]. It would be interesting to explore if the sensing activity is a plausible mechanism for explaining transitions from unimodality to bimodality.

One important point of interest is the range of validity of regarding the timescale separation between the cell decision and the cell’s microenvironmental variables. In particular, we assumed that the internal state characteristic time is slower than the microenvironmental one, which can be true for decision timescales related to the cell cycle duration. Sometimes, cell decisions may seem to be happening within one cell cycle, but the underlying molecular expressions may evolve even over many cell cycles [33,34]. During the cell cycle time, we can safely assume that external variables, such as chemical signal concentrations or migrating cells, will be in a quasi-equilibrium state. However, for cell decisions with shorter timescales, such as migration-related processes, which are at the order of one hour, this assumption needs to be relaxed. In the latter case, the discrete-time dynamics presented in Section 2 are still valid.

Here, we assumed that the fast timescale environmental variables can be influenced by the current state of cellular internal variables. However, we did not consider the influence of the past time states. This would imply non-Markov dynamics for internal cellular state evolution. It would be interesting to study how this assumption could impact the information flow dynamics between environmental states and cellular internal variables.

The outlined theory is related to single-cell decision making. Our ultimate goal is to understand how Bayesian learning impacts the collective behavior of a multicellular system. An agent-based model driven by Bayesian learning dynamics could be used to analyze the collective dynamics as in [22]. Interestingly, we expect a Bayesian learning multicellular theory to produce similar results to the $\mathit{rattling interactions}$ introduced in [44]. Similarly, in rattling dynamics, an approximation of the mutual information between neighboring individuals is minimized, leading to the emergence of a self-organized active collective state.

Regarding cell sensing, we took an agnostic approach, where a generic function was assumed. Linearizing the sensing function leads to steady-state dynamics, which could be seen in the ligand–receptor dynamics, e.g., [45], by assuming our sensed environment variable $Y|X$ is the ligand–receptor complex and the variable X the receptors. It will be alluring to further investigate the non-linear relationship between internal and external variables, which means considering a few more terms in the Taylor series expansion of conditional variance to simulate a greater variety of biological sensing scenarios.

Our decision-making approach is a dynamic theory based on Bayesian learning of cellular internal states upon variations of the microenvironment distribution. The classical Bayesian decision-making methods are of a static nature relying on Bayesian inference tools [16]. Belief updating networks resemble the ideas of Bayesian learning; however, such algorithms are treated typically computationally, and to our knowledge there have been not many attempts of deriving dynamic equations [46]. The oldest life science field where such ideas have been developed is human cognition. This dates back to 1860 when Hermann Helmholtz postulated the Bayesian brain hypothesis, where the nervous system organizes sensory data into an internal model of the outside world [47]. Recently, Karl Friston and collaborators formulated the brain free energy theory deriving a variational Bayesian framework for predicting cognitive dynamics. Friston’s ideas were recently translated into the Bayesian mechanics approach [48]. The latter resembles our approach, but it requires concepts of Markov blankets and control theory. The main difference is that all of the above attempt to model human cognition and not cell decision making.

Finally, assuming Bayesian learning/LEUP as a principle of cell decision making, we can bypass the need for a detailed understanding of the underlying biophysical processes. Here, we showed that even by using an unknown cell sensing function, we can infer the state of the cell with a minimal number of parameters. Building on these concepts, we can create theories and predictive tools that do not require the comprehensive knowledge of the underlying regulatory mechanisms.