Thermodynamics of the Ising Model Encoded in Restricted Boltzmann Machines

Abstract

The restricted Boltzmann machine (RBM) is a two-layer energy-based model that uses its hidden–visible connections to learn the underlying distribution of visible units, whose interactions are often complicated by high-order correlations. Previous studies on the Ising model of small system sizes have shown that RBMs are able to accurately learn the Boltzmann distribution and reconstruct thermal quantities at temperatures away from the critical point $T_c$. How the RBM encodes the Boltzmann distribution and captures the phase transition are, however, not well explained. In this work, we perform RBM learning of the $2d$ and $3d$ Ising model and carefully examine how the RBM extracts useful probabilistic and physical information from Ising configurations. We find several indicators derived from the weight matrix that could characterize the Ising phase transition. We verify that the hidden encoding of a visible state tends to have an equal number of positive and negative units, whose sequence is randomly assigned during training and can be inferred by analyzing the weight matrix. We also explore the physical meaning of the visible energy and loss function (pseudo-likelihood) of the RBM and show that they could be harnessed to predict the critical point or estimate physical quantities such as entropy.

1. Introduction

The tremendous success of deep learning in multiple areas over the last decade has really revived the interplay between physics and machine learning, in particular neural networks [1]. On the one hand, (statistical) physics ideas [2], such as the renormalization group (RG) [3], the energy landscape [4], free energy [5], glassy dynamics [6], jamming [7], Langevin dynamics [8], and field theory [9], shed some light on the interpretation of deep learning and statistical inference in general [10]. On the other hand, machine learning and deep learning tools are harnessed to solved a wide range of physics problems, such as interaction potential construction [11], phase transition detection [12,13], structure encoding [14], physical concepts’ discovery [15], and many others [16,17]. At the very intersection of these two fields lies the restricted Boltzmann machine (RBM) [18], which serves as a classical paradigm to investigate how an overarching perspective could benefit both sides.

The RBM uses hidden–visible connections to encode (high-order) correlations between visible units [19]. Its precursor—the (unrestricted) Boltzmann machine—was inspired by spin glasses [20,21] and is often used in the inverse Ising problem to infer physical parameters [22,23,24]. The restriction of hidden–hidden and visible–visible connections in RBMs allows for more efficient training algorithms and, therefore, leads to recent applications in Monte Carlo simulation acceleration [25], quantum wavefunction representation [26,27], and polymer configuration generation [28]. Deep neural networks formed by stacks of RBMs have been mapped onto the variational RG due to their conceptual similarity [29]. RBMs are also shown to be equivalent to tensor network states from quantum many-body physics [30] and interpretable in light of statistical thermodynamics [31,32,33]. As simple as it seems, energy-based models like the RBM could eventually become the building blocks of autonomous machine intelligence [34].

Besides the above-mentioned efforts, the RBM has also been applied extensively in the study of the minimal model for second-order phase transition—the Ising model. For the small systems under investigation, it was found that RBMs with an enough hidden units can encode the Boltzmann distribution, reconstruct thermal quantities, and generate new Ising configurations fairly well [35,36,37]. The visible → hidden → visible ⋯ generating sequence of the RBM can be mapped onto an RG flow in physical temperature (often towards the critical point) [38,39,40,41,42]. However, the mechanism and power of the RBM to capture physics concepts and principles have not been fully explored. First, in what way is the Boltzmann distribution of the Ising model learned by the RBM? Second, can the RBM learn and even quantitatively predict the phase transition without extra human knowledge? An affirmative answer to the second question is particularly appealing, because simple unsupervised learning methods such as principal component analysis (PCA) using configuration information alone do not provide quantitative prediction for the transition temperature [43,44,45] and supervised learning with neural networks requires human labeling of the phase type or temperature of a given configuration [46,47].

In this article, we report a detailed numerical study on RBM learning of the Ising model with a system size much larger than those used previously. The purpose is to thoroughly dissect the various parts of the RBM and reveal how each part contributes to the learning of the Boltzmann distribution of the input Ising configurations. Such understanding allows us to extract several useful machine learning estimators or predictors for physical quantities, such as entropy and phase transition temperature. Conversely, the analysis of a physical model helps us to obtain important insights about the meaning of RBM parameters and functions, such as the weight matrix, visible energy, and pseudo-likelihood. Below, we first introduce our Ising datasets and the RBM and its training protocols in Section 2. We then report and discuss the results of the model parameters, hidden layers, visible energy, and pseudo-likelihood in Section 3. After the conclusion, more details about the Ising model and the RBM are provided in the Appendix A, Appendix B and Appendix C. Sample codes of the RBM are shared on GitHub at https://github.com/Jing-DS/isingrbm (accessed on 18 November 2022).

2. Models and Methods

2.1. Dataset of Ising Configurations Generated by Monte Carlo Simulations

The Hamiltonian of the Ising model with $N=L^d$ spins in a configuration $\mathbf{s}={[s_1,s_2,\cdots ,s_N]}^T$ on a d-dimensional hypercubic lattice of linear dimension L in the absence of a magnetic field is

$$\begin{array}{c}\hfill \mathcal{H}\left(\mathbf{s}\right)=-J\sum _{\langle i,j\rangle }{s}_i{s}_j\end{array}$$

(1)

where the spin variable $s_i=\pm 1$ ($i=1,2,\cdots ,N$), the coupling parameter $J>0$ (set to unity) favors ferromagnetic configurations (parallel spins), and the notation $\langle i,j\rangle$ means to sum over nearest neighbors [48]. At a given temperature T, the configuration $\mathbf{s}$ drawn from the sample space of $2^N$ states follows the Boltzmann distribution

$$\begin{array}{c}\hfill p_T\left(\mathbf{s}\right)=\frac{e^{-\frac{\mathcal{H}\left(\mathbf{s}\right)}{k_{B}T}}}{Z_T}\end{array}$$

(2)

where $Z_T=\sum _{\mathbf{s}}{e}^{-\frac{\mathcal{H}\left(\mathbf{s}\right)}{k_{B}T}}$ is the partition function. The Boltzmann constant $k_B$ is set to unity.

Using single-flip Monte Carlo simulations under periodic boundary conditions [49], we generate Ising configurations for two-dimensional ($2d$) systems ($d=2$) of $L=64$ ($N=4096$) at $n_T=16$ temperatures $T=0.25,0.5,0.75,1.0,\cdots ,4.0$ (in units of $J/k_B$) and for three-dimensional ($3d$) systems $(d=3)$ of $L=16$ ($N=4096$) at $n_T=20$ temperatures $T=2.5,2.75$, $3.0,3.25$, $3.5,3.75,4.0$, $4.25,4.3,4.4,4.5$, $4.6,4.7,4.75,5.0,5.25$, $5.5,5.75,6.0,6.25$. After being fully equilibrated, $M=$ 50,000 configurations at each T are collected into a dataset $D_T$ for that T. For $2d$ systems, we also use a dataset $D_{\cup T}$ consisting of 50,000 configurations per temperature from all Ts.

Analytical results of the thermal quantities of the $2d$ Ising model, such as internal energy $\langle E\rangle$, (physical) entropy S, heat capacity $C_V$, and magnetization $\langle m\rangle$, are well known [50,51,52,53]. Numerical simulation methods and results of the $3d$ Ising model have also been reported [54]. The thermodynamic definitions and relations used in this work are summarized in Appendix A.

2.2. Restricted Boltzmann Machine

The restricted Boltzmann machine (RBM) is a two-layer energy-based model with $n_h$ hidden units (or neurons) $h_i=\pm 1$ ($i=1,2,\cdots ,n_h$) in the hidden layer, whose state vector is $\mathbf{h}={[h_1,h_2,\cdots ,h_{n_h}]}^T$, and $n_v$ visible units $v_j=\pm 1$ ($j=1,2,\cdots ,n_v$) in the visible layer, whose state vector is $\mathbf{v}={[v_1,v_2,\cdots ,v_{n_v}]}^T$ (Figure 1) [55]. In this work, the visible layer is just the Ising configuration vector, i.e., $\mathbf{v}=\mathbf{s}$, with $n_v=N$. We chose the binary unit $\{-1,+1\}$ (instead of $\{0,1\}$) to better align with the definition of Ising spin variable $s_i$.

The total energy $E_{\theta }(\mathbf{v},\mathbf{h})$ of the RBM is defined as

$$\begin{array}{cc}\hfill E_{\theta }(\mathbf{v},\mathbf{h})& =-{\mathbf{b}}^T\mathbf{v}-{\mathbf{c}}^T\mathbf{h}-{\mathbf{h}}^T\mathbf{W}\mathbf{v}\hfill \\ \hfill & =-\sum _{j=1}^{n_v}{b}_j{v}_j-\sum _{i=1}^{n_h}{c}_i{h}_i-\sum _{i=1}^{n_h}\sum _{j=1}^{n_v}{W}_{ij}{h}_i{v}_j\hfill \end{array}$$

(3)

where $\mathbf{b}={[b_1,b_2,\cdots ,b_{n_v}]}^T$ is the visible bias, $\mathbf{c}={[c_1,c_2,\cdots ,c_{n_h}]}^T$ is the hidden bias, and

$${\mathbf{W}}_{n_h\times n_v}=\left[\begin{array}{c}-{\mathbf{w}}_1^T-\\ -{\mathbf{w}}_2^T-\\ ⋮\\ -{\mathbf{w}}_{n_h}^T-\end{array}\right]=\left[\begin{array}{cccc}|& |& & |\\ {\mathbf{w}}_{:,1}& {\mathbf{w}}_{:,2}& \cdots & {\mathbf{w}}_{:,n_v}\\ |& |& & |\end{array}\right]$$

(4)

is the interaction weight matrix between visible and hidden units. Under this notation, each row vector ${\mathbf{w}}_i^T$ (of dimension $n_v$) is a filter mapping from the visible state $\mathbf{v}$ to a hidden unit i, and each column vector ${\mathbf{w}}_{:,j}$ (of dimension $n_h$) is an inverse filter mapping from the hidden state $\mathbf{h}$ to a visible unit j. All parameters are collectively written as $\theta =\{\mathbf{W},\mathbf{b},\mathbf{c}\}$. “Restricted” refers to the lack of interaction between hidden units or between visible units.

The joint distribution for an overall state $(\mathbf{v},\mathbf{h})$ is

$$p_{\theta }(\mathbf{v},\mathbf{h})=\frac{e^{-E_{\theta }(\mathbf{v},\mathbf{h})}}{Z_{\theta }}$$

(5)

where the partition function of the RBM:

$$Z_{\theta }=\sum _{\mathbf{v}}\sum _{\mathbf{h}}{e}^{-E_{\theta }(\mathbf{v},\mathbf{h})}.$$

(6)

The learned model distribution for visible state $\mathbf{v}$ is from the marginalization of $p_{\theta }(\mathbf{v},\mathbf{h})$:

$$p_{\theta }\left(\mathbf{v}\right)=\sum _{\mathbf{h}}{p}_{\theta }(\mathbf{v},\mathbf{h})=\frac{1}{Z_{\theta }}{e}^{-{\mathcal{E}}_{\theta }\left(\mathbf{v}\right)},$$

(7)

where the visible energy (an effective energy for visible state $\mathbf{v}$ (often termed as “free energy” in the machine learning literature)):

$$\begin{array}{cc}\hfill {\mathcal{E}}_{\theta }\left(\mathbf{v}\right)& =-{\mathbf{b}}^T\mathbf{v}-\sum _{i=1}^{n_h}ln\left(e^{-{\mathbf{w}}_i^T\mathbf{v}-c_i}+e^{{\mathbf{w}}_i^T\mathbf{v}+c_i}\right)\hfill \end{array}$$

(8)

is defined according to $e^{-{\mathcal{E}}_{\theta }\left(\mathbf{v}\right)}=\sum _{\mathbf{h}}{e}^{-E_{\theta }(\mathbf{v},\mathbf{h})}$ such that $Z_{\theta }=\sum _{\mathbf{v}}{e}^{-{\mathcal{E}}_{\theta }\left(\mathbf{v}\right)}.$ See Appendix B for a detailed derivation.

The conditional distributions to generate $\mathbf{h}$ from $\mathbf{v}$, $p_{\theta }\left(\mathbf{h}\right|\mathbf{v})$, and to generate $\mathbf{v}$ from $\mathbf{h}$, $p_{\theta }\left(\mathbf{v}\right|\mathbf{h})$, satisfying $p_{\theta }(\mathbf{v},\mathbf{h})=p_{\theta }\left(\mathbf{h}\right|\mathbf{v})p_{\theta }\left(\mathbf{v}\right)=p_{\theta }\left(\mathbf{v}\right|\mathbf{h})p_{\theta }\left(\mathbf{h}\right)$, can be written as products:

$$\begin{array}{cc}\hfill p_{\theta }\left(\mathbf{h}\right|\mathbf{v})& =\prod _{i=1}^{n_h}{p}_{\theta }\left(h_i\right|\mathbf{v})\hfill \\ \hfill p_{\theta }\left(\mathbf{v}\right|\mathbf{h})& =\prod _{j=1}^{n_v}{p}_{\theta }\left(v_j\right|\mathbf{h})\hfill \end{array}$$

(9)

because $h_i$ are independent of each other (at fixed $\mathbf{v}$) and $v_j$ are independent of each other (at fixed $\mathbf{h}$). It can be shown that

$$\begin{array}{cc}\hfill p_{\theta }(h_i=1|\mathbf{v})& =\sigma \left(2(c_i+{\mathbf{w}}_i^T\mathbf{v})\right)\hfill \\ \hfill p_{\theta }(h_i=-1|\mathbf{v})& =1-\sigma \left(2(c_i+{\mathbf{w}}_i^T\mathbf{v})\right)\hfill \\ \hfill p_{\theta }(v_j=1|\mathbf{h})& =\sigma \left(2(b_j+{\mathbf{h}}^T{\mathbf{w}}_{:,j})\right)\hfill \\ \hfill p_{\theta }(v_j=-1|\mathbf{h})& =1-\sigma \left(2(b_j+{\mathbf{h}}^T{\mathbf{w}}_{:,j})\right)\hfill \end{array}$$

(10)

where the sigmoid function $\sigma \left(z\right)=\frac{1}{1+e^{-z}}$ (Appendix B).

2.3. Loss Function and Training of RBMs

Given the dataset $D={[{\mathbf{v}}_1,{\mathbf{v}}_2,\cdots ,{\mathbf{v}}_M]}^T$ of M samples generated independently from the identical data distribution $p_D\left(\mathbf{v}\right)$ ($\mathbf{v}\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}.}{\sim }{p}_D\left(\mathbf{v}\right)$), the goal of RBM learning is to find a model distribution $p_{\theta }\left(\mathbf{v}\right)$ that approximates $p_D\left(\mathbf{v}\right)$. In the context of this work, the data samples $\mathbf{v}$s are Ising configurations, and the data distribution $p_D\left(\mathbf{v}\right)$ is or is related to the Ising–Boltzmann distribution $p_T\left(\mathbf{s}\right)$.

Based on maximum likelihood estimation, the optimal parameters ${\theta }^{*}=arg\underset{\theta }{min}\mathcal{L}\left(\theta \right)$ can be found by minimizing the negative log likelihood:

$$\mathcal{L}\left(\theta \right)={\langle -ln{p}_{\theta }\left(\mathbf{v}\right)\rangle }_{\mathbf{v}\sim p_D}={\langle {\mathcal{E}}_{\theta }\left(\mathbf{v}\right)\rangle }_{\mathbf{v}\sim p_D}+ln{Z}_{\theta }$$

(11)

which serves as the loss function of RBM learning. Note that the partition function $Z_{\theta }$ only depends on the model, not on the data. Since the calculation of $Z_{\theta }$ involves summation over all possible $(\mathbf{v},\mathbf{h})$ states, which is not feasible, $\mathcal{L}\left(\theta \right)$ cannot be evaluated exactly, except for very small systems [56]. Special treatments have to be devised, for example by mean-field theory [57] or by importance sampling methods [58]. An interesting feature of the RBM is that, although the actual loss function $\mathcal{L}\left(\theta \right)$ is not accessible, its gradient:

$${\nabla }_{\theta }\mathcal{L}\left(\theta \right)={\langle {\nabla }_{\theta }{\mathcal{E}}_{\theta }\left(\mathbf{v}\right)\rangle }_{\mathbf{v}\sim p_D}-{\langle {\nabla }_{\theta }{\mathcal{E}}_{\theta }\left(\mathbf{v}\right)\rangle }_{\mathbf{v}\sim p_{\theta }}$$

(12)

can be sampled, which enables a gradient descent learning algorithm. From step t to step $t+1$, the model parameters are updated with learning rate $\eta$ as

$${\theta }_{t+1}={\theta }_t-\eta {\nabla }_{\theta }\mathcal{L}\left({\theta }_t\right).$$

(13)

To evaluate the loss function, we used its approximate, the pseudo-(negative log) likelihood [59]:

$$\tilde{\mathcal{L}}\left(\theta \right)={〈-\sum _{i=1}^{n_v}ln{p}_{\theta }\left(v_i\right|v_{j\ne i})〉}_{\mathbf{v}\sim p_D}\approx \mathcal{L}\left(\theta \right)$$

(14)

where the notation:

$$\begin{array}{c}\hfill p_{\theta }\left(v_i\right|v_{j\ne i})=p_{\theta }\left(v_i\right|v_j\mathrm{for}j\ne i)=\frac{e^{-{\mathcal{E}}_{\theta }\left(\mathbf{v}\right)}}{e^{-{\mathcal{E}}_{\theta }\left(\mathbf{v}\right)}+e^{-{\mathcal{E}}_{\theta }\left([v_1,\cdots ,-v_i,\cdots ,v_{n_v}]\right)}}\end{array}$$

(15)

is the conditional probability for component $v_i$ given that all the other components $v_j$$(j\ne i)$ are fixed [37]. Practically, to avoid the time-consuming sum over all visible units $\sum _{i=1}^{n_v}$, it is suggested to randomly sample one $i_0\in \{1,2,\cdots ,n_v\}$ and estimate that:

$$\tilde{\mathcal{L}}\left(\theta \right)\approx {〈-n_{v}ln{p}_{\theta }\left(v_{i_0}\right|v_{j\ne i_0})〉}_{\mathbf{v}\sim p_D},$$

(16)

if all the visible units are on average translation-invariant [60]. To monitor the reconstruction error, we also calculated the cross-entropy $\mathrm{CE}$ between the initial configuration $\mathbf{v}$ and the conditional probability $p_{\theta }\left({\mathbf{v}}^{\prime }\right|\mathbf{h})$ for reconstruction $\mathbf{v}\stackrel{p_{\theta }\left(\mathbf{h}\right|\mathbf{v})}{⟶}\mathbf{h}\stackrel{p_{\theta }\left({\mathbf{v}}^{\prime }\right|\mathbf{h})}{⟶}{\mathbf{v}}^{\prime }$ (see Appendix C for the definition).

For both $2d$ and $3d$ Ising systems, we first trained single-temperature RBMs (T-RBM). $M=$ 50,000 Ising configurations at each T forming a dataset $D_T$ are used to train one model such that there are $n_T$T-RBMs in total. While $n_v=N$, we tried various numbers of hidden units with $n_h=400,900,1600,2500$ in $2d$ and $n_h=400,900,1600$ in $3d$. For $2d$ systems, we also trained an all-temperature RBM ($\cup T$-RBM) for which 50,000 Ising configurations per temperature are drawn to compose a dataset $D_{\cup T}$ of M = 50,000$n_T=8\times {10}^5$ samples. The number of hidden units for this $\cup T$-RBM is $n_h=400,900,1600.$ Weight matrix $\mathbf{W}$ is initialized with Glorot normal initialization [61] ($\mathbf{b}$ and $\mathbf{c}$ are initialized as zero). Parameters are optimized with the stochastic gradient descent algorithm of learning rate $\eta =1.0\times {10}^{-4}$ and batch size 128. The negative phase (model term) of the gradient ${\langle {\nabla }_{\theta }{\mathcal{E}}_{\theta }\left(\mathbf{v}\right)\rangle }_{\mathbf{v}\sim p_{\theta }}$ is calculated using CD-k Gibbs sampling with $k=5$. We stopped the training until $\tilde{\mathcal{L}}$ and CE converged, typically at 100–2000 epochs (see the Supplementary Materials). Three Nvidia GPU cards (GeForce RTX 3090 and 2070) were used to train the model, which took about two minutes per epoch for a $M=$ 50,000 dataset.

3. Results and Discussion

In this section, we investigate how the RBM uses its weight matrix $\mathbf{W}$ and hidden layer $\mathbf{h}$ to encode the Boltzmann distributed states of the Ising model and what physical information can be extracted from machine learning concepts such as the visible energy and loss function.

3.1. Filters and Inverse Filters

It can be verified that the trained weight matrix elements $W_{ij}$ of a T-RBM follow a Gaussian distribution of zero mean with the largest variance at $T\sim T_c$ (Figure 2a) [62]. The low temperature distribution here is different from the uniform distribution observed in [35], which results from the uniform initialization scheme used there. This suggests that the training of RBMs could converge to different minima when initialized differently. According to Equation (10), the biases $c_i$ and $b_j$ can be associated with the activation threshold of a hidden unit and a visible unit, respectively. For example, whether a hidden unit is activated ($h_i=+1$) or anti-activated ($h_i=-1$) depends on whether the incoming signal ${\mathbf{w}}_i^T\mathbf{v}$ from all visible units exceeds the threshold $-c_i$. The values of $c_i$ (and $b_j$) are all close to zero and are often negligible in comparison with the total incoming signal ${\mathbf{w}}_i^T\mathbf{v}$ (and ${\mathbf{h}}^T{\mathbf{w}}_{:,j}$) (see the Supplementary Materials for the results of constrained RBMs where all biases are set to zero). The distribution of $c_i$ and $b_j$ should in principle be symmetric about zero (Figure 2b,c). A non-zero mean can be caused by an unbalanced dataset with an unequal number of $m>0$ and $m<0$ Ising configurations. The corresponding filter or inverse filter sum may also be distributed with a non-zero mean in order to compensate the asymmetric bias, as will be shown next.

Since $\mathbf{v}=\mathbf{s}$ is an Ising configuration with $\pm 1$ units in our problem, ${\mathbf{w}}_i^T\mathbf{v}$ will be more positive (or negative) if the components of ${\mathbf{w}}_i^T$ better match (or anti-match) the signs of the spin variables. In this sense, we can think of ${\mathbf{w}}_i^T$ as a filter extracting certain patterns in Ising configurations. Knowing the representative spin configurations of the Ising model below, close to, and above the critical temperature $T_c$, we expect that ${\mathbf{w}}_i^T$ ($i=1,2,\cdots ,n_h$) wrapped into an $L^d$ arrangement exhibits similar features. In Figure 3a, we show sample filters of T-RBMs with $n_h=400$ trained for the $2d$ Ising model at three temperatures $T=1.0,2.25$, and $3.5$ (see the Supplementary Materials for more examples of filters). At low T, the components of ${\mathbf{w}}_i^T$ tend to be mostly positive (or negative), matching the spin up (or spin down) configurations in the ferromagnetic phase. At high T, filters ${\mathbf{w}}_i^T$ possess strip domains consisting of roughly equal numbers of well-mixed positive and negative components, like Ising configurations during spinodal decomposition. Close to $T_c$, the ${\mathbf{w}}_i^T$ patterns vary dramatically from each other, in accord with the large critical fluctuation. In particular, some even exhibit hierarchical clusters of various sizes. The element sum of the filter—filter sum $\mathrm{sum}\left({\mathbf{w}}_i^T\right)={\displaystyle \sum _{j=1}^{n_v}}{W}_{ij}$—plays a similar role as the magnetization m. The distribution of all the $n_h$ filter sums at each T changes with increasing temperature as the Ising magnetization changes, from bimodal to unimodal with the largest variance at $T_c$ (Figure 3b). This suggests that the peak of the variance $〈{\left|{\displaystyle \sum _{j=1}^{n_v}}{W}_{ij}\right|}^2〉-{〈\left|{\displaystyle \sum _{j=1}^{n_v}}{W}_{ij}\right|〉}^2$ as a function of temperature coincides with the Ising phase transition (inset of Figure 3b). More detailed results about the $2d$ and $3d$ Ising models are in the Supplementary Materials.

When a hidden layer $\mathbf{h}$ is provided, the RBM reconstructs the visible layer $\mathbf{v}$ by applying the $n_v$ inverse filters ${\mathbf{w}}_{:,j}$ ($j=1,2,\cdots ,n_v$) on $\mathbf{h}$. The distribution of the inverse filter sum$\mathrm{sum}\left({\mathbf{w}}_{:,j}\right)={\displaystyle \sum _{i=1}^{n_h}}{W}_{ij}$ is Gaussian with a mean close to zero (Figure 3c), where a large deviation from zero mean is accompanied by a non-zero average bias ${\displaystyle \sum _j}{b}_j/n_v$, as mentioned above (Figure 2b). We find that this is a result of the unbalanced dataset, which has ∼60% $m<0$ Ising configurations. Because the activation probability of a visible unit $v_j$ is determined by ${\mathbf{w}}_{:,j}$, the correlation between visible units (Ising spins) is reflected in the correlation between inverse filters. This is equivalent to the analysis of the $n_v\times n_v$ matrix ${\mathbf{W}}^T\mathbf{W}$ or its eigenvectors as in [38,42], whose entries are the inner product ${\mathbf{w}}_{:,j}^T{\mathbf{w}}_{:,j^{\prime }}$ of inverse filters. We can therefore locate the Ising phase transition by identifying the temperature with the strongest correlation among the ${\mathbf{w}}_{:,j}$s, e.g., the peak of ${\mathbf{w}}_{:,j}^T{\mathbf{w}}_{:,j^{\prime }}$ at a given distance $r_{j{j}^{\prime }}$ (inset of Figure 3c). See the Supplementary Materials for results in $2d$ and $3d$.

In contrast, the filters of the $\cup T$-RBM trained from $2d$ Ising configurations at all temperatures have background patterns like the high temperature T-RBM (in the paramagnetic phase). A clear difference is that most $\cup T$-RBM filters have one large domain of positive or negative elements (Figure 4a), similar to the receptive field in a deep neural network [29]. This domain randomly covers an area of the visual field of the $L\times L$ Ising configuration (see the Supplementary Materials for all the $n_h$ filters). The existence of such domains in the filter causes the filter sum and the corresponding bias $c_i$ to be positive or negative with a bimodal distribution (Figure 4b,c). The inverse filter sum and its corresponding bias $b_j$ still have a Gaussian distribution, although the unbalanced dataset shifts the mean of $b_j$ away from zero.

3.2. Hidden Layer

Whether a hidden unit uses $+1$ or $-1$ to encode a pattern of the visible layer $\mathbf{v}$ is randomly assigned during training. In the former case, the filter ${\mathbf{w}}_i^T$ matches the pattern (${\mathbf{w}}_i^T\mathbf{v}$ is positive); in the latter case, the filter anti-matches the pattern (${\mathbf{w}}_i^T\mathbf{v}$ is negative). For a visible layer $\mathbf{v}$ of magnetization m, the sign of ${\mathbf{w}}_i^T\mathbf{v}$ and the encoding $h_i$ is largely determined by the sign of $\mathrm{sum}\left({\mathbf{w}}_i^T\right)$ (

Table 1. When $\mathrm{sum}\left({\mathbf{w}}_i^T\right)>0$, a visible layer pattern $\mathbf{v}$ with magnetization $m>0$ (or $m<0$) is more likely to be encoded by a hidden unit $h_i=+1$ (or $h_i=-1$). When $\mathrm{sum}\left({\mathbf{w}}_i^T\right)>0$, the encoding is opposite.

	$\mathbf{sum}\left({\mathit{w}}_{\mathit{i}}^{\mathit{T}}\right)>0$	$\mathbf{sum}\left({\mathit{w}}_{\mathit{i}}^{\mathit{T}}\right)<0$
$m>0$	$h_i=+1$	$h_i=-1$
$m<0$	$h_i=-1$	$h_i=+1$

). Since the distribution of $\mathrm{sum}\left({\mathbf{w}}_i^T\right)$ is symmetric about zero, the hidden layer of a T-RBM roughly consists of an equal number of $+1$ and $-1$ units—the “magnetization” $m_h=\frac{1}{n_h}{\displaystyle \sum _{i=1}^{n_h}}{h}_i$ of the hidden layer is always close to zero and its average $\langle m_h\rangle \approx 0$. The histogram of $m_h$ for all hidden encodings of visible states is expected to be symmetric about zero (Figure 5). We found that, for the smallest $n_h$, the histogram of $m_h$ at temperatures close to $T_c$ is bimodal due to the relatively large randomness of small hidden layers. As more hidden units are added, the two peaks merge into one and the distribution of $m_h$ becomes narrower. This suggests that a larger hidden layer tends to have a smaller deviation from $m_h=0$.

The order of the $h_i=\pm 1$ sequence in each hidden encoding $\mathbf{h}$ is arbitrary, but relatively fixed once the T-RBM is trained. The permutation of hidden units together with their corresponding filters (swap the rows of the matrix $\mathbf{W}$) results in an equivalent T-RBM. Examples of hidden layers of T-RBMs with $n_h=400$ at different temperatures are shown in the inset of Figure 5, where the vector $\mathbf{h}$ is wrapped into a $20\times 20$ arrangement. Note that there are actually no spatial relationships between different hidden units, and any apparent pattern in this $2d$ illustration is an artifact of the wrapping protocol.

As a generative model, a T-RBM can be used to produce more Boltzmann-distributed Ising configurations. Starting from a random hidden state ${\mathbf{h}}^{\left(0\right)}$, this is often fulfilled by a sequence of Markov chain moves ${\mathbf{h}}^{\left(0\right)}\to {\mathbf{v}}^{\left(0\right)}\to {\mathbf{h}}^{\left(1\right)}\to {\mathbf{v}}^{\left(1\right)}\to \cdots$ until the steady state is achieved [31]. Based on the above-mentioned observations, we can design an algorithm to initialize ${\mathbf{h}}^{\left(0\right)}$ that better captures the hidden encoding of visible states (equilibrium Ising configurations), thus enabling faster convergence of the Markov chain. After choosing a low temperature $T_L$ and a high temperature $T_H$, we generate the hidden layer as follows:

• At low $T\le T_L<T_c$, if $\mathrm{sum}\left({\mathbf{w}}_i^T\right)>0$, $h_i=+1$; if $\mathrm{sum}\left({\mathbf{w}}_i^T\right)<0$, $h_i=-1$. This will be an encoding of an $m>0$ ferromagnetic configuration. To encode of an $m<0$ ferromagnetic configuration, just flip the sign of $h_i$.
• At high $T\ge T_H>T_c$, randomly assign $h_i=+1$ or $-1$ with equal probability. This will be an encoding of a paramagnetic configuration with $m\approx 0$.
• At intermediate $T_L<T<T_H$, to encode an $m>0$ Ising configuration, if $\mathrm{sum}\left({\mathbf{w}}_i^T\right)>0$, assign $h_i=+1$ with probability $p_h\in (0.5,1.0)$ and $h_i=-1$ with probability $1-p_h$; if $\mathrm{sum}\left({\mathbf{w}}_i^T\right)<0$, assign $h_i=-1$ with probability $p_h\in (0.5,1.0)$ and $h_i=+1$ with probability $1-p_h$. $p_h$ is a predetermined parameter, and the above two algorithms are just the special cases with $p_h=1.0$ ($T\le T_L$) and $p_h=0.5$ ($T\ge T_H$), respectively. In practice, one may approximately use $p_h=(\langle |m|\rangle +1)/2$ or use linear interpolation within $T_L<T<T_H$, $p_h=0.5+0.5(T-T_L)/(T_H-T_L)$.

Below, we compare the (one-step) reconstructed thermal quantities using two different initial hidden encodings with results from a conventional multi-step Markov chain (Figure 6). The hidden encoding methods proposed here are quite reliable at low and high T, but less accurate at T close to $T_c$.

3.3. Visible Energy

When a T-RBM for temperature T is trained, we expect that $p_{\theta }\left(\mathbf{v}\right)\approx p_D\left(\mathbf{v}\right)\approx p_T\left(\mathbf{s}\right)$—the Boltzmann distribution at that T. Although formally related to the physical energy in the Boltzmann factor (with temperature absorbed), the visible energy ${\mathcal{E}}_{\theta }\left(\mathbf{v}\right)$ of an RBM should be really considered as the negative log (relative) probability of a visible state $\mathbf{v}$. For single-temperature T-RBMs, the mean visible energy $\langle {\mathcal{E}}_{\theta }\left(\mathbf{v}\right)\rangle$ increases monotonically with temperature (except for the largest $n_h$, which might be due to overfitting) (Figure 7a,b). The value of $\langle {\mathcal{E}}_{\theta }\left(\mathbf{v}\right)\rangle$ and its trend, however, cannot be used to identify the physical phase transition. In fact, ${\mathcal{E}}_{\theta }\left(\mathbf{v}\right)$ can differ from the reduced Hamiltonian $\frac{\mathcal{H}\left(\mathbf{s}\right)}{k_{B}T}$ by an arbitrary (temperature-dependent) constant while still maintaining the Boltzmann distribution $p_{\theta }\left(\mathbf{v}\right)\approx p_T\left(\mathbf{s}\right)$ (if the partition function $Z_{\theta }$ is calibrated accordingly).

The trend of $\langle {\mathcal{E}}_{\theta }\left(\mathbf{v}\right)\rangle$ for T-RBMs can be understood by considering following approximate forms. First, due to the symmetry of $+1$ and $-1$, the biases $b_j$ and $c_i$ are all close to zero. A constrained T-RBM with zero bias has a visible energy:

$$\begin{array}{c}\hfill {\mathcal{E}}_{\mathbf{W}}\left(\mathbf{v}\right)=-{\displaystyle \sum _{i=1}^{n_h}}ln\left(e^{-{\mathbf{w}}_i^T\mathbf{v}}+e^{{\mathbf{w}}_i^T\mathbf{v}}\right)\end{array}$$

(17)

that approximates the visible energy of the full T-RBM, i.e., ${\mathcal{E}}_{\theta }\left(\mathbf{v}\right)\approx {\mathcal{E}}_{\mathbf{W}}\left(\mathbf{v}\right)$. Next, unless ${\mathbf{w}}_i^T\mathbf{v}$ is close to zero, one of the two exponential terms in Equation (17) always dominates such that ${\mathcal{E}}_{\mathbf{W}}\left(\mathbf{v}\right)\approx {\tilde{\mathcal{E}}}_{\mathbf{W}}\left(\mathbf{v}\right)$, where

$$\begin{array}{c}\hfill {\tilde{\mathcal{E}}}_{\mathbf{W}}\left(\mathbf{v}\right)=-\sum _{i=1}^{n_h}\left|{\mathbf{w}}_i^T\mathbf{v}\right|=-{\displaystyle \sum _{i=1}^{n_h}}\left|\sum _{j=1}^{n_v}{W}_{ij}{v}_j\right|.\end{array}$$

(18)

Equation (18) can further be approximated by setting $\mathbf{v}=\mathbf{1}$ with all $v_j=+1$, i.e., ${\tilde{\mathcal{E}}}_{\mathbf{W}}\left(\mathbf{v}\right)\approx {\tilde{\mathcal{E}}}_{\mathbf{W}}\left(\mathbf{1}\right)$ with

$$\begin{array}{c}\hfill {\tilde{\mathcal{E}}}_{\mathbf{W}}\left(\mathbf{1}\right)=-{\displaystyle \sum _{i=1}^{n_h}}\left|\mathrm{sum}\left({\mathbf{w}}_i^T\right)\right|=-{\displaystyle \sum _{i=1}^{n_h}}\left|{\displaystyle \sum _{j=1}^{n_v}}{W}_{ij}\right|.\end{array}$$

(19)

In summary, ${\mathcal{E}}_{\mathbf{W}}\left(\mathbf{v}\right)$, ${\tilde{\mathcal{E}}}_{\mathbf{W}}\left(\mathbf{v}\right)$, and ${\tilde{\mathcal{E}}}_{\mathbf{W}}\left(\mathbf{1}\right)$ are all good approximations to the original ${\mathcal{E}}_{\theta }\left(\mathbf{v}\right)$ (Figure 7a). The increase of mean $\langle {\mathcal{E}}_{\theta }\left(\mathbf{v}\right)\rangle$ with temperature coincides with the increase of $-\left|\mathrm{sum}\left({\mathbf{w}}_i^T\right)\right|$ with temperature, which is evident from Figure 3b. At fixed temperature, the decrease of $\langle {\mathcal{E}}_{\theta }\left(\mathbf{v}\right)\rangle$ with $n_h$ is a consequence of the sum ${\displaystyle \sum _{i=1}^{n_h}}$ in the definition of visible energy. The variance $\langle {\mathcal{E}}_{\theta }^2\rangle -{\langle {\mathcal{E}}_{\theta }\rangle }^2$ is a useful quantity for phase transition detection, because it reflects the fluctuation of the probability $p_{\theta }\left(\mathbf{v}\right)$. In both low T ferromagnetic and high T paramagnetic regimes, $p_{\theta }\left(\mathbf{v}\right)$ is relatively homogeneous among different states. When T is close to $T_c$, the variance of $p_{\theta }\left(\mathbf{v}\right)$ and ${\mathcal{E}}_{\theta }\left(\mathbf{v}\right)$ is expected to peak (Figure 7d,e). The abnormal rounded (and even shifted) peaks at large $n_h$ could be a sign of overfitting.

For the all-temperature $\cup T$-RBM, the Ising phase transition can be revealed by either the sharp increase of the mean $\langle {\mathcal{E}}_{\theta }\left(\mathbf{v}\right)\rangle$ or the peak of the variance $\langle {\mathcal{E}}_{\theta }^2\rangle -{\langle {\mathcal{E}}_{\theta }\rangle }^2$ (Figure 7c,f). However, this apparent detection can be a trivial consequence of the special composition of the dataset $D_{\cup T}$, which contains Ising configurations at different temperatures in equal proportion. Only configurations at a specific T are fed into the model to calculate the average quantity at that T. Technically, a visible state $\mathbf{v}$ in $D_{\cup T}$ is not subject to the Boltzmann distribution at any specific temperature. Instead, the true ensemble of $D_{\cup T}$ is a collection of $n_T$ different Boltzmann-distributed subsets. Many replicas of the same or similar ferromagnetic states are in $D_{\cup T}$, giving rise to a large multiplicity, high probability, and low visible energy for such states. In comparison, high temperature paramagnetic states are all different from each other and, therefore, have low $p_{\theta }\left(\mathbf{v}\right)$ (high ${\mathcal{E}}_{\theta }\left(\mathbf{v}\right)$) for each one of them. Knowing this caveat, one should be cautious when monitoring the visible energy of a $\cup T$-RBM to detect phase transition, because changing the proportion of Ising configurations at different temperatures in $D_{\cup T}$ can modify the relative probability of each state.

3.4. Pseudo-Likelihood and Entropy Estimation

The likelihood $\mathcal{L}\left(\theta \right)$ defined in Equation (11) is conceptually equivalent to the physical entropy S defined by the Gibbs entropy formula, apart from the Boltzmann constant $k_B$ difference (Appendix A). However, just as entropy S cannot be directly sampled, the exact value of $\mathcal{L}\left(\theta \right)$ is not accessible. In order to estimate S, we calculated the pseudo-likelihood $\tilde{\mathcal{L}}\left(\theta \right)$ instead, which is based on the mean-field-like approximation $p_{\theta }\left(\mathbf{v}\right)\approx {\prod }_{i=1}^{n_v}{p}_{\theta }\left(v_i\right|v_{j\ne i})$. Similar ideas to estimate the free energy or entropy were put forward with the aid of variational autoregressive networks [63] or neural importance sampling [64]. The true and estimated entropy of the $2d$ and $3d$ Ising models using T-RBMs with different $n_h$ are shown in Figure 8a,b. As a comparison, we also considered a “pseudo-entropy” with a similar approximation:

$$\tilde{S}=-k_B{〈{\displaystyle \sum _{i=1}^N}{p}_T\left(s_i\right|s_{j\ne i})〉}_{\mathbf{s}\sim p_T}\approx S$$

(20)

where the conditional probability:

$$p_T\left(s_i\right|s_{j\ne i})=\frac{e^{-\frac{\mathcal{H}\left(\mathbf{s}\right)}{k_{B}T}}}{e^{-\frac{\mathcal{H}\left(\mathbf{s}\right)}{k_{B}T}}+e^{-\frac{\mathcal{H}\left([s_1,\cdots ,-s_i,\cdots ,s_N]\right)}{k_{B}T}}}$$

(21)

and the ensemble average ${〈\cdots 〉}_{\mathbf{s}\sim p_T}$ is taken over states obtained from Monte Carlo sampling. In both $2d$ and $3d$, $\tilde{S}$ is lower than the true S, especially at high T, because a mean-field treatment tends to underestimate fluctuations.

While increasing model complexity by adding hidden units is usually believed to reduce the reconstruction error, e.g., of energy and heat capacity [35,36] (see also the Supplementary Materials), a recent study suggested that a trade-off could exist between the accuracy of different statistical quantities [65]. Here, we found that the pseudo-likelihood of T-RBMs with the fewest hidden units in our trials ($n_h=400$) appears to provide the best prediction for entropy. Increasing $n_h$ leads to larger deviations from the true S at higher T. The decreasing of $\tilde{\mathcal{L}}$ with $n_h$ at fixed temperature agrees with the trend of the visible energy. A lower ${\mathcal{E}}_{\theta }\left(\mathbf{v}\right)$ corresponds to a higher $p_{\theta }\left(\mathbf{v}\right)$ and, thus, a lower $\tilde{\mathcal{L}}$ according to its definition. The surprisingly good performance of $\tilde{\mathcal{L}}$ in approximating S could be due to the fact that visible units $v_i$ in RBMs are only indirectly correlated through hidden units, which collectively serve as an effective mean-field on each visible unit. We also calculated $\tilde{\mathcal{L}}\left(\theta \right)$ with the all-temperature $\cup T$-RBM in $2d$ (Figure 8c). Compared with single-temperature T-RBMs of the same $n_h$ (Figure 8a), the $\cup T$-RBM predicts higher $\tilde{\mathcal{L}}\left(\theta \right)$ with considerable deviations even at low T. The trend of $\tilde{\mathcal{L}}\left(\theta \right)$ also agrees with that of $\langle {\mathcal{E}}_{\theta }\left(\mathbf{v}\right)\rangle$ (Figure 7c).

The knowledge of the entropy allows us to estimate the phase transition point according to the thermodynamic relation $C_V=T\frac{dS}{dT}$. We constructed this estimated $C_V$ as a function of temperature using $\tilde{\mathcal{L}}\left(\theta \right)$ and its numerical fitting, whose peaks are expected to be located at $T_c$ (Figure 9). The predicted $T_c$s are compared with the results from entropy and pseudo-entropy, as well as the Monte Carlo simulation results for our finite systems and the known exact values for infinite systems in

Table 2. $T_c$ estimated according to the peak of $T\frac{d\tilde{\mathcal{L}}}{dT}$ obtained from single-temperature T-RBMs and the all-temperature $\cup T$-RBM with different numbers ($n_h$) of hidden units. Predictions from numerical derivatives $T\frac{dS}{dT}$ and $T\frac{d\tilde{S}}{dT}$ are also shown for comparison. Results extracted from the peak of $c_V$ obtained by Monte Carlo simulations of finite systems are listed under “MC”. “Exact” refers to analytical or numerical results for infinite systems.

Model	${\mathit{n}}_{\mathit{h}}=400$	900	1600	2500	S	$\tilde{\mathit{S}}$	MC	Exact
$2d$T-RBM	2.240	2.291	2.316	2.367	2.267	2.367	2.28	2.269
$2d$$\cup T$-RBM	2.189	2.163	2.214	-	2.267	2.367	2.28	2.269
$3d$T-RBM	4.444	4.434	4.444	-	4.390	4.383	4.44	4.511

. It can be seen that single-temperature T-RBMs capture the transition point fairly well within an error of about 1–$3\%$.

4. Conclusions

In this work, we trained RBMs using equilibrium Ising configurations in $2d$ and $3d$ collected from Monte Carlo simulations at various temperatures. For single-temperature T-RBMs, the filters (row vectors) and the inverse filters (column vectors) of the weight matrix exhibit different characteristic patterns and correlations, respectively, below, around, and above the phase transition. These metrics, such as filter sum fluctuation and inverse filter correlation, can be used to locate the phase transition point. The hidden layer $\mathbf{h}$ on average contains an equal number of $+1$ and $-1$ units, whose variance decreases as more hidden units are added. The sign of a particular hidden unit $h_i$ is determined by the signs of the filter sum $\mathrm{sum}\left({\mathbf{w}}_i^T\right)$ and the magnetization m of the visible pattern. However, there is no spatial pattern in the sequence of positive and negative units in a hidden encoding.

The visible energy reflects the relative probability of visible states in the Boltzmann distribution. Although the mean of visible energy is not directly related to the (physical) internal energy and does not reveal a clear transition, its fluctuation, which peaks at the critical point, can be used to identify the phase transition. The value and trend of the visible energy can be understood from its several approximation forms, in particular the sum of the absolute value of filter sums. The pseudo-likelihood of RBMs is conceptually related to and can be used to estimate the physical entropy. Numerical differentiation of the pseudo-likelihood provides another estimator of the transition temperature because it provides an estimate of the heat capacity. All these predictions about the critical temperature were made by unsupervised RBM learning, for which human labeling of the phase types is not needed.

As a comparison, we also trained an all-temperature $\cup T$-RBM, whose dataset is a mixture of Boltzmann-distributed states over a range of temperatures. Each filter of this $\cup T$-RBM is featured by one large domain in its receptive field. Although the visible energy and pseudo-likelihood of the $\cup T$-RBM show a certain signature of the phase transition, one should be cautious, as this detection could be an artifact of the composition of the dataset. Changing the proportions of Ising configurations at different temperatures could bias the probability and the transition learned by the $\cup T$-RBM.

By extracting the underlying (Boltzmann) distribution of the input data, RBMs capture the rapid (phase) transition of such a distribution as the tuning parameter (temperature) is changed, without knowledge of the physical Hamiltonian. Information about the distribution is completely embedded in the configurations and their frequencies in the dataset. It would be interesting to see if such a general scheme of RBM learning can be extended to study other physical models of phase transition.