Bayesian-Based Hybrid Method for Rapid Optimization of NV Center Sensors

Abstract

NV centers are among the most promising platforms in the field of quantum sensing. Magnetometry based on NV centers, especially, has achieved concrete development in areas of biomedicine and medical diagnostics. Improving the sensitivity of NV center sensors under wide inhomogeneous broadening and fieldamplitude drift is a crucial issue of continuous concern that relies on the coherent control of NV centers with high average fidelity. Quantum optimal control (QOC) methods provide access to this target; nevertheless, the high time consumption of current methods due to the large number of needful sample points as well as the complexity of the parameter space has hindered their usability. In this paper, we propose the Bayesian estimation phase-modulated (B-PM) method to tackle this problem. In the case of the state transforming of an NV center ensemble, the B-PM method reduced the time consumption by more than $90\%$ compared with the conventional standard Fourier basis (SFB) method while increasing the average fidelity from $0.894$ to $0.905$. In the AC magnetometry scenario, the optimized control pulse obtained with the B-PM method achieved an eight-fold extension of coherence time $T_2$ compared with the rectangular $\pi$ pulse. Similar application can be made in other sensing situations. As a general algorithm, the B-PM method can be further extended to the open- and closed-loop optimization of complex systems based on a variety of quantum platforms.

1. Introduction

The nitrogen-vacancy (NV) center in diamond shows bright prospects in the quantum sensing of magnetic fields [1,2,3], electric fields [4], temperature [5] and strain [6]. In these areas, research on NV center-based ultrasensitive magnetometry has achieved fast development [7,8] and is on the road to practical and commercial applications in biomedicine and diagnostics [9,10,11,12]. Inhomogeneous broadening due to ambient nuclear spins and external bias fields is one of the main obstacles to further improving the sensitivity of sensors. To alleviate the problem, dynamical decoupling (DD) is widely applied in sensing strategies based on NV centers [13,14,15], and various optimal methods are used to increase the performance of DD sequences [16,17,18], since inhomogeneous broadening also damages the fidelity of the control pulses with finite pulse length that construct DD series. Adiabatic strategies can significantly prolong decoherence time $T_2$ and sensitivity compared with conventional flat $\pi$ pulses [17], while smoothly shaped pulse designs based on numerical optimization [18] can adapt to a wider pulse length range where the adiabatic condition is not satisfied. One major drawback of numerical optimal designs is their time efficiency. On one hand, multiple sample points need to be measured to obtain an accurate value of the objective function denoting the average fidelity over frequency and field amplitude broadening, so the processing time for calling the objective function once is prolonged. On the other hand, the total objective function calling times increase along with the number of parameters, while in most cases, a larger number of parameters (over 10) are needed to guarantee fair fidelity. A rapid but well-performed optimization method is anticipated to improve the usability of such control strategies.

In this work, we propose the Bayesian estimation phase-modulated (B-PM) method to overcome the time consumption problem. Unlike the developed Bayesian methods [19,20], the B-PM method grafts the Bayesian estimation model onto the direct search method, thus circumventing the complex process to calculate the acquisition functions [21]. Further taking advantage of the phase-modulated method, the B-PM method makes itself an efficient hybrid optimization method for robust quantum control against noise. The objective function of the optimization process under consideration is the average value of multiple functions with different detuning and amplitude values of the control field. The values of these multiple functions can specifically refer to the fidelity between the final and target states, gate operators and so on. Using the Bayesian-based estimation model, we make an accurate prediction of the average fidelity based on a small number of sample points, so the computation time for calling the objective function once is reduced. In addition, adopting a phase-modulated basisallows the control field to comprise multiple frequency components with fewer parameters, which leads to a significant decrease in the necessary total number of objective function calling times to find the local optimal results. We further verify that the computation time of the estimation process is negligible compared with the sample measuring process. Overall, the B-PM method decreases the total time consumed during the entire optimization process. We firstly applied the B-PM method to the state flipping of an NV center ensemble. Compared with the conventional standard Fourier basis(SFB) method without estimation, the B-PM method increased the average fidelity from $0.894$ to $0.905$ with only $9.3\%$ of total time consumption. When applied to the sensing strategy of AC magnetic signals, the B-PM-shaped pulse prolonged decoherence time $T_2$ by 8 times compared with the conventional rectangular $\pi$ pulse. The B-PM method could be extended to DC sensing strategies [22], and open- and closed-loop optimization processes for open systems [23,24] and many-body systems [25,26,27].

2. Methods

2.1. Optimal Control Model of NV Center Ensemble

The nitrogen-vacancy center (NV center) in diamond [28] has a triplet ground state with electron spin $S=1$ and zero-field splitting $D=2\pi \times 2.88$ GHz. The energy gap between the ground and excited states is $1.945$ ev (637 nm), and due to uneven radiation probability from excited states $|m_s=\pm 1\rangle$ and $|m_s=0\rangle$ to the radiationless metastable intermediate state, the NV center can be optically initialized and read out. The structure of the NV center and energy level scheme are shown in Figure 1a,b. The Hamiltonian of the triplet ground state of the NV center can be expressed as

$$H=D\left(S_{\mathrm{Z}}^2-\frac{2}{3}\right)+\gamma \mathbf{B}·\mathbf{S}+H_{\mathrm{ele}}+H_{\mathrm{hf}},$$

(1)

where D is the zero-field splitting, $\mathbf{B}$ is the magnetic vector, ${\gamma }_{\mathrm{NV}}=2\pi \times 2.8$ MHz/G is the gyromagnetic ratio of the NV center, $H_{\mathrm{ele}}$ is the electric interaction term with coupling coefficient ∼ Hz/(V/m) and $H_{\mathrm{hf}}$ is the hyperfine coupling term expressing the interactions between the NV center and ambient nuclear spins. Under the magnetic field along the z-direction, $\mathbf{B}=B_z$, and by neglecting the electric coupling term, the Hamiltonian of the two-level subspace formed by $m_s=0$ ground state $|{\psi }_0\rangle$ and $m_s=-1$ (or $m_s=1$) ground state $|{\psi }_1\rangle$ can be written as

$$H=\frac{D+\gamma B_z}{2}{\sigma }_z.$$

(2)

One simple but requisite control target is to flip all spins in the NV ensemble from one state to another with high average fidelity. Let us consider an ensemble of the two-level system described by Equation (2), which is controlled by a time-dependent field $g\left(t\right)$; the Hamiltonian of each spin in the ensemble can be represented as

$$H\left(t\right)=\frac{{\omega }_0+\delta }{2}{\sigma }_z+\kappa g\left(t\right){\sigma }_x,$$

(3)

where ${\omega }_0=D+\gamma B_z$ is the unperturbed energy gap between $|{\psi }_0\rangle$ and $|{\psi }_1\rangle$, $\delta$ is the normally distributed detuning term as a result of inhomogeneous local ambient among the ensemble and $\kappa$ is the amplitude drift factor varying with different control trials. The probability density of $\delta$ and $\kappa$ can be noted as

$$p\left(\delta \right)=\frac{1}{\sqrt{2\pi }{\sigma }_{\delta }}{e}^{-\frac{{\delta }^2}{2{\sigma }_{\delta }^2}}$$

(4)

and

$$p\left(\kappa \right)=\frac{1}{\sqrt{2\pi }{\sigma }_{\kappa }}{e}^{-\frac{{\kappa }^2}{2{\sigma }_{\kappa }^2}}$$

(5)

respectively, where we take the full width at half maximum (FWHM) of $p\left(\delta \right)$ as $2\pi \times 26.5$ MHz, corresponding to a dephasing time of $T_2^{*}\approx 20$ ns [17], and the FWHM of $p\left(\kappa \right)$ as $0.5$. Figure 1c–e depict fidelity

$$f(\delta ,\kappa )={\left|〈{\psi }_g\mid \psi (\delta ,\kappa ,T)〉\right|}^2$$

(6)

as a function of detuning $\delta$ and amplitude drift $\kappa$ of rectangular control field $g\left(t\right)=\pi /T=2\pi \times 10$ MHz, where T is the evolution time, $\psi (\delta ,\kappa ,T)$ is the final state, and the initial state and the target state are taken as $\psi (\delta ,\kappa ,0)={\psi }_0$ and ${\psi }_g={\psi }_1$, respectively. Based on Figure 1c, we took the sample ranges as $\delta \in 2\pi \times [-10,10]$ MHz and $\kappa \in [0.5,1.5]$, as is shown in Figure 1f, and the optimization target was to improve the average fidelity in these ranges.

Taking into account inhomogeneous detuning as well as amplitude drift, the explicit form of the average fidelity can be represented as

$$\mathcal{F}=\int \int d\delta d\kappa p(\delta )p(\kappa )f(\delta ,\kappa ).$$

(7)

For fixed $\delta ,\kappa$ and T, $\psi (\delta ,\kappa ,T)$ is the function of control field $g\left(t\right)$; therefore, it is the function of parameters $\mathbf{\lambda }$ that construct $g\left(t\right)$. For example, $g\left(t\right)$ can be constructed based on the standard Fourier basis as

$$g_{\mathrm{SFB}}\left(t\right)=\sum _{j=1}^{N_{\mathrm{D}}}{a}_{j}cos\left({\omega }_{j}t+{\varphi }_j\right)cos\left({\omega }_{0}t\right)$$

(8)

where $\mathbf{\lambda }=\{\mathbf{a},\mathbf{\omega },\mathbf{\varphi }\}$, or on the phase-modulated Fourier basis [29] as

$$g_{\mathrm{PM}}\left(t\right)=\sum _{j=1}^{N_{\mathrm{D}}}{a}_{j}cos\left[{\omega }_{0}t+\frac{b_j}{{\nu }_j}sin\left({\nu }_{j}t\right)\right]$$

(9)

where $\mathbf{\lambda }=\{\mathbf{a},\mathbf{b},\mathbf{\nu }\}$. The optimization target, therefore, is to find parameters $\mathbf{\lambda }$ that give the highest value of $\mathcal{F}$, while the maximum amplitude of $g\left(t\right)$ is limited to $g_{\max }$. We concisely describe this optimization model as

$$\begin{array}{cc}min\hfill & 1-\mathcal{F}\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & g\left(\mathbf{\lambda }\right)-g_{\max }\le 0.\hfill \end{array}$$

(10)

Plenty of optimization methods can be used to solve this problem, with $\mathbf{\lambda }$ being the optimization parameters and $\mathcal{F}$ being the objective function. Ideally, the value of $\mathcal{F}$ should be calculated by averaging the fidelities of a large number of samples taken from the whole ensemble with several repeated control processes. In practice, this time-consuming process is usually replaced by taking the weighted average of several equidistant samples in a certain range, which can be represented as

$${\mathcal{F}}_{\mathrm{obj}}=\mathcal{N}\sum _{k=1}^M\sum _{j=1}^{N}p\left({\delta }_k\right)p\left({\kappa }_j\right)f\left({\delta }_k,{\kappa }_j\right)$$

(11)

where the distribution of $\delta$ and $\kappa$ is assumed to be known; M and N are the numbers of different values of $\delta$ and $\kappa$, respectively; and $\mathcal{N}={\left[{\sum }_{k=1}^M{\sum }_{j=1}^{N}p\left({\delta }_k\right)p\left({\kappa }_j\right)\right]}^{-1}$ is the normalization constant. Even so, this process still consumes a substantial amount of time. Below, we will show how the Bayesian-based estimation method can be used in the optimization process and combined with the PM method to efficiently reduce the total execution time.

2.2. The Estimation Model

One essential task of the optimal process is to build a computationally cheap estimation model, also known as metamodel [30,31,32], of the expensive function. Bayesian statics and analysis methods provide powerful tools to complete this task. From the viewpoint of Bayesian statistics, any event can be endowed with a probabilistic property, whether or not it is a stochastic event. So, undetermined parameter $\theta$ in a static model can be taken as a realization of stochastic process $\mathbf{\theta }$ subject to a prior distribution $p\left(\theta \right)$, which represents the available knowledge or simple beliefs about $\mathbf{\theta }$ before any observation of the samples. This prior knowledge needs to be updated with the information in observed sample data $s$ to give the final description of $\mathbf{\theta }$, represented by posterior distribution $p\left(\theta \right|s)$. This relationship is explicitly formulated by the Bayesian theorem as follows:

$$p\left(\theta \right|s)=\frac{\mathcal{L}\left(\theta \right|s\left)p\right(\theta )}{p\left(s\right)},$$

(12)

where likelihood function $\mathcal{L}\left(\theta \right|s)=p(s\left|\theta \right)$ is defined as the conditional probability distribution of the given parameters of the data [33].

We limit the considered target true functions to be deterministic, which means that all trials with the same input parameters give identical function values. Following the principle of Bayesian statistics, the deterministic response function $y\left(x\right)$ of a k-dimension variable $x={\{x_1,x_2,\dots ,x_k\}}^{\prime }$ can be treated as a realization of stochastic process $Y\left(x\right)$ [34].

$$Y\left(x\right)=\sum _h{\beta }_h{f}_h\left(x\right)+ϵ\left(x\right),$$

(13)

where $f_h\left(x\right)$ is a function of $x$; ${\beta }_h$ are unknown coefficients to be estimated; and $ϵ\left(x\right)$ is the error term, which is a random process with zero mean and covariance

$$\mathrm{Cov}\left[x_i,x_j\right]={\sigma }^{2}Corr\left[x_i,x_j\right],$$

(14)

where $x_i$ and $x_j$ are two sets of variables, $\sigma$ is the process variance and $Corr\left[x_i,x_j\right]$ represents the correlation. Equation (13) can be regarded as the Bayesian prior of the true response function, in which the right-hand side resembles the form of the linear regression model, except that the errors of different $x$ are correlated rather than independent from each other. We consider the common and simple case where the stochastic process is stationary, and the correlation takes the specific form of [34]

$$Corr\left[x_i,x_j\right]=exp\left[-d\left(x_i,x_j\right)\right],$$

(15)

where

$$d\left(x_i,x_j\right)=\sum _{h=1}^k{\alpha }_h{\left|x_{i,h}-x_{j,h}\right|}^{p_h}\left({\alpha }_h\ge 0,p_h\in [1,2]\right),$$

(16)

where ${\alpha }_h$ and $p_h$ are parameters that need to be determined. This specific form corresponds to a product of the Ornstein–Uhlenbeck process at $p_h=1$, which is widely used in physical science models. Furthermore, with the correlation form of Equations (15) and (16), regression parts ${\sum }_h{\beta }_h{f}_h\left(x\right)$ in the stochastic process model can be simply replaced with a constant, $\mu$, without undermining the predictive performance [35]. Hereafter, we use the following simplified stochastic process model:

$$Y\left(x\right)=\mu +ϵ\left(x\right).$$

(17)

A typical method for analyzing such stationary Gaussian process model is the Kriging method [36], which is flexible and robust for global estimation [37] and is widely applied in fields of spatial statistics [36,38], the design of computer experiments [34] and Bayesian optimization [35]. Besides the ordinary Kriging method, various analysis methods based on the Kriging method as well as the Bayesian approach are also well studied for Gaussian processes with non-stationary correlation and other kinds of stochastic processes [37,39,40,41,42]. Here, we adopt the best linear unbiased predictor (BLUP) of Equation (17) obtained with the Kriging method to form the estimation model. Supposing that the response function has been evaluated at n samples $s=\{s_1;s_2;\dots ;\dots s_n\}$ and that the corresponding function values are $y\left(s\right)={\{y\left(s_1\right),y\left(s_2\right),\dots y\left(s_n\right)\}}^{\prime }$, the BLUP of $y\left(x\right)$ can be represented as [35]

$$\widehat{y}\left(x\right)=\widehat{\mu }+r^{\prime }\left(x\right){\mathbf{R}}^{-1}(y\left(s\right)-\mathbb{1}\widehat{\mu }),$$

(18)

where $\mathbf{R}$ is an $n\times n$ correlation matrix with entries $R_{i,j}=\mathrm{Corr}\left[s_i,s_j\right]$; $r\left(x\right)=[R\left(s_1,x\right),\dots ,$$R\left(s_n,x\right){]}^{\prime }$ is the $n\times 1$ vector of correlation between the errors of the sample and untried input $x$, with $R(s_1,x)=\mathrm{Corr}\left[s_1,x\right]$; $\widehat{\mu }={\left({\mathbb{1}}^{\prime }{\mathbf{R}}^{-1}\mathbb{1}\right)}^{-1}{\mathbb{1}}^{\prime }{\mathbf{R}}^{-1}y\left(s\right)$ is the generalized least-squares estimation of $\mu$, and $\mathbb{1}$ is the $n\times 1$ vector with all entries equal to 1.

By properly selecting the parameters in Equation (16), we can expect a predict function that describes true function $y\left(x\right)$ well by only using the limited number of sample points $s$. One useful method to determine these parameters is maximum likelihood estimation (MLE). According to the Gaussian process we adopt here, the likelihood function of the samples is [35]

$$\frac{1}{{\left(2\pi \right)}^{n/2}{\left({\sigma }^2\right)}^{n/2}{\left|\mathbf{R}\right|}^{\frac{1}{2}}}exp\left[-\frac{{(y\left(s\right)-\mathbb{1}\mu )}^{\prime }{\mathbf{R}}^{-1}(y\left(s\right)-\mathbb{1}\mu )}{2{\sigma }^2}\right],$$

(19)

where $\left|\mathbf{R}\right|$ represents the determinant of $\mathbf{R}$. The analytical solutions of $\mu$ and $\sigma$ maximizing Equation (19) are

$$\widehat{\mu }={\left({\mathbb{1}}^{\prime }{\mathbf{R}}^{-1}\mathbb{1}\right)}^{-1}{\mathbb{1}}^{\prime }{\mathbf{R}}^{-1}y\left(s\right),$$

(20)

which is exactly the generalized least-squares estimation of $\mu$, and

$${\widehat{\sigma }}^2=\frac{{(y\left(s\right)-\mathbb{1}\widehat{\mu })}^{\prime }{\mathbf{R}}^{-1}(y\left(s\right)-\mathbb{1}\widehat{\mu })}{n}.$$

(21)

Therefore, we only need to find the values of ${\alpha }_h$ and $p_h$ that give the maximum value of Equation (19), which can be completed using any optimization method at hand.

Corresponding to the specific model of the two-level NV electron spin system, once control field $g\left(t\right)$ and control time T are fixed, target function $f(\delta ,\kappa )$ is the deterministic response function of two-dimension variable $x={\{\delta ,\kappa \}}^{\prime }$ and can be approximately estimated using predict function $\widehat{f}\left(x\right)$, which takes the same form of $\widehat{y}\left(x\right)$ in Equation (18). Specifically, we write the Hamiltonian in the interaction picture with respect to $\frac{{\omega }_0}{2}{\sigma }_z$ as

$$H_{\delta ,\kappa }\left(t\right)=\frac{\delta }{2}{\sigma }_z+\kappa H_c\left(t\right),$$

(22)

where $H_c\left(t\right)$ is the time-depended Hamiltonian. By neglecting counter-rotating terms in rotating-wave approximation, $H_c\left(t\right)$ can be expressed as

$$H_c\left(t\right)={\mathrm{\Omega }}_x\left(t\right){\sigma }_x+{\mathrm{\Omega }}_y\left(t\right){\sigma }_y,$$

(23)

where ${\mathrm{\Omega }}_x\left(t\right)$ and ${\mathrm{\Omega }}_y\left(t\right)$ take the forms of

$${\mathrm{\Omega }}_x^{\mathrm{SFB}}\left(t\right)=\sum _{j=1}^{N_{\mathrm{D}}}\frac{a_j}{2}cos\left({\omega }_{j}t+{\varphi }_j\right)cos\left({\phi }_j\right),$$

(24)

$${\mathrm{\Omega }}_y^{\mathrm{SFB}}\left(t\right)=\sum _{j=1}^{N_{\mathrm{D}}}\frac{a_j}{2}cos\left({\omega }_{j}t+{\varphi }_j\right)sin\left({\phi }_j\right)$$

(25)

with the SFB method and the forms of

$${\mathrm{\Omega }}_x^{\mathrm{PM}}\left(t\right)=\sum _{j=1}^{N_{\mathrm{D}}}\frac{a_j}{2}cos\left[\frac{b_j}{v_j}sin\left(v_{j}t\right)\right],$$

(26)

$${\mathrm{\Omega }}_y^{\mathrm{PM}}\left(t\right)=\sum _{j=1}^{N_{\mathrm{D}}}\frac{a_j}{2}sin\left[\frac{b_j}{v_j}sin\left(v_{j}t\right)\right]$$

(27)

with the PM method. The target response function can be expressed as

$$f(\delta ,\kappa )={\left|〈{\psi }_g\mid U_{\delta ,\kappa }\psi \left(0\right)〉\right|}^2,$$

(28)

where

$$U_{\delta ,\kappa }=\mathcal{T}exp\left[-i{\int }_0^{T}H\left(\delta ,\kappa ,t\right)dt\right]$$

(29)

is the evolution operator and $\mathcal{T}$ is the time-order operator.

So far, we established the Bayesian-based estimation model of the final-state fidelity under certain frequency detuning $\delta$ and amplitude bias ratio $\kappa$. With this usable tool, we could further construct a high-efficiency optimization method to explore the optimal control field of an ensemble system.

2.3. The Hybrid Optimization Method

Now, we introduce the hybrid B-PM optimization method for solving the problem described by Equation (10). As represented in Figure 2, the whole optimization process can be divided into three parts: initial construction of the available predict function; then, search for optimal parameter $\mathbf{\lambda }$ using the predict function as the objective function; lastly, calculation of the true value of the objective function in Equation (11) with $M\times N=2500$ sample points. The first segment is the essential part of this method, and the following strategies are adopted to ensure convergence and improve the optimization results.

• Sample position: In the sample selection step, instead of completely randomly picking the sample points, we add random bias to evenly distributed coordinate values to obtain the randomly yet uniformly distributed sample positions. This choice improves the estimation performance, especially in cases with small sample numbers.
• Model parameter: Model parameters $\alpha$ and p are selected based on a random initial field $g^{\mathrm{ini}}\left(t\right)$ and are fixed during one optimization process. This tactic ensures the monodromy of the objective function and the convergence of the subsequent search process, reduces the computation time spent building the predict function and does not damage the estimation accuracy.
• Cross validation: After constructing a predict function, cross validation is applied to eliminate the low-performing ones. Each function value of the n sample points $y\left(s\right)$ is successively regarded as an unknown quantity and is predicted based on model parameters $\alpha$ and p, and other $n-1$ samples, given a corresponding predict vector $y_{\mathrm{pred}}\left(s\right)$. Then, linear fitting is performed on those points located at $\left(y\left(s\right),y_{\mathrm{pred}}\left(s\right)\right)$, and we use slope $p_{\mathrm{fit}}$ as a criterion. In principle, $p_{\mathrm{fit}}$ far away from 1 indicates bad performance of the predictor, and by examining the fitted slope of bad predict models occurred in the optimization process, we found that bad models correspond to small values of $p_{\mathrm{fit}}$. Therefore, we set $p_{\mathrm{fit}}>0.6$ as the threshold. If this condition is fulfilled, the predict function can be used as the objective function in the following direct search process. Otherwise, a new model needs to be built from the very beginning.

After the valid predictor was constructed, it was used as the objective function of the following direct search process, where we used the phase-modulated method to further improve the efficiency. Initial parameter ${\mathbf{\lambda }}_{\mathrm{ini}}$ was taken in the range of $a_j^{\mathrm{ini}}\in [0,{\mathrm{\Omega }}_{\max }]$, and $b_j^{\mathrm{ini}},{\nu }_j^{\mathrm{ini}}\in [0,2\pi /T]$, where ${\mathrm{\Omega }}_{\max }$ is the maximum field amplitude and T is the evolution time. The SFB method was also investigated for comparison, with initial parameters $a_j^{\mathrm{ini}}\in [0,{\mathrm{\Omega }}_{\max }]$, ${\omega }_j^{\mathrm{ini}}\in [0,2\pi /T]$ and ${\varphi }_j^{\mathrm{ini}},{\phi }_j^{\mathrm{ini}}\in [0,2\pi ]$. In both methods, we took ${\mathrm{\Omega }}_{\max }=2\pi \times 10$ MHz and $T=100$ ns, and the total field amplitude was constrained as $\sqrt{{\mathrm{\Omega }}_x^2\left(t\right)+{\mathrm{\Omega }}_y^2\left(t\right)}\le {\mathrm{\Omega }}_{\max }$; frequency parameters $b_j,{\nu }_j$ and ${\omega }_j$ were constrained in range $[0,5\times 2\pi /T]$; and phase parameters ${\varphi }_j$ and ${\phi }_j$ were constrained in range $[0,2\pi ]$. An extra constraint on $b_j/{\nu }_j$ can be added to avoid an infinite value when ${\nu }_j$ goes close to zero, e.g., $b_j/{\nu }_j\ge {10}^{10}$, but in our simulation, the constraint on $b_j/{\nu }_j$ was not used, and no infinite value of $b_j/{\nu }_j$ appeared during the optimization process.

In the last step, given optimization parameter ${\mathbf{\lambda }}_{\mathrm{opt}}$, the average fidelity in Equation (11) was calculated based on 2500 uniformly spaced sample points in regions $\delta \in 2\pi \times [-10,10]$ MHz and $\kappa \in [0.5,1.5]$, with $M=N=50$.

2.4. AC Magnetometry

Magnetometry is one of the most promising fields in which the NV center can play an important part, with its outstanding characteristics such as fine biological compatibility, normal pressure and temperature operating conditions, and relatively long spin lifetimes. Pulsed and continuous dynamical decoupling (DD) has been applied in the coherent control and magnetic sensing of NV centers. In practice, the large inhomogeneous broadening of an NV center ensemble decreases the controllability of the DD sequence and ultimately hinders the sensitivity in magnetometry. Adiabatic strategies have been applied to tackle such problem and have achieved good results. When the frequency of the signal is not low enough to fulfill the adiabatic condition, finding the optimal field with limited pulse length to achieve certain gate operation becomes a key approach to improve the sensitivity. Here, we used the B-PM method to optimize the control field and obtain robust Pauli-X and Pauli-Y gates under inhomogeneous broadening, which composes the XY-8 DD sequence in the AC magnetometry process.

During the pulse period, control Hamiltonian $H_{\mathrm{c}}$ in Equation (22) takes the form of

$$H_{\mathrm{PM}}^{\mathrm{X}}\left(t\right)={\mathrm{\Omega }}_1\left(t\right){\sigma }_x+{\mathrm{\Omega }}_2\left(t\right){\sigma }_y$$

(30)

for the Pauli-X gate and

$$H_{\mathrm{PM}}^{\mathrm{Y}}\left(t\right)={\mathrm{\Omega }}_1\left(t\right){\sigma }_y-{\mathrm{\Omega }}_2\left(t\right){\sigma }_x$$

(31)

for the Pauli-Y gate, where

$${\mathrm{\Omega }}_1\left(t\right)=\sum _{j=1}^{N_{\mathrm{D}}}\frac{a_j}{2}cos\left[\frac{b_j}{v_j}sin\left(v_{j}t\right)\right],{\mathrm{\Omega }}_2\left(t\right)=\sum _{j=1}^{N_{\mathrm{D}}}\frac{a_j}{2}sin\left[\frac{b_j}{v_j}sin\left(v_{j}t\right)\right].$$

(32)

For gate optimization, the objective function takes the form of

$${\mathcal{F}}_{\mathrm{obj}}=\mathcal{N}\sum _{k=1}^M\sum _{j=1}^{N}p\left({\delta }_k\right)p\left({\kappa }_j\right)f_g\left({\delta }_k,{\kappa }_j\right),$$

(33)

where

$$f_g(\delta ,\kappa )=\frac{1}{2}+\frac{1}{3}\sum _{ϵ=x,y,z}Tr\left(U_{\mathrm{Tar}}\frac{{\sigma }_{ϵ}}{2}{U}_{\mathrm{Tar}}^{†}{U}_{\delta ,\kappa }\frac{{\sigma }_{ϵ}}{2}{U}_{\delta ,\kappa }^{†}\right),$$

(34)

is the gate fidelity of fixed $\delta$ and $\kappa$; $U_{\mathrm{Tar}}$ is the target quantum gate; $U_{\delta ,\kappa }$ is the actual operator, which takes the same form as in Equation (29); and $\mathcal{N},p\left({\delta }_k\right)$ and $p\left({\kappa }_k\right)$ take the same forms and values as in Equation (11). We applied the B-PM method to search for the robust control field with the amplitude constraint of $\sqrt{{\mathrm{\Omega }}_1^2\left(t\right)+{\mathrm{\Omega }}_2^2\left(t\right)}⩽{\mathrm{\Omega }}_{max}$ and obtained an optimized shaped pulse, which constructs the XY-8 DD sequence in the AC magnetometry strategy.

The AC magnetic signal to be measured oscillates at frequency ${\omega }_{\mathrm{s}}$, and the control pulses are applied on time nodes where the AC signal changes its direction. Due to the power limitation of the microwave field, each pulse possesses a finite pulse length $T_{\mathrm{pulse}}$, whose value follows the relationship ${\omega }_s=\pi /\left(T_{\mathrm{pulse}}+{\tau }_{\mathrm{pulse}}\right)$, where ${\tau }_{\mathrm{pulse}}$ is the time separation between two adjacent pulses. The amplitude of AC magnetic signal $g_{\mathrm{ac}}$ can be read out from the Ramsey oscillation of the NV center sensor. At the beginning of the measurement, the NV centers in the ensemble are initialized into state $|0\rangle$ and flipped with a $\pi /2$ pulse, and begin to rotate around the ${\sigma }_z$-axis. After a time period t, one $3\pi /2$ pulse is applied to the sensors, and their population on $|0\rangle$ is measured. Under the ideal condition with instantaneous gate pulses, the population can be expressed as $P_0=\left(1+cos\left(2\chi \right(t\left)\right)\xi \left(t\right)\right)/2$, with phase term $\chi \left(t\right)\equiv {\int }_0^{t}g\left|cos\left({\omega }_{\mathrm{s}}{t}^{\prime }\right)\right|d{t}^{\prime }$ and exponential decay term $\xi \left(t\right)$ being induced by inhomogeneous broadening and dynamic noise. At coherent time $t=T_2$, the decay term goes down to $1/e$. In practice, the finite pulse length leads to a deviation in the real phase term from $\chi \left(t\right)$.

Specifically, the total Hamiltonian in the rotating-wave approximation is

$$H_{\delta ,\kappa }\left(t\right)=\frac{\delta +{\delta }_d\left(t\right)}{2}{\sigma }_z+g_{\mathrm{ac}}cos\left({\omega }_{s}t\right){\sigma }_z+\kappa H_c,$$

(35)

where a time-dependent dynamic noise term $\delta \left(t\right)$ is considered, obtained with the Ornstein–Uhlenbeck process

$${\delta }_d(t+\Delta t)={\delta }_d\left(t\right)e^{-\Delta t/\tau }+{\left[\frac{c\tau }{2}\left(1-e^{-2\Delta t/\tau }\right)\right]}^{1/2}n,$$

(36)

where $\tau$ and c are the correlation time and the diffusion constant of noise, respectively, valued as $\tau =20\mathsf{\mu }\mathrm{s}$ and $\sqrt{c\tau /2}=2\pi \times 50\mathrm{KHz}$ [17]; and $n\sim \mathcal{N}(0,1)$. Controlled Hamiltonian $H_{\mathrm{c}}$ only takes a non-zero value during pulse duration $T_{\mathrm{pulse}}$. We took the frequency of the signal to be ${\omega }_s=\pi /\left(T_{\mathrm{pulse}}+{\tau }_{\mathrm{pulse}}\right)=2.5\pi \mathrm{MHz}$, where ${\tau }_{\mathrm{pulse}}$ is the time separation between two adjacent pulses. We took the maximum limit of the field amplitude as ${\mathrm{\Omega }}_{max}=2\pi \times 10\mathrm{MHz}$; so, for the rectangular pulse, $T_{\mathrm{pulse}}=50\mathrm{ns}$, during which $H_c={\mathrm{\Omega }}_{max}{\sigma }_x$ for the Pauli-X gate and $H_c={\mathrm{\Omega }}_{max}{\sigma }_y$ for the Pauli-Y gate, and the time separation was ${\tau }_{\mathrm{pulse}}=350\mathrm{ns}$. For the optimization pulses, we set $T_{\mathrm{pulse}}=100\mathrm{ns}$ and ${\tau }_{\mathrm{pulse}}=300\mathrm{ns}$.

3. Results

3.1. Feasibility of the Estimation Model

We first demonstrate the feasibility of the estimation model based on its estimation accuracy and time efficiency. Figure 3 shows its performance in predicting the fidelity of the final state under certain frequency detuning $\delta$ and amplitude drift factor $\kappa$. For the fixed control field shown in Figure 3a, the value of $f(\delta ,\kappa )$ varied with $\delta$ and $\kappa$, which we show with a 2D color distribution image in Figure 3b, where $50\times 50$ sample points were taken to ensure its resolution. If the available sample number decreased to 9 and 16, respectively, the resolution ratio clearly dropped, as shown in Figure 3c,d. However, based on the same number of sample points, the images depicted with 2500 prediction values $\widehat{f}\left(x\right)$ in Figure 3e,f show an obvious improvement in resolution ratio and accuracy compared with their counterparts in Figure 3c,d, respectively. By applying the estimation method, more information about how the target function distributed in the parameter space was extracted based on the knowledge provided by the sample points.

Another concern is how much time this prediction process takes, or if it has advantage in terms of computing time and cost. Here, we used the average computing time of 100 prediction processes to give a general description. The control field in each process took the form of $g\left(t\right)=acos\left[{\omega }_{0}t+\frac{b}{\nu }sin\left(\nu t\right)\right]$, with evolution time $T=100$ ns, and parameters $a,b$ and $\nu$ were randomly taken from the ranges of $a\in [0,10\times 2\pi ]$ MHz, $b\in [0,2\pi /T]$ and $\nu \in [0,2\pi /T]$. Each prediction process used 16 sample points, and the mission was to calculate the average fidelity (see Equation (11)) in ranges $\delta \in 2\pi \times [-10,10]$ MHz and $\kappa \in [0.5,1.5]$. We represent the estimated fidelity as

$${\widehat{\mathcal{F}}}_{\mathrm{obj}}=\mathcal{N}\sum _{k=1}^M\sum _{j=1}^{N}p\left({\delta }_k\right)p\left({\kappa }_j\right)\widehat{f}\left({\delta }_k,{\kappa }_j\right),$$

(37)

where the formation and values of $\mathcal{N},p\left({\delta }_k\right)$ and $p\left({\kappa }_k\right)$ are the same as in Equation (11), and $M\times N$ represents the number of different prediction points used in the calculation. Two strategies were separately applied: in Figure 4a, the estimation model was updated in each trial, while in Figure 4c, a fixed estimation model served in all the 100 trials. To make a comparison, we also show the results of calculating the true ${\mathcal{F}}_{\mathrm{obj}}$ with $M\times N$ sample points. It is natural that for the true function-based objective function ${\mathcal{F}}_{\mathrm{obj}}$, the computation time was approximately proportional to $M\times N$, the total number of function $f(\delta ,\kappa )$ calling times. In contrast, the processing time of the predict function-based objective function remained stable as $M\times N$ increased, causing distinct time retrenching as $M\times N$ increased to 2500. Notice here that for $M\times N\le 64$, the computation time of ${\widehat{\mathcal{F}}}_{\mathrm{obj}}$ was longer than that for ${\mathcal{F}}_{\mathrm{obj}}$, because the values of 16 sample points needed to calculated, and an extra amount of time needed to be spent on selecting the parameters that constructed the estimation model. Figure 4c shows that once the estimation model was built, the process for calculating the value of the predict function became rapid, and its corresponding time consumption could be neglected, so the total computation time was decided by the number of true sample points, i.e., $M\times N=16$.

Figure 4b,d show the estimation accuracy of ${\widehat{\mathcal{F}}}_{\mathrm{obj}}$ and ${\mathcal{F}}_{\mathrm{obj}}$ with different calculated sample numbers $M\times N$. In Figure 4b, we used the same data as in Figure 4a, and in Figure 4d, we used the same data as in Figure 4c. We used the value of ${\mathcal{F}}_{\mathrm{obj}}$ with $M\times N=2500$ as the benchmark to calibrate the accuracy, denoted as ${\mathcal{F}}_{\mathrm{obj}(M\times N=2500)}$. The estimation accuracy was expressed as the value deviation from ${\mathcal{F}}_{\mathrm{obj}(M\times N=2500)}$, that is, $|{\widehat{\mathcal{F}}}_{\mathrm{obj}}-{\mathcal{F}}_{\mathrm{obj}(M\times N=2500)}|$ with the estimation objective function method and $|{\mathcal{F}}_{\mathrm{obj}}-{\mathcal{F}}_{\mathrm{obj}(M=50)}|$ with the true objective function method. In general, the deviation decreased as $M\times N$ increased, and the estimation deviation values of Figure 4b,d were in the same scope, showing that the fixed estimation model strategy did not damage the estimation accuracy.

3.2. Optimization Efficiency of the B-PM Method

The results in Figure 4 demonstrate that by using the fixed estimation model, the total computation time is decided by the function $f(\delta ,\kappa )$ calling times. Figure 5 shows a comparison among the B-PM method, the PM method, the Bayesian estimation SFB (B-SFB) method and the SFB method in terms of the final value of ${\mathcal{F}}_{\mathrm{obj}}$ and the calling times of function $f(\delta ,\kappa )$ during the search process. The results with $N_{\mathrm{D}}=1$ are shown in Figure 5a,c, and the results with $N_{\mathrm{D}}=2$ are shown in Figure 5b,d. Each result is based on 100 trials with random initial parameters. On the whole, the B-PM method gave the result of ${\mathcal{F}}_{\mathrm{obj}}=0.905$ with 1252 average calling times, while the SFB method gave the result of ${\mathcal{F}}_{\mathrm{obj}}=0.894$ with 13,479 average calling times, so B-PM resulted in improved average fidelity using only $9.3\%$ of computation resources compared with SFB method.

Figure 6 shows the detailed optimization results of the B-PM method with $N_{\mathrm{D}}=1$ and $n=9$ and those of the SFB method with $N_{\mathrm{D}}=2$ and $M\times N=16$. Figure 6a,d show the final objective function values of 100 trials with random initial parameters $\mathbf{\lambda }$, in which we marked the value used in the optimization process with black points and their corresponding, more accurate values calculated with $M\times N=2500$ true sample points with red points. As is shown in Figure 6a, among 100 trials of the B-PM method, 42 results gave ${\mathcal{F}}_{\mathrm{obj}}\ge 0.9$, and 86 results gave ${\mathcal{F}}_{\mathrm{obj}}\ge 0.87$, while with the SFB method (Figure 6b), only 3 results out of 100 trials gave ${\mathcal{F}}_{\mathrm{obj}}\ge 0.87$. Figure 6b,e display the shape of the optimized control field in the interaction picture, and Figure 6c,f display the fidelity distribution in regions $\delta \in 2\pi \times [-10,10]$ MHz and $\kappa \in [0.5,1.5]$.

3.3. Sensitivity Improvement in AC Magnetometry

Figure 7a shows the sketch of the XY-8 sequence and AC signal to be sensed. Figure 7b shows the population of the sensor in $|0\rangle$, measured at every terminal of a single XY-8 period so that the time interval between two data points was $3.2\mathsf{\mu }\mathrm{s}$. The 1000 measurements made under different $\delta$ obeyed the Gaussian distribution with zero mean value and FWHM = $2\pi \times 26.5\mathrm{MHz}$, and the $T_2$ time was identified as the time when the maximal value of $P_0$ dropped below $(1+1/e)/2$, the value of which is marked with a reference gray dashed line in Figure 7b. Because of the inhomogeneous broadening of $\delta$ as well as dynamic noise $\delta \left(t\right)$, the populations decayed over time, i.e., $T_2\approx 180\mathsf{\mu }\mathrm{s}$ for the rectangular XY-8 pulse and $T_2\approx 1500\mathsf{\mu }\mathrm{s}$ for the PM XY-8 pulse, respectively.

4. Discussion

Our work shows that the Bayesian-based estimation model can be effectively applied to estimating the fidelity of the state transformation and the gate construction of NV centers under wide inhomogeneous broadening noise. The estimation accuracy as well as the time efficiency make it an ideal tool to constitute a new practical optimization method, which we denote as Bayesian estimation phase-modulated (B-PM) method. The B-PM method can be combined with various of search algorithms, i.e., the direct search algorithm [43,44], the genetic algorithm [45] and the gradient-based algorithm [46,47,48,49]. Here, we adopted the widely used Nelder–Mead direct search method [50] and related optimization tools that are easily available on the most common programming platforms. The Nelder–Mead method seeks the best parameter point by successively constructing new points to replace the worst point following a heuristic simplex approach. Consequently, the objective function needs to be frequently called to rank the points during the searching process, which consumes the majority of the total processing time. The hybrid Bayesian estimation phase-modulated (B-PM) method reduces the processing time in two ways. On the one hand, by setting the predict function as the objective function, the time needed to compute the objective function value once is reduced, as shown in Figure 4. On the other hand, the phase-modulated method can significantly decrease the total number of times in which objective functions are called, by introducing a simpler landscape of the parameter space. This is based on its ability to comprise more frequencies with the same number of parameters than the conventional SFB and other amplitude-modulated methods. A detailed description about this point can be found in reference [29], and recent works [51] using other kinds of phase-modulated control methods also show their usability in protecting the coherence of samples with large inhomogenities.

Taking these two factors together, the advantage of the B-PM method over the commonly used standard Fourier basis(SFB) method becomes obvious. To be specific, compared with the best result obtained with the SFB method, the B-PM method increased the optimal value of the objective function from $0.894$ to $0.905$, with a more than $90\%$ decrease in the average time consumption. Moreover, among 100 results obtained with random initial parameters, only 1 result of the SFB ($N_{\mathrm{D}}=2$ and $M\times N=16$) method gave ${\mathcal{F}}_{\mathrm{obj}}\ge 0.89$, while 42 results of the B-PM ($N_{\mathrm{D}}=1$ and $n=9$) method gave ${\mathcal{F}}_{\mathrm{obj}}\ge 0.9$, indicating that the B-PM method is much more robust and can reach a fine result when the amount of total trials is limited. When making a comparison between the B-PM and PM methods, people may query that the improvement induced by the Bayesian estimation model is not obvious. This is true for the current system studied in this article, which needs to be tested on more physical models and systems. However, we stress here that based on the results showed in Figure 5, the B-PM method guarantees a higher value of objective function than the PM method when the computation resources are on the same scale. This is crucial when the computation resources are severely stressed and when the control performance is sensitive to the precise value of the objective function.

We further displayed the utility of the B-PM method in NV center-based AC magnetometry, especially in high-frequency cases where adiabatic strategies are not viable. The numerical simulation result shows that the optimized pulse achieved an eight-fold extension of coherence time $T_2$ compared with the conventional rectangular $\pi$ pulses with the same maximum amplitude. A similar optimization strategy can applied to other sensing cases, e.g., the spin bath driving [22,52,53] process for DC magnetometry sensing. We used the XY8 sequence in our simulation, but the optimization pulses can be applied to other DD sequences, including XY16 [54], periodic dynamical decoupling (PDD) [55], concatenate dynamical decoupling (CDD) [56] and so on [57]. Comparing the performance of optimization pulses in these sequences could provide a useful prescription for DD optimization, which will be completed in our subsequent studies. We denote that the normal distribution in Equation (3) results from a variety of disturbance factors, including the interaction between NV center spins, nearby nuclear spins or spin bath, and external bias fields, as well as the frequency bias of the control field. We took the FWHM of the broadening as $2\pi \times 26.5$ MHz in our simulation, corresponding to a dephasing time of $T_2^{*}\approx 20$ ns. To check the feasibility of the B-PM method under other conditions with longer $T_2^{*}$, we simulated the fidelity of the Pauli-X gate for $T_2^{*}\approx 20$ ns, $T_2^{*}\approx 260$ ns (corresponding FWHM $=2\pi \times 2$ MHz) and $T_2^{*}\approx 2600$ ns (corresponding FWHM $=2\pi \times 0.2$ MHz). The B-PM control field gave the results of $0.8454$, $0.9671$ and $0.9804$, respectively, while the rectangular pulse gave the results of $0.8047$, $0.9583$ and $0.9628$. These results show that our method is feasible under different $T_2^{*}$ conditions.

One important potential application area of the B-PM method is the closed-loop optimization [25,26,27] of complex systems, such as many-body [58] and many-electron systems [59]. In these cases, numerical simulations fail to accurately describe the system due to a lack of information of the complex system, and the experimental results are directly used as the values of the objective function in the optimization process. The advantage of the B-PM method in reducing the requisite total sample number during the process could be more prominent in such conditions, since the time needed to experimentally obtain one sample dataset is generally much longer than that of the simulation process. Besides NV centers, the B-PM method is also expected be applied to the optimization of other prevailing quantum platforms, such as trapped ions [60,61,62], cold atoms and superconducting qubits [63,64].