Real-Time Strain and Elasticity Imaging in Phase-Sensitive Optical Coherence Elastography Using a Computationally Efficient Realization of the Vector Method

Abstract

We present a real-time realization of OCT-based elastographic mapping local strains and distribution of the Young’s modulus in biological tissues, which is in high demand for biomedical usage. The described variant exploits the principle of Compression Optical Coherence Elastography (C-OCE) and uses processing of phase-sensitive OCT signals. The strain is estimated by finding local axial gradients of interframe phase variations. Instead of the popular least-squares method for finding these gradients, we use the vector approach, one of its advantages being increased computational efficiency. Here, we present a modified, especially fast variant of this approach. In contrast to conventional correlation-based methods and previously used phase-resolved methods, the described method does not use any search operations or local calculations over a sliding window. Rather, it obtains local strain maps (and then elasticity maps) using several transformations represented as matrix operations applied to entire complex-valued OCT scans. We first elucidate the difference of the proposed method from the previously used correlational and phase-resolved methods and then describe the proposed method realization in a medical OCT device, in which for real-time processing, a “typical” central processor (e.g., Intel Core i7-8850H) is sufficient. Representative examples of on-flight obtained elastographic images are given. These results open prospects for broad use of affordable OCT devices for high-resolution elastographic vitalization in numerous biomedical applications, including the use in clinic.

1. Introduction

Optical Coherence Elastography (OCE) is one of the most actively developing extensions of Optical Coherence Tomography. Various aspects OCE and the proposed approaches to its realization (based on several variants of quasistatic and dynamic approaches) have been discussed in recent years in several reviews [1,2,3,4,5,6,7,8]. These reviews indicate that breakthrough results in OCE have been demonstrated mostly in the last 5–6 years, although the studies on strain and elasticity imaging in OCT have been carried out over two decades. They were triggered by J. Schmidt in the seminal paper [9], the latter in turn was stimulated by the earlier initiated elastography-related studies in medical ultrasound [10].

By analogy with [10], in [9] it was proposed that the initial stage of OCE realization should be the reconstruction of displacements of scatterers using comparison of structural OCT scans. Then, using the reconstructed distribution of the displacements, their spatial gradients should be found to estimate local strains in the tissue. The mechanically produced local strains can then be related to the Young’s modulus of the tissue [10].

In the seminal works [9,10] and some other earlier OCE-related studies, such as [11,12], it was supposed that the reconstruction of the displacements should be made using the correlation approach. This approach implies that a fairly small subset (processing window) should be taken in the reference image. Then the cross-correlation coefficients should be found for the reference subset (window) in the initial image and a similar-in-size window in the deformed image. The cross-correlation should be maximized by moving the second window over the deformed image. If the displacements of the scatterers are non-zero, the so-estimated correlation coefficient reaches its maximum for non-coinciding positions of the cross-correlated windows. The displacement of the window taken from the deformed image corresponds to the displacement of the scatterers located near the center of the reference window.

It should be taken into account that OCT images are pixelated, so that in a straightforward manner, the correlational search can be made only for the displacements of the processing window by an integer number of pixels. Usually, such an accuracy is insufficient, such that various possibilities to estimate displacements with a subpixel accuracy have been considered in the literature. For example, using interpolation methods similar to those used in mechanical engineering [13,14], or recalculation of pixelated patterns using the shift theorem for Fourier transforms [15]. The general feature of all correlation-based approaches is that for finding the field of the local displacements, it is necessary to repeat the correlation-based search for every initial position of the processing window in the reference image. Furthermore, for every initial position, multiple calculations of the cross-correlation should be performed to determine the final position of the processing window, in which the correlation coefficients demonstrate the maximum value. These features, intrinsic to correlation-based methods, make them rather computationally demanding.

An alternative approach to the elastographic estimation of local strains in OCT is based on analysis of interframe phase variations [16], from which the axial displacements can be reconstructed. Unlike the correlation-based techniques, a very important feature of the phase-resolved approach is that determining of local interframe phase variations (and, therefore, local displacements) does not require multiple calculations over a sliding window. The sought local phase variations can be found by directly comparing the phases of the corresponding pixels in the pixelated reference and deformed OCT images. In fact, this comparison can be made by performing a single matrix operation with the entire compared OCT scans. In principle, such phase-based determination of local displacements has its own limitations/challenges. In particular, an important issue is the correct accounting for phase wrapping that occurs when the displacements have a supra-wavelength scale. Nevertheless, the possibility to fairly easily obtain local phase-variations without multiple search operations enables much faster estimation of the corresponding local axial displacements. This advantage of the phase-based approach very strongly accelerates the mapping of local displacements in comparison with the correlation-based searching.

The next step after mapping displacements proportional to the local phase variations is finding axial gradients of the displacements to obtain genuine local strains. This step requires the estimation of axial gradients of the initially determined interframe phase variations. If any measurement noise/error was completely absent, then for finding these gradients it could be sufficient to compare the phases of only two neighboring pixels on the pixelated phase-variation map. However, in reality, because of various measurement noises, such as noises of the receiving array of photodetectors and the “decorrelation noise” [10] induced by the strain itself, such pair-wise comparison of pixels along the vertical coordinate is unacceptably noisy. In this regard, for estimation of the phase-variation gradient along the depth of OCT scans, it was proposed to use the least-square method for estimating the local slope of the dependence of the interframe phase variation on the axial coordinate [16]. This least-square-based estimate is performed with averaging over some processing window (the latter may be either a sufficiently large 1D segment along a single A-scan in the phase-variation map or a 2D-window on the B-scan of interframe phase difference). The least-square estimations may additionally be supplemented with signal-amplitude weighting to further improve the signal-to-noise (SNR) ratio in the reconstructed local-strain map [16]. Such least-square estimation of the phase-variation slope within a sliding processing window (similarly to estimating gradients using displacements found by the correlation method) also requires multiple operations for every position of the sliding averaging window. However, due to the above-mentioned much faster estimation of local vertical displacements using the OCT signal phase, the overall computational demand is significantly reduced in comparison with the correlation-based method, so that even real-time visualization becomes possible, albeit using GPU-computations [17].

More recently, an alternative approach to estimating local gradients of interframe phase variations was proposed in [18,19]. This phase-based approach is called “vector” because in this method OCT signals characterized by phase and amplitude are considered as complex-valued quantities and treated as vectors in the complex plane. In comparison with the least-square method of finding the phase-variation gradients the vector approach may demonstrate better tolerance to measurement noises.

Another important advantage of this approach is the possibility of significantly enhancing the computational efficiency of finding local phase-variation gradients and, therefore, local strains. For improving SNR, the vector approach also uses averaging over a chosen area on the OCT scan. Certainly, this averaging may be realized using a sliding processing window, which requires repeated local calculations for every position of this window. However, unlike the other above-mentioned approaches the usage of a sliding window is not indispensable for the vector approach. In this paper, we will demonstrate that all stages of the vector approach can be realized in a matrix form that operates with the OCT scan as a whole without the necessity to introduce a real sliding processing window. For such a form, any subsequently performed local computations within a gradually moved processing window are completely eliminated; nevertheless, the vector method allows one to obtain genuine spatially resolved distributions of local strains over the OCE scan. In such a matrix form the vector approach is especially computationally efficient, which significantly simplifies real-time obtaining of elastographic maps of both local strains and Young’s modulus.

In what follows, we explain in more detail the difference between the vector method and previously used approaches. Then we describe the vector method in the modified form that is especially computationally efficient. Its realization in a medical OCT scanner is also described and representative examples of strain and tissue-elasticity maps obtained on-flight using quite a “typical” CPU (such as Intel Core i7-8850H, Intel Corporation, Santa Clara, CA, USA) without any GPU calculations are presented.

2. Materials and Methods

2.1. Materials That Can Be Characterized by the Discussed OCE Technique

Concerning the materials for which the discussed OCT-based elastographic technique is operable, it is necessary to clearly distinguish the two destinations of OCE: (i) visualization of local strains in the material; and (ii) visualization of elasticity of the material using the compression principle. The visualization of strains has no special limitations in terms of specific materials and origin of strains; the latter may be mechanically produced deformations, thermally produced strains, osmotic strains, etc. (see examples in [8]). Concerning characterization of elasticity using compression OCE, in which the mechanical loading is combined with visualization of the resultant strains, the described technique is operable for a broad class of sufficiently soft materials. These may be soft biological tissues (e.g., cancerous ones [20], and even such tissues as cartilage) or soft plastics. Previously, obtained strain and elasticity maps for such materials (as in review [8]) were obtained using elastographic post-processing. In contrast, in this section we describe the modified fast elastographic processing method and its realization enabling similar OCE imaging in real time. For demonstration of operability of the new method we will present elastographic images obtained for the dorsal-side skin of a volunteer’s hand.

2.2. Basic OCT Setup Used to Realize the Elastographic Modality in Real Time

Since a broad practical application of such OCT devices is foreseen, an important requirement to their design is to make them affordable for biomedical users. By this reason, the use of fairly inexpensive solutions for the OCT setup itself, as well as for the controlling computer system, is of key importance. In view of this, in what follows, we consider the realization of the developed OCE technique for a fairly inexpensive OCT device with “typical” parameters that are far from record values. This is a spectral-domain medical OCT device designed and produced at the Institute of Applied Physics (Russian Academy of Sciences, Nizhny Novgorod, Russia) with the central wavelength of 1.3 μm, spectral width 90 nm, and a receiving array enabling 20 kHz rate of A-scan acquisition. Due to the moderate intensity of the data flow, the signal transmission to the controlling computer acquisition was organized without specialized high-speed devices using only the standard USB interface (see more detail in [21]). The signal processing by the controlling computer also did not require specialized GPU calculations and was performed by the central processing unit (CPU) based on the developed procedures of multi-thread parallel computations in the CPU itself as described in [22].

The above-mentioned requirements of the OCT hardware and controlling computer impose rather strong limitations on the choice of the elastographic signal-processing method with which real-time realization would be possible. To better elucidate the differences among the main approaches to the visualization of local displacements, strains and elastic modulus in OCT elastography, we first briefly recall the conventional methods starting from the correlational method initially proposed in OCT by Schmitt [9] by analogy with the medical ultrasound [10]. Then we point out the challenges/limitations of its realization in OCT and turn to the phase-resolved approach that has become the most widely used for mapping axial strains the Young’s modulus in compression OCE. We start from its earlier form [16], then formulate the basic idea of its vector form [18,19] and finally describe the new fast variant of the vector approach to strain mapping in phase-resolved OCT.

2.3. Main Steps of the Correlation-Based Approach

In this section, we briefly recall the main steps of correlation-based speckle tracking. The correlation method of determining local displacements that has been discussed in OCT since the Schmitt’s paper [9] is based on maximization of the cross-correlation coefficient that has the following form:

$$C_{x,z}(x^{\prime },z^{\prime })=\frac{{\displaystyle \iint I_1(x,z)I_2(x-x^{\prime },z-z^{\prime })dxdz}}{{\left[{\displaystyle \iint I_1^2(x,z)dxdz}{\displaystyle \iint I_2^2(x-x^{\prime },z-z^{\prime })dxdz}\right]}^{1/2}}$$

(1)

Here, the intensity distributions $I_1(x,z)$ and $I_2(x,z)$ relate to the compared reference and deformed images (where the lateral and axial coordinates in the in-depth B-scans are denoted as $x$ and $z$, respectively). The integrals in the cross-correlation coefficient in Equation (1) are found over a processing window $S$ of the size $m_1\times m_2$. In the reference scan the window $S$ is centered at the point $(x,z)$. The corresponding fragment $S^{\prime }$ of the same size in the deformed image is centered at $(x-x^{\prime },z-z^{\prime })$. The position of the center of this window can be changed within a certain search region, to find coordinates $(x^{\prime },z^{\prime })$, for which the correlation coefficient $C_{x,z}(x^{\prime },z^{\prime })$ reaches a maximum. It is important to point out that for the intensity $I_1(z,x)$ in Equation (1) (and similarly for $I_2(x-x^{\prime },z-z^{\prime })$ in the deformed image) the average value $I_1^0={\displaystyle \iint I_1^{}(x,z)dxdz}$ over the current position of the processing window should be subtracted from the initial intensity, $I_1(z,x)=I_1^{init}(z,x)-I_1^0$. Consequently, the quantities $I_1(z,x)$ and $I_2(x-x^{\prime },z-z^{\prime })$ in Equation (1) have zero mean values. For identical images (not subjected to deformation) and in the absence of any noises, the correlation coefficient (1) reaches its maximum equal to unity for $x^{\prime },z^{\prime }=0$. For a deformed image, the positions of scatterers are displaced and coefficient $C_{x,z}(x^{\prime },z^{\prime })$ reaches a maximum for certain coordinates $(x^{\prime }*,z^{\prime }*)$. The latter coordinates define the displacement vector for the new position of scatterers initially located within the reference window. This approach has analogies both in ultrasound elastography [10] and engineering applications based on the analysis of photographic images [14]. As mentioned above, for pixelated OCT scans, the processing window can be straightforwardly shifted only in discrete steps, so that additional procedures are required for finding sub-pixel displacements (e.g., based on some interpolation procedures [13] or using the shift theorem for Fourier transforms [15]).

The process of correlation-based reconstruction of local displacements is schematically shown in Figure 1. It is important that displacements produced by tissue straining in addition to approximately translational displacement of various groups of scatterers also cause mutual displacement of scatterers, as well as affect the brightness of speckles due to interference of mutually displaced sub-resolution scatterers (the so-called blinking/boiling of OCT speckles). These effects, as well as measurement noises of the photo-detectors) inevitably cause errors in the correlation-based estimation of the displacements. Also, the result of the correlation search depends on the size of the processing window that produces a smoothing effect and reduces the spatial resolution of the so-found “local” displacements in comparison with the resolution of the initial structural OCT images. Such a correlation-based approach to reconstruction of displacement fields in the deformed tissues was demonstrated in OCT in several studies, for example, in the earlier works [11,12]. However, the approximate reconstruction of displacements was not yet the complete realization of the local strain reconstruction. The latter requires subsequent differentiation of the preliminary reconstructed noisy displacement fields. The degrading effect of the noises can be reduced by increasing the size of the processing window over which the gradients of displacements are estimated, but the increased window size additionally reduces the spatial resolution of the resultant maps of strain distribution (see a detailed discussion in [15]). The strain-induced decorrelation can also be mitigated by using super-broadband OCT sources like in [23]. Although after the earlier tests the correlation method has been used in some more recent works, e.g., [24,25,26,27,28,29,30,31], nevertheless, in recent years, in the development of OCT-based methods for strain and elasticity imaging, the main attention has shifted to phase-resolved methods [8,16,32,33].

In the context of comparison with the phase-based approaches, it is important to emphasize once again that the correlation-based reconstruction of local displacements (and then strains) indispensably requires the application of sliding “local” processing windows. The correlation coefficient (1) should be estimated for every position $(x,z)$ of this window in the reference image and then the computation of this correlation should be multiply repeated for various positions $(x^{\prime },z^{\prime })$ of the corresponding window in the deformed scan to maximize $C_{x,z}(x^{\prime },z^{\prime })$. The subsequent estimation of the displacement gradients also requires calculations over a sliding window. Certainly, such calculations can be made independently for various parts of the entire OCT scan, which allows for using parallel computations. However, for obtaining a spatially resolved image by such correlation-based processing, it is impossible to represent such procedures in a matrix form, in which the same operation(s) are applied to the pixelated OCT scan as a whole without search operations and calculations over sliding windows.

2.4. Phase-Resolved Approach to Reconstructing Local Displacements without Local Calculations over a Sliding Window

An important step in the realization of imaging of axial displacements and strains in OCT for subsequent mapping of the elastic modulus was publication [16]. Although the signal phase in OCT is sensitive almost exclusively to axial displacements (and, thus, axial strains), those axial strains can be sufficient for estimating the shear modulus of the visualized deformed tissues. This possibility is based on the idea formulated in [9,10] that for biological tissues with the Poisson’s coefficient close to the “liquid” limiting value 0.5, the shear modulus G with a high accuracy is proportional to the Young’s modulus E, G = E/3. Thus, if the tissue is allowed to freely expand laterally, it is the Young’s modulus which determines the axial strains produced by applied axial stress. Bearing in mind that the interframe phase variations are determined almost exclusively by axial displacement of scatterers, the estimation of these interframe phase variations opens a convenient possibility for estimating axial displacements and strains. Consequently, when a nearly uniaxial mechanical stress is produced in the tissue in the vicinity of the compressing OCT probe, this opens the possibility to visualize the spatial distribution of the Young’s modulus (at least in the relative sense). Such an elastographic principle is called compression OCE. In combination with the additional application of pre-calibrated layers of a soft tissue-like material placed between the OCT probe and the tissue, quantification of the Young’s modulus becomes possible [8,32,33].

Thus, for realization of compression OCE it was pointed out in [16] that mapping of axial displacements and then strains can be based on observing interframe phase variations. Actually, similar ideas about the possibility of phase-resolved detection of axial displacements of scatterers in OCT have been proposed for angiographic purposes (visualization of microcirculation) since the beginning of 2000s [34,35] and have even been discussed in the context of simplified realizations of OCE [36,37]. The basic equation relating axial displacement $U$ and phase variation $\mathsf{\Phi }={\phi }_2-{\phi }_1$ of the OCT signal is well known:

$$U=\frac{{\lambda }_0\mathsf{\Phi }}{4\pi n}$$

(2)

Here, ${\lambda }_0$ is the central optical wavelength of the OCT signal in vacuum, and $n$ is the refractive index of the tissue. In the following discussion instead of amplitude $A$ and phase $\phi$ of the OCT signal we will represent it via the complex-valued amplitude $a$:

$$a=A\exp (i\phi )=A\cos (\phi )+iA\sin (\phi )$$

(3)

A reference and deformed pixelated OCT scans then can be represented as complex-valued matrices $a_1(m,j)=A_1(m,j)\exp [i{\phi }_1(m,j)]$ and $a_2(m,j)=A_2(m,j)\exp [i{\phi }_2(m,j)]$, respectively. Next, the spatially resolved interframe phase variation $\mathsf{\Phi }(m,j)={\phi }_2(m,j)-{\phi }_1(m,j)$ can be found using a single matrix operation applied to the entire reference and deformed OCT scans:

$$\mathsf{\Phi }(m,j)=\arg \{a_2\cdot a_1^{*}\}$$

(4)

where the asterisk * denotes phase conjugation and the matrices are multiplied in the element-by-element sense. According to Equation (2), the so-found interframe phase variations $\mathsf{\Phi }(m,j)$ immediately yield the field of local axial displacements $U(m,j)={\lambda }_0\mathsf{\Phi }(m,j)/(4\pi n)$.

We emphasize that in Equation (4), no sliding local operations were required to obtain the field of the local displacements $U(m,j)$ with the maximal spatial resolution. This is an important advantage of the phase-resolved approach as schematically illustrated in Figure 2.

As shown in Figure 2, an important limitation of this approach is related to the fact of phase wrapping for displacements exceeding the half-wavelength scale. For such displacements, Equation (4) gives the estimate of the displacements with an uncertainly proportional to the entire number of $\lambda /2$, because the phase difference can be directly measured only with an uncertainty proportional to the entire number of periods of $2\pi$ rad. This uncertainly, however, is not critical in many cases. In particular, in realization of compression OCE, when the tissue is deformed by the pressed OCT probe, the displacement in the tissue bulk is increased gradually from zero values at the boundary of the OCT probe window. This allows one to perform phase unwrapping to lift the phase-variation ambiguity. Furthermore, in what follows, we will demonstrate that even the necessity of unwrapping can be obviated when mapping the local strains.

2.5. Least-Square Reconstruction of Local Strains Based on the Preliminary Reconstructed Interframe Phase Difference

After obtaining the initial map of interframe phase variations $\mathsf{\Phi }(x,z)$ and displacements $U(x,z)$ based on Equations (2) and (4), it was proposed in [16] to find the local axial strains $\epsilon =\partial U/\partial z$ using the least-square method to estimate the slope $\partial \mathsf{\Phi }/\partial z\propto \partial U/\partial z$ within a certain processing window as illustrated in Figure 3. In the initial version proposed in [16], such estimation was performed with averaging over a 1D segment of every individual A-scan to improve the SNR. However, the slope can also be estimated with averaging over a 2D rectangular window to obtain a reasonable compromise between the axial and lateral resolution [18,19,38,39]. An additional way to improve SNR proposed in [16] was the use of amplitude weighting when performing the least-square estimation of the slope. The amplitude weighting allows one to reduce the contribution of small-amplitude pixels that usually have the noisiest phases. A drawback of the least-square procedures is that such a way of calculations requires phase unwrapping to eliminate ambiguities in the displacement estimation; another drawback is the necessity to perform local calculations within a sliding processing window. The latter should be gradually moved to cover all pixels in the entire OCT scan (which is somewhat similar to correlation processing), such that repeated calculations increase the computational requirements. However, in contrast to the correlation processing, for the phase-resolved approach, such use of sliding windows is not mandatory and can be obviated in the vector method of finding local phase-variation gradients.

2.6. Vector Approach to Reconstruction of Local Strains in Phase-Sensitive OCE

The vector method proposed in [18,19] exploits the idea that the complex-valued amplitudes of pixels in phase-sensitive OCT scans can be treated as vectors in the complex-valued plane. Correspondingly, the pixel amplitude is represented by the absolute value of the vector and phase corresponds to its direction. In principle, comparison of phases for the considered pixels could be performed in the explicit form. However, in such a case it would be necessary to count each of the compared phases from some identical initial value (that can be chosen arbitrary). In this case, one should bear in mind the necessity of accurately considering the situations in which the compared phases for certain choices of the initial value may occasionally get into neighboring $2\pi$-intervals of phase periodicity. A simpler way is enabled by obtaining the matrix $a_2\cdot a_1^{*}$ where element-by-element multiplication is performed for the complex-conjugated vectors $a_2$ and $a_1^{*}$. The matrix $a_2\cdot a_1^{*}$ contains the information about the interframe phase difference. However, without explicit extraction of individual phases one does not need to explicitly define any reference value for these phases and does not need to control that the phases fall in the same interval of periodicity. Still, using the matrix $a_2\cdot a_1^{*}$, it is possible to find the vertical gradient of the interframe phase variation without the necessity to express this phase variation explicitly.

To this end, in recent works [18,19], it has been proposed to apply the vector representation not only for finding the interframe phase variations, but also for finding their axial gradients without the necessity to explicitly single out the phase. To improve the SNR, this vector estimation of the gradient can also be made with averaging over a certain processing window as was discussed in [18,19]. This averaging in the vector approach may be literally realized using local calculations within a sliding processing window by analogy with the least-square method and other similar methods, for which local calculations are indispensable. However, in what follows we will demonstrate that the reconstruction of local strain distribution based on the vector approach can be realized in a matrix form, operating with entire OCT scans without use of real sliding windows. We also emphasize that the vector approach for finding local strains may be realized in a form that does not require phase unwrapping even if the phase variation may exhibit multiple phase wrapping other the vertical coordinate of the OCT scan (like in the examples of phase-variation maps shown in Figure 2 and Figure 3).

The vector principle of finding the vertical phase gradients can be schematically explained as shown in Figure 4. In this schematic, following the original description in [18,19], the operations are represented in the conventional form with a sliding processing window.

An exception is the first step shown in Figure 4a, which corresponds to the pixel-by-pixel multiplication of the entire reference and deformed complex-conjugated OCT scans to obtain a complex-valued matrix in which the pixels contain information about the interframe phase variation $\mathsf{\Phi }(m,j)={\phi }_2(m,j)-{\phi }_1(m,j)$, yet without explicit use of this phase variation:

$$a_2(m,j)a_1^{*}(m,j)\equiv b(m,j)=B(m,j)\exp [i\cdot \mathsf{\Phi }(m,j)]$$

(5)

The so-found complex-valued quantities $b(m,j)$ (also represented as vectors in the complex-valued plane) usually noticeably fluctuate due to strain-induced decorrelation and other noises. To reduce this uncertainty and obtain a more regular array $\overline{b}(j)$, quantities $b(m,j)$ can be very conveniently averaged in the vector form over a certain averaging area.

In Figure 4b, this averaging is illustrated for the averaging area in the form of a horizontal segment of $N_x$ pixels in length:

$$\begin{array}{l}\overline{b(j)}\equiv B(j)\exp [i\cdot \mathsf{\Phi }(j)]={\displaystyle {\sum }_{m=1}^{N_x}b(m,j)=}\\ {\displaystyle {\sum }_{m=1}^{N_x}{A}_2(m,j)A_1(m,j)\exp \{i\cdot [{\phi }_2(m,j)-{\phi }_1(m,j)]\}}\end{array}$$

(6)

Certainly, instead of averaging over a one-dimensional segment in Equation (6), averaging can be performed over a rectangular window, which can additionally improve the SNR of the resultant strain map. The graphical representation of such averaging corresponding to summation of individual vectors $b(m,j)$ as illustrated in Figure 4b clearly demonstrates that the strong phase errors are automatically suppressed in such an averaging procedure, so that pixels with the strongest phase errors (~$\pi$ rad.) do not affect the direction (i.e., phase angle) of the resultant vector.

In addition to the vector averaging of the initial interframe phase difference shown in Figure 4b, another even more important feature of this method is that the vector representation is extended to finding the axial phase-variation gradients instead of the least-square of the vertical gradients used in [16]. The corresponding procedures in the vector form are illustrated in Figure 4c,d. First, as shown in Figure 4c, the inter-pixel phase difference is estimated for pixels separated in the vertical (axial) direction, with the vertical indices $j+k$ and $j$ (in the simplest case $k=1$, but if there is no phase wrapping on a scale of $k>1$ pixels, a larger $k>1$ may significantly improve the quality of the gradient estimation). In such a way, the matrix

$$d(m,j)=\overline{b}(m,j+k)\overline{b}*(m,j)$$

(7)

containing information about the sought vertical phase-variation gradients can be obtained, as illustrated in Figure 4c. Certainly, for k > 1, for estimating the phase gradient, the value $\arg [d(m,j)]/k)$ should be taken instead of $\arg [d(m,j)]$.

To further improve the SNR in the estimated phase gradients, additional averaging of the complex-valued vectors $d(m,j)$ over some processing area (window) can be performed as shown in Figure 4d. Then matrix $d(m,j)$ of the complex-valued vectors, for which the argument corresponds to the sought vertical gradients, can additionally be averaged over the chosen area to obtain even better SNR of the reconstructed strains.

Here, an important remark can be made about the preliminary averaging of the initially found phase-variation matrix $b(m,j)$. In real situations, for deformed tissues, the isophase lines corresponding to matrix $b(m,j)$ may be pronouncedly non-horizontal (see an experimental example of the phase-variation map in Figure 2). In more detail, such a situation is illustrated in Figure 5, showing especially simulated elastographic B-scans obtained for a deformed sample, in which strain-induced displacements are a superposed with translational displacements. The reference and deformed OCT scans are simulated using the model presented in [40]. This model can easily account for interframe motion of scatterers and describes the OCT scan formation for the often-used weakly focused illuminating beams with approximately depth-independent diameter. For such beams, the lateral inhomogeneity of the OCT beam phase can be neglected without appreciable worsening of the simulation quality, which was verified by comparison with a more complex and computationally demanding model [41] in which the beam focusing was rigorously accounted for. In the simulation, the following main parameters typical of OCT were used: the beam diameter was 15 μm, the central wavelength (in the tissue) was 1 μm, the spectral width was 90 nm, the B-scan depth was 2 mm and the width 4 mm. Initially, 65,000 scatterers with equal scattering strength were randomly distributed in the reference scan. For imitating measurement noises, random complex-valued numbers with a Gaussian distribution were added to every pixel of the structural scan, with the average intensity equal to the average intensity of the noiseless structural B-scan (i.e., the noise was rather strong, with SNR = 0 dB for the structural image).

To simulate the deformed complex-valued scan, the scatterers were displaced assuming that the strain in the three layers is somewhat different as shown in Figure 5(a4). The strain-induced displacements are superposed with pure translational displacements in the right half of the scan, the magnitude of these displacements gradually increases as schematically shown by the red curve in Figure 5(a4). These translational displacements do not affect the axial strain and axial phase-variation gradients, but cause bending of the isophase lines (see the interframe phase-variation maps in Figure 5(a1–a3). The raw (unaveraged) pixel-to pixel phase-variation map in Figure 5(a1) looks rather noisy because of the combined effect of “decorrelation noise” due to straining and the other superposed measurement noises. The quality of the phase-variation map can be improved using the vector averaging of matrix $b(m,j)$ over a pre-chosen area. However, in regions of inclined isophase lines the phase may vary fairly rapidly in the lateral directions, so that the positive effect of averaging of matrix $b(m,j)$ (like in Equation (6)) can be obtained only for sufficiently small areas with sizes ~several pixels.

This statement is clearly seen from the comparison of Figure 5(a1) showing the initially found map of pixel-to-pixel interframe phase variation with the averaged maps in Figure 5(a2,a3). Figure 5(a2) is obtained after vector averaging of matrix $b(m,j)$ over an area 8 × 8 pixels. After such averaging the left half with horizontal isophase lines in Figure 5(a2) looks much clearer, but the right-hand part with inclined isophase lines becomes strongly degraded because the chosen averaging area is too large. Comparison of the reconstructed strain maps in Figure 5(b1,b2) also shows the effect of such averaging in another form. Namely, Figure 5(b1) is directly obtained from the phase-variation map Figure 5(a1) using the vector estimation of axial phase gradients via Equation (7) with parameter $k=5$ for the vertical separation of compared pixels. Thus, Figure 5(b1) demonstrates that without any preliminary averaging the reconstructed strain is completely masked by noises. In Figure 5(b2) obtained for the preliminary averaged phase-variation map Figure 5(a2), the reconstructed strain becomes fairly well visible in the left-hand part of the scan where the isophase lines are fairly horizontal. On the contrary, in the right-hand side, where the isophase lines are pronouncedly inclined, the averaging does not give a positive effect because the averaging area 8 × 8 pixels is too big. Next, Figure 5(a3) shows the phase-variation map corresponding to averaging of vectors $b(m,j)$ over a fairly small area 3 × 3 pixels, which results in a moderate improvement of the phase-variation map in comparison with Figure 5(a1) over the entire scan (including the region of inclined isophase lines). The corresponding reconstructed strain in Figure 5(b3) is not completely masked in contrast to Figure 5(b1), but is still rather noisy. To improve the quality of strain reconstruction over the entire scan, after the preliminary small-area averaging of $b(m,j)$, the next procedure of averaging can be applied to complex-valued quantities $d(m,j)$ containing information about vertical phase gradients. When these gradients contained in matrix $d(m,j)$ are sufficiently uniform in the lateral direction, the averaging of $d(m,j)$ can be made over much larger areas than 3 × 3 pixels used for preliminary averaging of $b(m,j)$. The secondary averaging of vertical gradient can significantly improve the quality of the reconstructed strain map as shown in Figure 5(b4), corresponding to the strain map obtained after the preliminary small-area averaging of vector $b(m,j)$ (for 3 × 3 pixels) and subsequent averaging of $d(m,j)$ over a larger area (16 × 16 pixels). The so-obtained strain map shown in Figure 5(b4) with much better quality over the entire scan demonstrates high similarity with strain distribution adopted in the simulations (Figure 5(a4)).

It should be emphasized once again that for the simulations shown in Figure 5, a rather strong measurements noise with SNR = 0dB was imitated for the structural scans. This example illustrates the above-made remark that the vector method of strain estimation features very high tolerance to various noises, which is confirmed by the results of its application in various groups (see, e.g., [42,43]). In the following section we discuss in more detail the possibility to realize averaging in the vector approach without the use of conventional sliding windows.

2.7. Realization of the Vector Approach to Reconstruction of Local Strains without the Necessity of Local Calculations over a Sliding Window

As is clear from the above discussion, certainly the vector approach can be realized by performing local calculations generically similar to the use of a sliding window in the correlation-based approach or in the above-discussed least-square estimation of the phase-variation gradients. In contrast to those approaches, for which the use of sliding windows is indispensable, here we demonstrate that the vector approach makes it possible to map the local strains without any sliding windows using matrix operations applied to the entire OCT scans.

We recall that the first step of finding the matrix $b(m,j)$ has already been represented in a matrix form in Equation (5) as an element-by-element multiplication of the entire reference and deformed matrices $a_1^{*}(m,j)$ and $a_2(m,j)$. The next step shown in Figure 4b is averaging of the matrix $b(m,j)$ over an area with sizes $p\times q$ pixels. The result equivalent to averaging over a sliding window of $p\times q$ pixels can be obtained by performing summation of the entire matrix $b(m,j)$ with the same matrix $b(m,j)$ shifted laterally in steps by 1, 2, … p pixels. Similarly, the vertical averaging corresponds to summation of the laterally averaged matrix with its replicas shifted vertically by 1, 2, … q pixels. These averaging procedures are schematically illustrated in Figure 6.

For the entire matrix with a size of $M\times N$ pixels (where $M>>p$ and $N>>q$), this matrix averaging requires the summations $(M-p)\times (N-q)\times (p+q)\sim M\times N\times (p+q)$ to be performed. This should be compared with $(M-p)\times (N-q)\times p\times q\sim M\times N\times p\times q$ summations using the sliding window. For example, even for moderate sizes $p=q=4$, the matrix form requires twice smaller number of operations; for $p=q=8$ the reduction is already four times, etc. Certainly, this is achieved due to the trade-off between the required memory and number of operations, but for acceleration of computations, the reduction in the number of operations often is of key importance.

The following remark relates to the step shown in Figure 4c corresponding to the vector estimation of the axial (vertical) gradient. This operation is given by Equation (7) and corresponds to the element-by-element multiplication of matrix $\overline{b}(m,j)$ with its complex-conjugated replica vertically shifted by $k$ pixels. As mentioned in the discussion of Figure 4c, even after preliminary averaging, the phases of neighboring pixels in the averaged matrix $\overline{b}(m,j)$ may still be fairly noisy. Thus, the quality of the gradient estimation can be strongly improved if instead of comparing the neighboring pixels $\overline{b}(m,j)$ and $\overline{b}(m,j+1)$ one chooses a larger vertical separation, i.e., taking $\overline{b}(m,j+k)$ with $k>1$. The origin of this improvement in the phase-gradient estimation is clear: for vertically separated pixels, the estimated phase difference between their phases initially growth proportionally to the separation distance, whereas the level of masking phase fluctuations remains approximately the same. Consequently, the SNR for the estimate phase gradient initially increases proportionally to the separation parameter $k$.

Certainly, the chosen $k$ should not be too large to avoid phase wrapping on the vertical scale of $k$ pixels. If the vertical strain is $\epsilon$, then the absence of wrapping requires

$$\epsilon \cdot k\cdot d_z<\lambda /2$$

(8)

where $d_z$ is the vertical size of the pixel. The absence of wrapping, therefore, limits the maximal allowable strain to the following value for a given $k$:

$$\epsilon <\lambda /(2k{d}_z)$$

(9)

Let us assume a typical vertical size of the pixel $d_z=5$ μm and the wavelength $\lambda =1$ μm (which is close to the wavelength in the tissue for often used OCT sources with the central wavelength in vacuum ~1.3 μm). Then, for $k=1$, condition (9) yields $\epsilon <0.1$. For OCT, this is a huge strain that usually causes almost complete decorrelation of the initial and reference OCT scans. For vertical separation an order of magnitude larger, $k=10$, the phase wrapping limits the strain to a smaller, but still rather high for OCT, value of $\epsilon \sim 0.01$. In the presence of noises, the phase wrapping may occur for even somewhat smaller strains, so that somewhat smaller vertical separation can be recommended, say $k=5$ for interframe strains $\epsilon <0.005$, which is quite acceptable in many applications. For smaller strains, even larger separation can be used, so that the optimal $k>1$ for a particular situation can be chosen empirically to give a reasonable compromise between the degradation of the strain-mapping quality for insufficiently large values of $k$ (when the sought gradient is masked by measurement noises) and too large values of $k$ (for which the quality again degrades because of phase wrapping for $k$ pixels).

It should also be understood that the supra-pixel separation $k>1$ corresponds to averaging of strains over the vertical scale $\sim k{d}_z$, so that simultaneously with increasing SNR for a larger $k>1$, one reduces the vertical spatial resolution in comparison with the resolution of initial structural images. In practice, the vertical separation about several pixels may be a reasonable compromise between the improvement of SNR and decrease in the vertical resolution of strain reconstruction.

2.8. Real-Time Realization of the Fast Vector Approach without GPU Calculations

In this section, we describe the main steps/principles of the real-time realization of the above described fast variant of the vector approach in a multimodal OCT system, the main parameters of the basic OCT unit are described at the beginning of Section 2.2. Please note that earlier the elastographic modality for this device was based only on post-processing, although the implemented principles of parallel calculations described in [22] already made it possible to realize real-time angiographic imaging [44] using the same basic OCT system. Such a real-time realization of the angiographic modality made it possible the use of this system on patients [45]. The modified vector method of elastographic processing described in the previous section opened the way to realize the elastographic modality along with angiography in this OCT device.

We emphasize once again that the intended destination of this OCT device for a broad practical usage dictated that it should be affordable and convenient for biomedical users. Therefore, the utilization of fairly inexpensive solutions for the OCT setup itself, as well as for the controlling computer system was of key importance. In view of the above-mentioned requirements, the considered spectral-domain OCT system has flexibly orientable probe based on the use of single-mode isotropic fiber optics with use of the common path scheme. The spectral analysis of the interference signal is carried out using a spectrometer enabling a sequence of spectral readings, for which the correction of non-equidistance in the optical wavenumbers was performed using the optical-compensation approach described in [46,47]. Such a method of non-equidistance compensation is very important for the discussed real-time multimodal OCT system, because it allows one to avoid computationally demanding nonequidistant Fourier transform. This is in contrast to other generally used methods like [48,49] that require sufficient additional computing power.

A desktop or mobile general-purpose personal computer can be used as the main control and computing system. To connect the OCT system with a computer via the standard USB interface, a specially developed hardware/software package for data acquisition and control of the OCT system is used [21]. Such a signal acquisition through USB port is especially advantageous for creation of affordable multimodal medical OCT devices, bearing in mind that OCE modality enables significantly improved diagnostic quality in comparison to standard structural imaging, as described in recent work [50]. It should be emphasized that the synthesis and real-time visualization of OCT images (2D B-scans) comprise several sequential stages of non-trivial processing of the initial data consisting of a set of interference signal spectra that correspond to individual A-scans. These preliminary operations require significant computational expenses, so that to additionally realize real-time elastographic processing, one may use only the residual computational power.

Namely, in the first stage, procedures are performed to suppress artifacts caused by the non-identity of the radio-technical characteristics of the output channels that read the signals of the array of photodetectors linear scanning sensor [51]. In the second stage, transformations are performed to obtain a complex-valued analytical signal (equivalent to Hilbert transform) with two quadratures enabled by specific modulation of optical paths in the OCT setup, so that “mirror” artifacts and motion artifacts are suppressed using the algorithm described in [52]. Furthermore, calculations are made to compensate for the effect of material dispersion and non-uniformity of the spectral density of a broadband source of the illuminating optical beam. The next stage of calculations is intended for the final compensation of the residual nonequidistance of the recorded spectrum of the interference signal by means of a special, computationally efficient realization of Fourier transform for nonequidistant samples developed in [53].

An additional challenge in realization of this computational flow is that the processing of an intense continuous data stream is carried out simultaneously with the continuous data collection via the USB interface. Computing should be designed so that the CPU has enough time to support continuous data transfer controlled by the USB device driver under the control of a general-purpose operating system (such as Microsoft Windows). In the case of using only the central processor for computations in real time, the crucial role plays the method of multiple mutual synchronization of computational threads described in [22]. A separate computational thread realizes the procedures for the formation of a palette of pseudo colors and the visualization of the current B-scans in the user application window.

3. Results

The above outlined procedures of the initial formation of complex-valued B-scans and the developed variant of elastographic processing with increased computational efficiency made it possible to implement real-time elastographic visualization using only the central processor (such as Intel Core i7-8850H or AMD Ryzen 7 5800H) of the controlling computer as in the above-mentioned OCE-based study [50]. The basic parameters of this OCT system were indicated in Section 2.2. Figure 7 shows the graph demonstrating proportions among the computational times corresponding to the described stages of the OCT image synthesis and the elastographic visualization.

It is clear from Figure 7 that the proposed method of elastographic imaging was realized in real time using about 17% of the total computation time for each B-scan. We emphasize that the realization of the computationally efficient elastographic processing described in Section 2.7 has not required any supplementary computational means (such as multicore graphical cards or a more powerful central processor of the control computer.

As a result of the realized real-time elastographic processing, the following images are visualized: (i) current structural B-scan, (ii) map of pixel-to-pixel phase difference with the previous B-scan, (iii) map of reconstructed interframe strains, (iv) map of the Young’s modulus. The Young’s modulus map is obtained via comparison of local strains in the studied tissue and in the layer of reference material with precalibrated Young’s modulus (such layers are made of translucent soft silicone). The principles of use of reference silicone layers for quantification of the tissue elasticity in compression OCE were described in ample detail in earlier publications [8,32,33].

An example of the thus-obtained structural and elastographic images is shown in Figure 8 (the presented images are obtained for the skin at the dorsal side of the hand of a volunteer). We emphasize that these maps of strain and Young’s modulus distribution were obtained using only pair-wise comparison of scans that were processed on-flight using built-in parameters. This real time elastographic imaging is intended to play only an auxiliary role during the examination, because a series of scans can be saved and readily reprocessed to obtain elastographic images with much better quality. Anyway, such on-flight elastographic visualization is very helpful in the course of OCT examination. The correspondence of the elastographic images and skin structure is already quite clear for the presented example. A softer layer of epidermis is clearly seen between significantly stiffer stratum corneum and derma.

It can also be added that although the acquisition rate of the OCT system is rather modest, the human brain is not able to directly perceive the elastographic images refreshed in real time at a rate of 20 Hz, because their variability in the deformed tissue is much faster than for the more robust structural images (see, e.g., [55] for a discussion of such perception issues). In view of this, the speed of variability of interframe strain maps and corresponding elasticity maps is artificially reduced by applying sliding time-averaging over the last 10 frames.

4. Discussion

The discussed elastographic imaging in OCT is generically similar to the elastographic imaging performed by ultrasound medical scanners. However, significantly higher spatial resolution of elastography based on OCT opens qualitatively new prospects for numerous biomedical applications [1,2,3,4,5,6,7,8]. Probably the most well-known are applications related to improved differentiation between cancerous and normal tissue, for which OCE enables much higher contrast in comparison with structural (and even polarization-sensitive) OCT images [50,56]. In this context, much attention is paid to the acceleration of OCE visualization to enable intraoperative OCE examinations [57], as well as for application of OCE imaging in laboratory experiments with animals. Owing to the fact that spatial resolution of compression OCE corresponds to the scales of ~5–10 biological cells (~several tens of micrometers) new prospects for OCE-based morphological segmentation of tumorous tissues have been demonstrated for in vivo studies in [58]. The results of such OCE-based segmentation using the differences in the Young’s modulus for various morphological components of tumorous tissues demonstrate striking similarity with conventional morphological segmentation of histological images. However, unlike invasive, time consuming and laborious histological procedures, the OCE-based segmentation can be performed for freshly excised tissue samples and even feasible in vivo.

For many practical applications of OCE (such as detection of tumor-norm boundary), it is very desirable to obtain elastographic images with the same (or nearly the same) speed as structural scans. Until recently such real time OCE imaging was either impossible or required rather special computational facilities like in [17]. The elastographic visualization in compression OCE became much faster when instead of correlational speckle tracking initially proposed in [9] the use of phase information was proposed in [16]. In the phase-resolved approach, the necessity for search operations was eliminated, although the least-square estimation of local phase-variation gradients proposed in [16] still required local calculations over a sliding window.

The vector approach used here to estimate axial phase-variation gradients, in addition to exceptional tolerance to various noises, enabled further acceleration of OCE imaging. Namely, in contrast to various previously discussed methods of obtaining spatially resolved information, in this paper we demonstrate that the vector method can be realized via matrix operations applied to the entire OCE scans, which makes it possible to exclude both search operations and multiply repeated local calculations over a sliding window. From the previous sections, it is clear that the vector approach allows one to flexibly tune the main parameters of elastographic processing (such as sizes of averaging areas and scales over which the gradients are estimated). Depending on the characteristic strains, forms of isophase lines, and level of noises, the best results may be obtained for quite different parameters.

In this paper, we described the realization of real-time OCE visualization for an OCT system with a moderate rate of A-scan acquisition (20 kHz), so that for all stages of the OCT image formation including elastographic processing, the use of quite a “typical” central processor was sufficient. In the case of faster OCT systems, elastographic visualization based on the same principles certainly can be realized using GPU computations. For example, for a laboratory system enabling 80,000 A-scans/s, we realized real-time OCE processing using a middle-class video card (such as NVidia GTX, RTX). In this case, the required matrix operations and Fourier transforms were performed using the corresponding CUDA libraries. An example of a similar usage of GPU computations can be found in [59]. The efficiency of such signal processing simultaneously with the data acquisition via the USB 3.0 interface was demonstrated in [21].

Overall, the achievements in the development of OCT-based methods of high-resolution elastography have already demonstrated rich unprecedented possibilities of this new OCT modality for a broad range of biomedical (and not only biomedical) applications. In this context, the described computationally efficient vector approach that enabled possibility to realize on-flight elastographic imaging using quite “typical” OCT means and middle-class controlling computers are important for faster advancing of this very promising visualization technique towards wide intraoperative use in clinic and routine usage in laboratory studies.