Advanced Image Post-Processing Methods for Photoacoustic Tomography: A Review

Abstract

Photoacoustic tomography (PAT) is a promising imaging technique that utilizes the detection of light-induced acoustic waves for both morphological and functional biomedical imaging. However, producing high-quality images using PAT is still challenging and requires further research. Besides improving image reconstruction, which turns the raw photoacoustic signal into a PAT image, an alternative way to address this issue is through image post-processing, which can enhance and optimize the reconstructed PAT image. Image post-processing methods have rapidly emerged in PAT and are proven to be essential in improving image quality in recent research. In this review, we investigate the need for image post-processing in PAT imaging. We conduct a thorough literature review on the latest PAT image post-processing articles, including both general and PAT-specific post-processing techniques. In contrast to previous reviews, our analysis focuses specifically on advanced image post-processing rather than image reconstruction methods. By highlighting their potential applications, we hope to encourage further research and development in PAT image post-processing technology.

1. Introduction

Photoacoustic tomography (PAT) is a non-invasive, non-ionizing biomedical imaging technique that enables the reconstruction of the spatial distribution of photoacoustic pressure in the body. The photoacoustic effect [1], which underlies photoacoustic imaging, was first reported by Alexander Graham Bell in 1880. This effect occurs when a pulse, such as an optical or radio frequency wave, is absorbed by tissue. PAT offers high ultrasonic resolution and strong optical contrast in a single imaging modality. It is capable of providing high-resolution structural and molecular imaging in vivo by optically scattering biological media. By utilizing the photoacoustic effect, PAT overcomes the scattering of high-intensity light photons within biological tissues. As shown in Figure 1a, short pulses of laser-generated energy are directed to the tissue, generating thermal and acoustic pulse responses. The absorbed light is converted into heat, causing a rise in pressure due to the thermoelastic expansion of the irradiated tissues. The increased pressure is associated with localized energy injection and absorption from the light, along with other thermal and mechanical characteristics of the tissues, resulting in the generation of a photoacoustic signal. This signal is recorded using the tissue-facing side of an ultrasonic transducer, which is then amplified and digitized. A computational algorithm is used to process the PA signal, thus forming a PAT image.

The quality of PAT images is limited by the imaging hardware, signal acquisition and processing operations, and image reconstruction algorithm. The size and bandwidth of transducer elements are typically limited, and the geometry of the constituent arrays has a significant impact on the imaging results. The inverse model in the reconstruction algorithm is often an approximate and simplified description of the transducer array and medium properties. Imperfect illumination conditions and discrete data acquisition also affect the quality of the collected PA signal, ultimately impacting the fidelity in the resulting images. As a result of these factors, PAT images can be noisy and blurred or even distorted. Numerous methods were previously suggested to deal with the above challenges, including improved image reconstruction [2,3,4,5] and image post-processing methods. Specifically, as shown in Figure 1b, image reconstruction involves turning the raw signal into a desired image, while image post-processing entails enhancing and optimizing the reconstructed images. While a majority of previous research in PAT imaging was dedicated to image reconstruction [6], there has been a recent surge in studies focused on PAT image post-processing methods. As researchers delve deeper into the potential of the image post-processing technique, it is becoming increasingly evident that it plays a crucial role in enhancing the accuracy and reliability of PAT.

In this review, we aim to provide a comprehensive understanding of the usability and effectiveness of image post-processing methods in PAT. To achieve this goal, we first identified relevant articles through database screening and manual classification, and then analyzed the sources of demand for post-processing methods to determine future research directions. As shown in Figure 1c, we then classified the methods into two categories: general image processing techniques and PAT-specific processing techniques. General image processing techniques are common image-enhancement methods used to improve the quality of an existing image, such as denoising, deconvolution, segmentation, etc. In contrast, PAT-specific processing techniques are post-processing methods that are unique in terms of PAT imaging, i.e., light fluence correction, speed-of-sound correction, etc. We showcase the effectiveness of these methods by analyzing their theoretical advancement and experimental results. Finally, we discuss the potential of image post-processing methods in the field of PAT and suggest some future research directions. To the best of our knowledge, this study is the first article to discuss image post-processing as a specific topic for PAT imaging. Overall, this review aims to contribute to the development of image post-processing research on PAT imaging and foster new insights and ideas in this exciting and rapidly evolving field.

2. Material

We conducted a search in four databases—PubMed, Web of Science, IEEE Xplore, and Google Scholar—for articles published from 1 January 2010 to 1 March 2023. Three sections of keywords were used (photoacoustic* OR photo-acoustic OR Optoacoustic OR photo-acoustics OR photoacoustic* OR photoacoustic OR opto-acoustics) AND (tomography* OR imaging) AND (post-processing OR image domain OR image processing OR image analysis) NOT (image reconstruction) NOT (endoscopy) NOT (microscopy). The keywords in the three sections must appear at least once in the controlled indexing title, publication titles, or abstracts before we consider them as documents that satisfy the search criteria. Articles matching the topic were initially filtered according to the advanced search function.

To ensure that we focused on relevant studies, we used specific inclusion criteria to narrow down the list of records. Firstly, we limited our search to articles that discussed photoacoustic tomography as an imaging method, rather than topics related to PAT instrumentation, diagnostic procedures, contrast agents, or other peripheral areas. Secondly, we only included studies where the primary focus was on image post-processing, rather than signal processing or image reconstruction. Using these criteria, we aimed to identify studies that were most relevant to our review topic and provided meaningful insights into the post-processing of photoacoustic tomography images.

To identify relevant literature on image processing techniques for PAT, we first eliminated overlapping articles across the four literature databases. Next, abstracts of remaining articles were screened to further refine the list of relevant literature. Upon analyzing the articles, we identified two main categories for PAT post-processing methods: (1) general image post-processing and (2) PAT-specific image post-processing. By organizing the methods in this manner, we better observed their progress and compared them to similar methods.

3. The Importance of PAT Image Post-Processing

In an ideal PAT imaging scenario, a stationary object is illuminated by a high-power light source in an acoustically homogeneous medium. A transducer array with a wide bandwidth and detection angle is used to detect the PA signal. The image reconstruction algorithm must be robust, taking into account tissue properties while performing real-time operation. However, real-world conditions do not always satisfy these criteria. The cost and technical level of the imaging hardware reduces the accuracy of the acquisition process. Also, the biological properties of the tissue are obscured in the reconstruction algorithm by the crude estimation of the simplified imaging model. This issue leads to a deterioration in the reconstructed images’ quality. Therefore, to compensate for these problems, image enhancement using post-processing methods is required.

In this chapter, we discuss the various factors that can impact the quality of PAT images, including hardware limitations, tissue heterogeneity, and image reconstruction algorithms. We highlight the specific ways in which each of these factors can lead to a degradation in image quality, as shown in

Table 1. Main limiting factors of PAT imaging quality. Bulleted symbol (√) indicates correspondence between limiting factors (table rows) and manifestations (table columns).

		Structural Distortion	Spatial Aliasing	Negative Value	Reflection Artifacts	Clutter	Noise
Limitations in hardware	Poor illumination						√
	Limited view	√		√
	Limited bandwidth	√	√	√
	Sparse sampling	√	√
	Motion	√
Limitations in tissue	Optical attenuation						√
	Out-of-plane absorption					√
	Acoustic attenuation	√		√		√
	Acoustic heterogeneity	√		√	√	√
Limitations in algorithms	Inappropriate algorithms	√	√	√

, underscoring the importance of image post-processing techniques for improving the final result.

3.1. Hardware

We provide an overview of the hardware limitations in PAT systems, which generally include three essential components: illumination source, ultrasound transducer, and signal acquisition unit.

In PAT imaging, the intensity of illumination is restricted by safety guidelines, necessitating that the imaged tissue be irradiated within a standard range to meet the non-invasive imaging requirements. Excessive optical irradiation can cause tissue damage due to the absorption of excessive heat, whereas inadequate illumination can result in weak signals being obscured by electronic or thermal noise [7].

Although PAT imaging allows the illumination of tissues at a certain depth, the absorbing molecules at that depth have different characteristics from those at the skin surface. As a result of the light fluence, tissues at depth receive less light, resulting in weaker pressure waveforms.

The photoacoustic wave produced by the excited tissue is broadband, and the bandwidths of the actual transducer probes used for detection are limited. The emitted signals from large-volume targets tend to be at lower frequencies, while those from small-volume targets tend to be at higher frequencies. Given that the area of interest in PAT imaging often contains targets of varying sizes, the limited receiving bandwidth of the transducers is clearly inadequate to capture the entire photoacoustic signal [8]. This issue results in information being missing from the image [9], which may be seen as the hollowing out of the middle part or the loss of finer structures at the image boundaries (Figure 2A).

The appearance of the transducer element and the geometry of the array can significantly affect the received signal. The surface of the transducer element, as well as its geometry, determines its sensitivity angle, which limits its ability to respond to signals from all directions [11]. Even if arranged in a circular array to achieve a complete imaging field of view, there may still be blind spots in the near-edge field section. Furthermore, a closely spaced array can be costly, which is another important limitation to consider.

The accuracy of raw data in PAT imaging is also significantly affected by the acquisition process. Due to hardware cost constraints, the arrangement of transducer elements is usually sparse, causing insufficient spatial sampling [12]. This issue can lead to information being missing and affect the overall accuracy of the final image [13]. Furthermore, unexpected movements during acquisition, such as breath or cardiac pulsatile response, can cause relative displacements, resulting in misalignment and blurred image features (Figure 2B) [10]. This problem is due to the fact that the resolution of the hardware is not sufficient to observe continuous small movements. Therefore, it is difficult to align or compare images at adjacent locations during sequential acquisition, which may negatively impact the accuracy of the final image.

3.2. Tissue Heterogeneity

Except for the limitations in imaging hardware, the imaged object also affects the quality of the PAT image. One of the key challenges in PAT imaging is imaging depth, which is primarily limited by light attenuation [14] in tissue caused by absorption and multiple scattering. The laser illuminated at the object surface must penetrate biological tissue to reach the desired imaging depth. However, in this process, the light energy is gradually absorbed by the tissue and is not uniformly absorbed, resulting in a non-uniform and unknown distribution of light fluence [15].

In addition, the resolution of PAT imaging is affected by the return of the acoustic signal after optical–acoustic conversion. The acoustic signal is subject to attenuation and scattering during propagation, resulting in lower amplitude and a wider waveform of the high-frequency component. Consequently, the imaging resolution is reduced (Figure 3A) [16], as the high-frequency information of fine structures is lost and cannot be displayed in the image. Moreover, biological tissues are acoustically heterogeneous due to variations in the types and concentrations of constituent molecules, leading to differences in SOS (speed of sound) as a medium. At locations where there are boundaries with high acoustic contrast, incident sound waves generate acoustic reflections. These reflections overlap with the original image features to form acoustic reflection artifacts and can be mistaken for a real existing structure (Figure 3B) [17], which can greatly impact the accuracy of the reconstructed image.

In addition, when there are molecules with strong optical absorption outside the imaging plane of interest, the pressure wave signals they generate cannot be ignored. These spurious signals can even obscure the weak signal from deep tissue, resulting in spatial blending or clutter artifacts formed by the same transducer acceptance [18].

3.3. Image Reconstruction Algorithms

The recorded PA signal needs to undergo a mathematical transformation to form the final PAT image, which is referred to as image reconstruction. Ideally, the reconstruction algorithm should account for the PA wave generation and propagation physics. However, due to the uncertainty and complexity of the process, the reconstruction algorithm is often simplified and approximated, leading to a reconstruction error and resulting in low-quality reconstructed images.

Currently, common PAT image reconstruction algorithms include delay and sum (DAS) [6,19], back projection (BP) [20], and model-based (MB) [21,22] methods. Under ideal conditions, no additional processing steps are required for image reconstruction. However, in real-world PAT-imaging scenarios, algorithms possess distinct characteristics and may produce images with different levels of degradation [9,23,24], as shown Figure 4. Direct reconstruction algorithms, such as BP and DAS, can suffer from artifacts [13] and noise, resulting in low contrast and resolution of reconstructed images. Model-based algorithms can address some of these limitations by incorporating prior knowledge of the tissue and improving the physical acoustic wave propagation model; however, they are sensitive to errors in the model matrix and require higher computational cost.

4. General Image Post-Processing Methods for PAT

Image post-processing can effectively improve the usability of photoacoustic images and is a necessary step in many PAT applications. In this chapter, we will explore the common image post-processing methods that can enhance image quality and expand the range of applications of PAT imaging. These methods are summarized in

Table 2. A brief summary of general image post-processing methods for PAT.

Methods	Category	Advantages	Major Limitations
Artifacts suppression	Identify absorbers and artifacts [11,17,25,26,27,28,29]	Capable of handling artifacts stemming from multiple causes	Lack of real reference images for validation
Deconvolution	Estimation of PSFs [30,31]	Simple and intuitive	PSF estimation error
	Developing the model matrix [32,33]	Can be combined with image reconstruction algorithms	Expensive computation
	Data-driven [34,35,36]	Wide range of applications	Lack of labeled training datasets
Segmentation	Threshold [17,37,38,39]	Simple and efficient	Lacks robustness
	Boundary [38,40,41,42,43]	Consider small variations in tissue properties	Limited by low structural contrast of PAT
Multimodal imaging	PAT-US [28,44,45,46,47,48,49]	High adaptability	Noisy US image
	PAT-MRI [50,51,52,53,54,55,56,57,58,59]	High structural contrast provided by MRI	High cost/difficult to integrate
	Multimodal image fusion [60]	Abundant information from other imaging modalities	Difficult image registration

.

4.1. Artifacts Suppression

PAT imaging artifacts [13,61,62] include negativity artifacts, streak artifacts, splitting artifacts, reflection artifacts, etc. They not only obscure important information, but also hinder the implementation of quantitative image analysis. Image post-processing methods were proposed to suppress these artifacts.

As for artifacts from strong optical absorption, Nguyen et al. [17] proposed a method to detect and eliminate in-plane reflection artifacts in multi-wavelength PAT imaging, as shown in Figure 5A. The approach involves segmentating and estimating their spectral responses using the remaining images to identify and group features in the clearest image as the real absorbers. Fainter features appearing at greater depths are considered to be reflection artifacts and are removed by setting the corresponding pixel values to zero. Jaeger et al. [28] proposed a distortion compensation (DC) method to reduce artifacts by applying a moving time average to the PA image sequence, as shown in Figure 5B. The signal from the light absorber located in the image plane persists throughout the PA sequence and is, therefore, unaffected by the averaging, while the de-correlated clutter is reduced to improve the contrast-to-clutter ratio (CCR).

For motion artifact correction, Erlov et al. [29] conducted a study to explore the applicability of a regional motion correction algorithm that enables the interpretation of internal tissue motion in handheld 2D PAT. As shown in Figure 5C, the technique leverages intensity phase tracking (IPT) of interlaced ultrasound images that are co-registered with the PAT images to avoid the motion artifacts.

For negativity artifacts, Shen et al. [11] investigated the formation mechanisms and evaluated two post-processing approaches to address them. As shown in Figure 5D, the first approach, which was forced-zeroing, involved setting negative values to zero. The second approach, which was envelope detection, involved reversing the negative components of the image and using Hilbert transform to extract amplitude profiles.

Moreover, deep learning (DL) shows promising results for PAT image artifact suppression. Different types of deep convolutional neural (CNN) architectures were previously applied and tested in various studies, such as U-net [25,26,63], Fully Dense (FD)-Unet [64], simple CNN [65], and generative adversarial network (GAN) [66]. One example was provided by Allman et al. [27], who used a CNN network to locate and classify source and reflection artifacts in the traditional beamformed image. The network was trained on simulated data and transferred to experimental images. Among them, most of these methods use simulation datasets created using k-Wave [67] for network training, only [25] using real PAT images acquired via a dedicated PAT imaging system [25]. In addition, ref. [26] also used an online dataset (the DRIVE database [68]) for simulation.

Due to the complex conditions in real-world PAT imaging, it is not possible to fully identify the multiple above-mentioned types of artifacts and remove them completely. Therefore, a standardized evaluation tool for artefact suppression performance will facilitate subjective verification, especially for real PAT images.

4.2. Deconvolution

Due to the signal detection geometric, the reconstructed PAT images usually suffer from the loss of resolution and contrast. In addition, the signal generated by strong absorbers outside the imaging plane degrades the resolution of PAT images, and the resulting artifacts exhibit anisotropy [69,70]. In terms of improving image resolution, one of the most effective solutions is image deconvolution, which aims to potentially recover clear results from the degraded images.

PAT imaging can be modelled via the convolution of the point spread function (PSF) of the imaging system with the underlying absorption image. The PSF of an imaging system can be determined through experimental measurements using objects that are significantly smaller than the imaging system’s resolution. Qi et al. [31] suggested utilizing measured spatially variant PSFs for PAT image restoration. By acquiring PSF data from a specific PAT system, a high-quality PAT image can be generated using a maximum a posteriori (MAP) framework, which enhances both the resolution and overall image quality, as shown in Figure 6A. In the case of unknown PSF, Jetzfellner et al. [30] conducted photoacoustic measurements on six-day-old mice using a broadband hydrophone in a circular scanning configuration and near-infrared imaging. The results indicate that the resolution and contrast of the images can be improved through blind deconvolution, which provides estimates of the unknown PSF. However, the blind deconvolution method depends heavily on the accuracy of the estimated PSF; therefore, its performance is usually not as good as that of image deconvolution with measured PSF.

In addition, a model matrix for deconvolution can be used as the step following image reconstruction. As shown in Figure 6B, Nakshatri et al. [33] proposed a two-step model–resolution matrix-based deconvolution approach to improve the reconstruction image quality. The model–resolution matrix was developed in the context of different penalty functions, such as Quadratic and Geman–McClure. In addition, Awasthi et al. [32] proposed to use a basis pursuit deconvolution method, which also includes a model resolution matrix to improve Total Variational (TV) regularization results, as shown in Figure 6C. However, model-based deconvolution is usually computationally expensive, and the hyperparameters used during iterative optimization require empirical fine-tuning.

Moreover, image super resolution techniques can be applied to PAT, for example, multiple PAT images can be integrated into a single high-resolution image after alignment using optical flow estimation [36]. Learning-based methods were also introduced to PAT image deconvolution. For example, Deep-E, which is a fully dense neural network, provides improved elevational resolution by only using the 2D slices in the axial and elevational plane [35]. For network training, Deep-E uses the 2D images generated via K-Wave [67] as input data, and ground truth vascular images are generated using the Insight Segmentation and Registration Toolkit (ITK) [34]. This work reveals the potential of DL-based deconvolution methods in PAT image post-processing.

4.3. Segmentation

Image segmentation is a commonly used image processing technique that separates the area of interest from the background. In PAT imaging, it is particularly useful for images where the target object and background exhibit weak contrast. It can be used for problems such as light fluence correction, artifact removal, identification of regions of interest, SOS correction, etc.

Threshold segmentation is a widely used algorithm [17,37,38,39]. For example, as shown in Figure 7A, Liang et. al. [38] extracted a rough body region of the animal through thresholding. However, local threshold segmentation based on pixel amplitude usually has poor precision. To solve this problem, several automatic global threshold segmentation methods are proposed to segment and describe the boundaries of biological tissues. Nguyen et al. [17] used the automatic segmentation, as shown in Figure 7B, which is based on the Sobel edge detection algorithm by applying a threshold value, to detect image features. As shown in Figure 7C, Khodaverdi et al. [39] presented an automatic threshold selection (ATS) algorithm that can accurately distinguish targets from the background in adaptive matched filter (AMF) detection images. In another application, Raumonen et al. [37] proposed a vessel segmentation technique in a probabilistic framework by utilizing image voxels and vessel clustering, as shown in Figure 7D. This method achieved more robust and accurate segmentation by calculating the probability of each voxel belonging to blood vessels.

In addition, Mandal et al. [43] employed the segmented mask to construct a two-compartment active contour model for boundary segmentation. Liang et al. [38] proposed an automatic 3D segmentation method for PAT images based on an optimal 3-D graph search. These methods model the segmentation target based on its geometric properties and, therefore, achieve good results for specific tissue. Recently, since the advance of DL technology, PAT image segmentation methods based on deep neural network received increased attention [40,41,42]. For example, Chlis et al. [40] proposed the Sparse-UNET (S-UNET) method to obtain a segmentation mask for automatic vascular segmentation using real-world PAT images labeled by a human annotator. Zhang [71] designed segmentation software that identifies the six grades of breast cancer, which used transfer learning for deep classifier training. Other authors trained their network model using images generated via the MCXYZ program [72]. Compared to traditional methods, these DL-based approaches are shown to be able to better handle low contrast, noise, and complicated artifacts in PAT images.

4.4. Image Post-Processing for Multimodal Imaging

PAT can be integrated with conventional medical imaging modalities, such as magnetic resonance imaging (MRI) and Ultrasound (US) imaging. This multimodal imaging strategy can provide complementary information about the imaging tissue. However, different imaging modalities have their own characteristics, and thus, post-processing of the images is often necessary to improve the quality of the images. Various image post-processing techniques can be employed for PAT-based multimodal imaging, such as feature extraction, image registration, image fusion and so on.

For multimodal PAT-US imaging, much research on image post-processing have been proposed [28,44,45,46,47,48,49]. PA-US imaging enables high quality multimodal images for a wide range of applications, where the alignment and coupling of the two modalities depends heavily on the application of post-processing techniques. As shown in Figure 8A, Jaeger et al. [28] proposed a deformation compensation method to achieve registration of features between PAT and US images acquired from a commercial medical ultrasound imaging equipment. Kim et al. [47] used a dual-modal US-PA contrast agent. The US images are used to construct a masking image that contains the location information about the target site and is applied to the PAT image acquired after contrast agent injection. Han et al. [44] proposed a 3D modeling method to calculate the optical fluence distribution based on a dual-modality PA/US system, and then used this information to compensate the PA imaging results.

For multimodal PAT-MRI imaging of rigid parts, such as the animal head [50,51,52,53,54,55,56,57,58,59], Ren et al. developed a toolbox “RegOA” [50] and proposed a fully automated registration method for PAT-MRI multimodal brain imaging empowered by deep learning [53]. The datasets for network training were acquired in the PAT system and MRI scanner experimentally. However, they only applied image registration to rigid areas, such as the animal head, but not to abdomen. To improve image registration of both body and tumor contours between PAT and MRI, Gehrung et al. [54] combined a novel MRI animal holder and a landmark-based software co-registration algorithm for deformable tissues, which achieved the first co-alignment of soft tissue. However, the custom silicone MRI holder cannot be reused and has relatively low registration accuracy. As shown in Figure 8B, Zhang et al. [56] developed a novel dual-modality animal imaging bed to achieve dual-modality successive data acquisition and co-registration of PAT and MRI data in in vivo imaging applications. Based on their design, they then employed an automated rigid image registration algorithm for PAT and MRI, as well as proposing a PAT image restoration technique that used MRI information as guidance [59]. They achieved robust whole-body MRI-PAT image registration, and their registration tool is reusable. Finally, an attempt was made to integrate a PAT imaging system into a MRI scanner, with Chen et al. [58] developing a parallel hybrid magnetic resonance and optoacoustic tomography (MROT) system to acquire MRI and photoacoustic signals simultaneously. Based on the MROT system, they proposed a tailored data processing pipeline to register MRI image volumes acquired using different sequences onto the corresponding vascular and oxygenation data recorded via PAT.

For other PAT-related dual-modality imaging methods [73,74], Ni et al. [73] introduced a multiscale optical molecular imaging approach combining fluorescence microscopy and multispectral PAT. It enabled unique transcranial imaging capacity, single-plaque resolution, and non-invasive real-time visualization of the entire mouse brain. Moreover, based on real-time volumetric PAT and electrophysiological recordings, Gottschalk et al. [75] visualized real-time thalamocortical activity non-invasively.

For multimodal image fusion, Park et al. [60] proposed a fusion imaging technique in which overlapping PA, US, and MR images could be displayed concurrently in real time via co-registration of pre-acquired MR and real-time PA/US images. In this work, an image fusion algorithm was proposed to seek the spatial relationships between the MR volume and the PA/US image by performing rigid transformation.

Multimodal imaging requires the coupling of different imaging modalities, as well as quantitative analysis [76]. To achieve this goal, image post-processing techniques must continue to play an important role in the rapidly developing field of PAT multimodal imaging.

5. PAT-Specific Image Processing Methods

PAT-specific image processing methods are proposed to deal with image degradation problems that are unique to PAT imaging techniques. Accounting for the optical and acoustic properties of the imaging tissue, these methods include light fluence correction, acoustic correction, and spectral unmixing. With these techniques, researchers can better understand the structure and function of biological tissues and improve the accuracy and effectiveness of PAT imaging.

Table 3. A brief summary of PAT-specific image post-processing methods.

Methods	Category	Advantages	Major Limitations
Light fluence correction	Model-based correction [77,78,79,80,81,82,83]	Easy implementation	Sensitive to optical property uncertainty
	Data-driven correction [84]	Accounts for real-world variation	Requires ground truth data
	Segmentation-based correction [38,59,84]	Considers optical properties of heterogeneous tissue	Limited by segmentation accuracy
Acoustic correction	Single SOS selection [85,86,87,88,89]	Easy implementation	Assumes a uniform SOS
	Heterogeneous SOS correction [90,91,92,93,94]	High accuracy	High computation cost and low algorithm stability
Spectral unmixing	Linear unmixing model [95,96,97,98,99]	Easy implementation	Requires known absorber composition
	Blind-source spectral unmixing [100,101,102,103,104]	Accounts for unknown biological tissue composition and spectral characteristics	Unstable performance

gives a brief summary of these methods.

5.1. Light Fluence Correction

Light fluence correction refers to the process of correcting for image variations of light fluence deposited in tissue, which can occur due to the tissue absorption and scattering along the light path. Light fluence correction is important for accurately quantifying the distribution of optical absorption properties in tissue, which is a key aspect of PAT imaging.

A number of mathematical and numerical models are proposed for light fluence correction. For example, optical propagation models [82,83], fixed-point iterative methods [80,81], logarithmic unmixing of multispectral datasets [79], and measurement-based methods [77,78] were used. Zhou et al. [82] developed a real-time correction algorithm based on application of the diffusion dipole model (DDM), which simulates the fluence distribution as the responses to a pair of point sources produced using a collimated pencil beam in a semi-infinite turbid medium. Deán-Ben et al. [77] used the switching kinetics of reversibly switchable fluorescent proteins (RSFPs) to correct for the dynamic light fluence distribution deep in scattering media.

In addition, data-driven approaches are proposed to compensate for the non-linear light fluence distribution. Madasamy et al. [84] trained DL models, such as U-Net, FD U-Net, Y-Net, FD Y-Net, Deep residual U-Net (Deep ResU-Net), and GAN, with blood vessel and numerical breast phantom datasets to evaluate the performance of optical absorption coefficient recovery. They accumulated datasets from different online repositories, such as Kaggle [105], the retinal fundus multi-disease image dataset (RFMID) [106], the optical and acoustic breast phantom database [107], and the 3D acoustic numerical breast phantoms [108]. However, all above supervised learning methods may suffer from the lack of ground truth for network training. To address this problem, as shown in Figure 9A, Li et al. [109] proposed quantitative optoacoustic tomography (QOAT)-Net, which is a dual-path convolutional network, to estimate the absorption coefficients after training with data-label pairs generated via unsupervised “simulation-to-experiment” data translation.

Moreover, the automatic extraction and segmentation of PAT images can be crucial in improving image analysis efficiency and enabling advanced PAT-specific image post-processing, such as light fluence correction. As shown in Figure 9B, Liang et al. [38] presented a volumetric method for estimating light fluence in PAT images. The method employs the 3-D optimal graph search (3-D GS) algorithm, which takes into consideration the continuity among image slices for volume image segmentation. Using the 3D segmentation results of the entire animal body, the simulation volumetric light fluence distribution can be obtained to correct PAT images. To improve the accuracy of light fluence estimation, Brochu et al. [110] suggested organ-level segmentation for performing light fluence correction on regions with distinct optical properties. However, the poor soft tissue contrast of PAT images poses a challenge that hinders accurate organ segmentation. As shown in Figure 9C, Zhang et al. [59] introduced a method for estimating and correcting light fluence in PAT using guidance from MRI structural information. The method involves segmenting the registered MRI image obtained from a dual-modality imaging approach and using the segmentation result to guide the estimation of light fluence distribution. In fact, organ-level segmentation on MRI images is easier than on PAT images and, therefore, has higher segmentation accuracy. This approach also leads to the improved accuracy of light fluence correction.

5.2. Acoustic Correction

Tissues are acoustically heterogeneous, meaning that there is variation in the distribution of SOS among different tissue types. The acoustic properties of tissue surfaces can also vary significantly, resulting in reflections of acoustic waves [111,112]. However, during image reconstruction, the algorithm typically assumes a constant and uniform SOS distribution, which contradicts the acoustic heterogeneity of tissues. This issue can lead to artifacts and spatial aliasing, as well as structural distortion, due to incorrect algorithmic assumptions [113,114].

Image distortion caused by heterogeneous tissues can be suppressed by choosing an optimal SOS. As shown in Figure 10A, Treeby et al. [85] employed the autofocus approach to choose a SOS optimized for visual assessment of image reconstruction. In this approach, the SOS used for image reconstruction is manually adjusted to maximize the sharpness of prominent image features. This approach is intuitive, though it is prone to noise and artifacts that produce similar high-intensity image features. Jeon et al. [89] introduces a DL-based end-to-end SOS correction algorithm that uses eight different SOS images as training inputs [4]. Dehner et al. [88] propose passing the adjusted input SOS value to the trainable network layers to improve image fidelity of deep-learning-based PAT reconstruction, using the k-wave toolbox [67] to obtain labelled datasets by employing a diverse collection of publicly available real-world images as the initial pressure distribution. However, data-driven approaches for acoustic correction primarily rely on high-fidelity simulated data and have not yet been successfully applied to an in vivo imaging study.

Automatic selection of SOS [85,86,87] is a straightforward but limited approach that assumes uniform velocities throughout the tissue. In contrast, partitioning different tissue regions is helpful for correcting their inhomogeneous SOS distributions [43,115,116,117]. Lafci et al. [115] demonstrated the use of inhomogeneous SOS maps for image quality enhancement. The image was segmented manually into two regions: foreground (representing the mouse body) and background (representing the surrounding coupling medium). In general, segmentation-based SOS correction methods require the knowledge of tissue/organ distribution and have certain requirements for segmentation accuracy; however, in terms of image quality, it can significantly improve correction performance.

In addition, joint correction approaches were proposed to simultaneously extract optical absorption information and SOS distribution [90,91,92,93,94]. As shown in Figure 10B, Cai et al. [90] proposed a method to jointly reconstruct the SOS and PAT image. They updated the SOS distribution by maximizing the similarity of images from the partial arrays and then performing image reconstruction. The simulation data for network training were generated using k-wave [67]. Shan et al. [91] proposed a simultaneous reconstruction network (SR-Net) to update the initial pressure and sound speed iteratively. Even though it may suffer from numerical instability, this method still significantly improves PAT image quality. Also, the SOS distribution can be obtained via experimentally measuring [118,119,120] using additional techniques. Xia et al. [120] incorporated ultrasonic computed tomography (USCT) into the full-ring array PAT system to measure the SOS distribution within the object. In addition, the SOS correction model used in this USCT-enhanced method was obtained through experiments and, therefore, is more accurate than simplified assumptions.

5.3. Spectral Unmixing

In multispectral imaging, as tissues are excited at two or more wavelengths, it is possible to resolve the distribution of various tissue molecules or biomarkers by unmixing the images based on their spectral signature.

Studies on the quantification of the concentration of single absorbers were conducted. The linear unmixing model is the most widely used model [99]. Xia et al. [98] proposed a dynamic method, which showed that the PA amplitudes measured at different sO₂ states canceled the contribution from optical fluence and allowed calibration-free quantification of absolute sO₂. Tzoumas et al. [97] found that the statistical sub-pixel detection methods can focus on the detection of a unique spectral target with up to five times enhanced sensitivity compared to linear unmixing approximations. Tzoumas et al. [96] introduced eigenspectra multispectral optoacoustic tomography (eMSOT) to account for wavelength-dependent light attenuation, estimating blood sO₂ within deep tissue with substantially enhanced performance compared to previous methods, as shown in Figure 11A. Despite the quantitative advantage offered by eMSOT, both the non-convex nature of the optimization problem and the possible sub-optimality of the constraints may lead to reduced accuracy. To address this issue, Olefir et al. [95] presented a neural network architecture composed of a combination of recurrent and convolutional layers, which improved the accuracy of sO₂ computation by directly regressing from a set of input spectra to the desired fluence values. In their work, the input data for network training are generated via simulation experiment.

The biological tissue composition and its accurate spectral characteristics are not always known during spectral separation, leading to the emergence of blind or semi-blind source spectral unmixing techniques. As shown in Figure 11B, Glatz et al. [104] investigated principal component analysis (PCA) and independent component analysis (ICA), which have no a priori information, in separating specific absorbers and backgrounds. Compared them to spectral fitting schemes, this method is shown to be a promising alternative technique for separating mixed components. Deán-Ben et al. [103] suggested using vertex component analysis (VCA) for a semi-blind unmixing of multispectral optoacoustic data, which include a priori information on the spectral signatures of absorbers. In the case of semi-blind spectral unmixing, the standard version of the VCA algorithm can attain similar sensitivity to the PCA-ICA approach and has a faster performance. Deán-Ben et al. [79] also present a blind unmixing approach capable of correcting for wavelength-dependent light fluence variations using a logarithmic representation of the images taken at different wavelengths. An et al. [102] applied an approximate fluence adjustment, which was based on spatially homogeneous optical properties equal to those of the background region, to the PAT images before accurate separation of the chromophores via ICA, further reducing the unmixing error. The effect of retaining different numbers of dimensions for ICA was also demonstrated [101]. For DL-based methods, Durairaj et al. [100] introduces a dual autoencoder neural network architecture designed to estimate the end-member spectra (wavelength-dependent absorption coefficient) and the abundance maps (unmixed images) of the constituent molecules. Their method accounted for the non-linearities present in the mixing models and reduced the dependency on linear unmixing.

Several factors may affect the effectiveness of spectral unmixing [121,122,123,124]. Dolet et al. [122] took into account both the spatial neighborhood and the spectral behavior of the pixels used for multispectral PAT image clustering, which can significantly improve the accuracy of estimations of the concentration. Tzoumas et al. [123] studied the impact of the number of excitation wavelengths employed for the sensitivity and accuracy, finding that the unmixing sensitivity exhibits a statistical increase trend with respect to the number of wavelengths employed. Moreover, negative values may cause significant artifacts, which also can make fine structures invisible to spectral unmixing techniques. To overcome this issue, Ding et al. [121] imposed non-negative constraints solely onto the optical absorbers of interest, thus avoiding negative pressure values. Taruttis et al. [124] proposed a technique that employs stationary wavelet decomposition prior to non-negative spectral unmixing. This method offers a systematic and automated approach for transforming images at multiple excitation wavelengths into multiscale representations of specific chromophores, thus facilitating the identification of hidden structures and reducing the effects of negative values on imaging results.

6. Discussion

In the above literature review, it was proved that image post-processing techniques are able to significantly enhance the quality of PAT images. Ranging from artefact suppression and image deconvolution to PAT-specific SOS correction and light fluence correction, image post-processing methods for PAT are starting to create a solid foundation for the development of advanced PAT imaging technology, as well as for the extension of pre-clinical and clinical applications.

Regarding the aim of improving PAT image quality, rapid developments in the field of computer vision provided novel ideas that may be introduced to PAT. For example, deep generative models, such as the generative adversarial network [125], can adapt to different downstream tasks, such as super-resolution [126,127], denoising [128], etc.. They may be used to perform PAT image enhancement in an unsupervised manner. Moreover, advance segmentation models, such as segment anything [128], can be introduced to perform more accurate PAT-based tissue segmentation. In the absence of sufficient-labeled datasets, self-supervised networks [129,130] that are able to learn from the input test image itself can be introduced to PAT image post-processing tasks, such as deconvolution.

Moreover, hardware systems for PAT imaging are constantly evolving. This fact creates new technical challenges for PAT image post-processing. For example, advanced imaging strategies were previously proposed for ring-shaped PAT, such as the interlaced sparse sampling PAT [131,132], which employs a rotational scanning scheme. Also, there are an increasing number of studies conducted based on 3D PAT systems, such as spherical array PAT [133,134,135,136,137,138]. In these systems, special care must be taken to efficiently and accurately recover high-quality PAT images.

Finally, the development of image post-processing techniques for PAT is still in its early stage. Current research is usually conducted using private datasets with custom codes. Therefore, standardizing the image processing operations [139] is expected to address the differences between studies and facilitates the unification of various PAT-specific processing techniques. Moreover, DL-based methods emerged as promising PAT image post-processing tools, and the datasets used for network training play a big role as the basis of these methods. Several public datasets for PAT imaging were previously released and made available for use [108,140,141,142]. Future research on DL-based methods performed on these datasets will facilitate fair performance comparison.

7. Conclusions

Following the exploration of the importance of image post-processing for PAT imaging, this review summarizes existing works that employ advanced image processing methods to enhance the quality of PAT images. We divide these studies into two major categories—general image processing and PAT-specific processing—to facilitate a comprehensive survey of recent technical advancements. We anticipate that this review will inspire subsequent research on PAT image processing and encourage innovation on this topic to advance PAT imaging technology.