An Edge Detection Algorithm for SEM Images of Multilayer Thin Films

Abstract

In processing multilayer thin film materials, scanning electron microscopy (SEM) is commonly employed for observation. In images of SEM, backscattered electron (BSE) images is particularly suitable for distinguishing different components and layers of the films. However, at high magnification levels, BSE images often have blurriness and noise, leading to low edge sharpness. This study proposes a method for improving the integrity and accuracy of the edges. First, we segment the image into different contrast regions using the masking algorithm. Second, we enhance the images in separate regions by the enhancement algorithm. Finally, we combine the regions by logical operations. In instantiation, we implement our approach on SEM-BSE images. It was found that the edges are significantly sharpened through the assessment of the edge evaluation algorithm.

1. Introduction

In the field of material science, multi-layer materials are composite structures consisting of two or more distinct layers arranged in a specific order. These layers can be composed of the same or different materials, and their thickness and arrangement can be tailored as per requirements. Multilayer materials typically exhibit superior properties compared to single-material counterparts, including enhanced strength, improved corrosion resistance, and superior thermal conductivity. Consequently, they find extensive applications in various industries such as aerospace engineering, automotive manufacturing, and electronic equipment production. Scanning electron microscopy (SEM) is a widely used surface analysis technique for characterizing these materials at the nanoscale level. Sikora [1] emphasized that obtaining reliable information about nanostructures within multilayer thin film materials in electronic devices primarily relies on acquiring both structural details and focused ion beam (FIB)/plasma focused ion beam (PFIB) cross sections from SEM images. Pandey et al. [2] employed image processing techniques to identify minute particles present on film surfaces using SEM images. Backscattered electron (BSE) imaging and secondary electron (SE) imaging are commonly utilized methods for visualizing SEM images [3]. BSE imaging involves generating backscattered electrons through energy loss from atoms or molecules on the sample’s surface under an incident electron beam; these electrons are then detected by a backscatter detector to form an image signal. On the other hand, SE imaging provides better morphological contrast of the original material surface while BSE imaging is typically employed to highlight regions with different components [4].

SEM images exhibit a suboptimal signal-to-noise ratio and limited definition at high magnification, particularly in the context of backscattered electron (BSE) imaging. Additionally, interference noise generated by other particles during image capture can further compromise graphic resolution and impede structural analysis [5]. Moreover, due to the distinctive nature of SEM images compared to general natural images, morphological changes in specific material areas can only be observed and analyzed through variations in grayscale intensity. Consequently, the boundaries between different regions on the material surface appear relatively indistinct, making it challenging to discern contrasting areas with clarity. Therefore, more specialized and systematic approaches are required to achieve improved recovery performance [6]. Henceforth, our objective is to mitigate noise levels, enhance image details significantly, and facilitate a comprehensive examination of intricate morphological alterations.

Currently, the prevailing approach for image processing involves employing deep learning methods to construct neural networks that facilitate tasks such as image feature extraction and noise reduction. While this method is efficient and accurate, it does present certain challenges. The primary issue lies in the reliance on the training set for constructing the neural network, which significantly influences the overall outcome of image processing. Typically, a large number of diverse images are required as a basis for the training set [7]. However, there are optimization algorithms available that mitigate this requirement by placing higher demands on specific image features within the training set and yielding favorable results only for particular image processing operations [8]. In regard to SEM image processing discussed in this paper, which encompasses noise reduction, enhancement, edge detection, and other operations, achieving desired outcomes with limited training sets proves challenging. Therefore, traditional image processing methods offer a more suitable choice for SEM images.

Image denoising is a well-established field with a wide range of available methods, and the primary challenge lies in identifying an effective technique for denoising images. Evaluating the performance of an image denoising algorithm typically involves using commonly employed standards such as the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) [8,9]. However, these standards serve as references and require manual confirmation to meet specific noise reduction requirements. The non-local means method has been experimentally proven to exhibit excellent performance in denoising SEM images of multilayer thin film materials [10]. Additionally, after obtaining a denoised image, it is necessary to enhance it further in order to emphasize variations across different regions effectively. Similarly, enhancement algorithms offer various approaches for modifying images to achieve visually satisfactory results. Unlike image denoising that benefits from reliable evaluation standards, image enhancement lacks precise assessment criteria. The choice of a specific enhancement method depends on the task at hand, characteristics of the image content, and observation requirements [11]. Following the aforementioned preprocessing steps, edge extraction is performed on SEM images. Edge detection serves as a fundamental operation in image processing aimed at identifying points or regions with significant changes in brightness or intensity that often correspond to edges within the image itself. By extracting edges from images, we gain better insights into the structural characteristics of battery panels and establish foundations for designing and optimizing multi-layer materials.

In the field of image enhancement, histogram equalization [12] is frequently mentioned and used as a method. However, despite being a classical approach, histogram equalization has certain limitations such as over-enhancement and insensitivity to low-frequency information [13]. Another classic image enhancement method is gamma transformation. Rahman et al. [14] proposed an adaptive gamma correction (AGC) method that effectively enhances sensitivity to darker pixels. Nevertheless, this method lacks consideration for neighboring pixels and fails to accurately reflect the true conditions of images under natural lighting [15]. Logarithmic transformation, although relatively limited in its application compared to other methods, can achieve grayscale value balance through nonlinear conversion [16]. In [17], logarithmic transformation is employed to enhance images while specifically focusing on observing darker areas.

Currently, there are diverse image enhancement methods, each with its own set of advantages and disadvantages. This article aims to explore the integration of different image enhancement techniques in order to achieve more comprehensive and efficient results. The primary focus of this study is on utilizing mask algorithms for image segmentation and applying corresponding enhancement methods to different segments, thereby obtaining clearer images. In the field of image processing, a mask is typically defined as a grayscale image that matches the size of the original image, where each pixel value determines whether it should be preserved or masked out. Techniques such as filtering, thresholding, and morphological operations are commonly employed for generating masks in image processing tasks. These generated masks can then serve as inputs for subsequent steps to accomplish specific effects or tasks. For instance, Dollar [18] transformed edge detection into a local mask segmentation problem using fast segmentation algorithms, while Polesel [19] utilized an improved masking algorithm for enhancing filtered images. Furthermore, there is currently a plethora of mask algorithms that integrate diverse image processing neural networks to optimize network performance. For instance, as demonstrated in [20], a mask algorithm based on semantic segmentation was implemented within the R-CNN network architecture. In essence, masks serve as invaluable tools in the realm of image processing by facilitating more efficient handling and comprehension of images.

After the aforementioned preprocessing steps, edge extraction is conducted on SEM images. Edge detection, being a fundamental operation in image processing, aims to identify points or regions exhibiting significant variations in brightness or intensity, which typically correspond to edges within the image. By extracting these image edges, a more comprehensive understanding of the structural characteristics of multilayer films can be attained and serve as a foundation for optimizing their design. Amongst various methods for edge detection, different operators such as the Sobel operator, Laplacian operator, and Canny operator [21,22] are employed to detect image edges based on changes in grayscale gradient.

The evaluation of image edges has always posed a formidable challenge, with various edge detection methods proposed in the literature [23,24]. However, these evaluation metrics heavily rely on ground truth edge information and are limited to synthetic images or overly simplistic real-world scenarios. Conclusions drawn from synthetic images cannot be easily extrapolated to real-world scenarios [25]. Another approach suggested in the literature is a subjective human-based evaluation method [26] for assessing the relative performance of edge-detection algorithms. In this paper, we adopt the use of an edge evaluation criterion [27] based on a lagged connection and predictive search-based edge detection method. This method leverages gradients and incorporates pixel gradient changes during the lagged connection process, enabling prediction of weak edges using local information to bridge gaps and detect fragmented segments of edges. Experimental results demonstrate that this metric effectively measures the performance of edge detection without relying on prior knowledge (no standard reference image), providing evidence for determining whether current results are suitable for subsequent image processing stages, ensuring reliable and efficient execution in subsequent stages.

This paper explores techniques for denoising, masking, enhancing, and extracting edges from SEM images of multilayer thin film materials. The study utilizes SEM-BSE images of solar panels as illustrative examples to assist in algorithm development. BSE images typically exhibit lower clarity compared to secondary electron (SE) images in SEM. Therefore, representative BSE images were selected for processing using the proposed method, which can also be applied to other grayscale levels of SEM images. Solar energy is currently one of the most widely used and popular green clean energies. Traditional solar panels rely on the photovoltaic effect where a voltage difference is generated between two semiconductor layers upon exposure to sunlight, resulting in current flow through an external circuit [28,29]. Hence, optimizing the performance of solar panels plays a crucial role in improving solar energy conversion efficiency. One approach to enhance their performance involves designing modern solar panel structures with an increased number of thin film layers that offer higher performance and economic benefits [30]. Additionally, analyzing the relationship between surface morphology features observed in SEM images of solar panels and their performance can provide valuable insights into cross-sectional morphology and structure [31], thereby facilitating improvements and optimizations. It is anticipated that this study will contribute useful tools and methods for further research and development related to multilayer thin film materials.

2. Materials and Methods

2.1. Original Image Analysis

The FIB-SEM double beam system (model: Auriga) from Zeiss was employed as the SEM equipment in this experiment. The ion source utilized gallium ions under a high voltage of 30 KV, and the imaging mode of SEM was ESB imaging. Figure 1 shows the original image, which was captured at a magnification of 50k× with a work distance (WD) of 5.0 mm. An extra high tension (EHT) of 2.00 kV was set for optimal performance. To effectively demonstrate the algorithm, a SEM image depicting a cross section of a multi-layer thin film solar panel was chosen as an illustrative example in this article. This representative SEM image exhibits distinct grayscale regions due to multiple materials present, non-smooth boundaries resulting from the coating process used during fabrication, and areas with significant grayscale contrast in its upper half region. However, it also encompasses regions with low grayscale contrast that pose challenges for edge detection when considering both types simultaneously.

Prior to conducting SEM image analysis, it is imperative to eliminate any interfering factors such as information bars and annotations that may impact the accuracy of the analysis. This process is commonly referred to as Region of Interest (ROI) selection in the field of image processing. To distinguish it from subsequent ROI selections, this initial step will be labeled as ROI 1.

Backscattered images exhibit distinct grayscale variations for different components, enabling the identification of regions with varying brightness and providing insights into the distribution of noise within the image [32]. When an image possesses low noise levels and clear contrast, the corresponding grayscale variation curve will appear smoother. In practical terms, a line can be drawn across the entire image to measure changes in grayscale along that line, allowing for a preliminary assessment of noise presence and approximate morphological features. This approach aids in evaluating whether post-denoising techniques meet our target requirements and assists in determining if post-denoised images fulfill our desired objectives.

A scan line was drawn on the original image to capture the grayscale variation curve along this specific line. Figure 2a depicts the precise placement of the line, while Figure 2b presents corresponding grayscale variations within the marked section of Figure 2a. Analysis of Figure 2b reveals significant changes in image grayscale occurring approximately at positions 260 px and 380 px on the x-axis, which correspond to areas where peaks and valleys intersect with the scan line. Furthermore, it is evident from Figure 2a that there is substantial noise in the image, necessitating denoising for further analysis.

2.2. Design of Noise Reduction Method

Multiple denoising methods have been selected for experimentation in this study to compare and determine the most suitable approach. These methods include bilateral filtering [33], mean filtering [12], median blur [34], and non-local means (NLM) filtering [35]. The image is subjected to these diverse approaches for noise reduction.

After conducting a comprehensive evaluation of the denoised images using commonly employed methods such as PSNR, SSIM, and grayscale curve analysis, two to three suitable images are selected for further processing. PSNR is primarily utilized to quantify the denoising effect, typically yielding results ranging from 10 to 30 dB. An optimal denoising performance is generally considered when the value exceeds 30 dB. Conversely, SSIM is predominantly employed to assess the similarity between the denoised image and its original counterpart, with values usually falling within the range of 0 to 1. A higher value closer to 1 indicates a greater resemblance in structural characteristics compared to the original image. By employing diverse approaches for denoising treatment on images, we have obtained final results as presented in

Table 1. Evaluation of denoise methods.

	Bilateral	Gaussian	Mean	Median	NLM
PSNR	31.064	32.439	32.181	30.922	31.922
SSIM	0.576	0.701	0.689	0.578	0.584
Run time	0.014 s	0.033 s	0.007 s	0.013 s	0.014 s

. In addition to the aforementioned data, we also recorded the duration (in seconds) for noise reduction, which is presented in the table.

Based on the PSNR and SSIM evaluation criteria presented in the aforementioned table, it is evident that all denoising methods exhibit effectiveness and yield commendable results. Gaussian filtering demonstrates superior indicators, followed by mean filtering, with the NLM algorithm ranking third. The grayscale variation curves obtained from these three denoising methods are depicted in the accompanying Figure 3, wherein NLM produces a smoother curve with virtually no noise, aligning with the findings of [10]. Consequently, we have selected the NLM algorithm as our preferred denoising method. It should be noted that, despite denoising efforts, residual noise still persists in the image.

2.3. Mask-Based Image Enhancement Algorithm

Due to the utilization of diverse materials in adjacent layers of multilayer thin film materials, the contrast exhibited in BSE images varies, leading to discrepancies in edge sharpness among different layers within the multilayer thin film. Consequently, both high-contrast and low-contrast regions are simultaneously presented in a single SEM image. To enhance edge detection performance, it is imperative to amplify the denoised image by accentuating distinct contrast regions and reinforcing edges. Logarithmic transformation proves effective in mitigating noise interference, particularly when Gaussian noise is present, as it diminishes noise disruption in high-frequency components and enhances image clarity. However, owing to its non-linear nature, logarithmic transformation may induce blurring of edges during brightness adjustment. If further enhancement of contrast is required subsequent to processing, supplementary operations may be necessary. In comparison, gamma transformation allows for more precise control over brightness and contrast levels; hence it is especially suitable for scenarios where the original image already exhibits very high or very low brightness and contrast levels. Nevertheless, gamma transformation entails intricate calculations and consumes considerable time; moreover, it does not effectively reduce noise-like logarithmic transformation.

In order to achieve optimal image enhancement, this article employs a mask division method to segment different regions and independently enhance each region. Subsequently, a logical algorithm is utilized to integrate the results from each region. Based on the aforementioned content, we have selected the logarithmic enhancement algorithm (Figure 4b) and Gamma algorithm (Figure 4a) for regional processing.

After logarithmic image enhancement, the edge detection results of the image (Figure 5) are minimally affected by noise interference, exhibiting distinct edges with reduced discontinuities. However, certain regions in the lower portion still exhibit undetected edges. In contrast, gamma transformation enhancement yields an image that is characterized by enhanced clarity and finer details in low contrast areas but suffers from more pronounced edge noise and incomplete recognition in high contrast regions.

Further adjustments were made to the gamma value in the gamma transformation with the aim of minimizing noise interference. By adjusting the gamma parameter to 0.02, the resulting image demonstrated comparable grayscale and structure as that achieved through logarithmic enhancement. Notably, the image exhibited clear edges and reduced noise levels, as depicted in Figure 6. Upon comparing it with images obtained using logarithmic enhancement algorithm, it can be observed that, after undergoing gamma image enhancement algorithm processing, finer details within specific patches of the image became more distinct, as indicated by red circles in Figure 7. However, there was also a slight compromise on detail preservation for larger patches, as highlighted by blue circles.

It was considered that two images can be blended to complement their limitations, thereby enhancing the overall image quality. To achieve this, we propose a region-based approach where gamma transformation is applied within a specified circular area, and logarithmic transformation is employed outside of it. This novel technique effectively combines the strengths of both transformations for superior image enhancement results. Specifically, our method utilizes a masking algorithm to generate a binary representation of the original image, with target regions depicted as white and non-target regions as black (Figure 8b). During the image processing stage, corresponding transformation operations are selectively applied based on the white regions in the mask while disregarding the black regions.

The corresponding-sized mask is generated by manually selecting the area within the red box (Figure 8a). Subsequently, a reverse mask is obtained through a logical NOT operation. Given that the images in this study exhibit relatively monotonous and simple characteristics without abundant color levels, image logical processing techniques such as AND processing and OR processing are considered. The region enclosed by the red box employs the gamma algorithm, while logarithmic enhancement algorithm is applied to other areas.

The specific procedural steps are as follows:

• Mask selection. A target area on the original image is manually selected (different shapes can also be selected according to specific needs). The relative coordinates of the selected area are recorded as Mask 1 (Figure 9a), while an inverse mask (Mask 2, Figure 9b) is generated through a logical NOT operation.
• Masking processing. This step is an optional operation, primarily aimed at improving the accuracy of image enhancement processing by reducing image complexity. To address the presence of black patches in the image, we initially employ grayscale thresholding to identify and extract black areas, resulting in Mask 3 (Figure 9c). Subsequently, a logical AND operation is performed between Mask 1 and Mask 3 to generate Mask 4 (Figure 9d). Finally, we apply another logical AND operation between Mask 2 and Mask 3 to obtain the desired result—Mask 5 (Figure 9e).
• Segmentation area. The image, enhanced using the gamma algorithm, is segmented based on Mask 4 and referred to as ’Mask Result Image 1’ (Figure 10a). Similarly, the image enhanced by the logarithmic operator undergoes segmentation with Mask 5 and is denoted as ’Mask Result Image 2’ (Figure 10b). The obtained results are illustrated in the figure. To perform the masking operation in the program, we utilized the open-cv library in Python to execute a min–max operation on the input image itself. This approach enables the convenient utilization of masks for extracting specific regions of interest from the image. Furthermore, applying a min–max operation on the single image does not affect the results obtained through gamma algorithm enhancement and logarithmic operator enhancement.
• Image restoration. The two resulting images from the previous step should be combined using a weighted addition operation, where corresponding grayscale pixels of the two images are added based on their respective proportions. The purpose of employing weighted addition is to mitigate conflicting backgrounds in the merged image (as exemplified on the right as an instance of conflicting image). The merging proportions for Mask Result Image 1 and Mask Result Image 2 are 0.41 and 0.59, respectively. Furthermore, it is necessary to readjust the contrast of Mask Result Image 1 by modifying its grayscale range, with adjustment parameters ranging approximately from 130 to 255 gray levels. After performing these operations, the final outcome will be further processed in the next section.

2.4. Image Edge Detection Algorithm

Edge detection is performed on the image obtained through the aforementioned enhancement algorithm (Figure 11b). For image edge extraction, we adopt the widely used and highly effective Canny edge extraction algorithm [21]. The principle of Canny edge detection for the enhanced image involves calculating its first and second derivatives of the gray function to detect edges. When applying the Canny edge detection algorithm, two thresholds are required to filter edges based on gradient amplitude in the image neighborhood. These thresholds should be determined according to specific characteristics of each image since they do not have fixed values. Research conducted by Jing et al. [36] demonstrates that Canny’s comprehensive edge detection criterion exhibits a unique response to edges, accurate localization, and robustness against noise interference. The resulting image is then fed into an edge image detection algorithm [27] for further analysis. The output from this step will be compared with previously generated edges to confirm if there is improvement in detecting more precise boundaries within the images. Finally, colorization is applied to highlight detected edges that are superimposed onto the original image, yielding a final result that enhances visibility and clarity of these boundaries. This process not only produces the ultimate outcome but also provides a means for evaluating whether detected edges are reasonable.

The complete algorithm flow chart is shown in Figure 12. In particular, the two denoises after ROI 1 in the flow chart prevent the flow chart from crossing, but they are the same operation.

3. Results

After the algorithm described in the previous section, we obtained the final result image. However, further exploration and research are needed for these result images. Firstly, we will apply edge evaluation criteria [27] to assess all edges.

3.1. Comparison of Results with Different Numbers of Steps

The algorithm [27] is an edge evaluation method based on lagged connections and predictive search. This method utilizes pixel gradient changes to establish lagged connections, and predicts the presence of weak edges between these connected pixels using local information. As a result, it can detect fragmented edge segments by bypassing interruptions in the edge detection process. A score closer to 1 indicates a higher level of excellence in edge effects. Based on this evaluation criterion as a reference, the optimal image is selected as the final result.

We performed edge detection on each image generated in the previous algorithmic process, followed by an evaluation of the image edges to validate the effectiveness of the proposed mask segmentation algorithm in this paper. These steps specifically include NLM denoising, gamma transformation (with a gamma value of 0.02), logarithmic transformation, and our algorithm (mask segmentation algorithm). Meanwhile, it also adds time monitoring for each individual step during execution. Based on the obtained results (Figure 13), it can be observed that, with an increase in steps, there is a gradual improvement in the evaluation indicators. The optimized image edges generated by this algorithm received the highest rating among all other methods evaluated. In terms of runtime, the algorithm takes longer mainly due to the time impact caused by manually selecting and masking areas. Specific data pertaining to these findings are presented in

Table 2. Perform edge detection in different steps and evaluate the obtained edges.

	NLM	Logarithmic	Gamma (Value = 0.02)	Our Algorithm
Score	0.425244	0.430102	0.441971	0.446945
Run time	0.303 s	0.009 s	0.017 s	12.211 s

.

3.2. Comparative Validation of Different Edge Detection Methods

This article employs the Canny operator for image edge detection and validates its rationality by comparing it with other operators, including the Sobel, Outline, Scharr, and Laplacian operators in Figure 11a, as shown in Figure 14. The evaluation results of different edges are listed in

Table 3. Result edge evaluation.

	Canny	Sobel	Outline	Scharr	Laplacian
Score	0.739579	0.647487	0.631331	0.634396	0.642871
Run time	0.0187 s	0.0854 s	0.0857 s	0.0771 s	0.0903 s

. Moreover, the execution time of each operator was monitored.

According to the analysis of the tabulated data, it can be concluded that the Canny operator demonstrates superior performance in terms of edge detection accuracy and program execution efficiency. Therefore, selecting the Canny operator as the preferred method for edge detection in this study is deemed justifiable.

3.3. Verification of Different Image Effects

This article further validates the algorithm by conducting an analysis on multi-layer thin film images with varying magnifications and diverse morphological features. The obtained results are presented in Figure 15, where each set of images comprises three pictures: Figure 15a,d represents the original image, Figure 15b,e indicates the selected mask position, and Figure 15d,f displays the processed edge detection image. The first set of images (Figure 15a–c) was captured at a 4K magnification, characterized by the presence of numerous small particles mixed in the middle layer of the image. The contrast in this area is relatively low, while the upper half exhibits a higher contrast. The second set of images (Figure 15d–f) was captured at a 50K magnification and has a relatively simple structure. It features a lower-contrast layer in the top left corner, while the center and lower half of the image display stronger contrast.

After image processing, the first set of images reveals a distinct hierarchical structure in the upper layer and effectively segments the granular region in the middle interlayer. Moreover, it accurately detects the boundary line of the lower layer. In the second set of images, while preserving the edges of the central area from the original image, it also successfully identifies the edges of the top left corner region. It can be observed that both sets of images exhibit smooth edges without noticeable noise interference, while precisely detecting and recognizing edges in regions with varying contrasts.

4. Conclusions

After conducting experimental testing, this article proposes an edge detection method based on mask operation to enhance the clarity of edges in SEM-BSE images of multilayer thin film materials. This method accurately captures material feature edges while reducing image noise and highlighting their morphological characteristics, thereby facilitating further research. Through analysis of PSNR, SSIM index, and grayscale variation curve, we selected non-local means filtering as the image denoising technique. Subsequently, a mask algorithm was employed to partition the image into regions and generate masks for low contrast regions and high contrast regions. For image enhancement purposes, gamma transformation was preferred for low contrast regions while logarithmic transformation was chosen for high contrast regions. The two processed images were then restored using weighted addition to obtain clear and comprehensive edges. Finally, a performance evaluation of the edge detection criteria was conducted using the Canny edge detection algorithm, which also involved evaluating the generated images during the algorithm process. Experimental results demonstrated that, with iterations of the algorithm, there was a gradual improvement in both edge clarity and relevant indicators. Furthermore, different edge detection algorithms were compared and tested, revealing that, among all aspects considered, the Canny edge detection algorithm proved most suitable. Lastly, extensive testing on various types of multilayer thin film images confirmed that this algorithm remains effective in producing satisfactory results.