A Review of Document Binarization: Main Techniques, New Challenges, and Trends

Abstract

Document image binarization is a challenging task, especially when it comes to text segmentation in degraded document images. The binarization, as a pre-processing step of Optical Character Recognition (OCR), is one of the most fundamental and commonly used segmentation methods. It separates the foreground text from the background of the document image to facilitate subsequent image processing. In view of the different degradation degrees of document images, researchers have proposed a variety of solutions. In this paper, we have summarized some challenges and difficulties in the field of document image binarization. Approximately 60 methods documenting image binarization techniques are mentioned, including traditional algorithms and deep learning-based algorithms. Here, we evaluated the performance of 25 image binarization techniques on the H-DIBCO2016 dataset to provide some help for future research.

1. Introduction

Image binarization is an important aspect of image analysis, such as scene text detection [1,2,3] and medical image analysis [4,5]. Especially in the field of document image processing, binarization has a wide range of applications as a basic method of digital image processing, including text recognition, document image segmentation, image morphological processing, and feature extraction [6,7,8,9]. It commonly serves as the primary stage in document analysis and recognition systems, as well as Optical Character Recognition (OCR), exerting a substantial impact on the efficacy of subsequent character segmentation and recognition.

Binarization is a technique used to separate the region of interest, such as text, from the background in an image, and it represents one of the fundamental methods of image segmentation. This process involves converting a grayscale image into a binary black and white image. Thresholding is a widely used tool in image segmentation [10], including global, local, and various automatic thresholding methods [11,12]. However, these approaches encountered difficulties in effectively handling complex images with uneven pixel distributions and noise interference. In recent years, the advancement of deep learning technology has led to the successful application of deep learning algorithms, including Convolutional Neural Networks (CNNs) [13,14,15,16], Generative Adversarial Networks (GANs) [17,18], and U-Net [19] in the field of document image binarization. The continuous emergence of new algorithms and technologies offers opportunities for further optimization in the field of document image binarization. Despite extensive research in digital image processing, there are still numerous unresolved challenges in handling degraded document images.

The main difficulties encountered in document image binarization are related to the non-uniform variations present in the image, as illustrated in Figure 1. Particularly in degraded document images, a range of issues such as aging, damage, blurring, and fading are frequently encountered. These challenges not only diminish the overall image quality but also render binarization algorithms more complex in handling these irregular alterations. Moreover, low-quality document images often exhibit various imperfections, including aged paper and documents, noise introduced during the scanning process (such as Gaussian noise, white noise, salt-and-pepper noise), ink stains, and contamination. These issues lead to the emergence of numerous isolated points and abnormal areas during the binarization process. Additionally, document images are susceptible to variations in lighting and contrast, resulting in uneven brightness and color distribution, thereby amplifying the complexity of binarization. Common problems such as spot defects and fractures also give rise to disconnected areas in the binarized output, making it challenging to accurately extract the image content. Displacement, skew, and deformation are prevalent issues during the document scanning process. These deformations can cause distortion in the document image during binarization, directly impacting subsequent document processing and analysis.

Previous reviews on document image binarization have been much narrower in scope compared to the current article. Some reviews [25,26,27] choose to delve into detailed discussions and comparisons of various binarization methods, aiming to provide readers with concise choices. Some reviews [28,29,30] focus on threshold-based binarization techniques, overlooking popular neural network algorithms. Therefore, we aim to provide a broader and more comprehensive review of binarization techniques, offering researchers an extensive overview of binarization technologies. This paper seeks to conduct a comprehensive review and synthesis of prevalent methods for document image binarization within an open research framework. The review encompasses both conventional algorithms and deep learning-based approaches, with the objective of furnishing valuable insights for prospective investigations in the field of document image binarization.

2. Traditional Binarization Techniques

The threshold method is an image segmentation technique that relies on the grayscale value of a pixel to separate the image based on a specified threshold. This method typically involves two technical approaches: the global thresholding method and the local thresholding method, which separate the image pixels based on their size relationship to the specified threshold.

2.1. Global Threshold Method

The Otsu [31] algorithm, developed in 1979, is a prominent method of a global thresholding technique. The algorithm aims to determine an optimal threshold value, denoted as T, by analyzing the grayscale properties of an image. This process involves partitioning the image into foreground and background segments. The objective is to minimize the gap between the two segments while maximizing the difference between them. The difference in grayscale distribution serves as a measure of the contrast between foreground and background, with a larger difference indicating an easier segmentation. The Otsu algorithm is also commonly known as the maximized difference between classes method. The optimal threshold for the desired image is the value that maximizes the gap between categories, and it can be expressed as follows:

$$T^{\prime }=arg\underset{0\le T\le L}{max}{\omega }_0\left(T\right){\omega }_1\left(T\right){\left({\mu }_0\left(T\right)-{\mu }_1\left(T\right)\right)}^2,$$

(1)

we represent the image pixel in the gray level of the image, the image has L-order gray level, ${\omega }_0\left(T\right)$ and ${\omega }_1\left(T\right)$ are the probability distribution of the target and background when the threshold value is T, ${\mu }_0\left(T\right)$ and ${\mu }_1\left(T\right)$ represent the average gray value of the pixel of the target and background, respectively, if the pixel value of the input image is greater than $T^{\prime }$. The pixel value is set to white, or otherwise it is black.

The Otus algorithm partitions the entire image based on a single threshold, allowing for the determination of the optimal threshold for the image at once. This approach generally yields improved separation for images with a uniform background. However, it may result in suboptimal image processing for images with uneven backgrounds, such as misidentification of the background in document images with significant ink penetration or insufficient grayscale contrast. Figure 2 shows a few examples of the Otsu algorithm applied on document images.

As demonstrated by the most representative global method, Otsu, such approaches are unsuitable for low-contrast or uneven images. However, using it on documents with a uniformly pure background may indeed be a good choice.

2.2. Local Threshold Method

The Niblack [32] algorithm was developed to address the limitations of a fixed threshold by introducing a local binarization method. This approach involves utilizing a local window to calculate the mean and standard deviation within a small neighboring domain of each pixel. These values are then used to adjust the threshold for binarizing the image. The threshold calculation formula is expressed as follows:

$$T=m+k\times s,$$

(2)

where m represents the average gray value of pixels in the local area, s represents the standard deviation, and k is a constant correction factor, which can be adjusted according to the foreground and background conditions of the image. Trier [33] believes that Niblack performs better than other local binarization methods in gray images with low contrast, noise, and uneven background intensity.

The Sauvola [34] algorithm is an improvement upon the Niblack algorithm, designed to address the problem of excessive noise levels. It introduces a new parameter, R, which is based on the dynamic range of the standard deviation. The threshold calculation formula is expressed as follows:

$$T=m\times \left[1+k\times \left(\frac{s}{R}-1\right)\right],$$

(3)

it can be seen from the formula that Sauvola introduces a new parameter R, which represents the dynamic range of standard deviation. In regions with high contrast in the pixel neighborhood, when s approaches R, the threshold value approximately equals the average value. This makes it unfriendly to high-contrast regions. Therefore, it is necessary to choose the optimal values based on the characteristics and distribution of the image. However, this requires manually determining the values of the factor k and the window size. Additionally, it faces challenges in handling targets of different sizes and accurately capturing all characters when different font sizes are present in the same text [35].

In addressing the issue of black noise in the Niblack algorithm, Khurshid et al. [36] introduced an algorithm named “NICK”, which is purported to be more effective for deteriorated and noisy antique documents. In comparison to Niblack, it offers the advantage of significantly improving the binarization of light-colored page region images by reducing the binary threshold. The formula for calculating the threshold is expressed as follows:

$$T=m+k\sqrt{\frac{\sum p_i^2-m^2}{NP}},$$

(4)

$p_i$ represents the pixel value of a grayscale image, while NP denotes the number of pixels. The presence of noise can be reduced when the k value approaches 0.2, but this may lead to interrupted or faint characters. When the k value is close to 0.1, the text can be extracted but some noise is retained. It also did not address the issue of manually determining the factor. Consequently, B. Bataineh [37] contends that the method does not outperform the Niblack algorithm in exceptional circumstances, such as very low-contrast images or the variations of text size and thickness.

Saxena [30] believes that the window size is the main defect of the local threshold method. Both large and small size windows will generate noise. Small windows are effective in removing noise but may distort the text, while large windows can effectively preserve the text but may introduce some noise. And even in windows without target pixels, the local threshold method can still detect target pixels. Bataineh et al. [37] proposed a threshold approach based on dynamic flexible windows. This method involves two approaches: dynamically segmenting images into windows based on image characteristics and determining the appropriate threshold for each window. The window size generally also affects the computation time. However, T. Romen Singh et al. [38] chose to utilize the dotted image as an initial stage in the calculation of the local mean. This approach allows for the calculation of the average value to be independent of the window size. Compared to other local threshold techniques, this method does not involve the computation of standard deviations, thereby reducing computational complexity and accelerating processing speed.

Chaki [39] suggests that a larger value of k adds more pixels to the document image, thereby reducing text readability. Conversely, a smaller k value results in missing or incomplete characters, which reduces the number of potential pixels. Consequently, determining the appropriate value of k also becomes challenging. Taking the Niblack method as an example, Figure 3 illustrates the impact of different window sizes and k values on the local threshold method. Even with an accurate k value, it still generates pepper noise in the shadowed areas of the image or in non-text regions.

Researchers have applied some adaptive improvements that enable these parameters to be automatically adjusted without human intervention. He [40] compared Niblack, Sauvola, and their adaptive threshold method in an article, and found that adaptive Niblack and adaptive Sauvola performed better than originals. In the paper [41], the authors combine the image contrast defined by local image minimum and maximum values with the computed Sauvola’s binarization step, without manually adjusting the user-defined parameters to the document content. Figure 4 shows an example of this method. Usually, the fixed threshold is manually set according to the specific situation of different tasks, whereas the adaptive threshold tends to estimate the background surface of the document first, and then the thresholds are calculated according to the estimated background surface. For example, Lu et al. [42] incorporates the step of estimating the document background surface. Krzysztof et al. [43] also perform fingerprint background estimation as a preliminary step in the binarization task. They all utilize polynomial smoothing to estimate the background surface. This method is a direct background subtraction approach that does not require any rough estimates of foreground and background regions. Moghaddam et al. [44] estimate the backdrop surface of the document by an adaptive and iterative image averaging approach. Gatos [45] roughly divides images into background and foreground pixels and estimates a background surface by interpolating neighboring background intensities. Creating a local threshold policy involves combining the calculated background surface with the original image and integrating image up-sampling techniques. However, in He’s study [40], it was found that removing the background from images before applying the Niblack and Sauvola algorithms did not bring any benefit in better binarization. This is because the two algorithms rely on certain information hidden in the background for local threshold calculation. Therefore, flexibility and adaptability should be exercised when applying them in practical applications.

In order to address the limitations of the global threshold method in dealing with complex structured documents, the local threshold method assigns separate thresholds to each window of the image. This method has low complexity and generally performs well in most cases. However, a drawback is the poor connectivity of segmented objects, which often results in the phenomenon of ghosting such as background areas exhibiting pseudo-strokes, leading to significant sensitivity to noise. Many studies first introduce additional techniques to handle such issues before utilizing global or local thresholding methods, as seen in Su’s approach [46,47]. By relying on a combination of other techniques, complex methods can address degradation in document images, but they often require more computational resources. Here, we summarize some common threshold methods in

Table 1. Traditional binarization techniques (1).

Classification	Algorithm	Description	Performance
Global Threshold	Otsu [31]	The gray level corresponding to the maximum inter-class variance is selected as the global threshold.	Low complexity and fast operation. However, it cannot handle complex degraded images and is suitable for processing high-quality document images.
Local Threshold	Niblack [32]	It calculates the mean and standard deviation of the pixel within a local window of an image, laying the foundation for local binarization methods.	The processing time has increased, and obvious noise can be seen in the output binarization image, which greatly increases the foreground region.
	Bernsen [48]	Compute a separate threshold for each pixel by estimating and utilizing the local contrast before classification.	When dealing with complex backgrounds in document images, pseudo-shadow artifacts may occur.
	Sauvola [34]	It improves the Niblack method by modifying the threshold calculation.	High complexity. Performance degrades in high contrast regions, but it can eliminate dark areas in the background.
	Wolf [49]	Global statistical normalization based on Sauvola for automatic detection of text regions.	It can mitigate the impact of background noise but degradation in performance occurs when there is a sharp change in background gray values across the image.
	Gatos [45]	Local adaptive-based approach to enhance degraded image binarization documents.	It performs well on degraded document images with issues such as shadows, uneven lighting, and smear, but leaves a slight amount of noise.
	NICK [36]	In order to solve the black noise problem in Niblack, an improved method of threshold calculations was used.	It lowers the threshold and performs better in low-intensity images.
	Su [46]	Image contrast technique defined using local image maxima and minima.	This method can handle general document images but performs poorly in specific cases such as the ink-bleeding.
	Bataineh [37]	Dynamic segmentation of the image into windows based on image features, determining the threshold value for each window.	It can address specific challenges, such as thin pen strokes and low-contrast images. However, it unavoidably retains excess background.
	T.R.Singh [38]	Local adaptive threshold segmentation uses the local mean and mean difference to remove the background.	Unable to recognize faint text, it loses some fine details but exhibits a certain degree of recovery for unevenly lit backgrounds.
	Su [47]	Combination method based on local image contrast and local image gradient.	Compared to several classical thresholding methods, it exhibits superior visual quality. However, there is still a small amount of noise present.
	Hadjadj [41]	An improved Sauvola’s Algorithm technique for document images is presented.	It improves the quality of binarization results without adjusting manually the user-defined parameters to the document content.
	WAN [26]	Improving lost detail strokes by increasing the binarization threshold based on Sauvola.	Performs well on a clean background without pollution.

.

2.3. Mixed Threshold Method

To make up for the limitations of global and local thresholding approaches, researchers have proposed hybrid thresholding binarization algorithms. For example, Yang et al. [50] integrated Otsu and Bernsen’s method. Zemouri et al. [51] enhanced the document picture using global thresholding before binarization and then applied a local thresholding strategy for binarization. Chaudhary et al. [52] developed a rudimentary estimation of the backdrop, constructed an image with a high contrast, and then thresholded it using the hybrid technique. Due to the low identification rate of blurred letters in handwritten document images, K. Ntirogiannis et al. [53] devised a blend of global and local adaptive binarization. First, a background estimate with picture normalization based on background compensation is applied. Then, global binarization is performed on the normalized image. In the binarized picture, typical attributes of the document image such as stroke width and contrast are determined. In addition, local adaptive binarization is performed on the normalized image. Finally, the results of the two binarizations are mixed. Liang [54] developed a hybrid thresholding technique and determined the trade-off between local and global content using variational optimization. Xiao et al. [55] suggested a model consisting of a global branch and a local branch that takes the global block of the downsampled picture and the local block of the source image as inputs correspondingly. The ultimate binarization is achieved by merging the findings of these two branches. Saddami et al. [56] employed an integrated technique such as local and global thresholding methods to extract text from the backdrop to recover the information on degraded ancient Jawi manuscripts. P. Ranjitha et al. [57] suggested a classification system to deal with degraded document photographs by blending the modification of local and global binarization algorithms.

It is easy to observe that these methods, which combine global and local thresholds, first use a global threshold to process the entire document and then use local thresholds to address the shortcomings of the global method. We summarize some mixed methods in

Table 2. Traditional binarization techniques (2).

Classification	Algorithm	Description	Performance
Mixed threshold	Yang [50]	A combined Otsu and Bernsen method for processing printed document images.	It can repair broken strokes and eliminate minor ghost artifacts, but with increased processing time.
	Zemouri [51]	Apply the global threshold to the whole document, and use the Sauvola method on the intermediate image.	Its recognition effect is better than that of a single threshold, but the effect of degraded document images is not good.
	Chaudhary [52]	Otsu’s method separates background pixels from the foreground. Sauvola’s method is then applied to eliminate pixels erroneously estimated as foreground.	Thin or weak strokes can be easily confused with the background, making them difficult to identify. Suitable for high-contrast document images.
	K.Ntirogiannis [53]	Combining the results of both Otsu and Niblack, incorporating post-processing in intermediate and final steps.	It cannot simultaneously consider text information and large noise, making it unsuitable for document images with folds or page splits.
	Liang [54]	Combining the Otsu and Sauvola methods to determine thresholds.	It can reach the general level in document image binarization.
	Saddami [56]	It combines the Niblack and Wolf methods. The Otsu method and standard deviation are combined.	It retains a significant amount of noise and degrades the performance in images with low contrast.

.

2.4. Image Feature Method

Document images contain a wealth of information and are highly complex. Especially for degraded images, relying solely on thresholding is often insufficient. Therefore, pre-processing or post-processing steps are commonly incorporated during binarization. In the work of Moghaddam et al. [44], these methods are categorized into four meta-levels: pixel-level, region-level, content-level, and global-level. Pixel-level characteristics involve factors such as grayscale values and gradients. Region-level is applicable to various image processing domains. Some basic examples include calculations such as mean and variance. Content-level, more suitable for document images, focuses on stroke information, including stroke grayscale and contours. The highest level is the global level, encompassing all data in the image.

In degraded document images, attention is paid to stroke continuity, especially in handwritten documents. Edge detection is a pixel-level method used for this purpose. Based on the threshold method of edge detection, it first identifies the edge pixels within the image and then uses these edges as partition boundaries to divide the image into different areas. Edge detection typically involves using differential operators to identify areas of significant variation in the grayscale values of images. The example of edge detection is shown in Figure 5. Commonly used edge detectors include Sobel, Prewitt, Roberts, Laplace of Gaussian, and Canny. The selection of the edge detector is determined by the specific characteristics of the image in practical applications. Santhanaprabhu et al. [58] applied the Sobel edge detection technique to extract text and perform document image binarization. Lu et al. [42] used the L1-norm image gradient to identify the edge of the font from the compensated document image. T. Lelore et al. [59] have described a quick solution for repairing document images by employing the edge-based method to locate text in degraded document images. However, it should be noted that the edges detected by the edge detection technique may not completely enclose the prospective text area, thus requiring further improvement. Holambe et al. [60] exploited adaptive image contrast in combination with Canny’s edge diagram to identify the edge pixels of the font. Jia et al. [61] used structural symmetry pixels (SSPs) to calculate local thresholds for the neighborhood. SSPs is defined as the pixel around the stroke, whose gradient size is large enough and the direction is symmetric and opposite. The author extracts SSPs by combining the adaptive gradient binarization and iterative stroke width estimation algorithm. This approach reduces the influence of degraded documents and ensures the appropriate field size when determining the direction. Multi-threshold voting is then used to determine whether the pixel belongs to the foreground text, handling inaccurate SSPs detection. Hadjadj et al. [62] introduced a method of document image binarization that is frequently applied to the active contour model used in image segmentation. The objective of their method is widely used in the field of image segmentation. It aims to convert the problem of image segmentation to solve the minimum energy functional. Hadjadj defines image contrast of the maximum and minimum values of the local image. Use it to automatically generate initialization graphics for active contour models. The average threshold value is selected to generate binarization, as it enables the active contour to effectively detect low-contrast regions. When the active contour remains stationary, the result is obtained by thresholding the level set function.

Compared to the simple threshold method, edge detection-based segmentation methods can effectively extract text contour information with fast detection speed. Generally, edge detection methods do not consider the font size, which is an advantage. However, since it relies on calculating pixel gradients within the image itself, it is more sensitive to changes in lighting and can be easily affected by variations in illumination.

Various paradigms are often used in image analysis and processing. Based on the characteristics of the objects, researchers employ statistical methods to classify target pixels. Fuzzy theory defines a fuzzy set and calculates the membership degree of each pixel belonging to each set through fuzzy logic operations. Finally, it classifies pixels based on their membership degree in order to achieve the purpose of segmentation. This process involves obtaining a classifier through a training dataset to distinguish different data, and then using the classifier to predict unknown data. For example, Support Vector Machines (SVMs) can be used to classify image blocks, which is a form of Supervised Learning. Xiong et al. [63] divided a grayscale image into several blocks and enhanced the local contrast of image blocks to roughly extract the foreground. Subsequently, SVMs are employed to classify image blocks into different categories based on statistical information such as region mean, variance, and histogram, thereby determining the optimal global threshold. After thresholding, the image is seamlessly spliced using image seamless splicing technology. Following this, the Canny operator is applied for stroke edge detection, and adaptive local thresholding is used to eliminate noise near stroke edges.

Another approach is to group similar data together, such as K-means, which fall under Unsupervised Learning. Lai et al. [64] proposed two binarization methods based on K-means clustering, one utilizing intensity and the other utilizing color information. After converting the color image to a grayscale image, they applied a median filter to blur the background and used the Sobel operator to emphasize the vertical edges of the text.

Soua et al. [65] proposed a hybrid binarization based on Kmeans method (HBK) and implemented real-time processing of the parallel HBK method in an OCR system. However, K-means is a hard clustering algorithm and cannot process uncertain information. Although K-means is widely used, researchers have proposed more flexible soft clustering algorithms because of its hard clustering nature and sensitivity to noise. Such an algorithm is the fuzzy C-means clustering algorithm, which can accommodate uncertainties related to data points. There are also clustering algorithms such as Possibilistic-Fuzzy C-means (PFCM) [66] and Kernel-based Fuzzy C-Means (KFCM) [67]. Tong et al. [68] combined the Niblack algorithm and the Fuzzy C-Means algorithm (FCM) to propose a camera-based document image binarization algorithm named NFCM. It is expected to address the issue of document image breakage or blurring, preserve the fine details of character strokes, and eliminate glint interference. Furthermore, there are many blur processing algorithms. For instance, Biswas et al. [69] apply a Gaussian filter to the input degraded image files for blur processing. In addition, there is the FuzBin-based binarization method, which Annabestani et al. [70] use to extract text information from document images. They enhance image contrast with fuzzy expert systems (FESs) and then combine FESs with a pixel counting algorithm to obtain a range of threshold values. The middle value is taken as the final threshold. Finally, a method based on mathematical morphology operations, as employed by Gatos et al. [71], is used to enhance blurred stroke information. We summarize these methods in

Table 3. Traditional binarization techniques (3).

Classification	Algorithm	Description	Performance
	Lu [42]	It detects the text stroke edges based on the local image variation. Combining L1 parametric image gradient and local threshold segmentation.	Suitable for scanned documents with uniform color and texture. Unable to handle situations where the document is skewed or folded.
	T.Lelore [59]	The improved Canny method for edge detection is employed to achieve rough text localization.	This method has relatively low computational costs and can be applied in real-time applications.
Edge detection	Santhanaprabhu [58]	This method first constructs an adaptive contrast map, and then combines the Sobel edge map to detect the edges of text strokes.	This method has few parameters and can adapt to various degradation types such as uneven illumination and document smear.
	Holambe [60]	Adaptive image contrast combined with Canny’s edge map to Edge detection identify stroke-edge pixels.	The approach is low complexity, convenient, and involves fewer parameters.
	Jia [61]	By deforming the Sobel operator to compute the image gradient map. Then, a voting framework is employed to compensate for inaccurate structural symmetry pixels (SSPs).	Due to its adaptive stroke width estimation, this method performs slightly better on printed document images compared to handwritten images.
	Lai [64]	Binarization is performed on images captured by mobile devices using the K-means clustering approach to address degradation issues.	It can reasonably restore Korean text, but the effectiveness of binarized images with fine strokes needs further investigation.
	Tong [68]	Niblack and Fuzzy C-Means (FCM) were used for clustering and to calculate the local thresholds, respectively.	It can effectively preserve strokes and alleviate ghost artifacts. But its performance is unsatisfactory in handling weak strokes.
Fuzzy Logic	Biswas [69]	Blurring of the input degraded file image with Gaussian filter.	Effective for images with a simple background, but may lead to stroke fragmentation when dealing with faint strokes.
	Soua [65]	A method based on hybrid binarization and implemented processing of the parallel binarization based on Kmeans method in an optical character recognition (OCR) system.	Suitable for high-quality printed document images and can be applied in real-time scenarios.
	Annabestani [70]	A global threshold selection method based on fuzzy expert systems (FESs).	It falls within the category of global threshold methods and may require manual adjustment of local threshold for degraded documents. It is more suitable for uniform background.

.

In addition to the methods mentioned above, there are numerous other techniques available for binarizing document images. Although these methods may not be as widely used as other commonly adopted algorithms, they still hold unique value in practical applications. These include histogram-based methods such as [72,73,74]; entropy-based methods such as [75]; space binarization-based methods such as [76]; and object property-based methods such as [77], etc. In general, when faced with the task of document image binarization, it is important to comprehensively consider the characteristics and final requirements of the image to choose the most suitable method. Sometimes, combining or layering multiple methods can be an effective approach to improve the binarization of document images. This integrated thinking and practice ensure good results in real situations.

3. Deep Learning Binarization Techniques

Researchers hope to design a model that can receive all image information and address document degradation. A good approach is to design and train a neural network model. Badekas et al. [78] learned from the binarization results generated by various techniques using neural networks. They propose an integrated system for binarizing normal and degraded printed documents to enhance the visualization and recognition of textual characters. While this approach is highly effective for files with complex backgrounds and images, it can also result in lengthy processing times [79]. Deep learning, as a prominent research area in the field of artificial intelligence in recent years, has demonstrated significant potential in the field of document restoration. Therefore, researchers apply deep learning technology to document image binarization. This method can not only compensate for the limitations of traditional algorithms in handling degraded documents but also offers significant benefits in enhancing the efficiency and accuracy of document processing.

3.1. Based on Convolutional Neural Networks (CNNs)

Convolutional Neural Networks (CNNs) are a deep learning model with a basic structure that includes a convolutional layer, a pooling layer, and a fully connected layer. The convolutional layer is the core component of CNNs. In the field of document image binarization, a convolutional neural network first performs a convolution operation on the input document image to extract the feature information within the image. Then, through the pooling operation, the size of the feature map is reduced to decrease computation, while still preserving important feature information. The collected feature information is then classified in the fully connected layer, resulting in a binary outcome. Usually, researchers combine CNNs with other techniques, such as regularization and data enhancement, to optimize performance and enhance the model’s generalization ability. Pastor-Pellicer et al. [80] compared the similarities and differences between Multilayer Perceptrons (MLPs) and introduced the practical application of CNNs in document image binarization. The experiments conducted on the (H)DIBCO [20,22,81,82] and Santgall [83,84] datasets demonstrated that CNNs outperformed MLPs in this task, with particularly notable performance on the Santgall dataset. Various network architectures of CNNs can be utilized. Different from traditional threshold methods, researchers mainly train neural networks to learn degradation and subsequently restore degraded images. In the work by Saddami et al. [85], three deep CNN architectures were compared: Resnet101 [16], Mobilenet V2 [86], and Shufflenet [87] in the task of degradation classification. Shufflenet achieved superior performance in terms of accuracy and computational efficiency. He et al. [88] combined a CNN and the Otsu algorithm to propose an image binarization model called DeepOtsu. Through the automatic feature extraction of deep learning, the threshold selection method in the Otsu algorithm is optimized, resulting in better binarization results. Compared to the Otsu algorithm, the DeepOtsu model can handle more complex image scenes and has a stronger performance against interference factors such as lighting and noise. In a more complex approach, Vo et al. [89] proposed a new supervised binarization method based on a deep supervised network (DSN). The layered DSN architecture is used to learn how to predict text pixels at different feature levels. The network distinguishes text pixels from background noise using higher-level features. The layered architecture helps the proposed approach to retain text strokes more efficiently and provides excellent visual quality. Meng et al. [90] proposed a framework based on deep convolutional neural networks (DCNN). Firstly, the degraded document images are decomposed into spatial pyramid structures by a decomposition network. This network learns character features from images of different scales. A deconvolution network is then used to reconstruct the foreground image from each of these layers in a coarse-to-fine manner.

In 2015, Long et al. [91] proposed Fully Convolutional Networks (FCNs), which for the first time applied deep learning to the field of semantic segmentation. FCN classifies images at the pixel level. It can integrate input image features of any size and then upsample them using deconvolution. The feature is restored to the original input image size, and a label can be generated for each pixel. Tensmeyer [92] described binarization as a task of pixel classification and proposed an algorithm for binarization of low-quality document images and palm leaf manuscript images based on FCN. The FCN algorithm can not only achieve binarization but also recognize and segment different types of objects in the image at the same time. Compared to traditional convolutional neural networks, the advantage of FCN is that it does not limit the size of the image. It also does not require the image to be the same size. The prediction is then realized at the pixel level through the deconvolution output, which produces binary results of the same size as the original image. Secondly, it avoids duplicate storage and computation, making it more efficient. Of course, the shortcomings of FCN are also evident. It is insufficiently sensitive to the details in image processing, and the results obtained are insufficiently refined. In tasks that require high levels of detail, such as image processing of ancient document images, FCN has potential for further improvement. Ayyalasomayajula et al. [93] proposed an end-to-end structure that combines the FCN and the Primal-Dual network (PD-Net [94]) to address the issue of the foreground category of FCN being either too high or too low in the binarization of document images. The performance and accuracy of the model have been improved. We summarize the document image binarization method based on CNN in

Table 4. Deep learning techniques (1).

Algorithm	Description	Performance
Pastor-Pellicer [80]	Describes a practical application of Convolutional Neural Networks (CNNs) in document image binarization tasks.	The method still lags behind advanced approaches, but their work validates the superiority of CNN over Multilayer Perceptrons (MLPs).
Meng [90]	Deep Convolutional Neural Network (DCNN)-based framework. Binarization is employed based on the characteristics or patterns of characters.	It performs well on degraded document images. However, defects may appear in the middle of wide color strokes, and deep stains may not be completely removed.
Vo [89]	A new supervised binarization method based on Deep Supervised Networks (DSNs).	The results retained only a small amount of noise, with few thin or weak strokes missing. Plus, its processing time needs improvement.
He [88]	Combining the Otsu algorithm and CNN results in a method named DeepOtsu.	Enhancing degraded document images before binarization leads to clear and uniform text.
Tensmeyer [92]	Fully Convolutional Network (FCN) based binarization algorithm for low-quality document images.	FCN has certain limitations, and there might be effective features that it cannot learn.
Ayyalasoma-yajula [93]	Combining a FCN and a Primal-Dual Network (PD-Net).	It is suitable for historical document images and transfer learning scenarios.

.

3.2. Based on Generative Adversarial Networks (GANs)

Generative Adversarial Networks (GANs) [17] are composed of generator networks and discriminator networks. GANs perform well in binarization, text region detection, and text recognition, among other applications. They have a wide range of applications in image processing, as mentioned in [95,96,97,98]. GANs transform the binarization task from a classification problem into an image generation problem. The generator’s task is to convert the input grayscale image into a binary image, while the discriminator’s task is to judge whether the generated binary image is correct. Through training, the generator can learn how to convert the original document image into a high-quality binarized image, and the discriminator can learn how to accurately distinguish the image generated by the generator from the real binarized image. However, there are still some problems in the training process of GANs, such as unstable training, which requires a significant amount of computing resources and time.

Suh et al. [99] proposed a two-stage GAN for document image binarization with color noise and background removal. In the first stage, the background information is removed, and the color foreground information is extracted to enhance the document image. In the second stage, the binarized image generated by the adversarial network is used to achieve the binarization of the document image. Bhunia et al. [100] built a Texture Augmentation Network (TANet) by introducing adversarial learning to transfer the texture elements of degraded reference document images into a clean binarized image. This method has various noise texture versions of the same text content and expands the training set. Bhowmik et al. [101] drew inspiration from game theory and utilized unsupervised learning in their approach. Specifically, they use the K-means clustering method to classify pixels into foreground and background. The quality of binarization is enhanced through pre-processing and post-processing. Kumar et al. [102] optimized Bhunia’s algorithm by introducing a joint discriminator to combine TANet and the unsupervised document binarization network (UDBNet). This enhancement addresses dataset bias, aiming to achieve improved performance on actual degraded images. To generate more challenging adversarial samples for UDBNet training, researchers utilized an Adversarial Texture Augmentation Network (ATANet) to create a pseudo image pair. Konwer et al. [103] used GAN to remove staff line to achieve binarization in the pre-processing step of optical music recognition. Zhao et al. [104] introduced conditional generation adversarial networks (cGANs) [105] to solve the problem of multi-scale information composition in binary tasks. Souibgui et al. [106] used cGANs to propose a pix2pix framework called Document Enhancement Generative Adversarial Network (DE-GAN) to restore severely degraded document images. The discriminator inputs the degraded image and the Ground Truth (GT), and it compels the generator to generate an output that is indistinguishable from the GT. After the training is completed, the discriminator becomes unnecessary, and only the generator network is used to enhance the degraded image. Figure 6 shows the binarization result of this method, which still retains some background noise and slightly widens the text strokes.

R. De et al. [107] propose a Dual Discriminator Generative Adversarial Network (DD-GAN) that utilizes Focal Loss as the generator loss. The model uses a network of two discriminators to capture information. The global discriminator is responsible for higher-level image features, such as image background and texture, while the local discriminator focuses on lower-level features like text strokes. Additionally, the model employs focal loss which is used to solve the issue of class imbalance among pixels. Rajesh et al. [108] argue that while most existing technologies concentrate on pixel images as input, they may not yield satisfactory outcomes when processing compressed images that require complete decompression. Therefore, Rajesh applied DD-GAN and proposed the direct use of JPEG for compressing document images to achieve binarization. Lin et al. [109] proposed a three-stage approach to enhance and binarize degraded color document images by using discrete wavelet transform (DWT) and GANs. This general model approach can be trained with different wavelet transforms and neural networks. This method can be effectively applied to the degraded color document image binarization task. We summarize the document image binarization method based on GANs in

Table 5. Deep learning techniques (2).

Algorithm	Description	Performance
Bhunia [100]	A texture enhancement network is constructed by introducing an adversarial learning approach.	It can provide better visual quality but might compromise stroke edge details, resulting in noise within thick strokes.
Zhao [104]	The conditional GANs (cGANs) are introduced to solve the multi-scale information combination problem in binarization tasks.	It performs well under various degradation scenarios, but there are minor noise dots in the middle of the strokes.
Bhowmik [101]	They proposed a document image binarization method based on game theory, using the K-means clustering algorithm for pixel classification.	It effectively eliminates artifacts while preserving some boundary pixels of stamps and stains.
Kumar [102]	The joint discriminator combines the Texture Augmentation Network (TANet) and unsupervised document binarization network (UDBNet) to address the issue of dataset bias.	In handwritten document images, stroke omissions and background noise may occur.
Suh [99]	A two-stage color document image enhancement and binarization method using GANs. The study focuses on the issue of multi-color degradation.	It can solve the problem of document degradation, including postmarks, bleed-through, uneven contrast, and so on.
Souibgui [106]	A conditional generative adversarial network for document enhancement named DE-GAN.	It removes most background imprints, but the method also preserves some noise, as well as traces like paper creases.
R.De [107]	A Dual Discriminator GAN (DD-GAN) is proposed. It utilizes both global and local information about pixel distribution.	The overall performance is good, but the performance in HDIBCO2018 is poor, mainly due to the inability to distinguish background pixels that do not belong to the manuscript.
Rajesh [108]	The proposed model directly binarizes document images in their compressed form without the need for decompression.	The most important point of this method is its ability to significantly enhance computing efficiency and save storage space.
Fathallah [110]	A historical document image enhancement model is proposed based on GANs.	Some stains, such as watermarks, can be removed, but the resulting text strokes will be thinner, which may lead to the loss of some details.

.

3.3. Based on Attention Mechanism

The Attention Mechanism is a special structure in machine learning that simulates the selective perception of certain information by human attention. It automatically selects the most important part of the input data, reducing the impact of noise. Additionally, it can be used to enhance the expression and generalization ability of the network. For example, in document image processing, the attention mechanism can learn to calculate the weight coefficients of different areas. This allows for more attention to be paid to the text or background areas. Guo et al. [111] proposed a novel Multi-scale Multi-attention Network (MsMa-Net) for the fresh moiré document image binarization task. Peng et al. [112] proposed a deep learning framework for inferring the probability of a text region using a multi-resolution attention model. This probability is then fed into a convolutional conditional random field (ConvCRF) to obtain a final binarized document image. The author uses a neural network to learn the features of degraded document images and employs ConvCRF to infer the relationship between text areas and the background. The author claims that this approach can result in stronger generalization ability.

The encoder-decoder structure is a common model structure in deep learning. The encoder converts the input data into an intermediate value that captures the key characteristics of the input data. The decoder receives the median value from the encoder and uses it to generate an output, such as a pixel value for an image or a sequence of words for text. In Natural Language Processing (NLP), common encoder-decoder structures include Seq2Seq and Transformer. In the field of image processing, the commonly used encoder-decoder structures are U-Net and VGG. In encoder-decoder structures, attention mechanisms are often used, as seen in the implementation of U-Net. This process helps extract useful features from the original document image and generate an accurately binarized document image.

In a document image binarization task, the encoder typically converts the document image into a sequence of vectors to capture the key features of the document image. The decoder then uses this sequence to generate a binarized document image. The U-Net was proposed in 2015 and was initially applied to image segmentation tasks in the biomedical field. It uses an encoder-decoder structure, in which the encoder is responsible for extracting features, while the decoder restores the image to its original resolution. Different from FCN’s feature addition mechanism, U-Net concatenates the up-sampled and down-sampled feature maps by skip connections to preserve more dimension and location information. This improves the segmentation effect of the network. Therefore, the structure of U-Net is highly suitable for document image segmentation. Bezmaternykh et al. [113] used the U-Net architecture to propose a CNN-based method called U-Net-bin, which won first place in the DIBCO’17 competition. Furthermore, they argue that the success of binarization is not crucial in the form of Chinese or English characters. Xiao et al. [55] also used U-Net architecture as the foundation to propose a method for document binarization that combines local and global features. Based on the attentional U-Net, Zhao et al. [114] proposed a binarization method for historical Tibetan document images. In this method, the input image is unsampled twice during the inference stage to alleviate pseudo-touching. Ke Ma et al. [115] combined U-Net and Transformer models to perform end-to-end training for geometric correction and binarization of document images. They used a stacked U-Net with intermediate supervision for this purpose.

Peng et al. [116] proposed a convolutional encoder-decoder model specifically designed for the binarization of document images. The encoder is constructed by stacking convolutional layers to learn the features of the middle layer of the document image. The low-resolution representation is then mapped to the original size using a decoder to generate the final binarized image. Souibgui et al. [117] adopted the Vision Transformers model for the binarization of document images and named it DocEnTr. The model captures high-level global remote dependencies through a self-attention mechanism and outputs binarized images of documents in an end-to-end manner.

Chaurasia et al. [118] proposed a network architecture called LinkNet, which drew inspiration from the U-Net model and adopted an Encoder-Decoder structure to create a lightweight network capable of real-time segmentation. Xiong et al. [119] proposed an improved semantic segmentation model called DP-LinkNet, which is based on the LinkNet and D-LinkNet [120] models. They introduced a Hybrid Dilated Convolution (HDC) module in the middle of the architecture to increase the receptive field and enhance the network’s ability to capture details and textures in images. The Spatial Pyramid Pooling (SPP) has also been introduced to improve the perception of features at different scales. The experimental results show that the proposed method performs well on document images with noise, such as stains and imprints, and achieves excellent speed and accuracy. As shown in Figure 7, the two models exhibit little visual difference. Both effectively extract text, with only minor shortcomings in the details. Specifically, there are traces of discontinuity in fine strokes, and the edges of blurry strokes are not as sharp. Despite this, they have achieved a satisfactory outcome.

In general, U-Net can assist in document image processing by preserving text information and eliminating background noise. When combined with an attention mechanism, it can effectively improve the efficiency of document image processing. In the document image binarization, the traditional threshold segmentation method and morphological operation method can be combined to optimize the binarization result. Additionally, they can also be used to post-process the output of U-Net for further optimization. For example, morphological operations can be used to perform dilation and erosion operations, which help in removing noise and small fragmented areas. Additionally, threshold segmentation can also be used to extract more detailed information. At the same time, U-Net can also be combined with other deep learning models using multi-task learning. This approach allows for simultaneous text detection and binarization, thereby improving the efficiency of the entire document processing. In many tasks, these neural network algorithms can achieve high accuracy in binarization. Most of them do not require the pre-processing of the document image. However, due to the complexity of the neural network, the calculations may take some time. We summarize the document image binarization method based on the attention mechanism in

Table 6. Deep learning techniques (3).

Algorithm	Description	Performance
Peng [116]	The probabilities of text regions are inferred from a multi-resolution attention model, which is then fed into a ConvCRF.	It performs well in document images with contrast variations and exhibits good generalization capability.
Ke Ma [115]	A hybrid model based on U-Net and Transformer for flattening and correcting distorted deformations in document images.	It significantly restores distorted document images, which is highly beneficial for the binarization of degraded documents.
Peng [112]	It extrapolates the probability of the text region using a multi-resolution attention model and feeds it into ConvCRF to obtain the final binarized document image.	It can effectively remove degraded stain marks, but it is easy to identify the page boundary.
Bezmaternykh [113]	It uses U-Net architecture to achieve more accurate historical document images binarization.	It can exhibit excellent results for document images, and this method secured the first position in the DIBCO ’17 competition.
Xiao [55]	It utilizes a combination of global and local branches, adopting U-Net as its architecture, and finally produces the ultimate binarization result through logical operator fusion.	Comparative experiments indicate that the combined approach can enhance the discriminative performance between text and background.
Guo [111]	A Multiscale Multi-Attention Network (MsMa-Net).	It can eliminate the majority of background noise, but it has some limitations as the edges of strokes may not be smooth enough.
Zhao [114]	A U-Net-based binarization method provides a solution to the degradation issues in historical Tibetan document images, particularly pseudo-touching strokes.	This method achieves the best results in enlarged Tibetan images, but it may produce holes in the strokes of faint characters.
Xiong [119]	Based on the LinkNet architecture, the model integrates the Hybrid Dilated Convolution (HDC) and Spatial Pyramid Pooling (SPP) modules between the encoder and the decoder.	It can effectively remove background stains, restore text information, and is much faster than most CNN-based methods.

.

4. Results

Different techniques are utilized for evaluating document image binarization. Firstly, the most common approach involves human visual observation [121], but this intuitive method lacks quantitative analysis. Secondly, an end-to-end approach may be employed, such as using OCR performance as a reference [45], but the results obtained through this method may also reflect the influence of other image tasks. Unsupervised metric methods typically assess the quality of binarization by analyzing properties of image segmentation, such as those based on gray-intensity variances [122]. However, the method may lead to misleading results. Therefore, for the evaluation criteria of binarization in both handwritten and printed document images, pixel-based binarization evaluation methods are widely employed. This paper utilizes several well-known evaluation metrics commonly used in DIBCO competitions for quality assessment.

4.1. Performance Measures

4.1.1. PSNR

PSNR, which stands for Peak Signal-to-Noise Ratio, is used to measure the similarity between the original image and the processed image. This index is calculated by determining the Mean Square Error (MSE) between the pixel values of the two images and converting it into decibel (dB) units. To quantify the relative error between two images, the larger the value, the higher the similarity, the smaller the relative error, the better the image quality. In document image binarization, the resulting image after binarization consists of only black and white pixel values. This binarization result can be considered as a compressed or distorted image. The PSNR value can be calculated by calculating the MSE between the binarized image and the original image.

$$PSNR=10log\left(\frac{MAX{I}^2}{MSE}\right),$$

(5)

$$MSE=\frac{{\displaystyle \sum _{x=1}^M}{\displaystyle \sum _{y=1}^N}{\left(I_{bin}\left(x,y\right)-I_{GT}\left(x,y\right)\right)}^2}{MN}.$$

(6)

where MAXI denotes the maximum image pixel value, usually 255, while MSE denotes the mean square error between the compressed and original images. $I_{bin}$(x, y) denotes the image pixel value after binarization and $I_{GT}$(x, y) denotes the image pixel value of the reference image (Ground Truth).

4.1.2. F-Measure

F-measure (FM) is a metric that combines precision (P) and recall (R), and it is the harmonic mean of the two values. The following equation can express it as follows:

$$FM=\frac{\left(1+{\beta }^2\right)PR}{{\beta }^{2}P+R},$$

(7)

where $\beta$ is a weighting factor, generally taking a value of 1, indicating that Precision and Recall are equally important.

When $\beta$ = 1, which is also called the F1-Score. Precision indicates the proportion of pixels that are binarized as foreground and truly belong to the foreground, while Recall indicates the proportion of pixels that truly belong to the foreground and are correctly binarized as foreground. They are defined by the following parameters: True Positive (TP) indicates the number of pixels correctly classified as foreground; False Positive (FP) indicates the number of pixels incorrectly classified as foreground; and False Negative (FN) indicates the number of pixels incorrectly classified as background.

$$FM=\frac{2PR}{P+R},P=\frac{TP}{TP+FP},R=\frac{TP}{TP+FN}.$$

(8)

In document images, there are typically more background pixels than foreground pixels. The F-measure can penalize methods that produce disproportionate false positives or false negatives.

4.1.3. Pseudo F-Measure

Pseudo F-measure $\left(F_{ps}\right)$ [123], an improved algorithm for F-measure, is mainly used for the evaluation of binarization, which is calculated as follows:

$$F_{ps}=\frac{2·R_{ps}·P_{ps}}{R_{ps}+P_{ps}}.$$

(9)

$F_{ps}$ introduces pseudo-recall $\left(R_{ps}\right)$ and pseudo-precision $\left(P_{ps}\right)$, considering local stroke width and the distance from the contour of the ground truth text, thereby more effectively capturing binarization performance.

4.1.4. DRD

Distance Reciprocal Distortion (DRD), is a metric for image quality evaluation (cf. [124]), mainly used to measure the sharpness and contrast of the image. It has a good correlation with the error perception of human visual detection, so DRD is used to measure the visual distortion in document images binarization.

$$DRD=\frac{{\displaystyle \sum _{k=1}^N}DR{D}_k}{NUBN},$$

(10)

where $DR{D}_k$ denotes the distortion of the kth flipped pixel, and NUBN denotes the number of non-uniform color blocks in the reference image (GT).

4.2. Experimental Result

In this subsection, we compare the quality results of 25 techniques using the dataset H-DIBCO2016 [23] from the Handwriting Document Image Binarization Competition. The first part involves traditional techniques, utilizing threshold algorithms from commonly used Python libraries such as OpenCV and scikit-image for quality assessment. The evaluation results for the second part, which are based on deep learning techniques, are extracted from the data provided in the respective original papers. The omission of results for other techniques is due to the unavailability of implementation source code and limited details provided in the original papers. These constraints influenced the selection of methods included in the quality assessment results.

As observed in

Table 7. Quality evaluation results of different algorithms on the HDIBCO2016 dataset.

Classification	Algorithm	F-Measure	${\mathit{F}}_{\mathit{p}}\mathit{s}$	PSNR	DRD
	Otsu [31]	86.59	89.92	17.79	6.89
	Niblack [32]	63.10	63.64	12.47	31.48
	Bernsen [48]	71.73	75.87	13.66	25.41
	Sauvola [34]	81.27	83.91	15.36	17.92
	Wolf [49]	86.36	90.46	17.11	9.01
Traditional	Gatos [45]	86.59	89.09	17.47	7.01
based	NICK [36]	80.86	82.65	15.15	19.24
	Su [46]	74.41	87.03	15.52	12.55
	Hadjadj [41]	83.68	87.03	16.20	13.05
	T.R.Singh [38]	84.47	87.56	16.99	9.73
	Bataineh [37]	83.27	84.43	16.02	15.61
	WAN [26]	75.14	76.06	13.61	30.33
	Vo [89]	90.10	93.57	19.01	3.58
	Tensmeyer [92]	89.52	93.76	18.67	3.76
	He [88]	91.4	94.3	19.60	2.9
	Meng [90]	89.90 ± 4.55	-	18.79 ± 3.36	-
	Ayyalasomayajula [93]	90.18	-	18.99	3.61
Deep learning	Zhao [104]	91.66	94.58	19.64	2.82
based	Bhowmik [101]	91.15	-	19.18	3.20
	Kumar [102]	93.4	96.2	20.1	2.2
	Lin [109]	91.46	96.32	19.66	2.94
	R.De [107]	89.98	95.23	18.83	3.61
	Xiao [55]	90.77	94.21	19.33	3.11
	Peng [112]	91.68	-	19.59	2.93
	Peng [116]	88.07 ± 4.86	-	18.13 ± 3.13	-

Bold font indicates the best result.

, it can be inferred that the performance of methods based on deep learning models tends to surpass that of traditional threshold-based binarization methods, especially in the context of the HDIBCO2016 dataset. According to the results, Kumar et al. [102] proposed that the unsupervised document binarization network had the best performance in terms of three indicators: F-Measure, PSNR, and DRD. In the Pseudo F-measure index, the best performance was achieved by the three-stage binarization of color document images proposed by Lin et al. [109]. Among threshold-based methods, the Otsu algorithm [31], Wolf algorithm [49], and Gatos algorithm [71] specifically demonstrate better comprehensive evaluation indicators. On the other hand, algorithms based on deep learning models exhibit better overall performance. Notably, the model algorithms proposed by He [88], Zhao [104], and Peng [112] have shown promising results.

5. Conclusions

Document image binarization is a complex and multi-level process. Due to the various types and degrees of damage to document images, the processing emphasis varies. Because of this, there is no universal binarization method that works for all types of document images.

This paper provides a detailed overview of approximately sixty document image binarization methods, evaluating the quality of twenty-five of them using the H-DIBCO2018 dataset. Based on the evaluation, traditional binarization algorithms perform well in handling document images with simple backgrounds but show limited effectiveness when dealing with images with complex backgrounds, especially those containing mixed noise or severe contamination. Deep learning methods, by employing pixel-level image segmentation, produce promising results while also addressing complex details within the images. However, it is worth noting that these methods may require more computational resources and time.

When choosing specific methods, one must consider the task requirements. This may include degradation issues in the document, the type of document (handwritten or printed), and the availability of sufficient computational resources. Additionally, the type of degradation in the document should be taken into account when choosing the most suitable method.

Given the potential variations in characters and numbers in document images from different regions, future research should focus on enhancing the cross-language capabilities of algorithms to effectively handle stroke characteristics of different texts. Depending on the requirements of different scenarios, future studies can design algorithms that combine knowledge from various fields, such as image enhancement and restoration, or fine-tuning existing advanced algorithms.

Due to the complex data characteristics of degraded document images, each image may exhibit different types and levels of damage or degradation. Therefore, the primary challenge in training current networks is the lack of ground truth, which leads to insufficient datasets. To address this challenge, future research can delve into the application of unsupervised techniques. We also plan to continue collecting and exploring the application of unsupervised techniques in document image binarization.

Additionally, to comprehensively assess the performance of document image binarization methods, it is necessary to further explore and introduce new evaluation metrics. While existing metrics are widely used, they may inherently possess inaccuracies, prompting the search for more precise and comprehensive performance measures. We are also committed to evaluating binarization techniques by introducing additional high-quality metrics and utilizing diverse datasets. In future research, we will delve into exploring and evaluating a select few highly practical techniques.