Imperceptible–Visible Watermarking to Information Security Tasks in Color Imaging

Abstract

Digital image watermarking algorithms have been designed for intellectual property, copyright protection, medical data management, and other related fields; furthermore, in real-world applications such as official documents, banknotes, etc., they are used to deliver additional information about the documents’ authenticity. In this context, the imperceptible–visible watermarking (IVW) algorithm has been designed as a digital reproduction of the real-world watermarks. This paper presents a new improved IVW algorithm for copyright protection that can deliver additional information to the image content. The proposed algorithm is divided into two stages: in the embedding stage, a human visual system-based strategy is used to embed an owner logotype or a 2D quick response (QR) code as a watermark into a color image, maintaining a high watermark imperceptibility and low image-quality degradation. In the exhibition, a new histogram binarization function approach is introduced to exhibit any watermark with enough quality to be recognized or decoded by any application, which is focused on reading QR codes. The experimental results show that the proposed algorithm can embed one or more watermark patterns, maintaining the high imperceptibility and visual quality of the embedded and the exhibited watermark. The performance evaluation shows that the method overcomes several drawbacks reported in previous algorithms, including geometric and image processing attacks such as JPEG and JPEG2000.

1. Introduction

Since the beginning of the digital era and the rise of social media, many digital images and videos have been manipulated and shared without restrictions, control, and protection, creating copyright protection and intellectual propriety problems [1,2,3,4]. As a solution, several visible and invisible digital watermarking algorithms have been proposed mainly for images [5,6,7], video, audio [8,9], and other related fields [10,11,12]. However, in real-world applications such as banknotes, official documents, etc., the watermarks are used to deliver information about its authenticity. In this way, a novel watermarking algorithm named invisible–visible watermarking has been proposed as digital representation of the watermarks in real-world applications; in this algorithm, the imperceptibility, payload, and robustness of the invisible and the visible watermarking algorithms are combined with the high quality, resolution [13,14], and the robust display processing capabilities [14,15,16,17] incorporated into several electronic devices. This new watermarking technology can be classified by its exhibition stage as unseen–visible watermarking (UVW) [18,19,20,21] and imperceptible–visible watermarking (IVW) [22,23,24,25], as shown in Figure 1.

The UVW algorithms [18,19,20,21] embed imperceptibly a watermark pattern through a visible watermarking strategy and take advantage of the image enhancement functions provided by the display devices, such as brightness, contrast, gamma, etc. [26,27], to exhibit without additional computational algorithms the watermark to the human visual system (HVS). The IVW algorithms [22,23,24,25] have been proposed as a solution to the several drawbacks presented in the exhibition stage of the UVW algorithms, where a QR code as a watermark is exhibited to the HVS in gray-color, which reduces its readability and the capacity to be correct decoded by common applications. Considering the mentioned drawbacks, the authors of [23] present a pioneer IVW algorithm that takes advantage of the processing capabilities incorporated into the capture devices to embed and exhibit a QR code as a watermark. This algorithm addresses and improves the drawbacks presented by the UVW algorithms; however, its applicability is restricted to professional capture devices with high storage memory and histogram modulation functions, and the exhibited watermark is shown to the HVS in color, which limits its reading and the information decoding by applications with the ability to read QR codes in color. Conversely, an improved IVW algorithm is presented in [24] that solves the above-mentioned drawbacks. To achieve this, a visible watermarking algorithm in the spatial domain is used to embed a QR code into a color image, and an invisible watermarking algorithm in the frequency domain [28] is used to embed the crucial parameters of the embedding stage. In the exhibition stage, the algorithm takes advantage of the processing capabilities of the electronics devices to exhibit the watermark pattern through a strategy named binarization function.

In this paper, the algorithm reported in [23] is improved to deliver additional information about the visual content of images via QR codes exhibited in black-and-white color, maintaining its capability to be exhibited in color; besides, it can be applied to information security tasks that do not require strong robustness. The proposed improvements consider in the embedding stage the human visual perception based on the just noticeable distortion (JND) [29] and the analysis of the histogram distortion induced by the embedding strategy [25]. In the exhibition stage, the algorithm takes advantages of the processing capabilities of the electronic devices to implement a histogram binarization function and exhibit the watermark in black-and-white color. The experimental results show that the proposed algorithm is capable of embedding a watermark maintaining a high imperceptibility and improving the performance of the previously reported algorithms [23,24], and its robustness to common geometric and image processing attacks is higher in several aspects compared with article [24], including JEPG and JPEG2000 compression.

The rest of this paper is organized as follows. Section 2 shows the related works of the IVW algorithms. The procedure of the proposed algorithm is presented in Section 3. The experimental results and a performance comparison with the algorithms [23] and [24] are shown in Section 4 and Section 5, respectively. A discussion is presented in Section 6. Finally, this work concludes in Section 7.

2. Related Works

In this section, the IVW algorithms [23] and [24] are described briefly. These algorithms are considered fundamental works and are used to compare the performance of the proposed algorithm.

2.1. Imperceptible Visible Watermarking (IVW)

The authors of [23] reported a pioneer IVW algorithm that is based on the observation of how several capture devices are equipped with robust image processing capabilities used to provide post-capture functions such as histogram modulation, which is considered as the exhibition strategy, as shown in Figure 2.

In the algorithm, the appropriate embedding region is selected as the largest homogeneous region with smallest variance R_k and size M × N, from the color image I_k, where k∈{R, G, B}. Here, the appropriate mean value m is satisfied by Equation (1).

$$m=\left\{\begin{array}{l}\min ({\overline{R}}_R,{\overline{R}}_G,{\overline{R}}_B), if \min ({\overline{R}}_R,{\overline{R}}_G,{\overline{R}}_B)\ge T,\\ \max ({\overline{R}}_R,{\overline{R}}_G,{\overline{R}}_B), otherwise,\end{array}\right.$$

(1)

where ${\overline{R}}_k$ and T = 2 correspond to the mean value of the embedding region in the color channel k and an empirical strength of the embedding strategy, respectively. To embed a binary watermark pattern W = {w_i|w_i = 0,1, and i = 1, 2, 3,…, M × N}, a color channel of the embedding region is selected among R_R, R_G, and R_B with the smallest mean value. Finally, the embedding strategy is satisfied by Equation (2) if $\min ({\overline{R}}_R,{\overline{R}}_G,{\overline{R}}_B)\ge T$ or by Equation (3) if $\min ({\overline{R}}_R,{\overline{R}}_G,{\overline{R}}_B)<T$, respectively.

$$R_w(p_i)=\left\{\begin{array}{l}m, if w_i=0 and R_k(p_i)<m,\\ m-T, if w_i=1 and R_k(p_i)>m-T,\end{array}\right.$$

(2)

$$R_w(p_i)=\left\{\begin{array}{l}m+T, if w_i=0 and R_k(p_i)<m+T,\\ m, if w_i=1 and R_k(p_i)>m,\end{array}\right.$$

(3)

where R_k(p_i) and R_w(p_i) are the i-th pixel value of the embedding and its corresponding watermarked region in the color channel k. Once the watermarked image I_w is obtained, to exhibit the watermark, the algorithm leverages the processing capabilities equipped into the capture devices to execute an exhibition strategy based on the histogram modulation function in the range [α, β], where the range value is obtained as shown in Equation (4).

$$\left\{\begin{array}{l}\alpha =m-T, \beta =m, if\min ({\overline{R}}_R, {\overline{R}}_G, {\overline{R}}_B)\ge T,\\ \alpha =m, \beta =m+T, otherwise.\end{array}\right.$$

(4)

2.2. Improved Imperceptible Visible Watermarking (IIVW)

The authors of [24] proposed an improved version of the IVW algorithm [23] that uses a visible watermarking algorithm in the spatial domain and an invisible water-marking algorithm based on the discrete cosine transform (DCT) [28] to embed imperceptibly a binary watermark into a color image I_k. As is shown in Figure 3, once the watermarked image is obtained, it is distributed to the end-users by on-demand or streaming services, and the exhibition stage leverages the processing capabilities of the display devices to exhibit the watermark to the HVS by the mechanism called binarization function.

In this algorithm, n binary watermarks W = {w_i|w_i = 0,1, and i = 1, 2, 3,…, M × N} are embedded into n non-overlapping embedding regions R_n∈I_k selected with the smallest variance from the color image I_k, k∈{R, G, B}, as shown in Equation (5).

$$R_w\left(p_i\right)=\left\{\begin{array}{l}\max \left(0,{\mu }_{R_n}-\Delta JND\left(p_i\right)\right), if w_i=0,\\ \min \left({\mu }_{R_n}+\Delta JND\left(p_i\right),255\right), if w_i=1,\end{array}\right.$$

(5)

where µ_Rn and R_w(p_i) are the mean value of the n-th region and the i-th pixel value of the watermarked region, respectively; ΔJND(p_i) is the JND-based embedding strength [29], where Δ = 0.5 corresponds to the imperceptibility factor. After this process, a binary expression of the crucial parameters {n, (μ_R1, k₁), (μ_R2, k₂), (μ_R3, k₃), …, (µ_Rn, k_n)} is obtained to be embedded into BN blocks of size (8 × 8) through the imperceptible DCT-based watermarking algorithm called Quantization Index Modulation Dither-Modulation (QIM-DM) [28]. Finally, to exhibit the watermarks to the HVS, the QIM-DM inverse process is used to recover the crucial parameters used by the mechanism called the binarization function that is given by Equation (6).

$$I_E=\left\{\begin{array}{l}1, if I_k\ge {\mu }_R,\\ 0, otherwise,\end{array}\right.$$

(6)

where I_E corresponds to the exhibited image.

2.3. Oportunities to Improve the IVW algorithms

In the algorithm [23], the performance evaluation is reported in terms of the PSNR in the range of 49 to 69 dB for watermarks with sizes of 320 × 320 and 48 × 48, respectively. However, its embedding strategy, which is given by Equation (2) or Equation (3), modifies the histogram of the embedding region in the range of [m − T, m] or [m, m + T], as is shown in Figure 4a,b respectively.

In the improved IVW algorithm [24], the performance evaluation is reported in terms of the PSNR in the range of 47.97 to 49.5 dB until three binary watermarks are embedded at the same time into a color image. However, in this algorithm, the embedding strategy given by Equation (5) modifies the histogram of the embedding region, as shown in Figure 5.

As is described in Section 1, the algorithm in [24] improves the algorithm in [23] mainly in the exhibition stage where a QR code as watermark is exhibited in color, limiting its capability to be read. However, both algorithms have a strong similarity in their embedding strategies, which can be analyzed through the histogram of their corresponding watermarked regions. Comparing Figure 4 and Figure 5, both algorithms accumulate the watermark information in the defined upper and lower boundaries with respect to the mean value; in the case of [23], there are [m − T, m] and [m, m + T] to 4a,b respectively, and [µ_Rn −ΔJND(p_i), µ_Rn + ΔJND(p_i)] to the improved IVW [24]; however, in [23], the watermark is exhibited in color to the HVS, representing an applicability limitation with respect to [24], where the watermark is exhibited in a black and white color that is a better option for QR codes as a watermark. Considering these similarities in the embedding stage, the histogram study of the IVW algorithms reported in [23,24], and observing Figure 4 and Figure 5, in this paper, a new improved version of the IVW algorithms is proposed, where several weaknesses of the algorithm in [23] are improved by the performance of the algorithm in [24]. The embedding strategy is improved, substituting the empirical embedding strength T ≥ 2 by the luminance-based just noticeable difference (JNDL), ensuring a high watermark imperceptibility under normal viewing condition. A new exhibition strategy called histogram binarization function is proposed as well, which exhibits the watermark pattern and binarizes the histogram of the watermarked image with respect to the mean value of the embedding region.

3. Proposed Framework

In the new improved IVW algorithm, a watermark pattern with several characteristics such as owner logotype or QR code can be embedded to provide in color copyright protection or deliver information about the image content; the procedure is divided into the embedding and exhibition stages. The embedding stage based on [23] is improved by the similarities of the algorithms reported previously [25], the histogram study of the invisible–visible watermarking algorithms [18,19,20,21,22,23,24,25], and the incorporation of the JNDL as embedding strength [29], as shown in Figure 6. The exhibition stage leverages the processing capabilities of the display devices to execute an exhibition strategy called histogram binarization function. In this section, both stages are described in detail.

3.1. Embedding Stage

Given an original color image I_k = {I_k(p_i)|0 ≤ I_k(p_i) ≤ 255, i = 1, 2, 3,…, w_I × h_I}, where I_k(p_i) corresponds to the i-th pixel in the color channel k∈{R, G, B}, w_I and h_I are the width and height of the image, and a binary watermark pattern W = {w_i|w_i = 0,1, and i = 1, 2, 3, …, M × N}, with size M × N, the embedding procedure is given by the following steps.

• Largest homogeneous region selection. In the beginning of the embedding procedure, an embedding region R, with the same size of the watermark, is selected as the largest homogeneous region from I_k; in other words, it must satisfy the minimum variance given by Equation (7).

$$\underset{ R\in I_k}{\min }{\left(\frac{1}{\left(M\times N\right)-1}{\displaystyle \sum _{i=1}^{M\times N}\left|R(p_i)-\mu *\right|}\right)}^2,$$

(7)

where R(p_i) and µ* are the i-th pixel of the largest homogeneous region and its corresponding mean value.

To ensure the homogeneity of the embedding region, a high percentage of its pixels must have the same intensity value; however, if the percentage of pixels is less than a predefined homogeneity threshold Th as shown in Equations (8) and (9), a smoothing operation can be used iteratively until minimize the variance among them.

$$\underset{ }{{\displaystyle \sum _{i=1}^{M\times N}G \left(R(p_i)-\mu *\right)}\le M\times N}\times Th,$$

(8)

where

$$\underset{ }{G(l)=\left\{\begin{array}{l}1, if l=0,\\ 0, otherwise.\end{array}\right.}.$$

(9)

Occasionally, the smoothing operation generates a strong visual degradation in the embedding region, which is perceived by the HVS under normal view conditions. To minimalize the visual degradation, instead of using a smoothing operation, the pixels with the most dispersed values are selected, and its values are replaced by others nearest to the mean value µ*; this strategy minimizes the variance and homogenizes the embedding region; empirical analysis shows that 10% of the pixels in R can be used without affecting the visual content.

2.

Appropriate color channel selection. Once the largest homogeneous region R is obtained, an appropriate color channel is selected to embed the corresponding watermark W. According to the previously mentioned, the mean values of each color channel of the embedding region in an RGB color model (µ_RR, µ_RG, and µ_RB) are obtained by Equation (10), and the appropriate color channel must satisfy Equation (11); for example, if the optimal mean value is satisfied by µ_R = µ_RG, in which µ_RG is the minimum or maximum mean value, the appropriate color channel corresponds to the green one.

$$\left\{\begin{array}{c}{\mu }_{RR}=\frac{1}{M\times N}{\displaystyle \sum _{i=1}^{M\times N}{R}_R(p_i),}\\ {\mu }_{RG}=\frac{1}{M\times N}{\displaystyle \sum _{i=1}^{M\times N}{R}_G(p_i),}\\ {\mu }_{RB}=\frac{1}{M\times N}{\displaystyle \sum _{i=1}^{M\times N}{R}_B(p_i),}\end{array}\right.$$

(10)

$${\mu }_R=\left\{\begin{array}{l}\min \left({\mu }_{RR}, {\mu }_{RG}, {\mu }_{RB}\right), if \min \left({\mu }_{RR}, {\mu }_{RG}, {\mu }_{RB}\right)\ge JNDL\left(\min \left({\mu }_{RR}, {\mu }_{RG}, {\mu }_{RB}\right)\right),\\ \max \left({\mu }_{RR}, {\mu }_{RG}, {\mu }_{RB}\right), otherwise,\end{array}\right.$$

(11)

where JNDL(x) corresponds to luminance-based JND [29,30].

3.

Embedding strategy. In the proposed algorithm, the empirical embedding strength T is improved by the just noticeable difference (JND), which represents the maximum luminance variation perceived by the HVS [29,30]. In the literature, several JND algorithms in the spatial and frequency domain have been reported to estimate a numerical value [30,31]. Its use is typically defined by the application [31,32,33,34], which in the proposed algorithm concerns the image watermarking. An embedding strength that allows embedding imperceptibly a visible watermark pattern is required; in this way, the JNDL function [29] is considered to estimate the embedding strength and ensure the watermark imperceptibility of the new improved IVW algorithm. Thus, if the appropriate color channel to embed the watermark pattern depends on ensuring that the largest homogeneous region R has a variance value nearest to 0, the JNDL-based embedding strength can be obtained with respect to the corresponding mean value µ_R, as shown in Equation (12).

$$JNDL\left({\mu }_R\right)=\left\{\begin{array}{c}-\frac{1}{8}{\mu }_R+6, {\mu }_R\in (0,32)\\ -\frac{1}{32}{\mu }_R+3, {\mu }_R\in (33,64)\\ \frac{1}{96}{\mu }_R+\frac{1}{3}, {\mu }_R\in (65,255)\end{array} , \right.0\le {\mu }_R\le 255.$$

(12)

Once the largest homogeneous regions and its appropriate color channel are selected, an embedding strategy based on the JNDL is adopted to embed imperceptibly the binary watermark W. The embedding strategy is given by two ways given by Equations (13) and (14), which are satisfied when µ_R = min(µ_RR, µ_RG, µ_RB) or µ_R = max(µ_RR, µ_RG, µ_RB), respectively.

$$R_w(p_i)=\left\{\begin{array}{l}\max (0,{\mu }_R-\Delta ), if w_i=0 ,\\ \min ({\mu }_R,255), otherwise ,\end{array}\right.$$

(13)

$$R_w(p_i)=\left\{\begin{array}{l}\max (0,{\mu }_R), if w_i=0 ,\\ \min ({\mu }_R+\Delta ,255), otherwise,\end{array}\right.$$

(14)

where R_w(p_i) corresponds to the i-th pixel of the watermarked region and Δ = round(JNDL(µ_R)) represents the embedding strength based on the JNDL with respect to the optimal mean value µ_R. According to the embedding strategy given by Equations (13) and (14), the histogram of the watermarked region is modified as shown in Figure 7, ensuring the same distortion as that of the algorithms reported previously [23,24]. In both embedding strategies, the bits w_i = 0 and w_i = 1 of the binary watermark are embedded imperceptibly into the pixels with low and high intensity respectively, and with respect to the optimal mean value µ_R. In Equation (13), the pixels with low intensity are far from the optimal mean value in −Δ, as is shown in Figure 7a; whereas in Equation (14), the pixels with high intensity are far from the mean value in +Δ, as is shown in Figure 7b.

4.

Finally, the watermarked image I_w = {I_w(p_i)|0 ≤ I_w(p_i) ≤ 255, i = 1, 2, 3, …, w_Iw × h_Iw} with size w_Iw × h_Iw is obtained through the reconstruction of the watermarked region R_w.

According to the embedding procedure survey in

Table 1. Embedding procedure survey.

Procedures	Details
Input	Original image Ik and binary watermark W pattern
Step 1	The largest homogeneous region R with the same size of the watermark pattern is selected from the original color image.
Step 2	The mean values µRR, µRG, and µRB of each RGB color channel are obtained, as formulated by Equation (10).
Step 3	According to Equation (11), the optimal mean value µR is obtained. Moreover, the minimum or maximum mean value selected as the optimal one µR, also determines the appropriate color channel to embed the binary watermark pattern W.
Step 4	Ensuring the homogeneity of the embedding region R, the embedding strength JNDL-based is obtained with respect to the optimal mean value µR, as shown in Equation (12).
Step 5	According to Equation (13) or Equation (14), the binary watermark pattern W is embedded into the appropriate embedding region.
Step 6	Finally, once the watermark pattern W is embedded, the watermarked region is reconstructed, and the watermarked image Iw is obtained.
Output	Watermarked image Iw

and considering that n non-overlapping homogeneous regions can be found into an image, these steps can be performed iteratively until they have embedded n binary watermark patterns imperceptibly to the HVS. Although the proposed algorithm is designed to embed a watermark with its original size, it is not limited to leverage the robustness of the QR codes and the characteristics of owner logotypes to resize it if the original image does not have homogeneous regions with the same size as the watermark.

3.2. Exhibition Stage

In the final stage of the proposed framework, the watermark is exhibited to the HVS by a novel histogram binarization function that leverages the characteristics of the embedding strategy and the histogram distortion induced with respect to the optimal mean value; also, this strategy can be easily implemented in several electronic devices.

Histogram Binarization Function

The two approaches of the embedding strategy given by Equations (13) and (14) modify the histogram of the appropriate color channel as shown in Figure 7, and its characteristics are leveraged to propose the novel histogram binarization function as the exhibition stage, as shown in Figure 8. The main idea of this function is to binarize the watermarked image I_w with respect to the main histogram characteristic of the watermarked region, such as the optimal mean value µ_R, through the two different functions formulated by Equations (15) and (16).

$$I_E\left(p_i\right)=\left\{\begin{array}{l}1, if {\mu }_R=\min \left({\mu }_{RR}, {\mu }_{RG}, {\mu }_{RB}\right) and I_w(p_i)\ge {\mu }_R,\\ 0, otherwise,\end{array}\right.$$

(15)

$$I_E\left(p_i\right)=\left\{\begin{array}{l}0, if {\mu }_R=\max \left({\mu }_{RR}, {\mu }_{RG}, {\mu }_{RB}\right) and I_w(p_i)\le {\mu }_R,\\ 1, otherwise,\end{array}\right.$$

(16)

where Equations (15) and (16) are satisfying the embedding strategy given by Equations (13) and (14), respectively. Finally, I_E(p_i) corresponds to the i-th pixel of the exhibited image I_E.

4. Experimental Results

According to [24], the robustness against malicious or non-malicious attacks becomes meaningless, since the main approach of the IVW algorithms is the additional information delivery to the image visual content. However, the copyright application of the proposed algorithm requires reconsidering a robustness evaluation [35]. The experimental results show their applicability to any class of images, and the performance evaluation is provided against common geometric and image processing attacks including JPEG and JPEG2000 compression. Additionally, this section includes a fair comparison against several previously reported algorithms.

4.1. Dataset Detail and Binary Watermark Pattern

The dataset used to evaluate the performance of the proposed algorithm is obtained from the uncompressed color image dataset (UCID) [36], which contains 1338 uncompressed color images with 512 × 384 size and TIFF format. As shown in Figure 9, three binary watermark patterns are used for the performance evaluation, where Figure 9a,c are two owner logotypes with sizes of 150 × 150 and 134 × 39, respectively. Figure 9b corresponds to a QR code of 116 × 116 size and error correction label L that corresponds to a 7% of loss information; additionally, to specifically evaluate QR codes, error correction levels M, Q, and H are capable of information decoding with 15%, 25%, and 30% of loss information [37], respectively.

4.2. Performance Metrics for Image Quality Degradation, Imperceptibility, and Robustness

To evaluate the performance of the proposed algorithm in terms of the watermarked image quality degradation, imperceptibility of the watermark pattern, and robustness against the common geometric and image processing attacks listed in

Table 2. Common geometric attacks used in the experiment.

Attack	Specification
Rotation	Angle 45°, {90°,180°,270°}
Rescaling	0.75×, {0.9×, 1.1×, 1.5×}
Translation with cropping	x = 50, y = 50; {x = −50, y = −50}
Flip transformation	Horizontal—{Vertical} direction
Affine Transformation	[1,0.3,0;0.3,2,0;0,0,1]
Aspect ratio	[0.8,0,0;0,1.5,0;0,0,1]
Shearing	x-direction [1 0 0; 0.2 1 0; 0 0 1] {y-direction [1 0.2 0; 0 1 0; 0 0 1]}
Radial	------------

{ } additional experimental results refer to [40].

and

Table 3. Common image processing attacks used for the experiment.

Attack		Specification
		Image Watermarked by
		FI	QR	UNAM
Filter	Average	7 × 7	3 × 3	5 × 5
	Median	5 × 5	3 × 3	9 × 9
	Gaussian	9 × 9	3 × 3	9 × 9
Noise contamination	Impulsive	density = 0.5	density = 0.3	density = 0.5
	Gaussian	µ = 0, σ2 = 0.0004	µ = 0, σ2 = 0.00004	µ = 0, σ2 = 0.0005
Sharpening	------------	Radius = 4, Amount = 1	Radius = 0.5, Amount = 2	Radius = 5, Amount = 2
Blurring	------------	Radio = 4	Radio = 1.7	Radio = 4
Histogram Equalization	------------	[0, 0.33]	[0, 0.33]	[0, 0.33]
Compression	JPEG	QF:90	QF:90	QF:75
	JPEG2000	CR:10	CR:10	CR:15

, the peak signal to noise ratio (PSNR), structural similarity index (SSIM) [38], normalized color difference (NCD) [39], normalized correlation (NC), and bit error rate (BER) are used.

Image quality degradation is generally measured by the performance metric PSNR, as shown in Equation (17).

$$PSNR(I_k,I_w)=10{\log }_{10} \frac{\max {(I_k(p_i))}^2}{MSE},$$

(17)

$$MSE\left(I_k,I_w\right)=\frac{1}{w_{Iw}\times h_{Iw}}{\displaystyle \sum _{i=1}^{w_{Iw}\times h_{Iw}}{\left[I_k(p_i)-I_w(p_i)\right]}^2},$$

(18)

where MSE given by Equation (18) is the mean square error between the original I_k and the watermarked image I_w, and w_Iw = w_I and h_Iw = h_I are the width and height of the watermarked image. Typically, when PSNR∈ (37 dB, inf), it means that the embedding process generates a low-quality degradation in the watermarked image, and a value nearest to 0 dB represents a higher-quality degradation [41].

The imperceptibility can be defined as a measure of similarity between the original and its corresponding watermarked image. In addition to the PSNR, other metrics to measure the imperceptibility are the SSIM and the NCD, where the SSIM represents an individual comparison of three image parameters such as luminance, contrast, and structure, as shown in (19).

$$SSIM\left(I_k,I_w\right)=\frac{\left(2{\mu }_{I_k}{\mu }_{I_w}+C_1\right)\left(2{\sigma }_{I_k, I_w}+C_2\right)}{\left({\mu }_{I_k}^2+{\mu }_{I_w}^2+C_1\right)\left({\sigma }_{I_k}^2{\sigma }_{I_w}^2+C_2\right)},$$

(19)

where µ_I and µ_Iw are the mean value of I_k and I_w, respectively, σ_I,_Iw and σ²_X are the corresponding covariance and variance between I_k and I_w, respectively, and C₁ = (k₁L)² and C₂ = (k₂L)² are two constants used to stabilize the denominator, where k₁ = 0.01, k₂ = 0.03 and L = 255 correspond to the dynamic range of 8 bits per pixel. The NCD is another performance metric of imperceptibility that measures in the CIELAB color space the similarity between the original image I_k and its corresponding watermarked image I_w; it is formulated as shown in (20).

$$NCD\left(I_k,I_w\right)=\frac{{\displaystyle \sum _{x=1}^{h_{Iw}}{\displaystyle \sum _{y=1}^{w_{Iw}}\left(\sqrt{{\left(\Delta L(x,y)\right)}^2+{\left(\Delta a(x,y)\right)}^2+{\left(\Delta b(x,y)\right)}^2}\right)}}}{{\displaystyle \sum _{x=1}^{h_I}{\displaystyle \sum _{y=1}^{w_I}\left(\sqrt{{\left(L(x,y)\right)}^2+{\left(a(x,y)\right)}^2+{\left(b(x,y)\right)}^2}\right)}}},$$

(20)

where ΔL(x,y), Δa(x,y), and Δb(x,y) of the numerator are the difference between the luminance L and the spaces a and b of the original image I and the watermarked image I_w at the position (x,y). In the denominator L, a and b are the original image color spaces. Generally, the SSIM and NCD values nearest to 1 and 0, respectively, represent a strong similarity between the original image and its corresponding watermarked image.

The NC and the BER are two metrics used in this paper to evaluate the similarity between the original W and the recover watermark patter W’ as well as the robustness against the common geometric and imagen processing attacks listed in Table 2 and Table 3, respectively. The NC and the BER are denoted by Equations (21) and (22).

$$NC(W,W^{\prime })=\frac{{\displaystyle \sum _{i=1}^{M\times N}\left[w_i\times {w^{\prime }}_i\right]}}{\sqrt{{\displaystyle \sum _{i=1}^{M\times N}{\left(w_i\right)}^2}}\times \sqrt{{\displaystyle \sum _{i=1}^{M\times N}{\left({w^{\prime }}_i\right)}^2}}},$$

(21)

$$BER(W, W^{\prime })=\frac{{\displaystyle \sum _{i=1}^{M\times N}\left(w_i\otimes {w^{\prime }}_i\right)}}{M\times N},$$

(22)

where ⨂, w_i, and w’_i are the XOR function, i-th pixel of the original W, and the recover watermark W’ = {w’_i|w’_i = 0, 1, and i = 1, 2, 3, …, M × N}, with size M × N, respectively. According to Equations (21) and (22), the values of NC and BER nearest to 1 and 0, respectively, represent a strong similarity and robustness against common geometric and imagen processing attacks.

Additionally, to evaluate the performance of the proposed algorithm with respect to the geometric and the image processing attacks listed in Table 2 and Table 3, a fair performance comparison with the most sophisticated and recent IVW algorithms [23,24] is provided.

4.3. Suitability for Any Class of Images

Similar to the most recent and sophisticated reported algorithms [24], the proposed algorithm can use any class of images to embed any binary watermarks, mainly QR codes. As mentioned above, to evaluate the performance of the proposed algorithm, the binary watermark patterns FI, QR, and UNAM are used.

Although not all images of the UCID dataset ensure the largest homogeneous region to embed the mentioned watermark patterns, the proposed algorithm is able to obtain a low visual quality degradation and high imperceptibility; e.g.,

Table 4. Visual quality degradation and imperceptibility of non-attacked watermarked images.

Watermark	FI, 150 × 150 (22,500 Bits)		QR code, 116 × 116 (13,456 Bits)		UNAM, 134 × 39 (5226 Bits)
Host Image	Largest homogeneous region		Largest homogeneous region		Largest homogeneous region
	Ensuring	Non-ensuring	Ensuring	Non-ensuring	Ensuring	Non-ensuring
	ucid00146.tif	ucid00160.tif	ucid00449.tif	ucid00775.tif	ucid00306.tif	ucid00262.tif
Watermarked image
µR	255	22	255	58	181	242
Δ	3	3	3	1	2	3
PSNR (dB)	56.52	36.01	59.70	36.01	63.12	36.75
SSIM	0.99864	0.97417	0.99930	0.97915	0.99989	0.98863
NCD	0.00144	0.01228	0.00085	0.01053	0.00053	0.00426
Exhibited watermark.
NC	1	1	1	1	1	1
BER	0	0	0	0	0	0

shows three images that ensure the largest homogeneous region with the minimum required variance and three that do not ensure it, and its corresponding exhibited watermark pattern. Additionally, its performance evaluation in terms of quality degradation, imperceptibility, and robustness is shown, wherein both kinds of images offer a good quantitative and visual performance in non-attacked images.

4.4. Image Quality Degradation and Imperceptibility Analysis

In an efficient watermarking algorithm, the image quality degradation is performed by the PSNR, which is strongly related with the imperceptibility that in terms of SSIM and NCD performs similarly between the original and its corresponding watermarked image. To obtain this evaluation, the binary watermark patterns FI, QR, and UNAM are embedded individually into the 1338 images contained within the UCID dataset, and the PSNR, SSIM, and NCD of the images with the same mean value 0 ≤ µ_R ≤ 255 in the embedding region are averaging. For the FI watermark, the average values of PSNR, SSIM, and NCD with respect to the mean value µ_R are shown in Figure 10a–c, respectively, those with respect to the QR watermark are shown in Figure 10d–f, respectively; and finally, those with respect to the UNAM watermark are shown in Figure 10g–i, respectively.

Generally, if PSNR ≥ 37 dB, the watermarked image quality is acceptable, and the watermark is considered invisible to the HVS [39]. In this way, Figure 10a,d,g show the average of the performance quality metric with a value greater than 37 dB. Considering all the images of the mentioned dataset, the average of quality for each embedded watermark pattern FI, QR, and UNAM is greater than 37 dB, 43.50 dB, and 53 dB, respectively; in this context, the general quality value (GQV) of the proposed algorithm is 37 dB ≤ GQV ≥ 53 dB.

In the experiment, if SSIM ≥ 0.93 and NCD ≤ 0.05, the original image and its corresponding watermarking are considered similar, and the embedded watermark is considered as imperceptible to the HVS. Figure 10b,c,e,f,h,i show the imperceptibility metric SSIM and NCD with respect to the mean value µ_R of the appropriate embedding region. As can be observed, the obtained SSIM and NCD values are greater than 0.93 and smaller than 0.05, respectively. Similar to the GQV, a general similarity value (GSV) and general imperceptibility value (GIV) for watermarks FI, QR, and UNAM are obtained. A survey of the general performance for the UCID dataset is shown in

Table 5. General performance for UCID.

Image Size		GQV (dB)	GSV	GIV
Watermark	Length (Bits)
FI	22,500	37	0.9795	0.0188
QR	19,600	43.5	0.9920	0.0073
UNAM	5226	53	0.9982	0.0017

.

4.5. Robustness against Common Geometric Attacks

The easy geometric edition of digital images by mobile applications [42] or dedicated software [43] requires the proposed algorithm performance evaluation against common geometric attacks [35], such as those listed in Table 2. In this paper, the QR codes are considered the most sensible watermark patterns, because additionally to the recognition, they also required the stored information decoding.

In this context, an image is watermarked by the QR watermark;

Table 6. Robustness against geometric attacks.

	Rotation 45°	Rescaling 0.75X	Affine Transformation	Aspect Ratio
Attacked image
Exhibited watermark
QR decoded	Yes	Yes	No	No
	Translation with cropping	Flip transformation	Shearing x-direction	Radial Transformation
Attacked image
Exhibited watermark
QR decoded	Yes	Yes	Yes	Yes

shows the robustness results of the attacked image and its corresponding exhibited watermark. However, due to paper length, several results regarding Rotation, Translation with cropping, Flip transformation, and Shearing attacks are omitted in Table 6 but can be accessed from [40], including similar experimental results to FI and UNAM watermarks.

The high performance of the proposed algorithm against geometric attacks allows the recognition and information decoding of the watermark pattern QR; however, robust affine transformation and aspect ratio attacks avoid the correctly contained information decoding from QR with error correction level L. To solve this drawback, a QR code with a higher error correction level could be used.

4.6. Robustness against Common Image Processing Attacks

To evaluate the performance of the proposed algorithm against malicious or non-malicious image processing attacks, an image is watermarked individually by the three proposed watermarks.

As example,

Table 7. QR watermarked image after image processing attacks.

Attack	Filter			Noise Contamination
	Average	Median	Gaussian	Impulsive	Gaussian
Attacked Image
PSNR (dBs)	30.10	31.44	38.49	9.92	45.05
Attack	Sharpening	Blurring	Histogram Equalization	Compression
				JPEG	JPEG2000
Attacked Image
PSNR (dBs)	39.31	29.95	8.02	36.72	41.07

shows the QR watermarked image after being attacked by the common image processing attacks listed in Table 3.

Table 8. Results of common image processing attacks.

Attack		Watermark
		FI, 150 × 150 (22,500 Bits)		QR code, 116 × 116 (13,456 bits)		UNAM, 134 × 39 (5226 Bits)
Filter	Average		Kernel = 7 × 7 NC = 0.609263 BER = 0.275933		Kernel = 3 × 3 NC = 0.663281 BER = 0.191032		Kernel = 5 × 5 NC = 0.580299 BER = 0.220100
	Median		Kernel = 5 × 5 NC = 0.813774 BER = 0.087754		Kernel = 3 × 3 NC = 0.964520 BER = 0.011331		Kernel = 9 × 9 NC = 0.975498 BER = 0.008761
	Gaussian		Kernel = 9 × 9 NC = 0.845513 BER = 0.089324		Kernel = 3 × 3 NC = 0.724451 BER = 0.155122		Kernel = 7 × 7 NC = 0.814519 BER = 0.105576
Noise Contamination	Impulsive		density = 0.5 NC = 0.500342 BER = 0.264525		density = 0.3 NC = 0.696087 BER = 0.141795		density = 0.5 NC = 0.610032 BER = 0.263219
	Gaussian		µ = 0, σ2 = 0.0004 NC = 0.372110 BER = 0.349014		µ = 0, σ2 = 0.00004 NC = 0.652150 BER = 0.225626		µ = 0, σ2 = 0.0005 NC = 0.335012 BER = 0.329954
Sharpening			Radius = 4, Amount = 1 NC = 0.567612 BER = 0.270045		Radius = 0.5, Amount = 2 NC = 0.665532 BER = 0.206101		Radius = 5, Amount = 2 NC = 0.898715 BER = 0.056818
Blurring			Radio = 4 NC = 0.620014 BER = 0.265244		Radio = 1.7 NC = 0.683341 BER = 0.180042		Radio = 4 NC = 0.470032 BER = 0.351380
Histogram Equalization			[0, 0.33] NC = 1 BER = 0		[0, 0.33] NC = 1 BER = 0		[0, 0.33] NC = 1 BER = 0
Compression	JPEG		QF:90 NC = 0.481014 BER = 0.335214		QF:90 NC = 0.372457 BER = 0.380524		QF:75 NC = 0.590581 BER = 0.228475
	JPEG2000		CR:10 NC = 0.654781 BER = 0.220441		CR:10 NCD = 0.460211 BER = 0.352850		CR:15 NC = 0.618700 BER = 0.276114

shows the visual and numerical robustness results obtained from the images watermarked by the three mentioned watermarks. For the experimental evaluation, the quality of the exhibited watermarks is performed in terms of the NC and BER, and the watermarked images have been attacked by the maximum attack strength that the watermarks can resist; in this way, for watermarked images by QR, a maximum attack strength allows the QR code information decoding to watermarked images by FI or UNAM, a maximum attack strength also allows recognizing the watermark; the specification to each attack is also provided. The obtained results show the robustness of the proposed algorithm against Filter, Noise contamination, Sharpening, Blurring, and Histogram Equalization; in all those cases, visual results of the exhibited watermarks can be recognized easily by the HVS. Numerical results in terms of NC and BER show diverse values nearest to 1 or 0, respectively, which can be considered as the maximum or minimum degradation to recognize and/or decode the watermark pattern. Obtained results from the QR watermarked image show that the watermark can be exhibited with enough quality to be decoded after being attacked; however, its compression robustness is limited due to the QR watermark characteristics and its error correction level; in this way, the QR is only recognized after compression attacks.

Similar to the results reported in [24], Table 8 shows that watermarks such as owner logotypes FI and UNAM are more robust against common image processing attacks than QR logotypes, as can be observed in the compression evaluation where a UNAM watermark is recognized after being compressed by JPEG and JPEG2000 with a quality factor QF:75 and compression rate CR:15, respectively, while to recognize FI and QR watermarks, the compression attack must be decreased. In general, the proposed algorithm preserves a good imperceptibility and robustness against geometric and image processing attack.

5. Comparison with Similar Algorithms

In this section, the performance of the proposed algorithm is compared with the algorithms [23,24] under the same characteristics such as hardwater capabilities, watermark, and image dataset.

To obtain a trustworthy result, 600 images randomly selected from the UCID dataset were watermarked by the algorithms [23,24], and the proposed watermark used in this evaluation is the QR code, because it is considered the most sensible watermark.

Table 9. Robustness comparison against image processing attacks.

Attack		Specification	Imperceptible Visible Watermarking
			Proposed			[23] IVW			[24] IIVW
			QR Code, 116 × 116			QR Code, 116 × 116			QR Code, 116 × 116
			PSNR (dB)	BER	NC	PSNR (dB)	BER	NC	PSNR (dB)	BER	NC
Filter	Average	7 × 7	30.0994	0.2086	0.6546	31.1002	0.8995	−0.7892	29.6237	0.1994	0.4205
	Median	5 × 5	31.4374	0.0213	0.9550	31.4369	0.9785	−0.9548	30.4635	0.3424	0.9288
	Gaussian	9 × 9	38.4886	0.1642	0.7169	38.4905	0.9971	−0.9939	38.7412	0.1922	0.7011
Noise contamination	Impulsive	density = 0.5	9.94921	0.1516	0.6860	9.9675	0.8506	−0.6917	9.5478	0.1820	0.6104
	Gaussian	µ = 0, σ2 = 0.0004	45.0596	0.2267	0.6301	45.0465	0.8259	−0.6299	45.3874	0.2163	0.5110
Sharpening	------------	Radius = 4, Amount = 1	33.6065	0.2163	0.6442	33.6062	0.8041	−0.5815	32.9437	0.2335	0.6157
Blurring	------------	Radio = 4	29.9460	0.1958	0.6721	29.9468	0.9061	−0.8031	28.3472	0.1742	0.6521
Histogram Equalization	------------	[0, 0.33]	8.0202	0	1	8.0200	0.4004	−0.0907	8.0742	0	1
Compression	JPEG	QF:90	36.7214	0.4067	0.3750	36.7312	0.6970	−0.3472	35.9855	0.3210	0.3350
	JPEG2000	CR:10	41.0719	0.3546	0.4566	41.0916	0.7330	−0.4242	40.4582	0.3308	0.4544

shows the evaluation results of quality and similarity of the watermarked images and the exhibited watermark patterns, respectively; this evaluation is obtained after treating the watermarked images by the image processing attacks listed in Table 3. As can be seen, the algorithm [23] shows better quality results due to its embedding strength being a fixed and empirical value of 2; in the case of the proposed algorithm, and in [24], the embedding strength is not a fixed value and could be selected by the corresponding strategy in the range of [0,6]. With respect to the similarity evaluation, [23] shows a poor performance due to the algorithm exhibiting the watermark in color. Finally, the proposed algorithm shows a better performance than [23] and one that is a little bit higher than [24].

Figure 11a,b shows a BER comparison of the exhibited QR watermark after compression by JPEG and JPEG 2000, respectively; in both images L, M, Q, and H correspond to the QR error correction level.

Table 10. Image quality and watermark imperceptibility comparison.

Algorithm	Image Quality	Image Imperceptibility
	PSNR	SSIM	NCD
[23]	50.4186	0.9978	0.0040
[24]	48.5215	0.9951	0.0074
Proposed	51.2145	0.9963	0.0060

compares the image quality degradation and watermark imperceptibility of the proposed algorithm against those of [23] and [24]. In this evaluation, we used the 600 images that have been watermarked by the QR code with the maximum embedding strength. As can be seen, the obtained results for [23] are similar to those proposed; however, as explained previously, the embedding strength of [23] corresponds to the empirical value of 2, while the embedding strength of [24] and the proposed algorithm corresponds to Δ = 5.

6. Discussion and Analysis

To improve the previously reported algorithm [23], the proposed algorithm takes advantage of the histogram distortion induced by the embedding strategy. The obtained results show that the proposed algorithm has better performance than the most sophisticated and recent algorithm [24]. These results are obtained considering that the embedding region is selected as the largest homogeneous region satisfying the minimum variance; however, in a real-world scenario, the captured scenes can integrate many homogeneous regions that make it possible to embed multiple watermarks. Figure 12a shows an image that once analyzed chooses three non-overlapping regions to embed the proposed watermarks; Figure 12b–d show the corresponding exhibited watermarks.

The relation between the embedding and the exhibition strategy of [23,24], and the proposed algorithm is analyzed by the histogram distortion of the embedding region; this relation makes it possible to share the exhibition strategies among them, mainly the histogram modulation-based strategy. Figure 13a is the same multi-watermarked image of Figure 12a, and Figure 13b–d show the results to share among them regarding histogram modulation as the exhibition strategy. These results are possible due to all the algorithms inducing the same histogram distortion in the appropriate embedding region; however, this strategy is not feasible to exhibit QR codes.

In [24], the reported algorithm is applied on streaming video and images services, where end-users can exhibit the watermark through clicking a media-player button, which executes the corresponding exhibition strategy. To exhibit the watermark, the exhibition strategy shares a crucial parameters set that was embedded previously by the invisible watermarking algorithm QIM-DM [28], the incorporation of this algorithm increasing the computational cost and decreasing the image quality. In this context and taking into account that the most popular on-demand and streaming services, such as Netflix [44], DisneyPlus [45], and PrimeVideo [46], require a dedicated platform to play their audiovisual content, the crucial parameters to exhibit the watermark can be transmitted by the platform and stored into the memory of the electronic display device. When the end-users want to exhibit the watermark, the proposed algorithm takes advantage of robust image processing capabilities; with this strategy, an additional complex computational algorithm is not required to exhibit the watermark. If the proposed algorithm considers the same crucial parameters to exhibit every watermark, the required memory and the computational cost are low.

7. Conclusions

In this paper, a copyright protection and information delivery algorithm is presented; the proposed one introduces a new improved version of the pioneer imperceptible–visible watermarking algorithm [23] that takes advantage of the histogram distortion induced by the embedding strategy to introduce a new embedding strategy, and the processing capabilities incorporated into the electronic display devices to propose a new histogram binarization function-based exhibition strategy. The experimental results show that the proposed algorithm improves several aspects of the most sophisticated and recent IVW algorithm [24]. In general, the algorithm can embed one or multiple watermarks, guaranteeing high watermarked image quality in the range of 37 dB ≤ PSNR ≥ 53 dB, and the watermark imperceptibility is reported with an SSIM ≥ 0.93 and NCD ≤ 0.05, preserving the high visual quality of the exhibited watermarks, mainly QR codes. The low complexity of the proposed algorithm makes it a feasible option to be applied to copyright protection and information delivery related to video or image visual content.