A Design of Multi-Purpose Image-Based QR Code

Abstract

A multi-purpose image-based QR code is designed in this paper. There are four purposes for the generated image-based QR code. In the first purpose, the basic image-based QR code with the look of a host image is with an ingenious layout to be identified easier. In the second one, a saliency region detection method is adopted for enhancing the quality of the image-based QR code. In the third one, the host image is embedded into the image-based QR code for further access to the host image; Finally, the visual cryptography-based watermarking method is applied to the host image embedded image-based QR code. In the case that the specific users need verification from the image-based QR code, the binary verified image can be retrieved when the public share is available. The experimental results demonstrate that the generated image-based QR code not only looked better than some previous works but also had high quality host image embedded and identification ability.

1. Introduction

The Quick Response code (QR code) is a two-dimensional barcode designed by Denso, Japan [1]. QR code has been defined as an open standard, consisting of a series of black and white matrix squares, called modules, placed in a specific order. The basic structure of the QR code is shown in Figure 1. QR codes can store more data than one-dimensional barcodes. Applications include goods identification, commodity advertising, sales, inventory management, ticketing systems, etc. In the general production resume mechanism, people can get the QR code on the fresh products and use the QR code decoding function built into smartphones, and production information of fresh products can be revealed. The image-based QR code is more friendly than the traditional QR code, and the line number of a company can be added to the company’s trademark or the design of a personal business card that can also use image-based QR code to show the company or personal style.

There are many works related to the QR code. For more capacity of QR codes, Vongpradhip [2] appended the message capacity of the QR code. Dai et al. [3] used the QR code to produce a linear enhancement hologram of a marble image. Huang et al. [4] designed a reversible data hiding with histogram-based difference expansion for several QR code applications. Huo et al. [5] introduced a QR image code recognition system based on an artificial intelligence algorithm. Because the look of traditional QR codes is meaningless, the design of friendly QR codes added on extra value, has recently become a hot research topic. Some researchers have started working on the auto-generation algorithms of image-based QR codes.

Salient Region Detection simulates the human visual system to rapidly detect foreground objects in the scene. In [6], Xu et al. detected significant areas of images, based on the significant characteristics of spatial limitation, color and similarity distribution, and used multi-layer structures to merge into salient region detecting. In order to make the salient region consistent, this paper combines multi-size segmentation techniques. Firstly, an image is used to calculate pixel importance, to split with multiscale dimensions, and then to use the mean division method to find out the salient region of multiscale dimensions, and to calculate the maximum and average significant values, and finally obtain the saliency map.

Because data hiding methods carry secret data by images or videos that are unremarkable, hackers do not pay attention to the secret data that are hidden in the host carriers. To effectively avoid the attention of hackers, the secret-embedded image must be very similar to the host one. The simplest data hiding method is the least significant bit (LSB) substitution method. The method replaces the least significant bits of the host image with the secret bits. For example, Thien and Lin [7] proposed an optimal LSB substitution method by using module-based computation.

The motivation of the proposed work is to build an art QR code with extra embedding ability. The proposed work will improve the visual quality of other image-based QR works. Although it is not popular now, on websites the service of cute QR code production can be seen. For example, https://www.qrcode-monkey.com/ (accessed on 7 November 2021).

This paper also develops a high-quality image restoring system with an image-based QR code in a symmetric way. This adopted technology allows the user to produce an image-based QR code from a QR code and a host image. The QR code can be decoded with the general QR code decoding APP, and a high-quality host image can be recovered from the host-image embedded image-based QR code. Figure 2a–d shows an example of using the proposed method to add a QR code to a well-known picture, Lena. The input URL is set to be “http://www.ygu.edu.cn” (accessed on 7 November 2021). Figure 2a is the QR code containing the input URL, Figure 2b is the host image, Figure 2c is the generated host-image embedded image-based QR code, and Figure 2d is the recovered image extracted from the image-based QR code.

Traditionally, watermarking techniques are used for protection from copying without permission. Liu and Wu [8] proposed a robust watermarking scheme for multiple cover images and multiple owners. Ahmadi et al. [9] provided essential and potential information for developing an efficient robust and hybrid SVD-based image watermarking system. Lai and Tsai [10] indirectly embed the watermark on the wavelet coefficients but rather than on the elements of singular values of the cover image’s DWT sub-bands.

Surekha and Swamy [11] adopt the idea of visual cryptography [12] introduced by Noar and Shamir for watermarking. The watermark, a binary secret image, is computed by XOR with the most significant plane of the host image. The resulting image of the computation is called a public share. The watermark can be obtained, when the public share is computed by XOR with the most significant plane of the host image.

The problem of this proposed method is to design a multi-purpose image-based QR code with a better look than other works and with extra abilities. The domain of this problem is to decide what is a suitable image for the image-based QR code. In this paper, the suitable domain is, thus, a set of common images. For data hiding ability, the suitable subdomain is a set of half sized images of the image-based QR code. For advanced usage, the binary image can be extracted, if the public share is available. Therefore, the suitable subdomain is, thus, a set of binary images with the same size as the image-based QR code.

Figure 3a shows the proposed encoding process by block structure and Figure 3b shows the proposed decoding process by block structure. Figure 3c shows the proposed embedding steps by images. In Figure 3a, the encoding process adopts traditional QR code embedding and host image embedding process for generating the host-image embedded image-based QR code, and then adopts the watermarking embedding process applying a secret binary color image for generating the stego-image that is called a public share.

In Figure 3b, the decoding process includes three sub-processes: (1) general QR code decoding process that adopts QR code decoder APP that is easy to find in every smartphone; (2) image extracting process that is used for extracting the host image from the generated image-based QR code; (3) VC-based watermarking extracting process that is used for extracting the binary secret color image from the public share and the generated image-based QR code. In Figure 3c, the encoding process is shown with generated images.

In the rest of this paper: Section 2 describes the proposed method. The experimental results are demonstrated in Section 3. Finally, the conclusions are made in Section 4.

2. Related Works

In the previous studies of image-based QR codes, Wakahara et al. [13] have proposed algorithms for adding trademarks directly to the padding areas of QR codes, as shown in Figure 4.

Wakahara and Yamamoto [14] proposed a method to make the QR code with the dotted pictures and develop the dotted pictures confirmed by the cellular phone barcode reader. Ono et al. [15] proposed a system for decorating a 2-dimensional barcode with some illustrations inside the code and without detracting machine-readability and stored information. Lin et al. [16] made QR code from a machine decodable only to a personalized form with a visually pleasing appearance. Chu et al. [17] proposed an approach to produce high quality visual QR codes, called halftone QR codes.

Fang et al. [18] defined the optimization of image-based QR codes as minimizing the visual distance between QR codes and trademarks. Although significant areas of the image are detected, there are still many damages to the significant parts of the background of the image-based QR code, as shown in Figure 5.

You et al. [19] proposed a strategy that is capable of transferring more than one artistic style to an art QR code. In order to migrate multi-style aesthetic QR codes, they improve a neural style migration network to suit the aesthetic QR code style migration tasks. One of the results of the generated art QR code is shown in Figure 6.

3. The Proposed Method

To present the proposed method more plainly, the generated process is divided into several stages: the binary lighting matrix generating stage, the QR code embedding stage, and the host image embedding stage. In the binary lighting matrix generating stage, a host full-color image is divided into several blocks with the same numbers as the number of modules of the QR code. For each block, a Gaussian distribution is adopted for generating the binary lighting matrix for the host image. In the QR code embedding stage, a given text or URL can be encoded to form a QR code. The generated QR code and the binary lighting matrix are combined using the proposed process. Moreover, a salient region process is adopted for making the generated image-based QR code with a better look. In the host image embedding stage, the host image can be symmetrically embedded into the generated image-based QR code. Finally, in the secret image embedding stage, the Visual Cryptography-based watermarking technique is adopted for embedding a secret image into the image-based QR code in order to append extra ability to the image-based QR code.

3.1. The Binary Lighting Matrix Generating Stage

Firstly, let a given text or URL be encoded into a QR code with version V. The number of total modules of the QR code are m × m, where m = 4 × V + 17. If the size of the host image is n × n, the host image is then divided into m × m blocks with each size (n/m) × (n/m). When the host image is a full color image, it can be converted to a gray-level image by transferring the RGB of the color image to Y (luminance). The transfer equation is shown as follows:

Y_ij = 0.299R_ij + 0.587G_ij + 0.114B_ij

(1)

where R_ij, G_ij, and B_ij are the red, green and blue values of the color image at (i, j) position.

After generating the gray-level image, for each block C_u,v in the host image H, the central luminance l can be computed by the following equation, where 1 ≤ u ≤ m, 1 ≤ v ≤ m. The binary lighting matrix M is then shown as follows.

$$l_{C_{u,v}}=\{\begin{array}{c}0 if {\displaystyle \sum }_{x\in C_{u,v}}{H}_x{G}_x<128\\ 1 if {\displaystyle \sum }_{x\in C_{u,v}}{H}_x{G}_x\ge 128\end{array}$$

(2)

where G_x is a Gaussian distribution, so ${\displaystyle \sum }_{x\in C_{u,v}}{G}_x=1$ and

$$G_x=G_{\left(q,r\right)}=\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{{\left(q-\frac{n}{2m}\right)}^2+{\left(r-\frac{n}{2m}\right)}^2}{2{\sigma }^2}}$$

(3)

where 1 ≤ q ≤ n/m, 1 ≤ r ≤ n/m.

$$M=\left[\begin{array}{ccc}{l}_{C_{0,0}}& \cdots & l_{C_{0,m-1}}\\ ⋮& \ddots & ⋮\\ l_{C_{m-1,0}}& \cdots & l_{C_{m-1,m-1}}\end{array}\right]$$

(4)

3.2. The QR Code Embedding Stage

In this stage, the encoded QR code is embedded into the host image to generate an image-based QR code. Firstly, because the common QR code decoders use the finder patterns and alignment patterns to locate the QR code, the finder patterns and alignment patterns are completely kept in the corresponding locations of the image-based QR code. For encoding other locations of the image-based QR code, the binary lighting matrix and the encoded QR code are used to evaluate the encoding results. The black module of the QR code has a value of 0, and the white module of the QR code has a value of 1. According to each module of the QR code except the finder patterns and alignment patterns, if the module value of the QR code is equal to the value in the corresponding location of the binary lighting matrix, the block of the image-based QR code is filled by the corresponding one of the host image. However, if the values are different, the following rule is used for adjusting.

Let C = [c_i,j]_(n/m)×(n/m) be a block of the host image. The adjusted block C = [$c_{i,j}^{\prime }$]_(n/m)×(n/m) of the image-based QR code is defined as follows.

$$c_{i,j}^{\prime }=\{\begin{array}{c}{c}_{i,j } if i\ne \frac{n}{2m} or j\ne \frac{n}{2m}\\ 255\times M_{i,j} otherwise\end{array}$$

(5)

Then, the salient region detection method proposed by Cheng et al. [20] is used for adjusting the embedding result. To efficiently detect the salient region, the host image is segmented into regions using an image segmentation method [21]. By measuring the color contrast of all regions in the host image, the saliency value function of each region r_k is computed by the following equation.

$$S\left(r_k\right)={\displaystyle \sum }_{r_k\ne r_i}w\left(r_i\right)D_r\left(r_k,r_i\right)$$

(6)

where w(r_i) is the weight of region r_i and D_r(r_k, r_l) is the color distance between two regions. Normally, the bigger region has the higher weight, and the color distance between two regions r_k and r_l is defined as:

$$D_r\left(r_k,r_l\right)={\displaystyle \sum }_{i=1}^{n_k}{\displaystyle \sum }_{j=1}^{n_l}f\left(c_{k,i}\right)f\left(c_{l,j}\right)D\left(c_{k,i},c_{l,j}\right)$$

(7)

where f(c_a,b) is the probability of the color c_a,b among all n_a colors in the region r_a, and D(c_k,i,c_l,j) is the distance between two colors in the RGB color space.

After computing the saliency values, the saliency map can be generated by normalizing the saliency values to [0, 1]. We can use the saliency map to avoid the most salient region damaged when generating an image-based QR code. RS (Reed Solomon) error correction mechanism is the kernel of QR code encoding. Due to the error correction capacity of the RS error correction mechanism, the block of the image-based QR code can be readjusted by referencing the saliency map.

For each block C = [c_i,j]_(n/m)×(n/m) of the host image. Let P be the saliency map of the host image. The readjusted block of the image-based QR code $C^{″}$ = [$c_{i,j}^{″}$] is defined as follows.

$$c_{i,j}^{″}=\{\begin{array}{c}{c}_{i,j } if i\ne \frac{n}{2m} or j\ne \frac{n}{2m}\\ 255\times M_{i,j} if i=\frac{n}{2m} and j=\frac{n}{2m} and P_{i,j}<th\\ c_{i,j} if i=\frac{n}{2m} and j=\frac{n}{2m} and P_{i,j}\ge th\end{array}$$

(8)

where th is the threshold that will be manually decided according to different images to get better aesthetic effects.

3.3. The Host Image Embedding Stage

In this stage, the adopted host image is symmetrically embedded into the generated image-based QR code. First, the host image can be divided into 24 bit planes (8 each in red, green, and blue), divided from most significant bit (MSB) planes to less significant bit (LSB) planes. Changing important bit planes will cause serious damage to images. On the contrary, the changing of less important bit planes will be not easy to be discovered by human eyes. Therefore, the MSB planes of the original image must be properly preserved for the high-quality restoration, while the LSB planes of the original image can be symmetrically used to preserve important material so that it can also be used to preserve the main part of the host image by the image-based QR code.

Because the MSB planes of the host image that needs to be preserved are mostly duplicate data, the MSB planes of the host image can be compressed and preserved into the image-based QR code by the method of Thien and Lin [7] to gain minimum distortion. To easily describe the method, we suppose that the host image is gray-level. Firstly, the z MSB planes of the host image H are extracted as the pre-embedded data E. For each datum E_i,j,

$$E_{i,j}=H_{i,j}\gg z$$

(9)

where $\gg$ is the right bit shift operator, 1 ≤ i, j ≤ n.

Then the data E, the MSB planes of the host image H, are embedded into the image-based QR code Q to form an embedded image-based QR code W as the following equations. For each position (i, j), $d_{i,j}$ is a temporary digit for recording the embedded datum $E_{i,j}$. The $d_{i,j}^{\prime }$ is adopted for adjusting $d_{i,j}$ for least distortion. At final, $W_{i,j}$ is an embedded datum of the embedded image-based QR code.

$$d_{i,j}=E_{i,j}-Q_{i,j} mod 2^z$$

(10)

$$d_{i,j}^{\prime }=\{\begin{array}{c}{d}_{i,j } if\left(-\frac{2^z-1}{2}\right)\le d_{i,j}\le \frac{2^z-1}{2}\\ d_{i,j}+2^z if \left(-2^z+1\right)\le d_{i,j}\le \left(-\frac{2^z-1}{2}\right)\\ d_{i,j}-2^z if \frac{2^z-1}{2}\le d_{i,j}\le 2^z\end{array}$$

(11)

$$W_{i,j}=Q_{i,j}+d_{i,j}^{\prime }$$

(12)

3.4. The Binary Image Embedding Stage

After the host image is embedded into the image-based QR code, for the further usage of the image-based QR code, a binary image can be also embedded into the image-based QR code by Visual-Cryptography based watermarking technique. In this stage, a share will be generated. Using a Visual-Cryptography based watermarking technique to embed a binary watermark, the image-based QR code should not be adjusted. The extra overhead is only a share S = [s_i,j]_n×n. Let B = [b_i,j]_n×n of the binary image. The share can be generated by the following equation.

$$s_{i,j}=b_{i,j}\left(W_{i,j}//128\right)$$

(13)

where is an XOR operator, and // is an integer division operator without remainder.

However, using the above process, the generated share will be an image with noise. For several applications, it is inconvenient. Using the data hiding technique, the share can be hidden into a normal image L = [l_i,j]_n×n to be a stego-image L’ = [$l_{i,j}^{\prime }$]_n×n, named as public share, by the following equation.

$$l_{i,j}^{\prime }=l_{i,j}//2\times 2+s_{i,j}$$

(14)

The generated image-based QR code can be printed on a personal business card. When people scan the business card by using a smartphone, the acquired image-based QR code will be with a loss. However, because the image-based QR code is unchanged during the binary image embedding phase, the most significant bits must seldom be changed, even through the print and scan process. Therefore, the binary image can be then solved by the following equation, if the stego-image L′ is available.

$$b_{i,j}=\left(l_{i,j}^{\prime } mod 2\right)\left(W_{i,j}//128\right)$$

(15)

4. Experimental Results

In this section, four different host images are tested. The input URL is set to be https://www.ygu.edu.cn/ (accessed on 7 November 2021). It is encoded into a QR code, as shown in Figure 7.

The selected host images are shown in Figure 8a–d. Through the saliency region detection process and error correction, the generated image-based QR codes are shown in Figure 8e–h.

For preserving the host image in the image-based QR code, the module-based LSB embedding process is adopted. Figure 9a–d are the embedding results that are embedded the host images Figure 8a–d into the image-based QR codes Figure 8e–h, respectively. When the host images need to be recovered, the high-quality host images Figure 9e–h can be revealed from the host-image embedded image-based QR codes, shown in Figure 9a–d, respectively.

The distortion of the modified image is calculated by the mostly used peak signal-to-noise ratio (PSNR). The formula is as follows:

$$PSNR=10\times lo{g}_{10}\left(\frac{{255}^2}{MSE}\right)$$

(16)

where MSE is the mean square error between the original image and the processed image.

In the following experiments, the qualities of the host-image embedded image-based QR code and the recovered host image are evaluated. In common, if the peak signal-to-noise ratio of a typical image is above 33 dB, it is difficult to distinguish the differences between the host image and generated image with the naked eye, so such embedded quality will not affect the QR code scanning results.

Table 1. The image quality of the host-image embedded image-based QR codes.

	Figure 9a	Figure 9b	Figure 9c	Figure 9d
PSNR	36.96	37.43	37.37	37.04

lists the PSNR values of the host-image embedded image-based QR codes, shown in Figure 8a–d, compared with Figure 7e–h, respectively.

Table 2. The image quality of the recovered host-image.

	Figure 9e	Figure 9f	Figure 9g	Figure 9h
PSNR	39.89	39.32	39.78	40.20

lists the PSNR values of the recovered host images, shown in Figure 8e–h, compared with Figure 7a–d, respectively.

To demonstrate the experiment of Visual Cryptography-based watermarking on the image-based QR code, the image-based QR codes Figure 9a–d are printed on cards and pictured by a smartphone. The experimental results are shown in Figure 10. Figure 10a shows the binary color image and Figure 10b–e shows the public shares of Figure 9a–d, respectively. Figure 10f–i show the results that are taken, pictured from the printed images of Figure 9a–d, and through the correction process. The image qualities of Figure 10f–i are listed in

Table 3. The image quality of picturing result from the printed images of Figure 10f–i.

	Figure 10f	Figure 10g	Figure 10h	Figure 10i
PSNR	20.68	19.78	19.69	21.01

. Although Figure 10f–i are of poor quality, the solved binary color images, shown in Figure 10j–m, are still able to be recognized.

To compare with the method of Wakahara et al. [13], the proposed method generates image-based QR codes with a higher-quality image. Figure 11a shows the image-based QR code generated by Wakahara et al. [13], and Figure 11b shows the one generated by the proposed method.

To compare with the method of Fang et al. [18], the proposed method also generates image-based QR codes with a higher-quality image. Figure 12a shows the image-based QR code generated by Fang et al. [18], and Figure 12b shows the one generated by the proposed method. Figure 12a is obviously of poorer quality than Figure 12b.

To compare with the method of You et al. [19], the proposed method also generates image-based QR codes with the higher-quality image provided by You et al. [19], shown in Figure 13c. Figure 13a shows the image-based QR code generated by You et al. [19], and Figure 13b shows the one generated by the proposed method.

To summarize the above experimental results,

Table 4. The comparisons among Wakahara et al. [13], Fang et al. [18], You et al. [19], and the proposed method.

	Full Image	Host Image Recovery	Watermark Extraction	Visual Quality
Wakahara et al. [13]	x	x	x	Good
Fang et al. [18]	∨	x	x	Poor
You et al. [19]	∨	x	x	Acceptable
The proposed method	∨	∨	∨	Good

lists the comparison results on whether the QR code is filled up with the image, whether the host image can be recovered from the image-based QR code, whether the watermarked image can be extracted from the image-based QR code.

5. Conclusions

In this paper, image-based QR codes are gracefully designed to have several abilities. The outward of the generated image-based QR code can be modified by salient region detection methods with a better look. The novelty of this paper includes reserving the host image without extra storage, the host image can be embedded into the image-based QR code; when the extra binary image is necessary for some application, the visual cryptography based watermarking technique can be applied to the image-based QR code. The experimental results demonstrate that the proposed method is competitive on the look of the image-based QR code. Even though printing and picturing on image-based QR code, the extracted result of watermarked image is well recognized.