We not only take account of the quality of the CS reconstructed images and the ability to fool the DNN models, but also have thought to reverse the adversarial reconstructed images to sanitized samples. However, to the best of our knowledge, there is no previous study dealing with the above tasks. Therefore, we use the reconstructed images from the CSNet as the benchmark, then evaluate the performance of the adversarial reconstructed images and the restored sanitized images . Specifically, , , , and that appear in the following paper represent the set of all , , , and on the test set, respectively.
For evaluation, we use 1500 grayscale images of size 96 × 96 from the Tiny-ImageNet test set and employ peak signal-to-noise ratios (PSNRs) and recognition accuracy as evaluation metrics.
At first, we test the performance of the pre-trained VGG16, ResNet50, and DenseNet121 classifiers for 1500 Tiny-ImageNet test images. Table 1
shows the recognition accuracy of the original test set on the three classification networks. The row of “Average” means the average recognition rate of the three classifiers. Compared with the trained one on all ImageNet database, the recognition rate of the three recognition networks we trained is much lower. This is because our training set is much less than the ImageNet database, accounting for less than 1/30. Therefore, getting relatively low recognition rate is a reasonable phenomenon.
Then, setting the sampling rate as 0.1, 0.2, 0.3 and 0.5, we employ the CSNet to implement reconstruction for the test set of the Tiny-ImageNet. Table 2
shows the image quality of the CSNet reconstructed images under different sampling rates with the metric PSNR. Table 3
shows the recognition rates of VGG16, ResNet50, and DenseNet121 for
One can see from Table 2
and Table 3
that with the increase in sampling rate, the PSNR values and recognition accuracy of CSNet reconstructed images
were improved. Compared with the original image, at different sampling rates, the recognition accuracy of the reconstructed image
on the three classifiers decreased to varying degrees. However, even when the sampling rate is 0.1, the reconstructed images
can achieve a recognition accuracy of 41–61% on the three classifiers. While the sampling rate is 0.5, their recognition accuracy is almost equal to that achieved on the original images. This means that malicious users can acquire a lot of sensitive information from CSNet reconstructed images,
, which poses a great privacy threat.
4.2.2. Performance Evaluation
The goal of the proposed IPPARNet is to take the machine recognition metric into account while retaining good visual quality. On the one hand, the adversarial reconstructed images can mislead the target classifiers, making it difficult for semi-authorized users to abuse them effectively, so as to achieve the goal of privacy protection. On the other hand, authorized users can restore sanitized images from , and the recognition accuracy can be improved as high as possible, which is helpful for the machine’s subsequent recognition tasks. That is to say, while keeping a reasonable PSNR, for the adversarial reconstructed images , the lower the recognition rate, the better the performance. However, for the sanitized images , the higher the recognition rate, the better the performance.
At different sampling rates, the recognition rates of VGG16, ResNet50, and DenseNet121 classifiers for adversarial reconstructed images
and sanitized images
are shown in Table 4
. It can be observed that for each sampling rate, the recognition rates of the adversarial reconstructed images
are significantly lower than that of the CSNet reconstructed images
. Take the sampling rate of 0.1 as an example. The recognition rates of
, which is generated by our adversarial reconstruction network, on the three classifiers VGG16, ResNet50, and DenseNet121 are not more than 1/10, which are 6.0%, 10.0%, and 7.6%, respectively. Compared with
, the recognition rates of
are relatively reduced by 85.3%, 83.6%, and 86.2%. When setting the sampling rate as 0.2, 0.3, and 0.5, the average recognition rate of adversarial reconstructed images
on the three classifiers drops from 71.5%, 74.0%, and 78.6% to 8.8%, 12.8%, and 13.5%, respectively. It can be seen that the relative declines are all greater than 82.0%, which results in semi-authorized machine users being unable to recognize these adversarial reconstructed images precisely. In other words, semi-authorized machine users are prevented from performing effective data analysis and model training tasks, and the privacy of images is protected.
However, authorized users can obtain sanitized reconstructed images with the additional restoration network. When the sampling rate is 0.1, the corresponding recognition rates on the three classifiers are 49.2%, 62.0%, and 55.6%, respectively, which are slightly higher than the recognition rates of reconstructed by the CSNet. At other sampling rates, the average recognition rates of the sanitized reconstructed images on the three classifiers can reach 69.6%, 70.3%, and 74.7%, which are approximate to that achieved by . Obviously, contributes to the machine recognition tasks.
Analysis of Image Visual Quality
shows the PSNR values of the CSNet reconstructed images
, the adversarial reconstructed images
, and the sanitized images
. Take the sampling rate of 0.1 as an example. Compared with
, the PSNR value of our adversarial reconstructed images
is only reduced by 0.39 dB. At different sampling rates, the PSNR values drops by 0.2–6.1%. When the sampling rate is 0.5, the PSNR value remains at 27.82 dB, which still provides a good visual effect for human eyes. In comparison, the PSNR values of the sanitized images
have a smaller decrease. In the case of sampling rate 0.1, it is only 0.06 dB lower than
. Since the PSNR value of
is greater than that of
at the same sampling rate, we infer that
has better visual quality than
To perceive the visual quality of the image intuitively, three images from the Tiny-ImageNet test set were randomly selected as representatives. Figure 3
illustrates the original images
, the corresponding initial reconstructed images
, the reconstructed images
of CSNet, our adversarial reconstructed images
, and the restored sanitized images
of the three images.
It can be observed that our adversarial reconstructed images are not disturbed significantly compared with CSNet reconstructed images and the sanitized images restored by authorized users are visually indistinguishable from . Both and have good visual quality for human beings and is better.
During the reconstruction, the network of the CSNet learns a linear mapping to obtain relatively good initial reconstructed images, . Then, with a nonlinear network , the residual between the initial reconstructed images and the original images is learnt, which can eliminate the block artifacts of and further improve the visual quality simultaneously. However, the adversarial reconstruction network - aims to learn a perturbation that makes the final reconstructed images have the ability to deceive DNN models with this nonlinear network. That is, by adding the perturbation, the final reconstructed images can induce the recognizer to make wrong judgments while ensuring it has good visual quality.
shows the perturbations learned by the network
of CSNet and the proposed IPPARNet. It can be seen that in our method, the learned perturbations cannot only eliminate block artifacts and supplement the contour details, but also acquire additional adversarial perturbations.
In summary, with the proposed IPPARNet, under the premise of ensuring good visual quality, the recognition rate of the adversarial reconstructed images can be reduced by more than 82% compared with CSNet reconstructed images , while authorized users can restore sanitized images which achieve the approximate recognition accuracy of the .