# Using Noise Level to Detect Frame Repetition Forgery in Video Frame Rate Up-Conversion

^{*}

## Abstract

**:**

## 1. Introduction

- This paper uses Gaussian distribution to model the noise in a video frame, and find the FR forgery by observing the temporal variation of the noise level.
- The proposed method directly uses the noise statistics to detect fake traces, avoiding the performance loss from noise.

## 2. Background

#### 2.1. Residual Detection

_{2}norm. If f[t] is a repetitive frame, v[t] is zero; otherwise v[t] is not zero. Once these repetitive frames appear periodically, the residual variation will be zero periodically, so the fact that the residual variation periodically decays to zero can prove that the video is forged by FR. In general, when the original frame rate of video is increased by FR, in order to reduce the amount of data, H.264 [12] or high-efficiency video coding (HEVC) [13] is used to compress the forged video, which would cause some error between the compressed frame and the video frame as follows,

#### 2.2. Similarity Detection

_{t}and μ

_{t+}

_{1}are the means of f[t] and f[t + 1] respectively, λ

_{t}and λ

_{t}

_{+1}are the variances of f[t] and f[t + 1] respectively, λ

_{t},

_{t}

_{+1}is the covariance between f[t] and f[t + 1], c

_{1}and c

_{2}are constants. The SSIM value ranges in [0, 1], if it is set to be 1, showing that the two frames are the same, and the smaller the value is, the less similar the two frames are. Periodically inserting repetitive frames, the SSIM value will be 1 periodically, so it can be proved that the video is forged by FR once the SSIM value is seen to increase to 1 periodically. When transmitting a forged video, the noise is mixed inevitably as follows,

## 3. Proposed Noise-Level Detection

#### 3.1. Noise-Level Estimation

**f**is the column vector of the t-th frame f[t] through raster scanning,

_{t}**ψ**

_{l}is Daubechis-4 orthogonal wavelet basis and L is the total number of wavelet bases,

**Ѱ**is the representation matrix whose l-th column is

**ψ**, and

_{l}**y**is the column vector composed of wavelet coefficients. The MAD value of wavelet coefficients can then be calculated as follows,

_{t}#### 3.2. Periodicity Detection

## 4. Experimental Results and Analyses

_{NS}and E

_{PS}are the counts of mistakes when detecting NS and PS respectively, C

_{NS}and C

_{PS}are the capacities of NS and PS respectively. In addition, the detection accuracy (DA) is defined as follows,

#### 4.1. Subjective Performance Evaluation

#### 4.2. Objective Performance Evaluation

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Tsai, T.H.; Shi, A.T.; Huang, K.T. Accurate frame rate up-conversion for advanced visual quality. IEEE Trans. Broadcast.
**2016**, 62, 426–435. [Google Scholar] [CrossRef] - Bian, S.; Luo, W.; Huang, J. Exposing fake bit rate videos and estimating original bit rates. IEEE Trans. Circuits Syst. Video Technol.
**2014**, 24, 2144–2154. [Google Scholar] [CrossRef] - Video Edit Magic. Available online: https://video-edit-magic.en.softonic.com (accessed on 5 December 2017).
- Bestagini, P.; Battaglia, S.; Milani, S.; Tagliasacchi, M.; Tubaro, S. Detection of temporal interpolation in video sequences. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, 26–31 May 2013; pp. 3033–3037. [Google Scholar]
- Xia, M.; Yang, G.; Li, L.; Li, R.; Sun, X. Detecting video frame rate up-conversion based on frame-level analysis of average texture variation. Multimed. Tools Appl.
**2017**, 76, 8399–8421. [Google Scholar] [CrossRef] - Bian, S.; Luo, W.; Huang, J. Detecting video frame-rate up-conversion based on periodic properties of inter-frame similarity. Multimed. Tools Appl.
**2014**, 72, 437–451. [Google Scholar] [CrossRef] - Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process.
**2004**, 13, 600–612. [Google Scholar] [CrossRef] [PubMed] - Yang, J.; Huang, T.; Su, L. Using similarity analysis to detect frame duplication forgery in videos. Multimed. Tools Appl.
**2016**, 75, 1–19. [Google Scholar] [CrossRef] - Yao, Y.; Yang, G.; Sun, X.; Li, L. Detecting video frame-rate up-conversion based on periodic properties of edge-intensity. J. Inf. Secur. Appl.
**2016**, 26, 39–50. [Google Scholar] [CrossRef] - Ding, X.; Yang, G.; Li, R.; Zhang, L.; Li, Y.; Sun, X. Identification of motion-compensated frame rate up-conversion based on residual signal. IEEE Trans. Circ. Syst. Video Technol.
**2018**, 28, 1497–1512. [Google Scholar] [CrossRef] - Ding, X.; Zhu, N.; Li, L.; Li, Y.; Yang, G. Robust localization of interpolated frames by motion-compensated frame-interpolation based on artifact indicated map and tchebichef moments. IEEE Trans. Circ. Syst. Video Technol.
**2018**. [Google Scholar] [CrossRef] - Advanced Video Coding for Generic Audio-Visual Service; ITU-T Rec. H.264 and ISO/IEC 14496-10 (AVC); ITU-T and ISO/IEC JTC 1; International Telecommunication Union: Geneva, Switzerland, 2003.
- Sullivan, G.J.; Ohm, J.R.; Han, W.J.; Wiegand, T. Overview of the High Efficiency Video Coding (HEVC) Standard. IEEE Trans. Circ. Syst. Video Technol.
**2012**, 22, 1649–1668. [Google Scholar] [CrossRef] - Papoulis, A.; Pillai, S.U. Probability, Random Variables and Stochastic Processes, 4th ed.; McGraw-Hill: New York, NY, USA, 2002. [Google Scholar]
- Bergland, G.D. Fast Fourier transform hardware implementations-A survey. IEEE Trans. Audio Electroacoust.
**1969**, 17, 109–119. [Google Scholar] [CrossRef] - Tai, S.C.; Yang, S.M. A fast method for image noise estimation using Laplacian operator and adaptive edge detection. In Proceedings of the 2008 3rd International Symposium on Communications, Control and Signal Processing, St Julians, Malta, 12–14 March 2008; pp. 1077–1081. [Google Scholar]
- Zoran, D.; Weiss, Y. Scale invariance and noise in natural images. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan, 29 September–2 October 2009; pp. 2209–2216. [Google Scholar]
- Leigh, A.; Wong, A.; Clausi, D.A.; Fieguth, P. Comprehensive analysis on the effects of noise estimation strategies on image noise artifact suppression performance. In Proceedings of the 2011 IEEE International Symposium on Multimedia, Dana Point, CA, USA, 5–7 December 2011; pp. 97–104. [Google Scholar]
- Donoho, D.L. De-noising by soft-thresholding. IEEE Trans. Inf. Theory
**1995**, 41, 613–627. [Google Scholar] [CrossRef] [Green Version] - Mallat, S. A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell.
**1989**, 11, 674–693. [Google Scholar] [CrossRef] [Green Version]

**Figure 5.**Illustrations of Fourier spectrums for both original and forged video sequences: (

**a**) Soccer sequence in QCIF format, (

**b**) Mobile sequence in CIF format, (

**c**) Mobcal sequence in 720P format, (

**d**) Tractor sequence in 1080P format.

**Table 1.**Average peak-mean ratio (PMR) values of amplitude spectrums for different detection methods.

Detection Method | QCIF | CIF | 720P | 1080P | Total | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

NS | PS | ∆ | NS | PS | ∆ | NS | PS | ∆ | NS | PS | ∆ | NS | PS | ∆ | |

Residual detection | 6.76 | 17.24 | 0.61 | 7.67 | 19.88 | 0.61 | 10.04 | 29.58 | 0.66 | 11.73 | 23.88 | 0.51 | 8.63 | 21.98 | 0.61 |

Similarity detection | 7.36 | 21.91 | 0.66 | 7.63 | 24.31 | 0.69 | 10.17 | 29.63 | 0.66 | 11.90 | 26.30 | 0.55 | 8.79 | 25.23 | 0.65 |

Noise-level detection | 2.42 | 27.32 | 0.91 | 2.45 | 30.17 | 0.92 | 2.93 | 28.32 | 0.89 | 2.92 | 27.95 | 0.889 | 2.62 | 28.85 | 0.91 |

_{1}by the average PMR on NS and R

_{2}by the average PMR on PS, and the relative difference between R

_{1}and R

_{2}is ∆ = 1 − R

_{1}/R

_{2}.

Detection Method | QCIF | CIF | 720P | 1080P | Total | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

FNR | FPR | DA | FNR | FPR | DA | FNR | FPR | DA | FNR | FPR | DA | FNR | FPR | DA | |

Residual detection | 0.70 | 0.10 | 0.60 | 0.71 | 0.00 | 0.64 | 0.90 | 0.00 | 0.55 | 1.00 | 0.00 | 0.50 | 0.80 | 0.02 | 0.59 |

Similarity detection | 0.70 | 0.00 | 0.65 | 0.67 | 0.00 | 0.67 | 0.80 | 0.00 | 0.60 | 1.00 | 0.00 | 0.50 | 0.76 | 0.00 | 0.62 |

Noise-level detection | 0.00 | 0.00 | 1.00 | 0.00 | 0.00 | 1.00 | 0.00 | 0.00 | 1.00 | 0.25 | 0.00 | 0.88 | 0.04 | 0.00 | 0.98 |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Li, Y.; Mei, L.; Li, R.; Wu, C.
Using Noise Level to Detect Frame Repetition Forgery in Video Frame Rate Up-Conversion. *Future Internet* **2018**, *10*, 84.
https://doi.org/10.3390/fi10090084

**AMA Style**

Li Y, Mei L, Li R, Wu C.
Using Noise Level to Detect Frame Repetition Forgery in Video Frame Rate Up-Conversion. *Future Internet*. 2018; 10(9):84.
https://doi.org/10.3390/fi10090084

**Chicago/Turabian Style**

Li, Yanli, Lala Mei, Ran Li, and Changan Wu.
2018. "Using Noise Level to Detect Frame Repetition Forgery in Video Frame Rate Up-Conversion" *Future Internet* 10, no. 9: 84.
https://doi.org/10.3390/fi10090084