Babak Mahdian*, Adam Novoz Amsky and Stanislav Saic
Institute of Information, Theory and Automation of the Czech Academy of Sciences, Prague, Czech Republic
Received date: March 30, 2015; Accepted date: July 29, 2015; Published date: July 31, 2015
Citation: Mahdian B, Amsky AN, Saic S (2015) Detecting Cyclostationarity in Re-Captured LCD Screens. J Forensic Res 6:294. doi: 10.4172/21577145.1000294
Copyright: © 2015 Mahdian B, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Visit for more related articles at Journal of Forensic Research
It is easy to display a video or picture on an LCD screen and recapture it by using a camera to hide traces of digital image manipulation or fool an access system based on face recognition technique. In this paper, we show that humans do not have a good performance in detecting recaptured data from LCD screens. Hence, it is important to have methods capable of distinguishing between natural videos and pictures and those recaptured ones. In this paper, we show that, typically, recaptured images and videos from LCD screens exhibit detectable periodic patterns that are caused by regular sampling grid of the LCD monitor and aliasing. We develop our method using the theory of cyclostationarity and experimentally validate it. The term cyclostationarity refers to a special class of signals which exhibit periodicity in their statistics. Our method will be based on the fact that a cyclostationary signal has a frequency spectrum correlated with a shifted version of itself.
Biometrics; Aliasing; Cyclostationarity; CFA; Image and video recapturing; Digital forensics
In recent years, we have observed a dynamic development in two essential areas of forensics analysis of digital images and videos integrity verification (genuineness analysis) and image ballistics (source device verification). Although past research in the areas of image integrity verification and image ballistics have mainly focused on data hiding and digital watermarking approaches [1,2], today there is a growing alternative approach called the passive one which does not need embedding any secondary data into the image . Methods published in this area have, for example, attempted to detect image splicing [4-6], traces of inconsistencies in color filter array interpolation [7-9], traces of geometric transformations [10-12], cloning [13-15], computer graphics generated photos [16-18], JPEG compression inconsistencies [19-21], file structure inconsistencies [22,23], etc. Typically, pointed out methods are based on the fact that digital image editing brings specific detectable inconsistencies and statistical changes into the image.
In this paper we will deal with automatic recognition of videos and pictures that have been recaptured from LCD monitors. Recent advances in digital camera technology have meant that high resolution images can easily be obtained at a relatively low cost by using digital camera and smart phones. Moreover, a widespread availability of high quality softcopy display mediums, such as LCD monitors have made it possible to reproduce digital images with ease by recapturing the photo or the video from a display using a digital camera.
The motivation of detecting re-captured images and videos can be several. For example, automatic distribution of illegally captures movies from LCD screens. Recapturing is an easy tool to eliminate copy-right related invisible watermarks hidden in images and videos. Another area that needs to be capable of detecting recaptured videos and images is the authentication area. Access systems using face recognition techniques are often vulnerable to spoofing attacks. In a spoofing attempt, a person tries to masquerade as another person to gain an access to the system.
Our motivation does come from the digital forensics point of view. A large portion of digital forensics methods are bases on searching for inconsistencies among pixel. This, they can easily be overcome by recapturing a digitally manipulated image or video. The re-captured image would not contain traces of digital manipulation and inconsistencies among pixels. In other words, it would act as an original image. In other words, the forger can display fake images on LCD display and recapture the manipulated digital image to overcome image forensic systems. Consequently, detecting recapturing can signify tampering.
This paper will introduce a method capable of detecting recaptured images and videos using a single image and a single frame analysis. Hence, from now on, we will only consider digital images. In case of videos, the method can easily be applied on individual frames separately.
We also will show the results of an experiment showing human performance in identification of LCD re-captured images. We will see that recaptured images from the LCD screen are often perceptually indistinguishable to humans. However, there are fine differences between LCD screen recaptured images and non-recaptured ones caused, for example, by regular monitor pixel grid projected into the recaptured image and aliasing. We will use these differences to develop the method. Specifically, we will detect periodic properties present in the LCD recaptured images by using theory and methods of cyclostationarity. The term cyclostationarity refers to a special class of signals which exhibit periodicity in their statistics. Our methods will be based on the fact that a cyclostationary signal has a frequency spectrum correlated with a shifted version of itself.
The rest of the paper is organized as follows. In section 2 we will show related work. In section 3 we will perform an experiment to better understand the human performance in identification of LCD recaptured images. In section 4 we show differences between LCD screen recaptured images and non-recaptured ones and introduce a method to detect them. In section 5 we run an experiment to validate the method. In the last section, we summarize this paper by discussing the results and pointing out conclusions.
The problem of detecting and identifying recaptured images from LCD monitors has received interest from researchers in recent years and we have been observing a growing number of publications in this area.
Ke Yongzhen et al. have used combinations of low-level features including texture, noise, difference histogram, and color information to train a support vector machine classifier capable of detecting recaptured images. Hani Muammar and Pier Luigi Dragotti  have searched for the presence of aliasing due to sampling of the monitor pixel grid to identify recaptured images. To validate their approach, an investigation into the aliasing introduced in a digitally recaptured image has been conducted. Xinting Gao et al.  have introduced a recaptured image database captured by smart phone cameras. Huacheng Liu and Rangding Wang have proposed to identify recaptured images using DCT coefficients. They have noticed that the low frequency of an image mainly reflects such information as texture and profile details. So, they described the difference between real images and recaptured images by using the low frequency of DCT coefficients. Thirapiroon Thongkamwitoon et al.  have used the fact that edge profiles of single and recaptured images are different in order to detect recaptured images. They have trained two alternative dictionaries using the K-SVD approach to achieve their goal. Neslihan Kose and Jean-Luc Dugelay  have proposed an anti-spoofing approach, which is based on analysis of contrast and texture characteristics of captured and recaptured images. A method based on a rotation invariant local binary pat- tern variance has been used. Xinting Gao et al.  have presented a physical model for image recapturing. Their motivation was to make robot vision more intelligent and make a single-image-based countermeasure for re-broadcast attack on a face authentication system feasible. M. Visentini-Scarzanella, Dragotti PL  has derived a curve model for straight lines deformed after single capture under a radial distortion model and recapturing. Bestagini P et al.  have proposed a detector based on the analysis of a characteristic ghosting artifact left by the recapture process. Xiaoyang Tan et al. have used the Lambertian model to propose strategies to extract the information about different surface properties of a live human face or a photograph. Jiangwei Li et al.  have used structure and movement information of live face to propose a live face detection based on the analysis of Fourier spectra. Hong Cao and Kot AC  have proposed to use a set of statistical features to capture the common anomalies introduced in the camera recapturing process on LCD screens. Jiamin Bai et al.  have focused on paper recapturing and used micro-textures present in printed paper to detect images recaptured from printed materials.
We created a database of 800 photos. 400 of them were original and natural photos and the other 400 were LCD recaptured photos. To better understand human capability of detecting LCD re-captured images, we decided to make a survey-based experiment. We created a web application and invited a number of people to take part in our experiment. 47 of them have accepted our invitation and took part in experiments. The experimentation had the following process. After user signed in, he was shown a sequence of photos. There was always five seconds to decide if the photo is natural or has been recaptured from an LCD display. Natural and recaptured photos have been shown in random orders. Participants were allowed to interrupt the test process at any time, re-login, and continue anytime later. As a result, some have been tested by, for example, just 40 images and some others for example by all 800 images.
It is also important to note that all participants were first trained by showing them a number of LCD recaptured images and a number of natural images. Recaptured pictures were taken by using three cameras and three different LCD monitors. The youngest participant was 22 years old. The oldest was 64 years old. The majority of testers were male (only 11 participants were females). Most of the participants have been either expert in the area of digital image processing or university students of various disciplines. Figure 1 shows typical examples of nonrecaptured and recaptured images that have been shown to testers. Specifically, first row shows non-recaptured images. Pictures shown in the second row are recaptured from LCD.
Results obtained show that human performance in differentiating the recaptured photos from natural photos is not very good and LCD recaptured images can be effectively used to overcome authentication and image forensic systems. Specifically, the type-I error which show the rate of natural images classified as recaptured was 24.5 percent. The type-II error rate saying the rate of re-captured LCD images classified as natural images was 34 percent. Despite the fact that we spent an equal time and resources to prepare and train testers, variety in their performance was high. There were individuals with exceptional performances. On the other hand, majority of the testers exhibited poor performance. Table 1 shows performance data of 20 different individuals that participant in our survey. Shown are their performances in terms of achieved Type I error and Type II error.
|Tester ID||Type I error||Type II error|
Table 1: Shown are performances of 20 different testers that participated in our experiment.
In this section, we will develop a method based on the theory of cyclostationarity. The method will use the periodic patterns and artifacts that are present in the recaptured images of LCD screens. Recapturing an LCD screen by using a CCD or CMOS sensor is a discrete sampling of a regular grid structure of the LCD display. The original image is first shown on the LCD screen and its regular grid and subsequently a camera is used to make a picture of it.
A visual inspection of recaptured pictures and frames of LCD display makes it possible to observe that there are often present specific periodic patterns and artifacts such as moire patterns [34,35]. They are caused by capturing the periodic sampling grid of the LCD monitor or associated aliasing. The exact appearance and intensity of the artifacts is related to several factors such as the sampling rate of the recapturing process or Color Filter Array (CFA) used in the camera .
According to Nyquist frequency, the sampling rate of the recapturing process must be greater than twice the highest frequency encountered. Sampling at a lower rate results in aliasing. Figure 2 shows various types of sampling of the LCD grid. Here, the last two images show a sampling process that brings a loss of information because of an insufficient sampling rate.
As pointed out , CFA also plays important role in appearance and intensity of periodic patterns occurred in recaptured images. Many digital cameras are equipped with a single charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) sensor . At each pixel location, only a single color sample is captured. The color images are typically obtained in conjunction with CFA (Figure 3). The most commonly used CFA is called Bayer CFA after the name of its inventor B.E. Bayer from Eastman Kodak. It consists of alternating red and green pixels on odd lines and green and blue pixels on even lines. Missing colors are computed by an interpolating process, called CFA interpolation. There are many CFA interpolation algorithms that bear into the image’s different levels of spatial correlations and lead to different appearances of aliasing artifacts (bilinear, bicubic, medianbased, gradientbased, SHT, adaptive, directional filtering, etc.). For some instances of typical aliasing artifacts corresponding to different types of CFA, please see Figure 4. Here, the lighthouse image is shown. The image is often used for comparing various demosaicing results .
We will use the following simple, linear and stochastic model describing the recapturing process:
f(x) = (u h)(x) + n(x) (1)
where f, u, h, *, and n are the measured LCD screen, original LCD screen, system PSF, convolution operator, and a random variable representing the influence of noise sources that are statistically independent from the signal part of the image. In the last half a century, a lot of work has been done in the field of cyclostationarity . Much of the initial work introducing and examining the use of cyclostationary models in the signal analysis was carried out by Gardner et al. [39-41]. A zero–mean signal f(x) is defined to be second order cyclostationary if its second order statistics are periodic. The autocorrelation function of f(x) can be defined as:
Because of its periodicity in x, we can represent it in the form of a Fourier series expansion:
where α is the cyclic frequency. The parameter is called Cyclic Autocorrelation Function (CAF) and it is a fundamental parameter of cyclostationarity. CAF is defined as:
An appropriate way of analyzing cyclostationary properties is by applying the Fourier Transform (FT) to . The result is called Spectral Correlation
Function (SCF) and is defined as:
As we will deal with discrete signals, the discrete version of CAF should also be defined here:
where N and Δm denote the number of samples of the signal and sampling interval, respectively. Equivalently, the discrete SCF can be obtained by:
CAF and SCF are analogous to the autocorrelation function and the power spectral density function for stationary signals. When α=0, the SCF can also be interpreted as the power spectral density of the signal. For other values of α, SCF is the cross–spectral density between the signal and the signal shifted in frequency by α. So, if the signal being analyzed exhibits cyclostationarity, the SCF will be non-zero for some α ≠ 0 . Otherwise, only for α=0, we will have non-zero values.
A cyclostationary signal has a frequency spectrum that is correlated with a shifted version of itself . Based on this, in our method, we focus on detecting the traces of cyclostationarity by estimating the spectral correlation function. To estimate the SCF, we can simply use equation (7). But, due to its computational complexity, we use a more computationally effective SCF estimation method based on Fast Fourier Transform (FFT). FFT algorithm has computational complexity O (nlog2n). Let’s say f (x, y) is the image being analyzed and F (n, u) is a matrix containing FFT of image’s rows (i.e., F (1, u) contains the one– dimensional FFT of the first row of f (x, y)). The SCF can be estimated in the following way:
where * denotes a complex conjugate and N is the number of image’s rows.t
Data obtained can be combined together to create the resulting correlation map:
Periodic patterns of LCD recaptured images are often weak and not strong enough to be easily detectable using the basic cyclostationarity methods. We overcome this passing the analyzed image through a bandpass filter. Let f (x, y) denote the image being analyzed. Then, Dx and Dy are band-passed images containing the horizontal and vertical derivative approximations:
Applying the equations (8) and (9) to Dn, where Dn is given by:
and n denotes the order of derivative filter, results in more accurate and robust outcomes.
Application of the derivative filter to (8) results in:
For an example of results obtained by Equation 12 (Figure 5). Here, three digital images and obtained results are shown. Red boxes highlight the analyzed area. The image on the left (Figure 5a) is a natural and non-recaptured image. Pictures shown in middle and on right (Figure 5b and 5c) are obtained by re- capturing of an LCD screen. Distinctive peaks for pictures in middle and on right are characterstics and signifying the recapturing process. We remind that when α=0, the SCF can be interpreted as the power spectral density of the signal. For other values of α, SCF is the cross–spectral density between the signal and the signal shifted in frequency by α. If the picture being analyzed exhibits cyclostationarity, the SCF will exhibit non–zero values for some α ≠ 0 .
Figure 5: The image on the left is a natural and non-recaptured image. Pictures shown in middle and on right are re-captured from an LCD screen and exhibit visible traces of aliasing. Corresponding correlation maps generated by Eq. 12 are shown either (θ= 0). Distinctive peaks for the pictures shown in middle and on right are characteristics and signifying the recapturing. Red boxes highlight the analyzed area.
If traces of recapturing from an LCD screen is not found, only α= 0 have
Non-zero values and no string and distinctive peak are generated. Results shown correspond to a derivative filter of order 2, D2.
Periodic patterns of recapturing have typically different intensities in various parts of the image. Thus, to successfully find traces of recapturing, we divide the image into non-overlapping blocks of R × R pixels. We denote these blocks by b (x, y). The developed method is always separately applied on each individual block.
Periodic patterns corresponding to recapturing an LCD screen often are rotated and have complex spatial distributions that are caused by the rotation angle of the LCD monitor in respect to the camera that has been used to make the recapturing as well as sampling rate. To this end, we estimate the SCF and the correlation map in various directions, θ. Specifically, we apply Equation 8 and 9 systematically at angles θ from 0 to 179?, in 1? increments (Figure 6). The new coordinate system corresponding to each θ is obtained in the following way:
This results in 180 vectors ρθ. If the investigated region has been recaptured from an LCD screen, typically, some of ρθ contain a specific strong peak corresponding to the cyclostationarity.
To validate theoretical assumptions and to measure the performance
of the developed method, we used a derivative filter of order 2, ρ D2 (α):
Eq. 13 was applied to green channels of 800 tested images. Size of non-overlapping blocks b(x, y) was set to 420 × 420 pixels. To automatically detect peaks corresponding to periodic artifacts of LCD recapturing, we used an automatic peak detector (P D) searching for local maximum at positions α ≠ 0 .
The P D detects the strongest peak that is significantly larger than the local average. If the size of the found peak lies within a particular threshold, the image is classified that contains patterns of LCD recapturing, otherwise we say no traces of re-capturing have been found.
Specifically, we used a hypothesis testing approach. By assuming that the observations in maps generated by Eq. 13 are normally distributed, we formulated the null and the alternate hypothesis as follows:
Ho: The peak does not correspond to recapturing
Ha: The peak corresponds to a recaptured image.
Hypothesis testing was carried out in conjunction with t-statistic and a one percent significance level. Not rejecting Ho, signifies that the peak is not a recaptured image. Type I error indicates that in one percent of the cases, we claim that an image is recaptured when it is not. In other words, we are creating a 99 percent confidence interval for the peak size to be a recaptured image.
The hypothesis testing was applied on 400 recaptured and 400 nonrecaptured pictures. Results obtained have shown type I error of 3.92 percent and type II error of 8.44 percent. By empirical analysis of results, it was straightforward to observe that the quality and content of tested images play an important role in this task. For instance, detecting recaptured pictures of a very dark (almost black) content was very challenging. Moreover, as pointed out in , it should be noted that there are a large number of possible settings of cameras, LCD screens, surrounding lighting, etc. that have an impact on properties of recaptured data. Under specific settings, it is possible to minimize the impact of aliasing and periodic structure of the monitor grid and obtain re-captured pictures that differ very slightly from original and natural digital images. This can be achieved, for example, by optimal settings of resolution, brightness, shutter speed, capturing mode, distance between the camera and LCD, etc.
In this paper, we developed a method detecting videos and pictures that have been recaptured from LCD screens. The method uses periodic patterns present in such signals. Such periodic patterns are usually caused by re-sampling of the regular pixel grid of the LCD monitor and aliasing. We used the periodicity present in statistics of such signals and automatically detected correlation in their frequency spectrum. Results obtained are promising and show that employing cyclostationarity methods can be effective in many applications in computer vision and pattern recognition including the detection of recaptured data from LCD screens.
This work has been supported by the Czech Science Foundation under Project no. GACR 13-28462S.
Make the best use of Scientific Research and information from our 700 + peer reviewed, Open Access Journals