Received March 15, 2016; Accepted May 06, 2016; Published May 13, 2016
Citation: Cheemalapati S, Adithya PC, Valle MD, Gubanov M, Pyayt A (2016) Real Time Fear Detection Using Wearable Single Channel Electroencephalogram. Sensor Netw Data Commun 5:140. doi:10.4172/2090-4886.1000140
Copyright: © 2016 Cheemalapati S, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Visit for more related articles at International Journal of Sensor Networks and Data Communications
Real time detection of emotional state has multiple applications for security, safety and identification of dangerous situations. Traditionally electroencephalogram (EEG) based emotion studies are conducted in controlled lab environment with multi-channel systems and large signal processing power. In order to be useful in real world situation the system for emotion detection has to be miniature, portable and working in real time supported by calculations that can be provided by a processor power of a mobile phone. Here we present our results on real time fear detection using portable single electrode EEG system conducted on 10 subjects. We studied possibility of translation of the markers previously identified for complex multi-electrode system - ratios of slow waves to fast waves into real time portable system. It was demonstrated using Student’s t-test that the average value of the monitored parameter during normal state was significantly higher than that of during a scary stimulus with a P value of (0.027) ~ 0.03 for Theta/ Beta. The framework for portable fear detection together with the markers discussed in this study can enable many applications important for the soldiers in the battlefield or police officers while being under attack as an indicator that help is urgently needed.
Fear detection; Electroencephalogram; Signal processing; Real time portable system
Emotions are very important for human existence, since they control our ability of decision making, problem solving, communication and negotiation and allow us to adapt to unpredictable changes in the environments [1,2]. There are many ways to classify emotions rom simple pleasure/unpleasure  to detailed - happy, sad, disgust, fear, joy and anger . Electroencephalography (EEG - recording of electrical activity along the scalp) have been demonstrated effective for differentiation between different emotional states [3,5]. EEG signals are commonly described in terms of rhythmic activity across different frequency bands: a) delta (1-3 Hz), b) theta (4-7 Hz), c) alpha waves (8-13 Hz), d) beta waves (13-30 Hz), e) gamma (30-100+ Hz) [5-8]. Analyzing the distribution of power in respective frequency bands will reveal features that correspond to a particular state of the human mind [5,9].
The ratio of slow waves to fast waves (SW/FW) has been demonstrated to indicate emotions, behavioral and medical conditions [6,7,10-16]. SW/FW ratio was identified as a marker for motivational balance between the reward and punishment systems . They were also shown to have a negative correlation with fear and anxiety . Increase in Beta waves was correlated with fear . Recently a decrease in SW/FW ratio was observed after fear stimulus was applied . Feedback related negativity was also shown to negatively correlate to SW/FW ratio . A negative relationship between theta/beta ratio and attentional control was recently identified .
Various stimulus mechanisms and algorithms were used for invoking and analyzing emotions. Murugappan et al. used clustering algorithms Fuzzy C-Means (FCM) and Fuzzy k-Means (FKM) for classifying the emotions like fear in their studies by showing commercial movie clips for stimulus . In  photographs with happy and fearful faces were shown and EEG data was processed for analysis. In  eye blinking rate and facial temperature were monitored for identifying fear along with EEG. Participant’s responses to angry expressions invoking fear were monitored using fMRI . Petrantonakis et al. used higher order crossings analysis for emotion detection  while stimulus was provided by showing other humans expressing different emotions. In an experiment conducted by Kwang et al.  four types of visual stimulations were given in randomized method for 10 min each with 5 min intervals and EEG data was processed.
The current limitation of the EEG-based emotion recognition is that most of the results demonstrated on sophisticated equipment without portability and real time detection capabilities. Standard electrode arrays consist of 16-32 channels . The multi-channel configurations consist of 128-256 electrodes for high density recording  and thus typically are not suitable for mobile applications. Research on usage of single channel EEG is very scarce in the literature to date.
Here we present fear detection using very compact single channel portable system. Very small size, no need to use gel for the contact with the skin and small computational power required for processing of the data from just one channel enable many important real time applications for the safety and security.
In this study, normal EEG activity without any additional stimuli was continuously recorded during 155 second intervals. It was followed by an EEG recording during which the subjects were watching a fragment of a PG-13 video that contained several scary episodes.
Ten healthy human subjects with ages ranging from 18 to 30 years (average age 23 years, ± 4 years) old took part in the experiment. Most of the subjects were students from the University of South Florida. EEG recordings were conducted using commercially available single channel electrode- NeuroSky at sampling rate of 512 Hz. The recordings were taken from left frontal lobe region of each subject during the experiments. This is done in accordance with earlier reported studies. Studies [6,7] analyzed EEG recordings from frontal regions have been used for correlation between SW/FW ratios and emotions such as fear and attention control. Kwang et al.  recorded EEG signals at the left temporal lobe and observed an increase in beta waves during fear.
After the experiment each subject was given a questionnaire to rate the level of fear that they experienced during the watching of the scariest moments from the video. The average rating was 7 out of 10, where 10 correspond to the highest level of fear, what means that the video was moderately scary.
The acquired data was initially band pass filtered between 1 Hz (lower cutoff frequency) and 40 Hz (upper cutoff frequency) and then was processed to extract features in specific frequency bands. All the analysis and processing was done in using EEGLAB, an open source Matlab toolbox . Additionally, normal EEG recording was conducted on one of the subjects four days in a row under controlled conditions to test the device reproducibility, followed by statistical analysis.
The raw data from the single channel EEG were pre-filtered to remove serious and obvious motion artifacts. Next, the short time power spectral distribution for the entire duration of the experiment was computed using welches spectral estimation technique for every 1 s segment of the artifact-free signal. Information about three frequency bands was extracted, including delta (δ: 1-3 Hz), theta (θ: 4-7 Hz) and beta (β: 13-30 Hz). Normalization for the time and the frequency range was done according to the formulas (1) and (2), where: i range from q=1 Hz to s=3 Hz, w=3 for Delta, and i range from q=13 Hz to s=30 Hz, w=18 for Beta waves, j was changing between 1 and 155 Seconds.
The power densities were log normalized for calculating the ratios. Then the ratio of mean power of slow waves and fast waves was continuously plotted with time. The ratios of slow waves to fast waves during the watching of the scary movie episode were compared to that ratio taken in a normal state for each subject. Scary video stimuli for one of the subjects was stopped after first scary episode and reported as it is. Additionally, ANOVA analysis was performed on the set of four times repeated experiments to test the reproducibility of the normal data for the same subject. Next Student’s t-test (paired sample t-test) was performed with a null hypothesis stating the average value of the SW/FW ratio in normal emotional state is same as that during watching the scary stimulus to prove that there is significant difference.
The statistical parameters from the reproducibility test on a single subject under normal conditions are provided in Table 1. The experiment was conducted during 4 consecutive days under identical conditions. The SW/FW ratio average over the whole testing period (155 seconds) for those four days had mean value 3.20 with a standard deviation of 6%. This indicates that the SW/FW ratio is very reproducible for the same subject under identical conditions. A single factor ANOVA to test for significant difference between means over four days, returned a P value 0.1 (>0.05) and thus failed to reject a null hypothesis that the means of ratios of slow waves to fast wave over four days in normal condition are equal. The power spectra for these experiments are provided in Figure 1, it can be clearly seen that there is a fluctuation of the signal power from day to day, and the maximum values vary substantially, what can be attributed to variability of contact between the electrode and the skin. On the other hand the shape is very consistent, what results in stable SW/FW ratio (Table 2).
Table 1: The results of the reproducibility experiment conducted on the same subject under normal conditions over four days in a row.
|Source of Variation||SS||dj||MS||F||P-value||F crit|
Table 2: ANOVA for the reproducibility experiment conducted during four consecutive days demonstrates a P value >0.05 which fails to reject the null of equal means thus supporting that the SW/FW measure is reproducible for the same subject under identical normal conditions.
Since fear is a subjective emotion, in order to take into account personal differences, subjects were given a questionnaire where they had to rate the level of fear they experienced on scale from 1 to 10 and also identify the scariest moments of the video. Figure 2 demonstrates the power spectra generated by fast Fourier transformation of EEG signal for episodes indicated as the scariest (b) and the not scary at all (a). Similar to what was previously described for the multi-channel EEG  significant increase in fast wave (beta) activity was observed in presence of the scary stimulus.
A plot of the ratio of the power of the slow waves over fast waves vs. time for all subjects is shown in Figure 3 (Delta/Beta) and Figure 4 (Theta/Beta). The data for the normal state are shown in blue and during the watching of the scary movie in red. The scariest moments of the video are indicated with red solid blocks at the top of the figure. The averaged value (155 s) of the SW/FW ratio goes down for the scary stimulus. For many subjects the moments of the most pronounced decrease corresponded to the scariest episode in the video, while for others the ratio was staying low even in between the scariest episodes, probably because of anticipation of the fear. Another explanation might be that some of the subjects were scared while others just startled.
A Student’s t test conducted on the data shown in Table 3 shows a significant difference with P value of 0.04 (<0.05) in the Delta/Beta (SW/FW) ratios for the normal and scared states (Table 4) while Table 6 shows the statistical analysis conducted on data for Theta/Beta (SW/ FW) (Table 5) with a significant difference value of 0.03 for the normal and scared states indicating the strong evidence against null hypothesis stating equal means for both states and thus proving that SW/FW ratio could be a valid marker for the fear detection with a single channel EEG. It can be noticed from the t-tests that the Theta/Beta ratio has shown more significant difference compared to the Delta/Beta ratios (Tables 4 and 6).
|Subject||Mean SR/FW at Normal Conditions||Mean SW/FW during watching the scary video|
|Mean||Standard Deviation||Maximum||Minimum||Mean||Standard Deviation||Maximum||Minimum|
Table 3: The summary of the experimental data displaying significant difference between SW/FW (Delta/Beta) ratios in normal and scared states.
|Mean SW/FW at Normal Conditions||Mean SW/FW during watching the scary video|
Table 4: The summary of the statistical analysis of the experimental data displaying significant difference between SW/FW (Delta/Beta) ratios in normal and scared states.
|Subject||Mean SW/FW at Normal Conditions||Mean SW/FW during watching the scary video|
|Mean||Standard Deviation||Maximum||Minimum||Mean||Standard Deviation||Maximum||Minimum|
Table 5: The summary of the statistical analysis of the experimental data displaying significant difference between SW/FW (Theta/Beta) ratios in normal and scared states.
|Mean SW/FRY at Normal Conditions||Mean SW/Fir during watching the scary video|
Table 6: The summary of the experimental data displaying significant difference between SW/FW (Theta/Beta) ratios in normal and scared states.
In this study we for the first time demonstrate real time fear detection using portable single channel EEG. The ANOVA test and standard deviation measurements show reproducibility of the SW/FW ratio metric conducted in the normal state measured four days in a row. The study is in agreement with previous studies on detection of emotional state using multi-channel non-portable traditional EEG systems. We demonstrated that not only specific emotion can be identified, but it can also be done dynamically, in real time, while traditional studies normally focus on a static identification with a large number of channels. The results are consistent with previous research indicating changes in SW/FW ratio during emotional stimulus [6,7,11,12,14]. Based on a multimodal experiment  for the evaluation of fear, it was shown that facial temperature and subjective evaluation were more reliable than EEG signal recorded afterwards. Therefore, we focused on real time monitoring of EEG change with respect to scary stimulus and found the statistically significant reduction in SW/FW ratio. Since only 10 subjects were used in this study, further controlled experiments should be conducted with more subjects. This might be incorporated in wearable emotion detection systems, potentially as one of the available sensing modalities. The ability to identify fear in real time and potentially transmit this information remotely brings many important applications related to safety and security, especially when the scared person cannot verbally express this emotion. One example can be police officer under attack, so this device can be used to detect the danger and send help if needed. Another example can be a child that can be so scared of somebody that would never confirm that the person can be harmful, while this device would be able to identify the source of the fear and help with the protection of the child.
The authors are thankful to all the participants of the study. The study is carried under IRB approval Pro00011490 at University of South Florida. Surya is supported by the University of South Florida Graduate School Signature Research Doctoral Fellowship.