Evaluation of electrocardiogram: numerical vs. image data for emotion recognition system.

Background: The electrocardiogram (ECG) is a physiological signal used to diagnose and monitor cardiovascular disease, usually using ECG wave images. Numerous studies have proven that ECG can be used to detect human emotions using numerical data; however, ECG is typically captured as a wave image rather than as a numerical data. There is still no consensus on the effect of the ECG input format (either as an image or a numerical value) on the accuracy of the emotion recognition system (ERS). The ERS using ECG images is still inadequately studied. Therefore, this study compared ERS performance using ECG image and ECG numerical data to determine the effect of the ECG input format on the ERS. Methods: This study employed the DREAMER dataset, which contains 23 ECG recordings obtained during audio-visual emotional elicitation. Numerical data was converted to ECG images for the comparison. Numerous approaches were used to obtain ECG features. The Augsburg BioSignal Toolbox (AUBT) and the Toolbox for Emotional feature extraction from Physiological signals (TEAP) extracted features from numerical data. Meanwhile, features were extracted from image data using Oriented FAST and rotated BRIEF (ORB), Scale Invariant Feature Transform (SIFT), KAZE, Accelerated-KAZE (AKAZE), Binary Robust Invariant Scalable Keypoints (BRISK), and Histogram of Oriented Gradients (HOG). Dimension reduction was accomplished using linear discriminant analysis (LDA), and valence and arousal were classified using the Support Vector Machine (SVM). Results: The experimental results indicated that numerical data achieved arousal and valence accuracy of 69% and 79%, respectively, which was greater than those of image data. For ECG images, the highest accuracy for arousal was 58% percent; meanwhile, the valence was 63%. Conclusions: The finding showed that numerical data provided better accuracy for ERS. However, ECG image data which shows positive potential and can be considered as an input modality for the ERS.