Remote Facial Emotion Recognition System

Григорий Романович Хазанкин, Иван Сергеевич Шмаков, Алексей Николаевич Малинин

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

Abstract

This paper describes a method for collecting information of user's emotional state based on its facial expressions and linking the data acquired to the user's actions in a time. Methods and Algorithms: At client side, we use reactive programming model implemented by RxJS library to manage data streams. Our client library includes logic to combining events from different sources such as MediaStream APIs, classic DOM event API. We have developed our own neural network architecture that allows us to map the facial expressions of people into a continuous space of arousal/valence. The same architecture for classification of emotions was used. We used inception and residual blocks and batch normalization layers to achieve higher accuracy. In addition, we augment train data by adding pose variations using 3D face reconstruction. Results: We have developed a system that allows us to collect various data such as video stream and events of user interaction with the webpage. We also trained a neural network that shows high accuracy. One part of the system is a client library that does not require the user to install any special software and can be easily integrated into the website. The other part is a server application that enables the site owner to analyze the data collected.
Original languageEnglish
Title of host publicationSIBIRCON 2019 - International Multi-Conference on Engineering, Computer and Information Sciences, Proceedings
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Pages975-979
Number of pages5
ISBN (Electronic)978-1-7281-4401-6
ISBN (Print)978-1-7281-4402-3
DOIs
Publication statusPublished - Oct 2019
Event2019 International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON) - Academpark, Dom Uchyonykh—TUSUR, Ural Hi-Tech Park, Novosibirsk, Tomsk, Yekaterinburg, Russian Federation
Duration: 21 Oct 201927 Oct 2019
Conference number: V
https://ieeexplore.ieee.org/xpl/conhome/1002416/all-proceedings

Publication series

NameSIBIRCON 2019 - International Multi-Conference on Engineering, Computer and Information Sciences, Proceedings

Conference

Conference2019 International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON)
Abbreviated title2019 SIBIRCON
Country/TerritoryRussian Federation
CityNovosibirsk, Tomsk, Yekaterinburg
Period21.10.201927.10.2019
Internet address

Keywords

  • Biometrics , Behaviometrics , Network Traffic Analysis , Identification , Signature-based Approach , Inference-Based approach , Hybrid approach , YAF , GMMs
  • facial expression
  • data streams
  • valence
  • machine learning
  • remote diagnostics
  • psycho-physiology
  • arousal

Fingerprint

Dive into the research topics of 'Remote Facial Emotion Recognition System'. Together they form a unique fingerprint.

Cite this