Multimodal Speech Sentimental Analysis

Abstract

In today’s world, communication is not just limited to text. Communication today can be in the form of various modes, such as speech, visual, and textual cues. With the help of these multiple modalities, sentiment analysis becomes more easy and accurate. This paper introduces Multimodal Speech Sentiment Analysis (MSSA), which integrates Convolutional Neural Network (CNN) and Bidirectional Encoder Representations from Transformers (BERT) algorithms for comprehensive analysis of the sentiments that are expressed through multiple modalities, such as acoustic features from speech. This paper sets a new standard for sentiment analysis by promoting speech data along with conventional textual analysis. We are not only focusing on multimodal approaches so that we can interpret the sentiments accurately but also seizing the complications of human expressions in the real world.