Enhancing Multimodal Affective Analysis with Learned Live Comment Features

LCAffect, a multimodal dataset, integrates live comments for sentiment, emotion, and sarcasm analysis in English and Chinese videos. It features synthetic live comment inference through a contrastive encoder, outperforming state-of-the-art methods. The dataset, comprising 11,356,212 live comments and 829 hours of video, offers a new benchmark for live comment analysis. A contrastive pre-training approach enhances a V2LC encoder for multimodal fusion, improving affective analysis tasks. Evaluated on six datasets, the proposed model demonstrates significant improvements in performance. The study also discusses advancements in self-supervised learning, multimodal analysis, and computational linguistics, highlighting key contributions and techniques.