POLLY: A Multimodal Cross-Cultural Context-Sensitive Framework to Predict Political Lying from Videos

Chongyang Bai, Maksim Bolonkin, Viney Regunath, V. S. Subrahmanian

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Politicians lie. Frequently. Depending on the country they are from, politicians may lie more frequently on some topics than others. We develop the novel concept of a tripartite "VAT"graph (Video-Article-Topic) with three types of nodes: videos (with a politician featured in each), news articles that mention the politician, and topics discussed in the videos or articles. We develop several novel types of audio and video deception scores for each audio/video, as well as a topic deception score and an edge deception score for each edge in the graph. Our POLLY (POLitical LYing) system builds upon past work by others to generate predictions for whether a politician is lying or not. We test POLLY on a novel dataset (which will be made publicly available upon publication of this paper) consisting of 146 videos and 6337 news articles involving 73 politicians from 18 countries from all major continents. We show that POLLY achieves AUC and F1 scores over 77%, beating out several baselines. We further show that POLLY is robust to translation errors made by Google Translate.

Original languageEnglish (US)
Title of host publicationICMI 2022 - Proceedings of the 2022 International Conference on Multimodal Interaction
PublisherAssociation for Computing Machinery
Pages520-530
Number of pages11
ISBN (Electronic)9781450393904
DOIs
StatePublished - Nov 7 2022
Event24th ACM International Conference on Multimodal Interaction, ICMI 2022 - Bangalore, India
Duration: Nov 7 2022Nov 11 2022

Publication series

NameACM International Conference Proceeding Series

Conference

Conference24th ACM International Conference on Multimodal Interaction, ICMI 2022
Country/TerritoryIndia
CityBangalore
Period11/7/2211/11/22

Keywords

  • Deception Detection
  • Deception Detection Dataset
  • Natural Language Processing
  • Political Deception
  • Video Understanding

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Fingerprint

Dive into the research topics of 'POLLY: A Multimodal Cross-Cultural Context-Sensitive Framework to Predict Political Lying from Videos'. Together they form a unique fingerprint.

Cite this