Social Processes: Self-Supervised Forecasting of Nonverbal Cues in Social Conversations. (arXiv:2107.13576v1 [cs.LG])

The default paradigm for the forecasting of human behavior in social
conversations is characterized by top-down approaches. These involve
identifying predictive relationships between low level nonverbal cues and
future semantic events of interest (e.g. turn changes, group leaving). A common
hurdle however, is the limited availability of labeled data for supervised
learning. In this work, we take the first step in the direction of a bottom-up
self-supervised approach in the domain. We formulate the task of Social Cue
Forecasting to leverage the larger amount of unlabeled low-level behavior cues,
and characterize the modeling challenges involved. To address these, we take a
meta-learning approach and propose the Social Process (SP) models–socially
aware sequence-to-sequence (Seq2Seq) models within the Neural Process (NP)
family. SP models learn extractable representations of non-semantic future cues
for each participant, while capturing global uncertainty by jointly reasoning
about the future for all members of the group. Evaluation on synthesized and
real-world behavior data shows that our SP models achieve higher log-likelihood
than the NP baselines, and also highlights important considerations for
applying such techniques within the domain of social human interactions.