Large-scale vision-language models (VLMs) have shown impressive versatility across multimodal tasks, yet humor comprehension remains a challenge. Humor often relies on subtle interactions among entities, events, and context across image and text, requiring complex reasoning chains that conventional prompting or linear chain-of-thought methods struggle to capture.
To address this, researchers introduce CaRGo-T (Causal Reasoning Graph-of-Thought), a framework that models the causal and contextual relationships underlying multimodal humor as a lightweight graph-based reasoning structure. The graph is serialized into a code-based representation generated by a VLM, which can then be interpreted by the same or a different VLM to produce final predictions in zero-shot or in-context learning settings.
Evaluated on four datasets covering satire, sarcasm, and memes, CaRGo-T consistently outperforms existing reasoning-based baselines with state-of-the-art commercial and open-source VLMs. The framework achieves performance gains of approximately 1-20% on humor understanding and 1-3% on humor detection. Further analysis using mutual information shows that CaRGo-T's reasoning representations contain more information relevant to the target output than baseline approaches.
The code is available on GitHub, and the paper is published on arXiv.