Multimodal large language models (MLLMs) continue to struggle with hallucinations, a problem that limits their reliability in practical deployments. According to a new research paper, existing attention-based mitigation approaches rely on indirect signals such as attention weights, which the authors argue do not accurately reflect the underlying information shift that produces hallucinations.
To address this, the researchers propose HEAL (Head-lEvel information disentAnglement and caLibration), a method designed to both identify and mitigate hallucinations. HEAL first applies causal noise intervention on multi-head outputs to filter out causally redundant heads, then disentangles the information distribution within the remaining heads using a counterfactual Difference-in-Differences approach that sorts heads into four categories.
The analysis yields a central finding: hallucinations occur when the information distribution drifts away from a healthy equilibrium in synergy heads, and this is not strongly correlated with the quantity or strength of modality-specific heads. In other words, the problem appears tied to how information is distributed rather than to how many dedicated visual or language heads exist.
Building on that insight, HEAL injects dynamic information calibration factors into the value vectors of synergy heads and actively regulates visual-language dependencies, steering the output distribution toward factual evidence. The authors report that extensive experiments show HEAL effectively reduces hallucinations across multiple MLLMs, describing it as a simple and interpretable pathway to improving model trustworthiness.