Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models

Junjie Wu, Tsz Ting Chung, Kai Chen, Dit-Yan Yeung·October 30, 2024

Summary

The paper introduces a unified framework for evaluating hallucinations in Large Vision-Language Models (LVLMs), focusing on both object and relation hallucinations. It proposes the Tri-HE benchmark to assess these issues, revealing that relation hallucinations are more severe. A training-free approach is proposed to mitigate hallucinations, outperforming open-sourced counterparts on Tri-HE. The framework highlights the need for a comprehensive study on hallucinations in LVLMs, emphasizing the importance of reliability in these models.

Key findings

6

Tables

7

Advanced features