Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling

The LLM-QL model uses large language models for dense retrieval, prioritizing query likelihood maximization. It includes an auxiliary task for contrastive learning, employing Attention Stop and Input Corruption to improve global document semantics. Experiments show superior performance over other LLM-based retrievers, with enhanced query likelihood ranking compared to word-based methods. This research advances information retrieval, language modeling, and unsupervised dense information retrieval, aiming to boost search relevance and entity search.