Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation

The paper "Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation" introduces RecLoRA, a model that enhances recommendation systems by incorporating personalized information from large language models (LLMs). The main limitations of current approaches, such as suboptimal performance due to shared LoRA parameters, inefficiency in handling long text sequences, and scalability issues with large datasets, are addressed by RecLoRA. The model uses a Personalized LoRA module for individual users and a Long-Short Modality Retriever for different history lengths. Additionally, it employs a Few2Many Learning Strategy to magnify small training spaces to full spaces. The model outperforms existing baselines in extensive experiments on public datasets. RecLoRA is a novel framework for LLMs in recommendation tasks that introduces a set of parallel, independent LoRA weights, a sequential CRM, and a soft routing method to aggregate meta-LoRA weights. This approach allows for parameter personalization, improves effectiveness with minimal time cost, and pre-trains the CRM on the full training space. The framework achieves a better trade-off between effectiveness and efficiency, requiring only a few-shot training data and extending sequence length without increasing time cost. Offline experiments on three datasets show promising results, validating the proposed model's effectiveness. The paper discusses the development of a personalized recommendation system using large language models (LLMs) for item preference estimation. It introduces two main paradigms for input data: ID modality, which involves one-hot encoding for conventional recommendation models, and textual modality, which uses hard prompt templates for LLMs. The paper then discusses conventional recommendation models, which capture collaborative patterns for precise user preference estimation. It also covers large language models as recommenders, detailing how they are adapted for recommendation tasks through low-rank adaptation (LoRA), a parameter-efficient fine-tuning method. The paper explains that LoRA maintains two trainable lightweight matrices attached to a frozen pretrained weight matrix to reduce resource consumption during fine-tuning. The proposed RecLoRA framework addresses three limitations: constructing personalized low-rank adaptation for LLMs, alleviating effectiveness and efficiency issues with behavior sequence extension, and expanding the receptive field of LLMs to the entire training space. The framework's network architecture and training process are described, with the overall goal of improving recommendation accuracy and efficiency. In summary, the paper presents RecLoRA, a lifelong personalized low-rank adaptation model for large language models in recommendation systems. It introduces a Personalized LoRA module, a Long-Short Modality Retriever, and a Few2Many Magnifier Strategy to enhance performance and efficiency. The model outperforms existing baselines in experiments on public datasets, demonstrating its effectiveness in recommendation tasks.