Demystifying Domain-adaptive Post-training for Financial LLMs

The paper introduces FINDAP, a systematic investigation into domain-adaptive post-training for finance, focusing on identifying core capabilities, designing an evaluation suite, and analyzing post-training stages. It proposes a novel preference data distillation method, leading to the Llama-Fin model, which outperforms others in financial tasks. Each post-training stage contributes to distinct capabilities, offering insights for domain adaptation of Large Language Models (LLMs).