LLMs Meet Finance: Fine-Tuning Foundation Models for the Open FinLLM Leaderboard

Varun Rao, Youran Sun, Mahendra Kumar, Tejas Mutneja, Agastya Mukherjee, Haizhao Yang·April 17, 2025

Summary

English summary within 100 words: This paper examines large language models (LLMs) in finance, focusing on Qwen2.5 and Deepseek-R1 fine-tuning. It showcases significant performance improvements across financial tasks, measures data scaling laws, and highlights LLMs' potential in finance. FinRL-DeepSeek combines reinforcement learning with LLMs for trading, while FinMind-Y-Me, using sequential fine-tuning and task-specific prompts, excels in financial regulations challenges. LLM-based multi-agent systems simulate market dynamics. The text covers six studies, exploring LLM finetuning dynamics, efficient model fine-tuning, system optimizations, scaling laws, inference scaling, and phase transitions in LLMs.

Introduction

Background

Overview of large language models (LLMs) in finance

Objective

To evaluate the performance of Qwen2.5 and Deepseek-R1 in financial tasks and explore their potential applications

Method

Data Collection

Selection of financial tasks for model evaluation

Data Preprocessing

Preparation of data for model fine-tuning

Case Studies

FinRL-DeepSeek

Integration of reinforcement learning with LLMs for trading

FinMind-Y-Me

Sequential fine-tuning and task-specific prompts for financial regulations challenges

LLM-based Multi-Agent Systems

Simulation of market dynamics through LLMs

Analysis

LLM Finetuning Dynamics

Exploration of fine-tuning strategies and their impact on performance

Efficient Model Fine-tuning

Optimization techniques for enhancing model performance

System Optimizations

Improvements in model architecture and training processes

Scaling Laws

Measurement of data scaling effects on model performance

Inference Scaling

Analysis of model performance with varying input sizes

Phase Transitions in LLMs

Examination of model behavior under different conditions

Conclusion

Summary of findings and implications for future research in LLMs in finance

Basic info

papers

computation and language

machine learning

artificial intelligence

Advanced features