DGRO: Enhancing LLM Reasoning via Exploration-Exploitation Control and Reward Variance Management

Xuerui Su, Liya Guo, Yue Wang, Yi Zhu, Zhiming Ma, Zun Wang, Yuting Liu·May 19, 2025

Summary

DGRO enhances large language models' reasoning, achieving 96.9% accuracy on Logic datasets. It outperforms base models on benchmarks like K&K and Math, balancing exploration and exploitation. Recent deep reinforcement learning focuses on integrating human preferences and logic reasoning. DGRO validates its effectiveness, surpassing base models in reasoning tasks.

Introduction

Background

Overview of large language models

Importance of reasoning in language models

Previous approaches to improving reasoning capabilities

Objective

The goal of DGRO in enhancing reasoning abilities

Key achievements of DGRO on Logic datasets and benchmarks

Method

Data Collection

Types of data used for training and evaluation

Sources of data for Logic datasets and benchmarks

Data Preprocessing

Techniques applied to data for improving model performance

Handling of specific challenges in Logic datasets and benchmarks

Integration of Human Preferences and Logic Reasoning

Strategies for incorporating human preferences into the model

Methods for enhancing logical reasoning within the model architecture

Deep Reinforcement Learning Approach

Overview of the reinforcement learning framework used by DGRO

How exploration and exploitation are balanced in the learning process

Results

Performance on Logic Datasets

Detailed accuracy metrics on Logic datasets

Comparison with base models

Benchmark Performance

Results on benchmarks like K&K and Math

DGRO's superiority over base models

Effectiveness in Reasoning Tasks

Specific reasoning tasks DGRO excels in

Analysis of how DGRO surpasses base models in these tasks

Conclusion

Summary of DGRO's Contributions

Recap of DGRO's improvements in reasoning capabilities

Future Directions

Potential areas for further research and development

Integration of DGRO with other AI techniques

Basic info

papers

machine learning

artificial intelligence

Advanced features

Insights

What are the core components of DGRO that contribute to its effectiveness in reasoning tasks?

In what ways does DGRO outperform base models on benchmarks like K&K and Math?

How does DGRO enhance reasoning capabilities in large language models?

How does DGRO balance exploration and exploitation in reasoning tasks?