PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedback

Kanika Goswami, Puneet Mathur, Ryan Rossi, Franck Dernoncourt·February 03, 2025

Summary

PlotGen, a multi-agent LLM framework, automates scientific data visualization, enhancing user trust and productivity. It iteratively refines plots, outperforming baselines by 4-6% on MatPlotBench. Key components include a Query Planning Agent, Code Generation Agent, and three Feedback Agents. PlotGen improves LLM-generated plot quality by 10-12%, making it a valuable tool for scientific visualization.

Key findings

2

Introduction
Background
Overview of scientific data visualization challenges
Importance of automation in enhancing user trust and productivity
Objective
To present PlotGen, a multi-agent LLM framework designed to automate scientific data visualization
Highlight improvements over existing methods, specifically a 4-6% outperformance on MatPlotBench
Method
Data Collection
Gathering datasets for various scientific domains
Utilizing diverse data types to ensure broad applicability
Data Preprocessing
Cleaning and formatting data for optimal visualization
Standardizing data structures to facilitate model training
Model Training
Training a large language model (LLM) on a diverse dataset of scientific plots
Incorporating feedback mechanisms to refine model outputs
Agent Design
Query Planning Agent: Determines the most effective visualization strategy based on the data characteristics
Code Generation Agent: Translates the visualization strategy into executable code
Feedback Agents: Continuously evaluate and improve the generated plots through user interaction and automated checks
Results
Performance Evaluation
Comparison with baseline methods on MatPlotBench
Quantitative metrics showing improvements in plot quality
User Trust and Productivity
Case studies demonstrating enhanced user satisfaction and efficiency
Quality Improvement
Detailed analysis of how PlotGen improves LLM-generated plot quality by 10-12%
Conclusion
Summary of Contributions
Recap of PlotGen's capabilities and achievements
Future Directions
Potential enhancements and research opportunities
Impact on Scientific Community
Discussion on how PlotGen can revolutionize data visualization practices in scientific research
Basic info
papers
computation and language
artificial intelligence
Advanced features
Insights
What is PlotGen and how does it enhance scientific data visualization?
How does PlotGen outperform baselines on MatPlotBench and what is the improvement in LLM-generated plot quality?
What are the key components of PlotGen and how do they contribute to its functionality?
What is the significance of PlotGen as a tool for scientific visualization?