The Fusion of Large Language Models and Formal Methods for Trustworthy AI Agents: A Roadmap

The text outlines a roadmap for integrating Large Language Models (LLMs) and Formal Methods (FMs) to enhance AI trustworthiness. It proposes a two-pronged strategy: using FMs to improve LLM reliability and leveraging LLMs to enhance FM tools' usability and efficiency. The goal is to develop LLM agents that are trustworthy, localized, and capable of generating outputs that are faithful and rigorously certified. The text discusses integrating FMs with LLMs to create a unified neural-symbolic AI, focusing on enhancing LLM reliability through integration with solvers and improving FM functionality using LLMs for automated verification, specification, and proof generation. The paper advances the research agenda for bi-directional enhancement, discussing implications, challenges, and future directions.