DraftNEPABench: A Benchmark for Drafting NEPA Document Sections with Coding Agents
Anurag Acharya (Pacific Northwest National Laboratory), Bishal Lakha (Pacific Northwest National Laboratory), Rounak Meyur (Pacific Northwest National Laboratory), Rohan Nuttall (OpenAI), Sarthak Chaturvedi (Pacific Northwest National Laboratory), Anika Halappanavar (Pacific Northwest National Laboratory), Leah Hare (Pacific Northwest National Laboratory), Lin Zeng (Pacific Northwest National Laboratory), Mike Parker (Pacific Northwest National Laboratory), Sai Munikoti (Pacific Northwest National Laboratory), Sameera Horawalavithana (Pacific Northwest National Laboratory)
Evaluation & Benchmarking
Abstract
Coding agents represent a transformative paradigm in software engineering, enabling automated coding, generation, and debugging through a natural language interface. Recent advancements in large language models (LLMs) and their ability to use external tools have expanded the potential of using these agents beyond software engineering tasks. In this work, we explore the application of coding agents in a noncoding domain: drafting environmental impact statement (EIS) sections. For that, we introduce DraftNEPABench: a challenging benchmark that requires coding agents to compose structured, coherent, and domain-specific drafts grounded in multiple complex regulatory and scientific reference materials. We evaluate various state-of-the-art commercial coding agents on this benchmark and demonstrate their promise in generating EIS documents. Our findings show that while coding agents outperform vanilla retrieval-augmented generation (RAG) setups for these tasks, there is still room for improvement. We highlight the potential and limitations of such agents in high-stakes, complex, and real-world tasks, and point toward future directions.
