How To Steer Your Multi-Agent System: Human-LLM Collaborative Planning

Abstract

In orchestrated multi-agent systems, humans often struggle to manage plans due to their complexity and limited transparency. Existing approaches rely on outcome-level supervision, where users verify only final outputs without visibility into intermediate reasoning. We investigate human–LLM collaborative planning with interaction paradigms that enable process-level supervision, allowing humans to iteratively inspect, steer, and refine plans. Our System-X prototype supports semantic and structural interactions, including high-level LLM-assisted structural edits, letting users modify individual subtasks or overall plan structures while the LLM interprets guidance and suggests revisions. Through a user study and complementary experiments, we characterize human collaboration patterns and evaluate how LLMs respond to feedback with varying scope and revision strategies. Our findings reveal hybrid workflows, effort–control–risk trade-offs, and design insights for more transparent, controllable, and effective human–AI co-planning.