LLMs Track Goals in Static Tests, Then Lose the Thread When Users Change Their Mind
A new framework reveals that strong single-turn LLM performance collapses when user intent shifts mid-conversation, exposing a blind spot in every static eval suite.
Every major LLM eval suite measures whether a model can complete a task given a complete, well-formed prompt. That assumption bakes in something that rarely holds in practice: that users know exactly what they want before the first message, and never change their mind.
The framework introduced here transforms existing single-turn benchmarks into dynamic multi-turn conversations where user intent shifts across turns, incrementally revealed, revised, and sometimes redirected entirely mid-conversation. No new annotation is required. The original evaluation protocol stays intact, so the same scoring applies, which means performance drops are directly comparable to the static baseline. The core design insight is that the gap it exposes is not a benchmark artifact. It is a measurement gap that static settings structurally cannot see.
The mechanism works by staging intent disclosure across turns rather than front-loading it. Think of it as the difference between handing a contractor a complete blueprint versus describing the renovation room by room, changing your mind about the kitchen halfway through. Models that score well in the blueprint setting are being tested on a skill, complete-instruction following, that is adjacent to but distinct from the skill that actually matters in agentic deployment: tracking and integrating a goal that keeps moving. When intent is revised mid-conversation, the model must decide whether to anchor to earlier context, defer to the latest instruction, or synthesize both, and the framework surfaces exactly where each model family breaks down.
Across multiple task types and model families, performance drops substantially when intent evolves. The gap is consistent, not model-specific. A model that handles a fully-specified task cleanly can fail the same task when the specification arrives in pieces or gets revised after partial completion. For teams building or evaluating conversational agents, the takeaway is direct: any eval suite built entirely on static, single-turn prompts is measuring a capability that does not transfer to the interaction patterns real users produce.
We're thinking: We find the most pointed implication here to be about what the field has been optimizing for. If the models that score best on today's agentic benchmarks are, in effect, very good at following complete instructions, then the training signal is systematically rewarding the wrong behavior for deployment. Every RLHF or SFT dataset built from single-turn preference pairs reinforces static-intent completion. The paper does not claim to fix this, but it does make the cost visible: teams shipping agents today are likely shipping systems that degrade in exactly the situations that matter most, when users are still figuring out what they want.
Key takeaways:
- The framework stages intent evolution across turns within existing benchmarks, exposing a failure mode that single-turn evaluation cannot detect by design.
- Performance drops substantially and consistently across model families when user intent shifts mid-conversation; the effect holds across multiple task types with no model family immune.
- Teams evaluating or fine-tuning conversational or agentic systems should add evolving-intent test conditions to their eval pipelines before treating static benchmark scores as deployment proxies.