Your Agent's Harness Matters More Than Its Model

The dominant assumption in agent development is that capability scales with the foundation model: bigger weights, better reasoning, better results. JIT-Agent exposes a different variable. The harness, meaning the memory management, planning strategy, action protocol, and tool orchestration wrapped around any given model, can dominate the contribution of the underlying model entirely. And until now, every team was writing that harness by hand.

The core mechanism is a trained model purpose-built to synthesize agent harnesses on the fly. JIT-Agent formalizes the harness as a composable, machine-generatable artifact governed by a fixed four-module protocol, then trains on that schema to do three things: customize harnesses for a given task, repair harnesses mid-execution when they break, and self-evolve by distilling performance signals from an expanding archive of prior configurations. Think of it as a compiler that reads a task, generates the scaffolding an arbitrary off-the-shelf LLM needs to solve it, then patches that scaffolding when execution drifts. The harness is not static infrastructure written once and deployed forever. It is generated fresh per task, then updated as evidence accumulates.

The self-evolution loop is the structural piece worth watching. Each harness configuration that runs against a task produces a performance signal. JIT-Agent distills that signal back into its own weights, meaning the harness generator gets better as it accumulates more prior configurations. The archive compounds. This is not fine-tuning the base agent model. It is a separate, orthogonal dimension of capability improvement that runs independently of model scaling.

Equipped with JIT-Agent as a harness helper, DeepSeek-V4-Flash surpasses GPT-5.6 on DeepSearchQA by 9.1 points and on OdysseyBench by 4.3 points. GLM-5.2, already a strong baseline, gains up to 20.2 points. JIT-Agent-generated harnesses match performance-competitive mature agent runtimes including OpenCode and Claude Code, and consistently improve multi-scale model families across DeepSeek V4, Mimo-V2.5, and Qwen3.6. For teams shipping agents in production, the takeaway is direct: the harness you deploy is as consequential as the model you select, and that harness is now trainable.

We're thinking: We find the compounding archive the most consequential detail here. The gains on DeepSeek-V4-Flash over GPT-5.6 are not just a benchmark win. They are evidence that a smaller, cheaper model with a well-generated harness can beat a larger, more expensive model with a generic one. That directly challenges the default procurement logic: buy a bigger model, get better agents. If harness quality compounds independently of model quality, then teams spending on frontier model access may be solving the wrong variable entirely. The more uncomfortable read is that every team currently hand-authoring agent scaffolding is accumulating technical debt that a trained harness model could eliminate, and the gap between teams that adopt this and teams that do not will widen with every task the archive grows on.

Key takeaways:

• JIT-Agent introduces harness intelligence as a trainable, transferable capability layer: a dedicated model generates task-adaptive scaffolding at runtime and self-evolves via a compounding archive of prior harness configurations, entirely orthogonal to base model scaling.
• DeepSeek-V4-Flash beats GPT-5.6 by 9.1 points on DeepSearchQA and 4.3 on OdysseyBench; GLM-5.2 gains up to 20.2 points. Results hold across three model families, though evaluations are controlled benchmarks and production generalization remains to be tested at scale.
• Teams building or maintaining agent systems should audit how much engineering effort goes into hand-authored scaffolding: if the harness is a primary performance driver, a trained harness generator is a higher-leverage investment than the next model upgrade.

Source: JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution