Also Worth Noting - 2026-07-24

Also Worth Noting

02 [Training] Multi-Turn On-Policy Distillation with Prefix Replay Full on-policy distillation for agentic tasks is expensive because every update demands fresh student rollouts and new teacher queries across entire trajectories. Replayed-Prefix On-Policy Distillation (ReOPD) sidesteps this by reusing pre-collected teacher trajectories as frozen prefixes, letting the student act only at selected steps while the teacher supplies dense per-step supervision at those points alone. This cuts teacher query cost without abandoning on-policy signal. Teams training long-horizon agents where environment interaction is slow or costly should look at prefix replay before scaling up full-rollout pipelines. link

03 [Theory] Dataset Distillation by Influence Matching Matching intermediate gradients or training trajectories is a proxy for what dataset distillation actually wants: a compact synthetic set that shifts converged model parameters the same way the full dataset does. Influence Matching (Inf-Match) targets that final-parameter outcome directly, using a fully differentiable, sample-level influence estimator that runs in linear time without inverse-Hessian products or convexity assumptions. It outperforms trajectory-matching methods while being cleaner to differentiate and operating at the sample level rather than the batch level. Teams using dataset distillation for continual learning or data-efficient fine-tuning have a more principled objective to work from. link

04 [Training] Predictive Divergence Masks for LLM RL PPO-style trust-region masks conflate two separate jobs: checking whether the policy has drifted too far from the behavior policy, and checking whether the current update pushes it even farther. Tangling them causes unnecessary clipping that wastes samples. The predictive divergence mask separates proximity from update direction, replacing the ratio-based direction test with a forward-looking divergence estimate. The result is better sample efficiency on LLM RL tasks without changing the underlying algorithm. This is a targeted fix worth applying before reaching for larger batch sizes or more rollouts. link

05 [Inference] SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation Pure linear attention scales well but loses the full-rank token interactions that make video generation coherent. SANA-Video 2.0 inserts full-softmax attention at boundary frames on a 3:1 ratio against gated linear attention blocks, recovering those interactions only where they matter most. At 5B and 14B scales, the hybrid matches quality benchmarks set by fully quadratic video DiTs while staying within single-GPU memory at 720p. The architecture is a concrete recipe, not just a scaling result, and the 3:1 ratio is a tunable starting point for teams building video generation infrastructure. link

06 [Eval] Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text Models that score well on text-coordinate spatial benchmarks fail when asked to mark, draw, or point directly on images, exposing a systematic answer-interface mismatch in how spatial reasoning is measured today. Current benchmarks force continuous visual tasks into discrete text outputs, which means scores reflect coordinate formatting ability as much as genuine spatial understanding. Evaluating spatial cognition through generative pixel outputs closes that gap and reveals capability levels that text-based scores consistently overestimate. Teams shipping vision agents into physical or spatial contexts should treat text-coordinate spatial benchmark scores with more skepticism than the numbers suggest. link