Also Worth Noting - 2026-08-10

Also Worth Noting

02 [Training] Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss Offline distillation matches online distillation quality while decoupling teacher inference from student training entirely. Caching only the top-K logits once means the expensive teacher forward pass never blocks the training loop, making the whole pipeline feasible on single-node setups where running a full teacher in lockstep was the compute ceiling. A fused chunked KL loss further cuts memory overhead during the student update step. Teams compressing large models for on-premises deployment should treat this as a practical reference before designing their next distillation run. link

03 [Theory] SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMs SFT degrades on multi-task setups not because the tasks are too different, but because gradient updates across tasks accumulate destructive interference at shared parameter layers. RL avoids this by inducing sparse, approximately orthogonal updates across tasks, a pattern confirmed at the parameter level and explained through multi-task gradient interference theory. The result reframes a common fine-tuning decision: when tasks conflict under SFT, the answer is not more data or better mixing ratios. It is a different training signal entirely. link

04 [Agent] PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say Agents over-acquire sensitive information before they ever produce a response, meaning output-level privacy audits miss the actual exposure window. PrivacyPeek is the first benchmark targeting the acquisition stage, where data first enters the agent's context, rather than what the agent discloses in its final action or reply. Over-acquired context sits one careless tool call or one adversarial prompt away from a full leak. Teams deploying agentic systems should audit context ingestion, not just output filtering. link

05 [Open-source] DCAS: Decoupling CLI Agent Scaffolding to Internalize Planning across Scaffolds Fine-tuning CLI agents on OpenHands trajectories quietly bakes scaffold-specific planning conventions into model weights, and performance degrades substantially when the model is deployed under any other scaffold. Untrained base models do not show this gap, confirming the degradation is fine-tuning-induced rather than a capability ceiling. The open SWE-agent ecosystem has built a single-environment dependency without naming it as such. DCAS proposes decoupling planning structure from scaffold conventions so that learned behavior transfers across deployment environments. link

06 [Inference] Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors A single bidirectional diffusion model, trained to step a dynamical system forward or backward via a direction flag, generates a self-supervised error signal at test time with no ground truth required. Rolling forward i steps then backward i steps must return to the start; the round-trip discrepancy serves as a proxy for unobservable rollout error. No ensembles, no held-out labels, no governing equations. Deployed simulators that previously had no diagnostic signal between ground-truth checkpoints now have one built into the model itself. link