Also Worth Noting - 2026-07-29

Also Worth Noting

02 [Training] Pass the Baton: Trajectory-Relayed On-Policy Distillation On-policy distillation breaks down not because the teacher is weak but because a bad prefix poisons every token that follows it. Relay-OPD detects a teacher-student continuation asymmetry on failed prefixes, where the teacher redirects while the student keeps going in the wrong direction, and uses that divergence as a label-free trigger to hand the trajectory off mid-generation. The student gets supervision grounded in a recoverable trajectory rather than a compounding mistake. Teams running OPD for reasoning fine-tuning should audit how much compute is currently burning on misdirected rollouts. link

03 [Inference] Reinforcement Learning for Code Optimization Adding execution time to a code-correctness reward signal sounds straightforward; in practice, timing noise alone is enough to collapse RL training into instability with solutions that barely improve and fail more often. The paper isolates three failure modes, measurement noise, reward sparsity, and GRPO instability, and addresses them through a staged pipeline that includes a new benchmark, DMC-Optim, built specifically around large optimization tests where timing signal is clean. Teams bolting execution-time rewards onto existing code-correctness pipelines will get noise, not speed gains, without this measurement discipline first. link

04 [Agent] CodeNib: A Multi-View Data System for Serving Repository Context to Coding Agents Coding agents rebuilding repository context from scratch on every task pay a hidden latency and token cost that benchmark pass-rates never surface. CodeNib constructs persistent lexical, dense, and structural views per commit, maps outputs to repository-relative source ranges, and maintains those views across edits, serving ranked search and bounded context through a single runtime. Across 100 repository snapshots it maps quality-cost frontiers across the full context lifecycle. The systems fix here may matter more for production agent cost than another point of benchmark accuracy. link

05 [Training] Constitutional Midtraining: Content Presence Drives Alignment Gains Alignment installed during pretraining at 120B scale does not erode under subsequent fine-tuning, directly challenging the assumption that RLHF post-training is the right place to install values. A 394M-token constitutional corpus built from Anthropic's Constitution was inserted at midtraining, with a 2x2 design crossing curriculum ordering against deliberative reasoning to produce four conditions plus a control. Content presence, not curriculum order or deliberative framing, drives the alignment gains. Teams relying on post-training alignment for safety-critical deployments should treat this as evidence that the intervention is happening too late in the pipeline. link

06 [Application] Shieldstral A 3B-parameter safety classifier matching or beating models nearly 7x its size is a deployable cost argument, not just a benchmark result. Shieldstral frames content moderation as binary question-answering, a formulation that unifies divergent safety taxonomies across heterogeneous datasets under a single training framework without requiring taxonomy alignment. It sets a new state of the art on multimodal safety classification. For content moderation teams, the inference cost drops by roughly 7x compared to the models it beats, with no accuracy penalty. link