Live harness evolution and long-horizon agent architecture

Yang Xiao, Yusong Sun, Haoyi Wu, Wenyang Hui, Wen Da, Zhaokai Luo, Mu Chuan, Yao Hu, Wenjie Li, Chengyue Jiang

PILOT introduces a supervisor-worker harness that enables live self-improvement: a separate supervisor can redirect or abort an active worker mid-execution while simultaneously distilling procedures and failure modes into reusable skills and memory. Across frozen backbones and multiple benchmarks it ranks first in five of six configurations, outperforming counterpart harnesses by up to 9.8 points on Terminal-Bench 2.0 and delivering double-digit gains plus large reductions in output tokens in the self-improvement setting.

Wenqian Ye, Ziwei Guan, Eric Xie, Bohan Liu, Shivani Modi, Buyun Zhang, Ellie Dingqiao Wen, Henry Kautz, Aidong Zhang

ProofEvolve evolves explicit, Lean-verified proof DAGs by letting a neural model propose decompositions, repairs and schema recombinations that the symbolic kernel checks at every step. Verified partial results are retained in a behaviorally indexed archive and extracted into a persistent schema library, so incomplete attempts contribute reusable structure without sacrificing soundness. On three competition-level Lean benchmarks it achieves the highest average solve rate among evaluated systems.

Sanket Badhe, Priyanka Tiwari, Jonghyun Chung

SKILLLS replaces append-only conversation history with an explicit mutable execution state: at each step the model sees only the immutable skill specification, the current structured state and the latest observation, discarding intermediate reasoning after a validated update. This prevents prompt growth and context poisoning over long horizons. Across datasets, models and environments it improves task accuracy while substantially cutting cumulative token consumption.

Sydney Lewis

Fixing the model and task while varying only harness context management (full history versus mechanical shortening of older tool results plus stall handling) produces large differences in coding-agent outcomes. Under tight context the treatment harness raises fail-to-pass fractions and complete solutions on SWE-bench Verified and Pro, and the same frozen treatment improves three additional models without retuning. The results show that coding-agent evaluations must treat model plus harness as the joint solver.

Xiaoxiao Lu, Yunlong Dong, Jiahao Shi, Ye Yuan

LEON models latent state evolution for world-action models via context-modulated operator propagation plus additive forcing, grounded in a controlled Koopman generator view. The architecture separates transition realization from representation learning and prediction-policy coupling. Across two WAM formulations it improves closed-loop performance and robustness while remaining effective under full transition replacement.

Hong Chen, Yudong Zeng, Yongwei Huang, Zuhao Ouyang, Junyan Zhang, Xuming Hu

A leave-one-out probe shows attention magnitude is essentially uncorrelated with a token's causal contribution, undermining common eviction heuristics. TwinKV is a training-free, attention-free repair pass that detects near-duplicate keys, identifies orphans and redundant donors, and swaps them while preserving the original budget and scoring rule. Composed with four recent policies it improves a majority of configurations on LongBench, LooGLE and RULER for multiple small models.

Kasper Nordenram, Amelie Dittmann

ClusterAttention uses fast recursive clustering of keys and queries per head to form fixed-size power-of-two blocks that run at dense attention latency on GPUs, plus centroid compensation that shrinks approximation error. It is training-free and works for unstructured inputs in a single forward pass. On TabPFN-3 it yields 2-6x speedups at 99%+ accuracy retention; on Wan 2.1 video generation it is both closer to dense attention and faster than a domain-specific baseline.

Maciej Besta, Leonard Schmidt, Lara Nonino, Robert Gerstenberger, Pierre Pang, Patrik Okanovic, Ales Kubicek, Tiancheng Chen, Baraq Lipshitz, Torsten Hoefler

The paper systematizes RL-for-LLM post-training (PPO, GRPO and variants) from a compute-centric view and builds a taxonomy of intra- and inter-model parallelism for multi-model RLM pipelines, including disaggregated placement, stage fusion and asynchronous execution. Insights are made rigorous via the work-depth model. It analyzes existing frameworks and distills practical guidelines plus open directions for scalable, cost-effective reasoning model training.

Himal Badu

PoP fuses intermediate hidden representations across depth inside a single forward pass to capture layer-transition uncertainty as a signal of factual error. On TruthfulQA and related benchmarks it reaches 75.5% AUROC while adding under 1.2% latency and requiring no extra generation passes. The approach targets overconfident false assertions that defeat output-stage uncertainty metrics without multi-sample overhead.

Ivan Kruzhilov

The paper shows that the DPO coefficient beta entangles inverse preference-noise scale with optimization step size, producing non-monotone policy deviation and incomparable loss values across beta at fixed learning rate. A centered-softplus reformulation is argmin-equivalent yet makes the two effects independently tunable and admits a continuous beta-to-0 endpoint. The result clarifies hyperparameter roles and simplifies scheduling for preference optimization.