Papers for 2026-11-02

10 papers
Zhiyuan Xiang, Xiang Deng, Wanjie Tao, ... , Zhao Jielun, Weili Guan
总结: 级联技能优化对齐分布并精炼机器人策略。
方法: 拒绝采样监督微调后冻结编码器做技能策略优化。
证据: 轻量策略使LIBERO成功率从80.1%升至97.5%。
为什么适合我: 技能级优化可结合模仿与强化学习用于全身控制。
推荐理由: 操作技能离散化与后训练优化,不是人形全身控制或腿足运动。
原摘要

Discretizing continuous actions into discrete skills using methods like VQ-VAE has emerged as a powerful paradigm for robotic manipulation. However, quantization errors yield a suboptimal distribution for the skill prior, degrading its performance. While reinforcement learning offers a path for refinement, its direct application faces unstable encoder updates and a granularity dilemma in importance sampling. To address these challenges, we introduce Cascaded Skills Optimization (CSO), a two-stage post-training framework. First, Rejection-Sampling Supervised Fine-tuning aligns the model's observation-to-skill mapping with the distribution of successful online trajectories via supervised fine-tuning. Second, with the observation encoder frozen, Skills Policy Optimization computes an independent, clipped importance ratio for each complete skill, enabling stable and efficient updates. With a lightweight QueST-style policy, CSO raises overall LIBERO success from 80.1% to 97.5%, improves performance on RoboTwin2.0, and achieves 61.0% success across five physical single-arm tasks. Additional comparisons with FAST, discrete BEAST, and fixed-buffer IQL further validate the effectiveness of our two-stage post-training strategy.

Nadeen Elhadad, Takuya Kiyokawa, Kensuke Harada
总结: 力驱动约束感知强化学习实现钩子非破坏拆卸。
方法: 物体中心仿真训练策略,粒子滤波整合力信息零样本部署。
证据: 四个真实任务验证,迁移至未见多阶段约束。
为什么适合我: 接触力感知契合接触丰富场景下的loco-manipulation。
推荐理由: 基于接触力的钩件拆解与物体中心强化学习,不是全身loco-manipulation或腿足运动。
原摘要

The growing volume of end-of-life products has intensified the need for automated disassembly, yet hook-type connections remain challengin: their orientations vary, they deform under load, and their geometry is hidden, so vision alone cannot identify a feasible non-destructive disassembly direction. This work proposes a constraint-aware reinforcement learning framework that infers feasible extraction directions online from contact forces. The policy is trained in a robot-agnostic, object-centric simulation on a simplified peg-in-hole pair and deployed zero-shot on a physical robot, where it transfers to real connections with unseen, multi-stage constraint structures. Because instantaneous contact forces are noisy and locally ambiguous, the policy is coupled with a particle filter belief that provides the policy with temporally integrated directional information and reduces redundant exploratory motion. The approach is validated on four real-world tasks, from peg-in-hole pairs to the inner propeller and front cover of an end-of-life air-conditioning unit. The full method achieves success rates of 100%, 100%, 90%, and 75% with the highest path efficiency on every task, and raises the success rate on the air-conditioning cover from 45% to 75% relative to a policy trained without the belief. These results indicate that force-driven reinforcement learning can enable disassembly without product-specific models or demonstrations.

Explore Where It Matters: Targeted Exploration for Efficient Rover Adaptation

2.0/5 偏低 裁判分 1.0 Candidate 当日相对
Jacob Levy, Arthur Bouton, Tristan D. Hassler, Hari Nayar, David Fridovich-Keil
总结: 目标探索高效适应漫游车可变形地形模型误差。
方法: 奖励加权损失适应世界模型,损失预测器引导MPC探索。
证据: 严重分布外下五次试验达约90%爬坡成功。
为什么适合我: 复杂地形在线适应与MPC利于腿足全身运动智能。
推荐理由: 可变形地形上的sim-to-real与MPC自适应最接近腿足感知运动,但对象是轮式星球车。
原摘要

Future planetary rovers will traverse steep, deformable terrain where slip, sinkage, and terrain deformation can threaten mobility. Although simulation can capture some wheel-soil effects, unmodeled deformable-terrain phenomena create sim-to-real gaps requiring deployment-time adaptation. Yet, rover operations leave little room for extensive trial-and-error. We introduce TERRA (Targeted Exploration for Reward-Relevant Adaptation), an online adaptation framework that efficiently seeks deployment data exposing task-relevant model errors and corrects them. TERRA adapts a simulation-trained world model with a reward-weighted prediction loss and trains a loss predictor to estimate where future performance-relevant errors are likely to occur, using this signal to guide exploration during MPC. In simulation, TERRA reaches approximately 90% slope-climbing success within five trials under severe out-of-distribution pretraining conditions, outperforming exploration baselines in reward, completion time, and energy to first success. On the ERNEST rover, TERRA recovers coordinated active-suspension locomotion on a deformable Mars-simulant slope within three to five real-world trials. Project website: https://terra-explore.github.io

Hooman Hamze, Benjamin Houshmand, Deepthi Pathare, ... , Morteza Haghir Chehreghani, Nikolce Murgovski
总结: 学习终端代价增强短视界MPC交互驾驶。
方法: 有限视界MPC用预训练RL估计终端代价,动态规划求解。
证据: 反应式周围交通仿真中评估该混合框架。
为什么适合我: RL终端代价与MPC结合可支持人形长视界全身控制。
推荐理由: 虽用RL终端代价改进短时域MPC,但对象是高速路重卡自动驾驶,属于明确不感兴趣的自动驾驶而非腿足/人形全身控制。
原摘要

Autonomous highway driving for heavy-duty trucks is a challenging control problem. The greater mass, dimensions and intricate dynamics increase the requirements for cooperation from surrounding traffic in critical maneuvers, requiring a careful balance between safety constraints and efficient long-horizon decision making. Model Predictive Control (MPC) provides rigorous constraint satisfaction but relies on terminal cost which can limit its effectiveness at short, computationally affordable horizons. Reinforcement Learning (RL) captures long-term driving behavior but offers no constraint guarantees and tends toward overly conservative policies when trained under sparse collision penalties. In this paper, we propose a hybrid MPC and RL framework for highway driving in autonomous trucks, where a finite-horizon MPC, solved via dynamic programming, uses a pre-trained RL model for terminal cost estimation. We evaluate the framework in a simulation environment with reactive surrounding traffic and compare it against pure MPC with zero terminal cost, rollout with terminal cost estimation using a base policy, and pure RL. Results show that the hybrid approach achieves zero collisions with the lowest cost of any configuration.

Yusen Feng, Shangbo Luan
总结: 多状态连续损伤粘塑性MPM支持RL灵巧操作。
方法: 工具中心局部化CD-MPM,双向耦合暴露阻力给策略。
证据: 六场景演示切割scooping与铲挖,结合PPO策略。
为什么适合我: 多状态接触仿真启发复杂地形与loco-manipulation。
推荐理由: 面向软体、脆性与粘塑性材料的灵巧操作仿真,落在不感兴趣的软体机器人方向。
原摘要

Robotic manipulation of multi-state materials needs a simulator that handles large deformation, topology change, and two-way contact between tools and materials while staying fast enough to drive online reinforcement learning. We present MG-CD-MPM, a tool-centered localized extension of CD-MPM. The implicit Ginzburg-Landau phase-field damage system is solved on a local patch by preconditioned conjugate gradients using a geometric-multigrid V-cycle as the preconditioner. The background MPM discretization remains uniform: only the damage solve is localized. Two-way rigid-tool coupling converts MPM contact impulses into forces and torques on the tool, exposing material resistance directly to the policy. The resulting Gym-like simulator covers soft solids, brittle materials, granular media, and viscoplastic or semi-fluid media within a unified per-particle constitutive layer. We demonstrate cutting, scooping, and shoveling in six scenes, combining scripted tool motions with PPO policies trained on two cutting tasks.

Aravind Venugopal, Grace Liu, Jeff Schneider, Benjamin Eysenbach
总结: 几何时间分数估计桥接自预测与生成占用模型。
方法: 沿概率路径学习占用密度,提取自预测表示。
证据: 多个运动与操作任务中显示性能提升。
为什么适合我: 改进RL占用估计利于全身运动策略与模仿学习。
推荐理由: 抽象强化学习占用密度估计,没有机器人运动或全身控制内容。
原摘要

In reinforcement learning (RL), temporal reasoning is essential for estimating the density of future outcomes under a policy's occupancy measure. While self-predictive representations have been a dominant paradigm for occupancy density estimation, we find that they yield surprisingly poor estimates. On the other hand, generative occupancy models can produce remarkably accurate samples from the occupancy measure. Yet, they cannot easily recover occupancy densities, limiting their use for control despite their expressivity. In this paper, we propose Geometric Time Score Estimators (GTSE), a framework that bridges the two paradigms by learning occupancy density estimators along probability paths. We analyze how GTSE implicitly induces better occupancy estimators through improved representation learning and show how to directly extract self-predictive representations from this framework. Through experiments across several locomotion and manipulation tasks, we show that GTSE improves performance over state-of-the-art methods by $2\times$ to $6\times$ in goal-conditioned RL settings, and that the representations learned using our framework transfer effectively to downstream RL tasks. Website: https://aravindvenu7.github.io/website/gtse/

Game-Theoretic Foundations for Undiscounted Bisimulation Metrics

1.0/5 偏低 裁判分 3.0 Candidate 当日相对
Matthew Zurek, Yudong Chen, Josiah P. Hanna
总结: 博弈论基础定义无折扣双模拟度量。
方法: 将度量视为零和博弈值函数,转移动态规划算法。
证据: 分析度量有限性依赖MDP结构并提出有向度量。
为什么适合我: 无折扣度量支持长视界全身运动强化学习。
推荐理由: 无折扣双模拟度量的博弈论基础,属于纯强化学习理论,与机器人运动无关。
原摘要

Bisimulation metrics offer a principled notion of task-relevant state similarity in reinforcement learning, but existing constructions and methods are largely confined to discounted MDPs, where contraction arguments ensure well-defined fixed points and downstream guarantees. We develop a unified foundation for bisimulation metrics in undiscounted settings, that is total- and average-reward problems, by replacing the standard operator-centric view with a game-centric one: we interpret prior metric operators as Bellman-Shapley operators of zero-sum turn-based games and define metrics as the corresponding game value functions, yielding unique definitions even when $γ=1$. This game-theoretic foundation facilitates transfer of dynamic-programming theory and algorithms. We also use this game-centric perspective to characterize how metric finiteness depends on structural properties of the underlying MDP and motivate certain directed metrics that remain finite in weakly communicating MDPs. We further introduce an abstract framework for modifying the games' action spaces, allowing general transition coupling sets, that recovers many existing metrics and guides new constructions. Finally, we prove that these metrics bound total-reward and, when the optimal gain is state-independent, optimal relative value differences. We also provide error and suboptimality guarantees for aggregation-based planning in total- and average-reward settings.

Yunchu Zhang, Ray Song, Junhan Jiao, ... , Siddhartha Srinivasa, Yiyue Luo
总结: 可互换末端与触觉传感实现工具中心操作。
方法: 统一接口加低成本适配器,集成触觉支持快速交换。
证据: 兼容多样工具与多平台,支持遥操作演示收集。
为什么适合我: 工具交换与触觉利于接触丰富loco-manipulation。
推荐理由: 家用工具快换与触觉末端执行器,不属于全身运动智能。
原摘要

Robot manipulation in everyday environments inherently involves substantial tool usage. Seemingly simple household tasks, such as cooking pan-fried tofu, require forceful cutting, delicate brushing, and precise placement using different tools under contact-rich interaction. Thus, the ability to autonomously swap and use task-appropriate tools is important for tool-centric manipulation. We introduce SwapBot, a modular, tool-centric end effector system featuring low-cost adapters that quickly convert common household tools into robot end effectors. Rather than pursuing a single universal gripper, SwapBot adopts a unified design principle that combines a fixed robot-side coupling interface with low-cost, tool-side adapters, enabling compatibility with diverse tools, including articulated ones, and multiple robot platforms. Each adapter integrates compact tactile sensors, supports fast autonomous swapping, and interfaces with an intuitive teleoperation system for efficient demonstration collection. User studies show that, during teleoperation, SwapBot reduces task completion time and operator workload. Using collected demonstrations on the robot, we train imitation learning policies to successfully perform tool-centric tasks like cooking tofu, stirring salad, flipping meat. Our ablation study shows that using tactile-augmented demonstrations improves policy performance across contact-rich skills, increasing success rates by 15 to 25 percentage points over RGB-only policies. These results highlight modular, tactile-equipped tools as a practical and extensible pathway toward robust and capable robot manipulation. Project website https://personalrobotics.github.io/SwapBot.

Decompose, Don't Expand: Capability-Independent AI Systems by Construction

1.0/5 偏低 裁判分 2.0 Candidate 当日相对
Tejaswi Gowda
总结: 分解任务而非扩展模型实现能力独立AI系统。
方法: 主机解决确定性部分,模型仅处理模糊残差。
证据: 九模型跨大参数范围,分解后子任务性能曲线平坦。
为什么适合我: 任务分解可优化全身感知运动中的模块化控制。
推荐理由: 通用AI系统任务分解设计,不涉及机器人控制或运动智能。
原摘要

A recurring design choice in building AI systems is whether, when a task is too hard for a model, to expand the model (make it larger, better-prompted, fine-tuned), or to decompose the task (extract the parts that admit a deterministic solution to a host, leaving the model only the irreducibly fuzzy residue). We make three claims. First, that many tasks commonly delegated whole to a model are in fact deterministic tasks with a small generative core. Second, and our central empirical result, that decomposing such a task yields a measurable and desirable property we call parity by construction: on the extracted sub-tasks the model-size performance curve becomes flat, because no model performs them at all. We measure this across nine models spanning a 494M-parameter local checkpoint to a frontier Opus-class model, a thirty-fold parameter range on the local family alone: once the host resolves the context, operation selection scores identically for all nine models and selector resolution for eight of nine, while the one task that does not decompose, labelling, remains sloped across the same range. Because the task leaves the model, a stock checkpoint suffices: no task-specific training and no fine-tuning. Third, that this is a general design discipline, not a domain trick: we show the same move recurring across physics setup, motion retargeting, and object-model editing, and in a Web-CLI tool family (same author, so not an independent replication). We give a method for locating the cut, the construction that makes a cut exist (a deterministic surface and a closed grammar over it), and a way to make the residue safe (resolve the deterministic context; verify the generative output), position the work against neuro-symbolic AI, tool-augmented language models, and generate-and-verify approaches, and are explicit about what is measured (one domain) versus illustrative (the rest).

Yi Liu, Xiangyue Zhang, Jianfang Li, ... , Yisheng He, Jianqiang Ren
总结: 过去作状态现在作注意力生成长视界共语运动。
方法: 块间持久状态总结风格,块内自注意力建模,流匹配。
证据: 解决长视界漂移与历史维护成本问题。
为什么适合我: 人体动作生成与重定向契合可跟踪全身控制。
推荐理由: 长时程共语动作生成偏角色动画,不是可上真机的物理全身控制,且接近不感兴趣的动画方向。
原摘要

Co-speech motion generation remains challenging over long horizons due to the need for temporal coherence and consistent style. Existing methods typically generate long motions segment by segment, either conditioning on a short seed of recent frames, which gradually drifts, or retaining ever-longer explicit histories, which are costly to maintain and hard to learn from limited data. We trace these limitations to a key asymmetry between past and present: past motion carries slowly varying style that should be compactly summarized, whereas the current segment exhibits fast-changing local dynamics that require fine-grained modeling. This asymmetry motivates a simple Past-As-State, Present-as-Attention principle for long-horizon co-speech motion generation. Following this principle, we propose PASPA, which summarizes past history with an inter-block persistent state and models the current segment with intra-block bidirectional self-attention. We instantiate PASPA within a blockwise flow matching framework, with multi-block supervision for effective training over variable histories, prefill-decode state caching for efficient rollout, and hybrid classifier-free guidance for enhanced history and speech conditioning. Extensive public-benchmark experiments show that PASPA achieves state-of-the-art FGD performance and preserves motion style across long-sequence generation.