Papers for 2026-10-19

10 papers
Zetao Du, Weiqing Wang, Hanbing Zhang, ... , Liang Wang, Jingya Wang
总结: 从人类演示重建物体中心4D手部,生成机器人动作监督。
方法: 提取交互片段重建物体几何,优化手物交互并重定向运动。
证据: 结合置信与时序约束选物体轨迹,分阶段2D对齐细化手部。
为什么适合我: 支持人体动作重定向,利于接触丰富操作与全身控制。
推荐理由: 最接近人体动作重定向与模仿及遥操作数据采集:从人类演示重建手物交互并重定向为机器人动作监督,但聚焦手部而非人形全身或腿足运动。
原摘要

Human demonstrations can capture task-relevant manipulation without using the target robot, but the recorded hand motions cannot directly serve as robot action supervision. Hand reconstruction and motion retargeting provide a direct route to such supervision, yet recovering hand motion alone does not ensure physically consistent hand-object interactions. To address this issue, we introduce Hand2Robot, an object-centric 4D hand reconstruction framework for robot policy post-training. Given videos of humans performing the target task, Hand2Robot extracts interaction clips, reconstructs object geometry, and generates robot action supervision through hand-object interaction optimization and motion retargeting. For object tracking, we combine observation confidence with temporal motion constraints to select a globally consistent object pose trajectory from per-frame pose candidates. For hand refinement, we use object geometry and motion as a reference and combine staged 2D alignment with object-local contact constraints to refine hand trajectories and finger joint configurations. Experiments on DexYCB and HO3D-v3 show improved object pose accuracy and hand-object interaction quality. Experiments on three real-robot dexterous manipulation tasks further show that reconstructed and retargeted human demonstrations can effectively supplement limited robot data for target-task policy post-training.

CSF: Contextual Safety Filtering for Motion Generators

3.0/5 一般 裁判分 3.0 Candidate 当日相对
Lizhi Yang, Yiling Hou, Yao Tang, ... , Blake Werner, Aaron D. Ames
总结: 为运动生成器提供场景依赖的上下文安全过滤。
方法: 用安全不安全参考轨迹定义仿射安全值,CBF-QP强制。
证据: 四种生成器上危险事件率降达90%,保留88-100%良性运动,G1真机验证。
为什么适合我: 提升全身运动安全,适用于人形复杂场景与真机部署。
推荐理由: 给可跟踪全身动作生成器做场景安全过滤,只是外围安全模块,不是运动学习或全身控制本身。
原摘要

Text-conditioned motion generators produce trackable whole-body motion, but they have no notion of scene-dependent safety: the same action may target an object or a person. Existing safeguards either inspect the prompt, require labeled motion data, or enforce geometric constraints; therefore, they do not directly account for how scene context changes a motion's meaning. We introduce contextual safety filtering (CSF), a training-free filter that grounds natural-language safety rules in safe and unsafe reference trajectories produced by the generator. For each active rule, safe and unsafe reference trajectories define an affine safety value that a safe reference tracking CBF-QP enforces. Across four pretrained generators with different architectures, CSF activates the intended rules in all explicit and scene-triggered unsafe cases and reduces the danger-event rate by up to 90%, while preserving 88-100% of benign motions. We demonstrate the complete system on a real-world Unitree G1, where it successfully prevents unsafe motions in a variety of scenarios, including interactions with humans and objects.

Wenxuan Zhu, Yuze Wang, Jiangran Lyu, ... , Zhizheng Zhang, He Wang
总结: 统一学习抓取与非抓取策略实现通用物体位姿到达。
方法: 预训练几何编码器预测接触运动,两阶段RL从可抓到不可抓。
证据: 跨多样物体夹爪联合学习,克服分离原语与几何敏感问题。
为什么适合我: 接触丰富操作协调,利于loco-manipulation与全身接触。
推荐理由: 最接近全身loco-manipulation与接触丰富场景交互:统一学习抓取与非抓取到达,但是夹爪物体操作,缺少腿足移动与人形全身协调。
原摘要

Object manipulation often requires combining non-prehensile actions with grasping. However, existing approaches typically develop these capabilities separately and integrate them by sequencing distinct primitives through prescribed intermediate objectives, limiting the coordination of contact motions and grasping. Their integration also remains sensitive to object and gripper geometry and depends on particular environmental supports, complicating real-world deployment. We present Uni-PNP, a unified policy that jointly learns prehensile and non-prehensile manipulation for object pose reaching across diverse objects and grippers. To achieve this, we first pretrain a geometry encoder to predict contact-induced object motion using a contact-dynamics dataset constructed from simulated interactions between diverse objects and grippers. We then train the policy in a two-stage reinforcement learning curriculum from graspable objects to ungraspable peers, encouraging the discovery of non-prehensile interactions and their integration with grasping. Experiments show 76.39% success on hundreds of seen objects with grippers, and 57.91% on unseen peers, outperforming the strongest pose-reaching baseline by 20.52 and 17.79 percentage points, respectively. We also analyze emergent behaviors, including nonprehensile and prehensile repositioning, grasp acquisition that requires grasping after tipping, and table-assisted reorientation while grasping, and transfer to the real-world scenario. Project page available at lgswdn.github.io/Uni-PNP.

Dexterity from Supervised Play

2.0/5 偏低 裁判分 7.0 Strong 当日相对
Botao He, Yiming Dou, Yiannis Aloimonos, ... , Akash Sharma, Ruoshi Liu
总结: 从人类玩耍学策略,自主收集数据训世界模型做灵巧操作。
方法: 行为克隆玩耍策略,MPC引导评估,自玩蒸馏闭环策略。
证据: 真机优于事后BC达69.6%,与RL相当但数据少490倍。
为什么适合我: 模仿学习与MPC结合,利于接触丰富真机全身控制。
推荐理由: 多指手内推与姿态重定向的玩策略和世界模型MPC,不属于全身人形/腿足运动智能。
原摘要

How can robots autonomously collect the contact-rich experience needed to learn dexterous manipulation? We present DSP: Dexterity from supervised play, a framework in which a robot learns a play policy from minutes of human play through behavior cloning, then deploys it autonomously to collect hours of interaction data to train a world model. During inference, the same play policy then guides model-predictive control (MPC) by proposing candidate action sequences for evaluation under the learned world model. To accelerate inference, we also introduce a method for distilling a closed-loop, goal-conditioned policy through self-play within the learned world model. We evaluate this framework on underactuated pushing and any-to-any in-hand pose reorientation with a multifingered hand, in simulation and real world. In real world experiment, our method outperforms hindsight behavior cloning by 69.6% in success rate and achieves comparable success to reinforcement learning with 490 times less interaction data. These results demonstrate that supervised play can autonomously collect interaction data, guide model-based planning, and enable policy learning without any task-specific demonstrations or sim-to-real transfer.

Guoping Pan, Xinmiao Yu, Wannan Cao, ... , Yi Cheng, Houde Liu
总结: 从单次人类视频提取轨迹,扩散生成可泛化操作数据。
方法: 6D位姿估计提取物体轨迹作中间表示,缓解域差距。
证据: 轨迹独立于视觉域与本体,支持高效数据生成与泛化。
为什么适合我: 人体视频转机器人,契合动作重定向与扩散模型应用。
推荐理由: 从人类视频抽物体轨迹做通用操作模仿,缺少人形全身控制、重定向或腿足运动。
原摘要

Imitation learning enables robots to perform complex tasks by leveraging human-collected demonstrations. However, most existing approaches require labor-intensive data collection with additional teleoperation systems, which hinders scalability. Human videos represent a natural source of data that implicitly contain knowledge of manipulation behaviors. However, transferring them to robots remains challenging due to the visual domain gap and the absence of explicit action labels. In this work, we introduce TDV, a trajectory-based framework that leverages data generation to enable generalizable manipulation. TDV focuses on task-relevant objects by leveraging plug-and-play 6D pose estimation techniques to extract object trajectories from human videos, thereby mitigating the impact of irrelevant backgrounds and varying viewpoints. As a modality-agnostic intermediate representation, trajectories are inherently independent of the visual domains and different embodiments, enabling efficient data generation. Our method outperforms previous approaches on RLBench simulation tasks, achieving near 100\% success rates across seven tasks. In real-world experiments, TDV enables cross-embodiment generalization manipulation from one-shot human demonstrations by leveraging efficient trajectory generation techniques.

Chen Cai, Steven Liu
总结: 物理信息可微框架预测多臂机器人间碰撞距离。
方法: 解析FK加可学习嵌入与残差网,边界挖掘与非对称惩罚。
证据: 集成非线性MPC作间隙项,验证近距多臂操作安全。
为什么适合我: 接触丰富碰撞建模,利于全身MPC与复杂场景运动。
推荐理由: 面向多臂协作的可微碰撞距离与 MPC,主体是工业机械臂而非人形或腿足全身运动。
原摘要

Close-proximity multi-arm manipulation requires collision models that are both geometrically accurate and differentiable enough for real-time optimization. Classical geometry checkers provide reliable distances but are difficult to use inside gradient-based model predictive control, while conservative proxy models can restrict tightly coupled motion. We present PI-UDF, a physics-informed unified differentiable framework for body-to-body collision distance prediction between articulated robots. PI-UDF combines analytical forward kinematics with learnable link-geometry embeddings and a shared residual network to predict pairwise inter-arm distances directly from robot configurations. To improve safety-critical fidelity, we combine quota-driven boundary mining with an asymmetric boundary-crossing penalty that emphasizes false-safe sign errors near the collision boundary. The learned distance field is integrated into nonlinear MPC as a differentiable inter-arm clearance term. We validate the framework on a real dual-Franka platform through high-speed close-proximity 14-DoF dual-arm swapping, sustained single-arm dynamic evasion, and dynamic-evasion planning configurations with frozen, predicted, and target-switching treatments of the moving arm. Hardware experiments and offline Drake/FCL replay show that PI-UDF provides a differentiable inter-arm clearance estimate suitable for closed-loop collision-aware collaborative robot motion generation.

KwangBin Lee
总结: 用专家隐藏状态几何在测试时引导提升模仿策略鲁棒。
方法: 学可逆映射衰减低方差坐标,再映射回隐藏状态空间。
证据: 四模拟任务中提升闭环鲁棒,无需额外数据或重训。
为什么适合我: 增强模仿学习鲁棒,利于人体动作跟踪与真机部署。
推荐理由: 通用模仿学习隐状态鲁棒性方法,不针对人形全身跟踪、重定向或真机运动控制。
原摘要

Small observation or action errors can compound during closed-loop execution and disrupt a trained imitation learning policy. We study whether the original expert demonstrations can improve robustness through test-time hidden-state steering without additional demonstrations or policy retraining. Our analysis of a trained policy finds that expert hidden-state variation is concentrated in relatively few dimensions, while the directions capturing this variation differ across regions. Motivated by this geometry, we propose Diffeomorphic Hidden-State Steering (DHS), which learns an invertible nonlinear coordinate map from expert hidden states obtained by replaying the demonstrations through the frozen policy. DHS attenuates coordinates with low expert variance more strongly than those with high variance, then maps the modified coordinates back to hidden-state space. This allows steering directions to vary with the current hidden state while retaining every coordinate. Across four simulated manipulation tasks, DHS improves mean success under observation noise over the unmodified Transformer policies. Controlled comparisons yield higher mean success with learned nonlinear coordinates than with linear coordinates under both hard projection and full-rank rescaling. Combining DHS with action smoothing improves mean success rate under observation noise in certain tasks compared with action smoothing alone.

Yanchen Ding, Jiayi Chen, Jianfeng Xu, ... , Li Yi, He Wang
总结: GPU加速可微准动态仿真合成接触丰富操作轨迹。
方法: 指定接触软成本引导优化,动力学决定几何摩擦无模式。
证据: Allegro基准17408试验成功率79.8%,时间减27至50倍。
为什么适合我: 接触丰富轨迹与MPC,利于人形腿足接触运动智能。
推荐理由: Allegro手内旋转的可微准动态仿真与MPC,属于灵巧手操作而非全身腿足/人形控制。
原摘要

Efficient synthesis of contact-rich manipulation trajectories is essential for downstream robot learning. Gradient-based planning offers a promising approach, but jointly optimizing motion and contact decisions remains challenging. Meanwhile, many existing planners run on CPUs with limited throughput. We present CQDC-Warp, a GPU-accelerated differentiable quasi-dynamic simulator for batched model predictive control (MPC). We use it to study contact-guided trajectory optimization, in which intended robot--object contacts are specified before planning and encouraged through soft costs, while the simulated dynamics determine contact geometry and frictional response without a prescribed contact-mode schedule. Across $17{,}408$ trials on the $17$-object Allegro in-hand rotation benchmark, our framework achieves $79.8\%$ success, compared with $73.7\%$ for CFree and $65.2\%$ for IMPACT, and reduces mean trajectory-generation time by factors of $27.0$ and $50.2$, respectively. On a single NVIDIA RTX 4090 with $2{,}048$ parallel worlds, forward and backward throughput reach $2.06\times10^5$ and $2.25\times10^5$ world-steps per second, respectively. Further experiments cover manipulation with LEAP and Shadow hands, environmental contact exploitation, bimanual manipulation, whole-body control, and real-world trajectory replay on a Sharpa hand.

Peijun Xu, Chuansen Nie, Yiyang He, ... , Jingyi Yu, Jiayuan Gu
总结: 30个真实居住家居数字孪生基准评估空间具身智能。
方法: 人机循环识别重建生成放置,人工对照源观测校正。
证据: 评估检测重建导航与loco-manipulation,揭示当前局限。
为什么适合我: 直接支持loco-manipulation与复杂家居感知运动研究。
推荐理由: 居家数字孪生与空间具身基准,侧重场景重建和评测,不是全身运动控制或动作重定向。
原摘要

Realistic household simulation must capture not only diverse environments but also the lived-in object arrangements and spatial constraints that shape robot motion and interaction. Existing resources often trade off scale, real-world correspondence, and interaction readiness, leaving a gap in faithful, interactive replicas of how real homes are actually arranged. To this end, we introduce LIVIN, a benchmark for spatial and embodied intelligence built on digital twins of 30 diverse lived-in homes. These replicas preserve observed room layouts, furniture configurations, and everyday belongings. To construct them, we design a human-in-the-loop workflow comprising instance recognition, architectural reconstruction, and object generation and placement, with intermediate results reviewed and corrected by humans against the source observations at each stage. We evaluate four tasks in LIVIN: 3D detection, 3D reconstruction, navigation, and loco-manipulation. Our evaluations show that current methods remain challenged by the dense object arrangements, occlusions, limited free space, and constrained interaction regions found in realistic lived-in homes. We hope LIVIN will help advance embodied AI in real-world homes, from spatial understanding to robotic interaction, and ultimately bring embodied intelligence into everyday home environments.

Value as Memory: Online Value-Guided Post-training for General Robot Policies

2.0/5 偏低 裁判分 3.0 Candidate 当日相对
Wenye Yu, Yijin Chen, Kaiyi Xiong, ... , Cewu Lu, Chuan Wen
总结: 价值缓存与在线RL实现通用策略真实世界后训练。
方法: 价值作进度记忆,结合演示干预滚动高效在线适应。
证据: 接触丰富任务价值缓存增益29.4%,在线学习增益32.8%。
为什么适合我: RL后训练接触操作,利于全身策略真机适应与泛化。
推荐理由: 通用机器人策略的在线价值后训练,场景是可变形与接触操作,不是人形全身运动智能。
原摘要

Real-world post-training of generalist robot policies faces two challenges: partial observability makes action selection ambiguous, while offline post-training methods delay feedback from deployment experience. We present Value as Memory (VaM), a framework that addresses these challenges through value caching and online real-world reinforcement learning (RL). Value caching conditions actions on recent value predictions as task-progress memory, while online updates enable timely policy improvement by combining offline demonstrations, off-policy interventions, and on-policy rollouts for sample-efficient adaptation. As a result, VaM transitions from offline RL warmup to online autonomous learning with sparse rewards. Across long-horizon tasks involving deformable objects and contact-rich manipulation, VaM yields a mean absolute success-rate gain of $29.4\%$ with value caching, while achieving a $32.8\%$ gain with online autonomous learning over matched offline training. The evaluated online schedule performs $8\times$ as many policy iterations under the same gradient-step budget, with more than $2\times$ lower recorded time per gradient step with concurrent data collection and training.