Papers for 2026-10-25

10 papers

OIL: An Object Interaction Layer for Humanoid Whole-Body Control

5.0/5 很相关 裁判分 8.0 Strong 当日相对
Pranay Dugar, Mohitvishnu S. Gadde, Rajesh Mangannavar, Aayam Kumar Shrestha, Alan Fern
总结: 为人形全身控制扩展物体交互层,支持推拉等非抓取操作。
方法: 增强运动学接口加入物体运动与接触,转化为协调全身行为。
证据: 单一策略可执行推拉、双臂拾放及移动交互。
为什么适合我: 直接赋能人形loco-manipulation与接触丰富全身控制。
推荐理由: 为人形全身控制器增加物体运动与接触指令层,使同一策略完成自由空间运动和物体交互。
原摘要

Recent progress in learned whole-body controllers for humanoid robots has enabled robust free-space motion by tracking kinematic targets such as joint or end-effector trajectories. This interface provides a useful abstraction for upstream systems, including motion capture, teleoperation, or generative models, which can specify desired motion without reasoning about forces or balance. However, this abstraction has largely been limited to robot motion, requiring task-specific controllers or retraining for object interaction. We introduce OIL, an Object Interaction Layer that extends the kinematic command interface of whole-body control to include desired object motion and contact. OIL augments a standard controller with object-centric commands and translates them into coordinated whole-body behavior while preserving locomotion and balance. This design enables a single learned policy to execute non-prehensile interactions, including pushing, pulling, two-arm pick-and-place, and locomotion-coupled transport, directly from object-level kinematic targets without task-specific retraining. In simulation, we demonstrate that object-level commands improve performance over robot-centric controllers while matching task-specific policies across variations in object properties such as mass and size. We further demonstrate zero-shot generalization to novel behavior combinations and sim-to-real transfer on the Digit humanoid robot.

FlowHMR: Physically Plausible Motion Capture from Video

4.0/5 相关 裁判分 5.0 Candidate 当日相对
Zhanke Wang, Chengfeng Zhao, Qing Shuai, ... , Chunchao Guo, Linchao Bao
总结: 从单目视频恢复物理合理的全局三维人体运动。
方法: 视频条件流匹配生成候选,再用GRPO以保真奖励后训练。
证据: 生成多样运动并优化,确保视频一致且可物理跟踪。
为什么适合我: 产出可跟踪人体动作,便于重定向至机器人全身控制。
推荐理由: 从视频恢复物理可行的全局人体动作,直接服务后续可跟踪、可上物理角色或真机的动作来源。
原摘要

We present FlowHMR, a framework for recovering physically plausible global 3D human motion from monocular video. Previous learning-based methods typically regress human motion directly from video and train the network with geometric supervision. However, recovering human motion from monocular video is inherently ambiguous in depth, and direct regression tends to collapse toward an averaged solution. Moreover, the recovered motions are not guaranteed to be physically plausible, so physics-based tracking of them often fails. To address these challenges, we formulate video motion capture as a video-conditioned motion generation problem and first pretrain a flow matching model for this task. Given an input video, the pretrained model generates diverse motion candidates, but not all of them are faithful to the video or physically trackable. We therefore post-train the model using Group Relative Policy Optimization (GRPO) with two rewards. A fidelity reward encourages consistency with the input video. A tracking reward favors motions that a physics-based controller can track successfully. Together, these rewards shift the model's output preference, so the post-trained model stays faithful to the input video while producing more physically plausible motion. We further introduce Wild-4K, a large and diverse dataset of about 4K internet videos, for evaluating human motion recovery in the wild. Qualitative and quantitative experiments on Wild-4K show that our method outperforms state-of-the-art methods in overall motion fidelity and achieves a physical tracking success rate of 82.47%, compared with 62.82% for the strongest baseline, GVHMR.

Load-Aware Quadruped Navigation with Online Contact-based Geometry Estimation

4.0/5 相关 裁判分 4.0 Candidate 当日相对
Qi Yin, XiaoLi Zhang, Yangming Shi
总结: 四足负载感知导航,在线从接触估计物体几何。
方法: RL训负载感知导航策略,IL训接触几何预测器。
证据: 杂乱环境不同负载下展现自适应轨迹与间隙行为。
为什么适合我: 腿足复杂地形接触丰富导航,契合loco-manipulation。
推荐理由: 最接近腿足感知运动:四足在携带物体时做负载感知导航,并用接触在线估计几何,但是导航与足迹而非人形全身跟踪或跑酷。
原摘要

Navigating cluttered environments while carrying objects remains a challenging problem for quadruped robots. Compared with works that focused on load-aware locomotion adaptation to maintain gait stability, existing autonomous navigation systems generally ignore the influence of carried objects' geometries and use fixed footprint inflation, leading to overly conservative navigation. To fill this knowledge gap, we propose a two-phase load-aware navigation learning framework. In Phase 1, we train LOADNav, a load-aware navigation policy via reinforcement learning, enabling the robot to adapt its trajectory and clearance behavior according to the geometric size of the carried object. In Phase 2, we train CoGE, an online geometry predictor via imitation learning, which estimates unknown payload geometry from occasional contact events. We evaluate the framework in cluttered environments with loads of varying sizes and shapes. The results show that our navigation policy exhibits clear load-aware behaviors, and the online geometry predictor estimates object geometry from occasional contact events, refining its estimate and adapting the navigation behavior accordingly. Without requiring any prior knowledge of the payload, it narrows the gap to geometry-aware policies.

Shutong Ding, Zhaoyang Liu, Yimiao Zhou, ... , Jiayuan Gu, Ye Shi
总结: 统一扩散策略框架,用无分类器引导分离探索利用。
方法: 探索与利用分两生成策略,采样时组合分数或速度场。
证据: 提升任务性能同时保留行为多样性与持续适应力。
为什么适合我: 扩散与RL结合,利于全身策略探索与多模态控制。
推荐理由: 方法上接近扩散策略与无分类器引导采样,但场景是通用操作、VLA与在线强化学习后训练,不是人形全身跟踪或腿足运动。
原摘要

Diffusion policies are increasingly used in robotic control and manipulation, vision-language-action models, and world-model-based agents. Their ability to represent multimodal action distributions offers rich opportunities for exploration in reinforcement learning (RL). However, optimizing diffusion policies with RL can reduce the behavioral diversity and further limit their exploration capability. In RL post-training, directly fine-tuning a pretrained policy may also disrupt its acquired capabilities, while action-space residuals may lack the expressive power for multimodal exploration. Together, these issues reveal a common challenge across online and offline-to-online diffusion RL: improving task performance while retaining the behavioral diversity and capabilities needed for continued adaptation. We introduce CFG-Policy, a unified, plug-and-play framework that separates exploration and exploitation into two generative policies and combines their score or velocity fields during sampling through classifier-free guidance. For online RL, a frozen exploration branch sustains behavioral diversity while an exploitation branch learns from task rewards; for offline-to-online RL, a pretrained exploitation branch remains frozen while a trainable branch steers the composed behavior. CFG-Policy preserves the underlying RL objectives and improves performance with QSM and QVPO on 5 MuJoCo tasks and with Flow-SDE+PPO on 4 RoboTwin manipulation tasks. Project page is at https://zhy-liu.github.io/cfg-policy-page.

World2Motion: Turning Video World Models into 3D Human Motion Generators

3.0/5 一般 裁判分 8.0 Strong 当日相对
Fangyuan Tu, Xiangyue Zhang, Yiyi Cai, ... , Haoran Xie, Haiyang Liu
总结: 将视频世界模型转为场景感知三维人体运动生成器。
方法: 构建合成真实视频运动对,提出移位解耦噪声调度。
证据: 单阶段从图像文本生成场景感知运动与对应视频。
为什么适合我: 场景感知人体动作生成,支持复杂环境重定向部署。
推荐理由: 把视频世界模型改成场景感知的3D人体动作生成,仍是动作先验上游,不是全身控制或loco-manipulation。
原摘要

We present World2Motion, a framework that generates scene-aware 3D human motion and corresponding video from a single image and a text prompt. While existing 3D motion generators learn from motion datasets, their generalization is constrained by limited coverage of environments. In contrast, video world models such as Cosmos 3 offer broader environmental priors but are not designed for full-body motion generation; recovering motion from their generated videos requires costly two-stage inference. To address these, we turn Cosmos 3 into a single-stage 3D motion generator. This adaptation has two challenges: the scarcity of paired video--motion data and temporal instability in the generated motion. First, we construct a training dataset combining synthetic video--motion pairs with real videos paired with estimated 3D motion. Second, we propose a shift-decoupled noise schedule that assigns different noise levels to video and motion through shared denoising progress. This design accommodates the different denoising requirements of the two modalities, reducing motion jitter. Experiments on a multi-source interaction benchmark show that World2Motion has better motion--text alignment and scene interaction compared with the evaluated 3D motion generators. It also matches the interaction success rate of the two-stage baseline while achieving approximately 3.3$\times$ faster inference. Our project page is available at https://fyantu.github.io/World2Motion/.

DexCast: Learning Real-World Dexterous Control from Human Videos

3.0/5 一般 裁判分 7.0 Strong 当日相对
Zihan Gao, Yuhong Zhang, Mengzhe Wang, ... , Zhifang Liu, Haoqian Wang
总结: 从人体视频直接学习真实灵巧手控制,无需遥操作。
方法: 利用HOI重建,DexHand模块捕获协调并条件扩散策略。
证据: 真实实验在多样灵巧操作任务中取得优异表现。
为什么适合我: 人体视频模仿接触操作,可扩展至全身loco-manipulation。
推荐理由: 从人体视频模仿灵巧手,接近人体动作模仿,但是手部操作而非全身重定向或loco-manipulation。
原摘要

Recent advances in 3D hand-object interaction (HOI) perception have opened new opportunities for learning dexterous robotic skills from human videos. However, existing dexterous imitation learning methods often rely on costly and manually teleoperated robot demonstrations, requiring substantial human effort. Therefore, we build DEXCAST, a framework that learns dexterous robotic hand control directly from monocular RGB-D videos by leveraging joint HOI reconstructions, without requiring any teleoperation. Furthermore, how to model high-dimensional robotic hand motion and extract temporal context to inform future action generation remains a key challenge. To this end, we propose a DexHand Motion Perception Module (DMPM) to capture inter-joint coordination and temporal dependencies in compact latent representations that condition our diffusion policy. Extensive real-world experiments demonstrate that DEXCAST achieves exceptional performance across diverse dexterous manipulation tasks.

Qing Yu, Kent Fujiwara
总结: 在直接运动空间做表示感知流匹配生成人体运动。
方法: 无编解码直接预测干净运动,用噪声缩放与RA-MMDiT。
证据: 提升文本驱动生成质量,支持帧关节直接操控。
为什么适合我: 高质量可操作人体运动,便于跟踪重定向真机控制。
推荐理由: 直接运动空间的人体动作生成,接近运动先验表示,但不是物理角色控制、跟踪或上真机。
原摘要

Recent advances in diffusion and flow models have substantially improved text-driven human motion generation. Yet most methods generate in low-dimensional, temporally downsampled latent spaces learned primarily for reconstruction, a bottleneck that can limit generation quality and preclude direct manipulation of individual frames and joints. We introduce MotionSpaceFlow (MSFlow), a representation-aware flow-matching framework that predicts clean motion directly in continuous motion space without a learned encoder or decoder. To account for the anisotropic structure of direct motion representations, we propose representation-aware noise scaling and show how the initial Gaussian source scale governs the covariance of intermediate probability-path marginals. We further introduce a Representation-Aware Multimodal Diffusion Transformer (RA-MMDiT), which jointly updates token-level language and full-resolution motion features through joint attention while adapting temporal information flow to the motion representation: causal attention for incremental features defined by frame-to-frame changes, and bidirectional attention for global features such as absolute joint coordinates. Across different datasets and motion representations, MSFlow achieves state-of-the-art text-to-motion performance. Its global representation variant additionally enables zero-shot, inference-time control over any joint or frame through projection sampling without control-conditioned training, delivering leading motion quality with exact constraint satisfaction.

Scaling Latent Motor Adaptation to Coordinated Bimanual Manipulation

3.0/5 一般 裁判分 4.0 Candidate 当日相对
Seung-Bin Joo, Alessandro Albini, Luca Wiehe, ... , Daniel Olmeda Reino, João F. Henriques
总结: 将潜在运动适应扩展至协调双臂接触丰富操作。
方法: 双臂RMA用观测动作历史与双腕深度条件潜在适应。
证据: 接触任务中保留特权性能,优于无潜在条件PPO。
为什么适合我: 接触丰富双臂适应,增强loco-manipulation鲁棒性。
推荐理由: 接触丰富双臂的潜在运动适应,接近接触交互,但不是全身loco-manipulation或腿足运动。
原摘要

Robust manipulation under uncertain object dynamics and geometry remains a core challenge in real-world robotics. Policies trained under fixed assumptions often degrade when object mass, friction, scale, or shape vary. Embedding-based adaptation methods such as Rapid Motor Adaptation (RMA) address this by conditioning actions on latent representations of environment dynamics inferred from interaction history. We investigate whether this paradigm scales from single arm settings to coordinated bimanual manipulation, where higher dimensional control, coupled contacts, and shared object forces introduce additional complexity. We evaluate bimanual RMA on three contact-rich ManiSkill tasks with randomized YCB object properties. Bimanual RMA, using observation-action history and dual wrist depth images, retains privileged policy performance on bimanual picking and lifting, while an otherwise matched stationary PPO policy without latent conditioning achieves low success; all methods remain limited on long-horizon stacking. We further demonstrate direct sim-to-real transfer on a dual Franka lifting task without real-world finetuning. Together, these results suggest that latent conditioning provides useful structure for scaling robust control to coordinated bimanual systems, while longer horizon tasks require additional temporal structure.

Disentangling Task-Relevant Latent Dynamics for Reward Modeling

2.0/5 偏低 裁判分 8.0 Strong 当日相对
Dayou Li, Jionghao Wang, Peihao Wang, ... , Wenping Wang, Zhiwen Fan
总结: 解耦任务相关潜在动态,从视频学习密集奖励。
方法: 非对称两阶段从多帧潜在学习连续任务相关表示。
证据: 抑制无关视觉变化,在相关空间建模有效密集奖励。
为什么适合我: 视频驱动RL奖励,支持人体动作模仿的全身训练。
推荐理由: 从视频学习操作任务奖励,只与强化学习奖励设计弱相关,不涉及腿足感知运动或全身控制。
原摘要

Reward design remains a major bottleneck for reinforcement learning in robotic manipulation. Handcrafted dense rewards can compensate for the limited guidance provided by sparse rewards, but require substantial task-specific reward engineering. Learning rewards from videos provides a scalable, data-driven alternative to manual reward design. However, existing video-based rewards are often computed in entangled visual spaces where task-relevant state changes are confounded with extraneous visual variation. As a result, the learned reward may respond to visual distractors instead of actual task execution. To address this limitation, we propose DiTaR (Disentangling Task-Relevant Latent Dynamics for Reward Modeling), a framework for learning dense rewards from heterogeneous videos. DiTaR learns a continuous task-relevant representation from multi-frame video latents through an asymmetric two-stage procedure that suppresses task-irrelevant visual variation. In the task-relevant latent space, we train a history- and language-conditioned rectified-flow model to capture task dynamics and derive a directional dense reward from temporal changes in dynamics compatibility. Together, these components enable reward modeling directly over task-relevant dynamics rather than generic visual variation. DiTaR substantially improves policy learning on MetaWorld-v3 and ManiSkill over video-based reward baselines, and transfers to real-world offline RL through reward relabeling. Reward analyses further show that DiTaR provides more reliable rewards.

Chunyu Luo, Hongyu Zhao, Guangyi Ni, Yue Song, Lingkai Kong
总结: 学习一步生成策略用于在线最大熵强化学习。
方法: 基于Drifting顺序学漂移与扩散更新,无需密度估计。
证据: MuJoCo上获最高聚合归一化回报,收敛至玻尔兹曼目标。
为什么适合我: 高效生成策略在线RL,适合全身运动智能快速适应。
推荐理由: 单步生成式策略的在线最大熵强化学习算法,缺少人形、腿足或接触丰富全身运动对象。
原摘要

Generative policies offer expressive action distributions for online reinforcement learning (RL), but many diffusion- and flow-based policies require costly iterative sampling. We propose DriftSpread, a one-step generative policy based on Drifting for online maximum-entropy (MaxEnt) RL. To incorporate entropy without a tractable policy density, we build on an energy-based view of Drifting and derive critic-guided Drift and entropy-induced Spread from the negative MaxEnt objective. Our method learns these updates sequentially in the same actor, using action regression for value improvement and distribution matching to capture the effect of entropy. The actor updates require neither direct samples from the critic-defined Boltzmann target nor policy-density or score estimation. Under exact fitting with a fixed critic, we prove convergence to the Boltzmann target in the small-step limit followed by the long-time limit. On MuJoCo, DriftSpread achieves the highest aggregate normalized return among all evaluated methods, while outperforming the diffusion- and flow-based baselines in both training and inference efficiency.