Papers for 2026-10-24

10 papers

Motion Retargeting with Spatiotemporal Preferences for Humanoid Loco-Manipulation

5.0/5 很相关 裁判分 10.0 Top pick 当日相对
Xiongbo Shi, Shihang Du, Zongyou Yu, ... , Junqiao Zhao, Guang Chen
总结: 时空偏好重定向提升人形loco-manipulation接触保真与效率。
方法: 视觉语言模型选关键信号部位,关键帧分配更多迭代与交互权重,加力学接触锚定。
证据: 针对物体交互准确保留关键阶段身体物体关系,优化计算效率。
为什么适合我: 实现人体演示到人形全身控制轨迹,支持loco-manipulation重定向。
推荐理由: 直接做人形动作重定向,并把关键帧与身体部位偏好用于全身控制与loco-manipulation,紧贴人体动作重定向与全身交互。
原摘要

Human motion retargeting bridges the embodiment gap between humans and humanoid robots by converting human demonstrations into reference trajectories for whole-body control and loco-manipulation. For object-interaction tasks, successful execution depends on accurately preserving body--object relations at critical stages. However, uniform per-frame iteration budgets do not account for these localized accuracy requirements, limiting the computational efficiency of retargeting. To address this limitation, we present Spatiotemporal Retargeting (STR), which uses keyframes and key body parts to allocate optimization effort. Specifically, a vision-language model selects relevant motion signals and body parts, and keyframes are identified from changes in the selected signals. Based on these selections, STR assigns additional solver iterations to keyframes and higher interaction weights to the selected parts. To further improve contact fidelity, we introduce Mechanics-Aware Contact Anchoring (MCA) to refine object-surface contact positions using an approximate object-wrench model while retaining the selected surface directions. Building on the resulting references, we develop Spatiotemporal Reinforcement Learning (STRL), which combines adaptive key-stage sampling with a contact reward based on target proximity and object contact forces for downstream control. Simulation experiments on a Unitree G1 and real-world experiments on a Unitree R1 demonstrate that STR improves geometric contact coverage and continuity at lower computational cost, while the resulting references improve downstream task completion across diverse object-interaction motions.

Dewei Wang, Jinrui Han, Jiyuan Shi, ... , Chenjia Bai, Xuelong Li
总结: 统一测试床覆盖人形loco-manipulation数据策略控制器与真机部署。
方法: 任务套件含多原子组合任务,共享控制器评估多策略与数据混合。
证据: 未见任务场景成功率下降,视觉扰动增加时鲁棒性差异显现。
为什么适合我: 提供基准评估复杂场景全身运动智能与loco-manipulation策略。
推荐理由: 统一评测人形loco-manipulation的数据、策略、控制器与真机部署,覆盖遥操作数据采集和全身场景交互。
原摘要

Humanoid robots must coordinate locomotion and manipulation in everyday environments, yet progress is difficult to assess across differing policies, controllers, and task settings. LocoManiLab provides an integrated testbed spanning data collection and generation, policy training, evaluation, and physical deployment. Its task suite covers 234 atomic and compositional tasks across five household scene categories, supported by 51.5 hours of simulated teleoperation. The evaluation covers twelve policies with a shared controller across task and scene splits and perturbation levels, two policies across five controllers, and studies of demonstration scale and data-source mixtures. Success rates fall on unseen tasks and scenes and generally decline as visual perturbations increase, with differences in generalization and robustness across policies. For data generation, a humanoid motion generation model is fine-tuned for loco-manipulation to study the alignment of its outputs with task execution requirements and its integration into the data generation pipeline. Three simulated loco-manipulation tasks closely match real-world setups, enabling evaluation of three policies in both domains. Similar policy rankings support the reliability of simulation-based evaluation.

Humanoid Mobility via Attention-based Contacts

5.0/5 很相关 裁判分 9.0 Top pick 当日相对
Junzhe He, Kehan Wen, Chong Zhang, ... , Vladlen Koltun, Marco Hutter
总结: 注意力接触规划桥接强化学习实现人形复杂地形稳健移动。
方法: 自监督注意力预测未来接触,接触条件跟踪器生成协调全身运动。
证据: 接触作感知控制接口,内化接触感知结构适应训练外轨迹。
为什么适合我: 强化复杂地形感知运动与跑酷全身协调,契合核心兴趣。
推荐理由: 用接触规划连接无参考RL与运动先验跟踪,面向复杂地形上的人形全身机动,属于腿足感知运动核心。
原摘要

Humanoid mobility over complex terrain requires accurate foot placement and coordinated whole-body motion capable of robust adaptation. Reinforcement learning (RL) policies trained without motion references generalize well across terrains, yet eliciting effective whole-body coordination through reward design and exploration alone remains challenging. In contrast, reference-based methods yield natural motions but struggle to adapt beyond the predefined trajectories in their training data. We introduce a learning framework that bridges these two approaches by using motion data strictly as a training prior. An explicit, self-supervised, attention-based contact planner predicts future body-environment contacts, while a contact-conditioned tracker translates these plans into coordinated whole-body motion. Together, they establish contact as an interface between sensory observations and control actions. This abstraction allows the policy to internalize the contact-aware structure of locomotion rather than directly replicating kinematics. Consequently, a single policy can exploit data-driven motion priors for natural whole-body coordination while retaining the flexibility to discover environment-dependent behaviors through reinforcement learning. Building on the learned contact planner and tracker, a subsequent fine-tuning stage bootstraps their acquired capabilities to explore environment variations beyond those covered by the original motion data. Operating entirely from onboard observations without any reference inputs at deployment, this framework enables versatile humanoid mobility in challenging environments.

Memory before Goal: Role of Pre-History Memory for Humanoid Loco-Manipulation

5.0/5 很相关 裁判分 9.0 Top pick 当日相对
Chenyuan Chen, Ran Ding, Lechen Zhang, Zewen He, Yoshihiko Nakamura
总结: 目标前物体引导记忆提升人形动态场景loco-manipulation。
方法: SAM3跟踪物体因果选关键帧,动作专家交叉注意读外观几何时间令牌。
证据: 保留正确帧提升基策略仿真与真机成功率,优于无记忆基线。
为什么适合我: 支持接触丰富动态场景感知运动与全身视觉语言动作策略。
推荐理由: 研究动态场景中人形loco-manipulation的物体历史记忆与全身VLA,并在G1仿真和真机任务上验证。
原摘要

Humanoid loco-manipulation in dynamic scenes is non-Markovian: cues vanish as objects change or are hidden, or leave the view as the robot turns and walks, yet whole-body vision-language-action (VLA) policies see only the present. We present WholeBodyMem, an object-guided perceptual memory that keeps the pre-history of object events seen before the robot acts on its goal. SAM3 tracks the task objects, and their events causally select key frames within an adaptive budget, which the action expert reads by cross-attention together with object tokens encoding appearance, geometry and time. We also contribute HuMeGo, a humanoid memory benchmark with six simulated and six real-world Unitree G1 tasks. Keeping the right frames rather than more frames, WholeBodyMem surpasses memoryless and uniform-memory baselines, raising its base policy's success from 29.1% to 72.6% in simulation and from 36.7% to 88.3% on the real robot, while also improving the standard SIMPLE tasks (70.2% to 86.9%).

Robot Motion Transition with Human Motion Prior

5.0/5 很相关 裁判分 8.0 Strong 当日相对
Taehyun Byun, Taemoon Jeong, Yoonbyung Chai, ... , Kyungjae Lee, Sungjoon Choi
总结: 人类运动先验时空对齐连接独立人形运动实现平滑过渡。
方法: 机器人到人到机器人过程,学习对齐模型确定位置朝向时机边界。
证据: 人类空间利用丰富先验跨实施例重用过渡知识避免意外动作。
为什么适合我: 连接人体动作序列到可跟踪全身控制,利于模仿与对抗先验。
推荐理由: 用人体运动先验对齐并衔接独立的人形跟踪动作,直接对应运动跟踪、动作过渡与跨本体重定向。
原摘要

Humanoid motion-tracking policies can reproduce diverse behaviors, yet connecting independently acquired motions into a continuous sequence remains challenging. Interpolation connects boundary poses without considering whether their relative position, heading, and timing provide suitable conditions for a transition. A generative motion prior can produce richer motion, but poorly arranged boundaries may cause it to generate excessive locomotion, turning, or another unintended action. We present Spatio-Temporal Alignment with a Motion Prior (STAMP), a framework for connecting independently acquired humanoid motions through a Robot-to-Human-to-Robot process. STAMP uses human motion space to draw on rich motion priors and reuse transition knowledge across robot embodiments. Its key component is a learned alignment model that determines the boundary conditions for motion inbetweening. Given the two motions retargeted to a human rig, the alignment model ranks candidate positions, headings, and transition durations for the following motion. A human motion prior generates the transition under the selected alignment. The completed sequence is retargeted to the robot and used as a reference to train a motion-tracking policy. We evaluate the tracking success of the generated transitions on Unitree~G1 and~ROBOTIS K1. Physical robot demonstrations further show that STAMP connects independently acquired motions into long-horizon sequences. Project page: https://childtoy.github.io/STAMP-project

Zejian Kang, Yifu Luo, Boyang Shen, ... , Wentao Yang, Xiangru Huang
总结: 类型化参考感知对话编辑框架生成修订人形扩散运动。
方法: 语言模型预测支撑根手事件,扩散生成结合文本编码,参考条件编辑。
证据: 重力参考状态与轨迹接触目标改善运动学一致性与可修订性。
为什么适合我: 扩散模型将人体动作变为可跟踪可重定向可编辑全身控制。
推荐理由: 面向Unitree G1的语言条件运动扩散生成与编辑,并约束支撑、轨迹和接触,接近扩散模型与人形全身运动生成。
原摘要

Language-conditioned humanoid motion generation enables users to create and refine robot movements through natural language. The central challenge is to connect an underspecified request to concrete motion structure that can subsequently be revised relative to an accepted reference. We present TRACE, a generation and editing framework for the Unitree G1 humanoid robot built around typed, measurable kinematic quantities. Our insight is to ground generation conditions in paired training motions and editing goals in the current motion. A language model learns to predict support, root, and hand events from motion-derived supervision; their time-aligned encodings complement the original text in a diffusion generator. Gravity-referenced states and trajectory--contact objectives improve kinematic consistency. For editing, typed requests define body--time scopes and source-relative goals. A separate reference-conditioned diffusion editor uses these goals and revised events to realize changes, while sampling-time projection preserves protected state coordinates. Quantitative evaluation and human assessment show improved semantic alignment and motion quality over the compared methods. Multi-turn editing evaluations demonstrate more reliable execution of successive relative requests under shared preservation constraints. Code is available at https://github.com/Multimodal-Perception-Action/TRACE.

Van Chung Nguyen, Hung Manh La
总结: 多势基奖励塑形统一人形运动奖励配置跨机器人通用。
方法: 势函数差塑形引导探索不改最优策略,统一结构仅需元数据。
证据: 九机器人上一致缩放行为降低调参敏感性优于标准塑形。
为什么适合我: 简化强化学习奖励设计助力人形全身运动策略跨平台训练。
推荐理由: 针对多款人形的统一RL奖励塑形与运动学习,属于人形全身运动控制,但偏奖励设计而非感知或操作。
原摘要

Designing and tuning reward functions remains one of the main challenges in building effective reinforcement learning (RL) pipelines for humanoid locomotion. While shaping rewards can accelerate learning, naively formulated terms often conflict with the desired behavior, overfit to a specific robot, or become unstable under small weight changes. Potential-Based Reward Shaping (PBRS) offers an alternative solution: shaping the reward via differences of a potential function to guide exploration without changing the optimal policy. In this paper, we study Multi-PBRS for humanoid locomotion and propose a unified reward configuration. We then evaluate the same reward structure and weights across nine humanoid robots, requiring only robot-specific metadata (joint/link mappings and default poses) rather than per-robot reward redesign. Our experiments show that, compared to standard shaping, MPBRS provides more consistent scaling behavior and reduced tuning sensitivity across robots, making it easier to tune without altering the optimal policy.

Maximilian St\"olzle, David M\"uller, Agon Serifi, ... , Espen Knoop, Moritz B\"acher
总结: 推理时噪声优化强制运动扩散模型物理可行输出。
方法: 代理约束含运动学检查与逆动力学锥规划,进化策略优化噪声。
证据: 无需重训模型,自回归链接优化段合成长物理可行轨迹。
为什么适合我: 提升扩散生成人体动作物理性便于重定向与真机全身控制。
推荐理由: 在推理时为运动扩散模型施加物理可行性约束,服务仿真与控制下游,贴近物理角色控制与运动先验。
原摘要

Motion Diffusion Models (MDMs) can generate diverse and visually appealing human motion; yet, the resulting trajectories frequently violate physical constraints, limiting their downstream use in simulation and control. We present PhysibleMDM, a framework that promotes physical feasibility of MDM output by optimizing at inference time the diffusion noise without retraining the generative model. We introduce a set of surrogate feasibility constraints that can be evaluated without full trajectory simulation. These constraints combine kinematic checks (e.g., ground penetration, slip, and self-collision) with dynamic feasibility tests computed via an inverse-dynamics inner loop formulated as a cone program. We solve the resulting black-box noise optimization problem using the Separable Covariance Matrix Adaptation Evolution Strategy (Sep-CMA-ES), thereby avoiding reliance on computationally expensive and discontinuous gradients. An autoregressive scheme then chains optimized segments to synthesize long motion sequences. Across 100 motion styles on an SMPL humanoid, PhysibleMDM reduces mean ground penetration, self-collision penetration, and ground slip by 75%, 92%, and 74%, respectively, relative to unoptimized diffusion output. Across 11 sequences for a custom bipedal robot, it achieves median reductions of 89.5% in per-sequence mean acceleration-reconstruction error and 79.5% in the mean inverse-dynamics residual.

Niklas Berndt, Xianghui Xie, Enes Duran, Yixiao Wang, Gerard Pons-Moll
总结: 潜在手物交互先验实现物理基灵巧操作策略高效学习。
方法: 跟踪mocap蒸馏到目标条件VAE先验,潜空间有界残差探索。
证据: 单奖励无复杂课程生成自然接触保真多样灵巧操作。
为什么适合我: 结合模仿对抗先验扩展接触丰富loco-manipulation到灵巧手。
推荐理由: 聚焦灵巧手-物体交互先验与残差策略,只在接触操作上部分相关,缺少人形全身运动与loco-manipulation。
原摘要

Synthesizing physically grounded dexterous hand-object interactions is important for VR/AR, gaming, and robotics, yet remains challenging because subtle contact dynamics, diverse object geometries, and target trajectories must be jointly respected while maintaining natural grasps. Existing methods either track mocap trajectories with limited transfer to novel objects, or rely on hand-only generative priors that fail to capture coupled hand-object motion, sacrificing contact fidelity and sample efficiency. To tackle these challenges, we introduce PiHand, a Physically grounded Interaction-aware Hand motion synthesis framework. PiHand first trains expert policies that track mocap demonstrations in simulation, distills them into a unified hand-object interaction prior using a goal-conditioned VAE, and finally solve diverse dexterous manipulation tasks through bounded residual exploration in its latent space. With only a single reward and no complex curriculum training, this yields general tabletop manipulation policies that generalize to unseen objects and trajectories on GRAB and DexYCB. The same prior transfers to object catching and bimanual handover, two tasks that are rarely seen in its training data. Across all tasks, PiHand consistently outperforms baselines in all metrics. We will release the processed data and model checkpoints.

Naru Suzuki, Satoshi Ikehata, Takehiko Ohkawa, Yoichi Sato, Shuhei Kurita
总结: 自我中心视频直接重建度量世界空间手部相机运动。
方法: 单阶段联合预测共享世界坐标,几何特征加流匹配处理遮挡。
证据: 避免多阶段误差解决尺度模糊与遮挡多轨迹歧义。
为什么适合我: 从人体交互视频提取可转移动作支持模仿学习全身控制。
推荐理由: 可为人体动作数据采集提供手部世界坐标重建,但是视觉重建而非全身重定向或人形控制。
原摘要

Recovering metric world-space hand motion from egocentric video supports understanding human interactions and transferring them to embodied agents. Three challenges complicate this task: multi-stage pipelines propagate errors from independently estimated hand and camera motion; monocular observations leave physical scale ambiguous; and occlusion and out-of-view hands admit multiple plausible motion trajectories. We present DewHand, a unified model for direct joint reconstruction of world-space hand and camera motion. To avoid compounding errors from multi-stage estimation, DewHand jointly predicts hand and camera motion in a shared world coordinate system within a single-stage pipeline. To resolve metric-scale ambiguity, it combines scene-level geometric features from VGGT-$Ω$ with metric-space supervision, while hand-specific features preserve fine-grained articulation. To handle occlusion and out-of-view hands, we formulate reconstruction as conditional motion generation with flow matching. Under the same architecture, this generative formulation outperforms direct regression in both accuracy and temporal smoothness with only two sampling steps. Experiments on HOT3D and ARCTIC demonstrate improved world-space accuracy and temporal consistency over state-of-the-art methods. DewHand reduces rigid-aligned world-space error relative to StableHand from 32.9 to 28.4 mm on HOT3D and from 76.2 to 36.2 mm on ARCTIC. It also recovers camera scale within 10% on two thirds of HOT3D clips, versus about a quarter for SLAM. Moreover, in zero-shot evaluations on HoloAssist, H2O, and TACO, captured with different devices, DewHand achieves the lowest rigid-aligned world-space error (WA-MPJPE) on each dataset.