Papers for 2026-11-11

10 papers

Kimodo-Policy: Learning Kinematic Action Models for Humanoid Loco-Manipulation

5.0/5 很相关 裁判分 10.0 Top pick 当日相对
Yunheng Wang, Jiaxi Zhang, Wei Lu, ... , Zecui Zeng, Renjing Xu
总结: 以全身运动生成为预训练目标,再条件化为人形移动操作策略。
方法: 冻结文本运动骨干,用轻量视觉条件与根身手层级闭环控制。
证据: 摘要提出动作块预测与执行反馈,但未给出定量结果。
为什么适合我: 直接面向人形全身移动操作与运动先验控制。
推荐理由: 直接做人形全身loco-manipulation,以全身运动生成为动作骨干并结合视觉与执行反馈,紧贴人形全身控制与移动操作。
原摘要

Humanoid loco-manipulation requires continuous coordination of high-dimensional whole-body motion to realize task objectives. While vision-language-action models and world action models transfer semantic and visual-dynamics priors to robot control, these priors do not necessarily capture the structure of robot motion. We introduce Kinematic Action Models (KAMs), a paradigm that treats whole-body motion as a direct pretraining target and uses a pretrained motion generator as the primary action-generation backbone. KAMs ground learned temporal and cross-body coordination priors through task conditioning, visual observations, and execution feedback. We present Kimodo-Policy as an initial instantiation, combining a frozen Kimodo text-to-motion backbone with lightweight ControlNet-based visual conditioning. A root-body-hand hierarchy coordinates locomotion and manipulation, while motion-history conditioning and repeated action-chunk prediction bridge motion synthesis and closed-loop control. Following large-scale loco-manipulation pretraining, the policy achieves the highest overall success among compared policies on multi-task HumanoidArena (72.14%), competitive performance on SIMPLE, and 60% mean success across eight real-world tasks, with 678M parameters (92M trainable). Ablations support the contribution of the pretrained motion backbone. Together, these results provide initial evidence for motion generation as a complementary foundation for humanoid action learning.

Hesham Shehata (Research, Development Division, Arithmer Inc., Tokyo, Japan)
总结: 把单目舞蹈视频经重定向与仿真跟踪迁到商用人形。
方法: 人体网格恢复、运动学重定向、强化学习跟踪并机载部署。
证据: 真机完成三次43秒、46秒及9.7分钟无绳表演。
为什么适合我: 覆盖视频到机器人重定向、运动跟踪与仿真到现实。
推荐理由: 从人体视频经运动重定向和强化学习运动跟踪部署到商用人形,并处理仿真到现实,属于全身运动跟踪与动作重定向的核心工作。
原摘要

We present a pipeline for transferring Awa Odori, a traditional Japanese festival dance, from monocular video to a commercial Unitree G1 EDU humanoid. The pipeline combines human mesh recovery, kinematic retargeting, reinforcement learning for motion tracking in simulation, and onboard deployment via ONNX. Training ran on a single RTX 5070 GPU. An eleven-version deployment ladder evaluated on hardware examines three interventions. We detect and repair silent corruption of deployment artifacts, model state-estimator drift during training, and edit the reference motion to support stable execution and a controlled ending. The final policies achieved three consecutive rope-free completions of a 43-second routine, a 46-second performance with automatic handoff, and a continuous 9.7-minute rope-free performance composed from the captured motion. In a live handover, a single remote request let the robot start from and return to the manufacturer's walking controller on stage, with a measured torque-free window of 45-140 ms. We report the training dynamics and hardware outcomes and discuss failure boundaries involving compliant flooring and occlusion in monocular capture.

Kevin Park, Izat Rasyad, Dvij Kalaria, Shankar Sastry
总结: 用生成运动引导仿真强化学习,蒸馏为人形视觉语言动作策略。
方法: 参考条件DAgger在失败后重启专家并聚合修正轨迹。
证据: 评估平台取物、开门等任务,摘要未给具体成功率。
为什么适合我: 结合运动生成、仿真强化学习和人形全身交互数据扩展。
推荐理由: 用生成的人体运动引导仿真强化学习并蒸馏为人形视觉-语言-动作策略,覆盖运动生成、模仿和人形全身交互。
原摘要

Training vision-language-action (VLA) policies typically depends on substantial robot demonstration data, making human teleoperation a costly and cumbersome bottleneck, particularly for whole-body humanoid interaction. Reinforcement learning in simulation offers an alternative for scaling this data, but designing task-specific rewards remains challenging. To address this, we propose DreamVLA, which uses generated motion trajectories to guide reinforcement learning (RL) in simulation, converting them into reliable task execution trajectories. These trajectories train VLA models in simulation without new task-specific teleoperation data. Our central contribution is a reference-conditioned DAgger distillation procedure: after a failed learner rollout, we restart the RL expert from a sampled learner state with the same reference suffix and aggregate successful corrective trajectories, addressing the distribution shift left by offline imitation. We evaluate picking from a platform, opening a drawer, pressing a button, and walking and picking from a platform, with ablations of DAgger iterations, residual RL, and generated dataset scale. Project website: ember-center-berkeley.github.io/DreamVLA.

Mayur Deshmukh, Aaron Tung, Junyi Li, ... , Michael Cho, Jiafei Duan
总结: 发布带第一视角与光学动捕的大规模全身运动数据。
方法: 十四人两百余小时,同步六相机、腕部传感与全身动捕。
证据: 覆盖多样运动基元与动态全身行为,摘要无模型指标。
为什么适合我: 为人形下肢与落足的视觉运动学习提供配对示范。
推荐理由: 面向人形loco-manipulation的大规模全身动捕与第一视角数据,直接支撑运动重定向、模仿学习和感知型全身控制。
原摘要

Humanoid robots must coordinate locomotion and manipulation to navigate complex environments and perform everyday tasks. Learning such behaviours from human demonstrations requires accurate whole-body motion paired with egocentric vision, particularly for lower-body movements and foot placements that are frequently invisible from a first-person perspective. However, existing egocentric datasets often lack high-fidelity optical motion capture or systematic coverage of diverse loco-manipulation behaviours. To address this gap, we introduce RoboMoCap, a large-scale multimodal dataset combining egocentric vision with marker-based optical motion capture, specifically designed for locomotion-centric humanoid learning. RoboMoCap comprises over 200 hours of recordings from 14 participants, featuring six head-mounted cameras, wrist-mounted visual and inertial sensing, and optical full-body motion capture at 120 Hz. The dataset systematically covers diverse locomotion primitives, dynamic whole-body movements, and loco-manipulation tasks, including variable-speed walking, turning, constrained navigation, terrain traversal, object transportation, and manipulation during locomotion. It provides temporally aligned multimodal observations and metric whole-body motion representations, enabling the study of human movement beyond directly observable body regions. We systematically evaluate existing methods on three benchmark tasks: egocentric full-body motion reconstruction, foot contact and placement estimation, and human-to-humanoid motion retargeting. By pairing egocentric vision with high-fidelity human motion across diverse locomotion and interaction scenarios, RoboMoCap provides a foundation for egocentric motion understanding and humanoid learning from human demonstrations.

Ziwon Yoon, Seok Hwan Kim, Yiyu Chen, ... , Lu Gan, Ye Zhao
总结: 用视觉语言模型与具身残差规划粗糙地形交互导航。
方法: 混合框架以高斯过程修正语言模型的物体可移动性预测。
证据: 摘要强调联合推理地形可通过性与物体可移动性,无数值。
为什么适合我: 关乎人形户外移动与可交互障碍,而非全身动作模仿。
推荐理由: 研究人形在崎岖地形上的可交互导航与可通行性推理,接近感知型足式运动,但侧重语义规划而非高动态运动跟踪。
原摘要

Humanoid robots have achieved significant progress in locomotion and manipulation, enabling traversal over rough terrain and richer interaction with the environment, such as moving obstacles. However, most traditional navigation methods assume non-interactive environments and focus only on traversal. While Navigation Among Movable Obstacles (NAMO) approaches relax this assumption, prior efforts have largely focused on flat environments and reactive interaction, rather than predicting object movability and planning ahead. These limitations become critical in outdoor rough-terrain scenarios, where robots must jointly reason about terrain traversability and object movability. In this work, we address NAMO for humanoid robots in cluttered outdoor rough terrain by proposing a hybrid planning framework that combines semantic priors from a Vision-Language Model (VLM) with embodied robot interaction data through a Gaussian process (GP) residual model. The GP corrects the VLM-derived movability prior to produce embodiment-grounded probabilistic movability estimates, which are then incorporated into hierarchical uncertainty-aware planning. The proposed approach allows humanoids to jointly leverage locomotion and manipulation for efficient and safe navigation, representing the first solution to NAMO in rough terrain through predictive movability reasoning and probabilistic VLM integration.

Xueyang Wang, Zehong Shen, Dingkun Wei, ... , Yujun Shen, Xiaowei Zhou
总结: 以多时间尺度异步协调实现长时程语言人形导航。
方法: 三层异步系统配合物理落地数据引擎训练运动。
证据: 摘要强调进度反馈与自然稳定全身运动,未列数值结果。
为什么适合我: 连接语言导航、视觉语言动作与全身控制,偏移动而非操作。
推荐理由: 长时程人形导航中协调语言推理、动作生成和全身执行,贴近人形全身控制与移动,但重点是导航进程而非动作模仿。
原摘要

We study humanoid navigation across multiple destinations under long-horizon language instructions. This task is challenging because robots must maintain progress across successive goals while continuously producing natural, stable whole-body motion through confined spaces. However, task decomposition in existing methods does not by itself ensure progress throughout execution, while waypoint-based navigation does not guarantee natural, route-faithful whole-body motion. Our key idea is to organize navigation into three complementary roles operating at different time scales: a Brain for task reasoning, a Cerebellum for local motion generation, and a Brainstem for whole-body execution. We realize this idea in LHNav, an asynchronous system that uses progress and arrival feedback to coordinate navigation goals, VLA motion-token prediction, and whole-body control. To train the Cerebellum for natural, route-faithful motion, we develop a physics-grounded data engine that improves physical tracking of motion references along collision-aware routes and pairs the resulting demonstrations with rendered visual observations. Together, these designs allow task planning to proceed alongside continuous movement and extend single-goal navigation to successive destinations with natural, route-faithful whole-body motion. In simulation, our navigation VLA outperforms OmniNav and DualVLN in single-goal success and motion smoothness, with fewer collisions and falls. Using a VLA trained only on single-goal demonstrations, LHNav achieves a 75.56% success rate on long-horizon navigation tasks. Deployment on the Unitree G1 further demonstrates single-goal navigation and zero-shot long-horizon task execution in real-world environments, including layouts with repositioned objects. Project website: https://robbyant-research.github.io/LHNav/.

Fangcheng Jin (University of Chinese Academy of Sciences)
总结: 提供文本生成运动、机器人重定向与可回溯校验的版本化流程。
方法: 用五十二关节骨架、运动库和不可变候选报告管理编辑。
证据: 含核心回归、CUDA案例、30帧及3600帧G1检查。
为什么适合我: 服务于文本到运动与人形重定向的可验证创作流程。
推荐理由: 虽含文本到运动和机器人重定向,但主体是版本化工作流与工程验证,缺少全身控制或运动跟踪算法贡献。
原摘要

Motion generation tools produce candidate trajectories, but interactive robot authoring also requires recoverable execution, traceable edits, explicit geometry, and model-specific validation. We present Kinora, a system that connects text-to-motion generation, a motion library, local agent tools, robot retargeting, and trajectory refinement through versioned artifacts. The current release uses an independently specified 52-joint authoring rig and procedural skeleton previews, separating the distributed human geometry from externally installed generation models and robot assets. Human-motion edits preserve frames outside a selected interval, while robot edits create immutable candidates whose reports govern acceptance and export. We evaluate server 0.5.1 and Gadget 0.15.2 with 156 core regression test executions, 49 Gadget executions, four CUDA generation/editing cases, a 30-frame G1 retargeting check, and a 3,600-frame synthetic G1 fixture paired with a suspended negative control. All four CUDA cases passed their numerical gates; the turning splice nevertheless retained a 14.3 cm support-foot drift proxy warning. The G1 positive fixture completed bounded simulation replay, whereas the suspended control was quarantined before replay. These results support selected integration and preservation contracts. They do not establish semantic-quality improvements, population-level feasibility, or hardware safety.

Athanasios Kyriakopoulos, Shubham S. Kumbhar, Kunal S. Narkhede, Abhijeet M. Kulkarni, Ioannis Poulakakis
总结: 在障碍环境中用可捕获性约束规划人形落足。
方法: 轴对齐安全走廊上的递推可行序贯模型预测控制。
证据: 高保真仿真验证,成功率和计算代价对预测时域不敏感。
为什么适合我: 契合人形落足稳定与模型预测控制,不是全身力矩控制。
推荐理由: 属于人形落足与步态MPC,和全身模型预测控制有关,但是线性倒立摆足步规划,而非全身运动跟踪或移动操作。
原摘要

A fundamental aspect of humanoid footstep planning is to jointly guarantee obstacle avoidance and locomotion stability. This paper presents a capturability-aware sequential Model Predictive Control (MPC) framework for Linear Inverted Pendulum (LIP)-based humanoid footstep planning in the presence of obstacles. The proposed method combines obstacle avoidance with capturability constraints through safe corridors constructed as sequences of intersecting rectangles, yielding a sequential MPC framework with recursive feasibility guarantees for the LIP-based planning layer. The planner autonomously adapts to constrained scenarios, switching to in-place walking when obstructions---such as doors assumed to be open during planning but unexpectedly found to be closed during execution---block forward motion. Performance is largely independent of the prediction horizon, allowing high success rates while keeping the computational cost low. The approach is validated in high-fidelity simulations with the humanoid robot Digit, demonstrating a practical solution for capturable humanoid footstep planning in cluttered environments.

Bingzhang Wang, Nathan Zhao, Jialeng Ni, ... , Kehua Chen, Jian Yao
总结: 解耦慢语义与快动作头,使人形操控频率不受骨干大小限制。
方法: 并发运行冻结骨干与流匹配动作头,用新视觉补偿陈旧语义。
证据: 摘要描述去同步训练与自适应动作块对齐,未给成功率。
为什么适合我: 针对人形灵巧操作的视觉语言动作闭环延迟问题。
推荐理由: 面向人形灵巧操作的异步视觉-语言-动作系统,接近人形操作,但贡献是延迟解耦而非全身运动控制或运动重定向。
原摘要

Vision-language-action (VLA) models pair a vision-language backbone with an action head, but in closed-loop control the backbone sits on the control-critical path: while it computes, the robot keeps moving, so each executed action is conditioned on an increasingly stale observation. A larger backbone therefore directly blocks reactivity, especially in humanoid dexterous manipulation. We present DASH, a Decoupled Asynchronous System for Humanoids that removes this coupling. DASH splits the policy into a slow vision-language model (System 2) and a fast flow-matching action head (System 1) that run as concurrent processes over the LCM communication protocol, so the control rate is set by the action head alone, independent of backbone size. The action head fuses stale semantics from a frozen backbone with fresh vision from its own ResNet encoder, desynchronized training exposes it to the staleness it meets at deployment, and an adaptive temporal aggregator aligns overlapping action chunks under variable inference latency. On an asynchronous RoboTwin benchmark that charges inference latency in simulated time, DASH holds 20 Hz control rate from 0.8B to 8B backbones while the synchronous rate falls from 9.7 to 4.1 Hz, with higher success at every size. On a real humanoid with dexterous hands, DASH achieves clearly higher pick-and-place success than the synchronous pipeline.

A Unified Inertial Encoder for Human and Humanoid Motion

3.0/5 一般 裁判分 8.0 Strong 当日相对
Yutian Chen, Shibo Zhao, Rocky Duan, ... , Yuheng Qiu, Koushil Sreenath
总结: 用统一惯性编码把人体运动理解零样本迁到人形状态估计。
方法: 在对应部位合成惯性测量,联合训练速度估计与运动文本对齐。
证据: 无需仿真或微调迁到真机,真实G1误差0.099米每秒。
为什么适合我: 有助人体到人形迁移,但是状态估计而非全身控制。
推荐理由: 借助运动学重定向打通人与人形惯性表征并做零样本状态估计,接近动作重定向,但不是全身控制、跟踪或策略学习。
原摘要

Learning humanoid state estimation and language-grounded motion understanding from robot recordings ties training coverage to the robot's motion repertoire. Human motion datasets offer broader coverage, yet transferring this knowledge requires overcoming differences in body geometry, sensing, and physical disturbances. To enable this transfer, we propose a unified inertial encoder whose shared inertial interface uses kinematic IMU synthesis to represent human and humanoid motion at corresponding body sites. Within this common input space, joint training on human and kinematically retargeted humanoid motion combines velocity estimation and motion--text alignment with continuous-time encoding and sensor and motion randomization. The resulting model transfers zero-shot to a physical Unitree G1 without physics simulation, real-robot training trajectories, or fine-tuning, achieving the lowest velocity error among evaluated methods across five datasets, including 0.099m/s on real G1. Its motion-description retrieval also extends to retargeted G1 motion categories observed only in human training, demonstrating how human motion knowledge can support the generalization of humanoid inertial odometry and language-grounded motion understanding.