Papers for 2026-10-23

10 papers
Zhaohui Yang, Quanbao Lin, Chengqi Shi, ... , Xiaokang Yang, Yao Mu
总结: 稀疏姿态驱动接口实现人形loco-manipulation全身控制。
方法: SP-VLA预测稀疏姿态手指,SP-MGM蒸馏生成协调全身运动。
证据: 结合任务姿态与潜在运动先验,实时重建全身体协调运动。
为什么适合我: 支持接触丰富loco-manipulation与全身控制,契合模仿学习。
推荐理由: 直接对应人形全身控制与loco-manipulation:稀疏姿态接口、运动先验和全身运动生成,是库内最核心方向。
原摘要

Humanoid loco-manipulation requires an effective interface between high-level task policies and whole-body control. Existing interfaces typically rely on a single action representation for both task intent and whole-body coordination. This may burden the high-level policy or leave one aspect underspecified. We present SP-HLM, a framework built on a sparse-pose-driven interface. The interface combines five task-critical body poses with a learned latent motion prior to represent spatial intent and whole-body coordination in complementary forms. SP-VLA, a vision-language-action model, first predicts task-relevant sparse poses and finger joint positions from multimodal observations and language instructions using asynchronous action-chunk inference. Given these sparse targets, SP-MGM, a motion generation model trained through two-stage distillation, reconstructs coordinated full-body motion in real time. A pretrained motion encoder maps this motion to a compact latent representation. SP-GMT, a general motion tracker, then integrates the explicit sparse targets with this implicit motion prior for whole-body control. Experiments in simulation and on a real humanoid robot demonstrate improved motion tracking, whole-body coordination, and loco-manipulation success over existing approaches. Additional results and videos are available on our \href{https://sp-hlm.github.io/}{\textcolor{blue}{project page}}.

SMR: Learning streaming motion retargeting for humanoids from human semantic

5.0/5 很相关 裁判分 10.0 Top pick 当日相对
Chun Gu, Juntong Zhang, Jingyu Li, Li Zhang
总结: 从人类语义学习流式运动重定向至人形机器人。
方法: 物理专家跟踪任务空间目标,训练严格因果Transformer。
证据: 模型在仿真执行运动并通过语义与物理有效性检查。
为什么适合我: 实现人体动作可跟踪可重定向可上真机,核心兴趣匹配。
推荐理由: 直接对应人体动作重定向与可执行全身跟踪:语义条件、物理专家和因果流式重定向,高度契合。
原摘要

Retargeting human motion to humanoids requires preserving motion semantics while accounting for the robot's physical constraints. Kinematic pipelines prescribe a robot trajectory through geometric matching, without establishing whether it can be executed under the robot's dynamics. Prior physics-supervised neural methods train bidirectional full-sequence models on physics-refined kinematic trajectories, remaining tied to prescribed kinematic solutions and requiring noncausal context. We propose SMR, a streaming neural motion retargeter learned from semantic-conditioned physics experts. We convert human motion into robot-aware task-space targets for body landmark positions, orientations, and velocities. Physics experts track these targets directly in simulation under contact, dynamics, and actuator constraints. We train a single strictly causal Transformer on their executed state trajectories that complete the motion and pass semantic and physical validity checks. At each incoming frame, the student uses only current and past human observations to generate a robot motion reference for a downstream whole-body controller. Experiments demonstrate that semantic-conditioned physics experts generate motion with higher semantic fidelity and physical validity than direct optimization-based retargeting, while the resulting streaming retargeter achieves improved closed-loop performance over kinematic and physics-refined baselines under a common whole-body controller.

Liu Cao, Botian Xu, Ruoqu Chen, ... , Zhongyu Li, Mengdi Xu
总结: 紧凑参考跟踪实现低延迟全身人形遥操作。
方法: 统一策略仅需根高度速度与四稀疏关键点SE(3)姿态。
证据: 通信推理延迟约20ms,跟踪保真度媲美密集参考方法。
为什么适合我: 低延迟稀疏跟踪利于真机全身控制与loco-manipulation。
推荐理由: 直接对应人形遥操作与全身运动跟踪:用稀疏关键点做低延迟全身参考跟踪,是库内核心问题。
原摘要

Real-time whole-body humanoid teleoperation remains challenging due to the computational latency introduced by optimization-based retargeting pipelines and future-frame conditioning. Existing motion tracking policies typically require dense reference targets including full joint positions, body-part transformations, and even future frames, which tightly couples the controller to expensive upstream modules. In this work, we present HuMiT, a whole-body teleoperation system built on a unified tracking policy that requires only a compact reference target, consisting of root height, root velocity, and the SE(3) poses of four sparse keypoints at the current frame, yet achieves competitive tracking fidelity compared to methods relying on more diverse reference states including complete joint positions, dense body-part transformations, and even future frames. Experiments show that our policy achieves a communication-to-inference latency of $\sim$20 ms measured between headset streaming and policy output and a video-based optical flow latency of 65 ms (median) across different motion types. We validate the system on a Unitree G1 humanoid, showcasing split-second reactive responses such as catching a tossed ball, as well as coordinated loco-manipulation capabilities. Project page: https://xiaohu-art.github.io/Track/.

$λ_0$: Scaling Open-World Egocentric Human Data for Humanoid Loco-Manipulation

5.0/5 很相关 裁判分 10.0 Top pick 当日相对
Lambda-humanoid Team, :, Chongyang Xu, ... , Mu Xu, Steven Hoi
总结: 扩展开放世界自我中心人类数据用于人形loco-manipulation。
方法: 构建HumanVerse数据集,管道对齐细化实现零样本迁移。
证据: 零样本转移G1完成全身操作与长距离loco-manipulation。
为什么适合我: 大规模人类数据支持模仿学习与全身loco-manipulation。
推荐理由: 直接对应人形loco-manipulation与人体数据采集:大规模自我中心人体数据、动作重定向并零样本上G1。
原摘要

Humanoid loco-manipulation requires coordinated whole-body interaction with the physical world, yet collecting robot demonstrations at scale remains costly. Human demonstrations offer a scalable alternative, but existing egocentric human supervision often omits locomotion and posture, while embodiment differences and controller dynamics hinder accurate end-effector execution. We introduce HumanVerse, a 500-hour dataset of open-world human loco-manipulation pairing egocentric video with synchronized body and hand motion. Our human-to-humanoid pipeline aligns camera viewpoints and arm appearance, and refines motion references and controller inputs offline to reduce end-effector errors while preserving whole-body coordination. We validate the pipeline through zero-shot transfer to Unitree G1 on representative whole-body manipulation and long-range loco-manipulation tasks, without target-task robot demonstrations. Building on this pipeline, we develop $λ_0$, a humanoid foundation model for whole-body loco-manipulation, with robot-aligned human demonstrations providing supervision for mid-training. Increasing whole-body human data consistently improves downstream task progress. $λ_0$ achieves 90.0% success on SIMPLE and 62.5% across real-world loco-manipulation tasks. We expect this study to guide scalable whole-body human data collection toward humanoid foundation models for richer interaction with the physical world. Homepage: https://lambdahumanoid.github.io/

Watch Your Step: Anticipated Perception for Humanoid Locomotion

5.0/5 很相关 裁判分 9.0 Top pick 当日相对
Matthias Heyrman, Chenhao Li, Ioannis Dadiotis, Marco Hutter
总结: 通过预期落脚点实现人形运动的高效感知。
方法: 落脚点预测模型从本体感觉预测,仅采样周围高程图。
证据: 与策略共训,感知预期落脚加速训练并改善地形泛化。
为什么适合我: 复杂地形感知运动优化,契合跑酷与接触丰富场景。
推荐理由: 直接对应腿足感知运动:用落足预测裁剪高程图,服务人形感知行走,方法与问题都契合。
原摘要

Perceptive locomotion policies typically observe a dense robot-centric elevation map, although only the terrain beneath and around the next footsteps informs the control decision. Learned encoders compress the whole visual input, leaving the policy to work out which inputs matter while learning the baseline locomotion task. Learning both at once slows training and, without computationally expensive attention mechanisms, makes generalization to new terrain harder. We instead remove redundant inputs from the elevation map before the policy sees it. A lightweight foothold prediction model (FPM) predicts future footholds from proprioception and the velocity command alone, and we sample the elevation map only in small patches around these predicted footholds. We train the prediction model alongside the locomotion policy, periodically retraining it on new policy rollouts as the robot's gait matures. This allows the perception to anticipate where the policy will step, while the policy learns to leverage terrain information around those steps. With the proposed lightweight preprocessing, the policy reaches a given terrain difficulty in roughly a third of the training iterations required with the complete map. It outperforms policies that observe the complete map where footholds are sparse, including in narrow walled corridors, and on unseen terrain, and it matches an attention-based encoder trained at several times the cost. The policy transfers zero-shot to a Unitree G1 humanoid traversing various obstacles, stairs, gaps, and step-downs beyond the training distribution in both indoor and outdoor environments.

Test-Time Motion Steering for Perceptive Humanoid Fall Recovery

5.0/5 很相关 裁判分 9.0 Top pick 当日相对
Osher Azulay, Cheng-Lin Hsieh, Stella X. Yu
总结: 测试时运动引导实现感知人形跌倒恢复。
方法: 学习恢复词汇,视觉扩散规划器预测未来状态并引导。
证据: 多样恢复由共享运动组成,任务目标推理时引导站立。
为什么适合我: 接触丰富场景感知恢复与扩散模型,相关全身运动智能。
推荐理由: 贴近感知人形全身控制与扩散运动先验:测试时按场景与任务引导跌倒恢复,接触丰富且方法匹配。
原摘要

Humanoid fall recovery requires selecting a motion that suits both the surrounding scene and the task. The same fallen pose can admit several ways to stand, with different routes, support transitions, and final headings. Yet reliable recovery alone does not provide control over which of these alternatives the robot executes. Steering this choice requires anticipating the physical consequences of a requested movement. We present RISE, an approach to perceptive humanoid recovery that lets task objectives steer how the robot returns to standing at inference time. It builds on two insights: diverse recoveries can be composed from shared movements, and anticipating how these movements unfold allows recovery to be steered toward a task objective. We learn a recovery vocabulary that connects intended movements to motor actions through sensory feedback. A visual diffusion planner learns from executions of this vocabulary to predict motion intent as future body states. Task costs guide this prediction toward a desired heading, rise speed, or path. The planner then generates motion latents conditioned on the steered intent, allowing the requested movement to shape its execution. Simulation experiments demonstrate diverse recoveries, terrain adaptation, and test-time steering without retraining, complemented by real-world demonstrations on a Unitree G1. Project page: https://osheraz.github.io/RISE/

Guillem Casadesus Vila, Sri Aditya Deevi, Grace X. Gao
总结: 评估地球训练人形控制器向月球重力地形土壤迁移。
方法: G1模拟训练多策略,变化重力地形接触并测试双向迁移。
证据: 地球策略月球存活0.64至0.75,月球策略无法地球站立。
为什么适合我: 关注复杂地形重力变化下腿足运动迁移,相关全身控制。
推荐理由: 贴近腿足/人形运动控制与sim-to-real:考察重力、地形和接触模型下的人形行走迁移,虽偏月球场景仍值得看。
原摘要

The Moon changes gravity, terrain, and soil at once. Learned locomotion has studied reduced gravity on rigid ground and granular terrain mostly at Earth gravity, but not humanoids under both, generalist-controller transfer, or the Earth cost of lunar adaptation. On a simulated Unitree G1, we train 12 policies with one recipe, varying gravity, terrain geometry, and ground contact (rigid or three-dimensional resistive force theory, 3D RFT), and including Earth-to-Moon fine-tuning and gravity randomization. We test them at both gravities on rigid ground and 3D RFT, and at their training gravity on three held-out soil models. We also test the frozen SONIC, a released generalist humanoid controller, at both gravities on rigid ground only. Gravity transfer is asymmetric. Earth-trained policies keep 0.64 to 0.75 survival at the Moon on flat rigid ground, Moon-trained and fine-tuned policies cannot stand at Earth, and SONIC survives at most 0.10 at the Moon. Rough training lifts combined-tile survival from 0.53 to 0.98 at Earth and 0.80 to 0.99 at the Moon. In 3D RFT, soil training helps on Earth sand, on tiles and for rough-trained policies also on flat sand. On the stiffer lunar soil it helps only flat-trained Earth policies at Earth and changes the survival of Moon-trained policies by at most a few hundredths. In the sampled rollouts of the held-out soil models, Moon-trained policies keep most of their rigid-ground progress on lunar regolith at the low G1 foot pressure, with per-seed progress ratios of 0.84 to 1.39 apart from one seed with a short rigid reference. A gravity-randomized policy trained on rough 3D RFT terrain has a seed-mean survival of at least 0.84 at Earth and 0.96 at the Moon on every key test cell up to 20 degrees. Lunar results are simulation predictions conditional on the contact model and soil stiffness.

Levin M\"uller, Oliver Fischer, Marco Trentini, Yusuke Tanaka, Marco Hutter
总结: 端到端强化学习实现月球重力四足点对点跳跃。
方法: 多阶段RL训练单一策略,仅用任务兼容机载传感。
证据: 仿真着陆误差小于身体尺寸,硬件转移成功执行跳跃。
为什么适合我: 腿足机器人复杂地形跳跃与RL,相关月球运动智能。
推荐理由: 只部分贴近腿足动态运动:四足月球跳跃的端到端RL,不是人形、也不是复杂地形感知跑酷主线。
原摘要

Legged robots can extend planetary exploration to rugged and discontinuous terrain that is inaccessible to conventional rovers. In lunar gravity, jumping further expands this mobility by enabling robots to cross obstacles and gaps that cannot be traversed by walking alone. We present a multistage reinforcement-learning approach that produces a single end-to-end control policy for commanded point-to-point jumping under lunar gravity. The resulting policy has a simple architecture and relies exclusively on mission-compatible onboard sensing, making its deployment on a real lunar mission plausible. In simulation, the learned policy performs accurate point-to-point jumps with landing errors smaller than the robot's body size. We analyze the torque and power demands of lunar jumping and characterize performance degradation under extreme outlier events, including control-update freezes of up to \SI{1}{s} during flight. Hardware transfer demonstrates that the policy can successfully execute a vertical jump under lunar-gravity offloading. Furthermore, the measured torque and power profiles are consistent with the simulation results and remain within the robot's hardware limits.

Ziqian Huang, Yuzhuo Chen, Shimin Zhang, ... , Xinyue Annie Yang, Animesh Garg
总结: 研究偏好聚合位置以实现可引导人形控制风格。
方法: 比较奖励优势损失与PCGrad聚合跨两个人形框架。
证据: 早期聚合响应广但泄漏多,后期聚合精度高响应弱。
为什么适合我: 可引导全身控制与偏好RL,支持多样运动风格调整。
推荐理由: 只部分贴近人形全身控制:讨论偏好条件下的风格可转向与目标聚合,不是跟踪、重定向或loco-manipulation。
原摘要

Modern robotic agents are increasingly expected to support steerable control, where a single policy can adjust its behavioral style at runtime under changing user preferences. Preference-conditioned reinforcement learning provides this interface, but conditioning alone does not ensure that a requested behavior change is strong, does not leak into unrelated behaviors, and does not compromise task performance. Our evaluation combines Preference Reward Responsiveness (PRR) with coverage and utility metrics. We use these measures to compare reward-level, advantage-level, loss-level, and PCGrad-based aggregation across two humanoid-control frameworks. No aggregation level performs best across all environments and criteria. Experiments show that aggregating objectives early produces a wider range of behavioral responses but greater unintended changes in other objectives, while aggregating them later improves control precision at some cost to responsiveness.

RLMuscle: Muscle-Driven Character Animation via Reinforcement Learning

3.0/5 一般 裁判分 7.0 Strong 当日相对
Chunlei Li, Siyuan Yu, Yang Gao, ... , Aimin Hao, He Wang
总结: 强化学习发现肌肉激活实现角色动画身体变形。
方法: RL控制器跟踪参考,1D代理加3D仿真并加EMG先验。
证据: 激活一致产生运动与变形,导向真实人类肌肉募集。
为什么适合我: RL肌肉驱动启发人形全身物理真实运动控制。
推荐理由: 边缘相关物理角色控制:用RL跟踪参考并驱动肌肉形变,重点是动画真实感而非可上真机的全身运动。
原摘要

Body deformation caused by muscle contraction plays a vital role in the visual realism of character animation, yet in most 3D animation pipelines artists have to manually specify muscle activations, which is time-consuming and gives no guarantee of physical realism. To address this, we propose a new method in which a reinforcement learning (RL) controller discovers per-muscle activations that track reference motions without manual tuning. Existing RL muscle controllers stop at 1D muscles; we go one step further, keeping the 1D model inside the RL control loop as a fast surrogate, and using a 3D volumetric simulation to produce the final deformation. Because both stages share the very same constitutive law, the muscle deformation is produced by the same activations that generated the motion. We further add a measured electromyography (EMG) prior that steers the redundant activation solution toward real human muscle recruitment. To achieve stable, efficient, and realistic simulation, we propose a novel geometric multigrid for Extended Position-Based Dynamics (XPBD) that is built on a coarse tetrahedral cage with barycentric-coordinate prolongation and closes a V-cycle on tetrahedral meshes. Within this GPU-parallelized solver, we inject the same Hill-type force-length law as an energy-based fiber constraint that turns per-fiber activation into realistic muscle contraction and bulging. We demonstrate volumetric muscle simulation on a 194-muscle human body with a diverse set of motions (walk, kick, swim, overhead-squat, etc.), simulating the full-body model at 18 ms per frame on a single RTX 4090. Our work takes a step toward character animation that is not only visually convincing but also biomechanically grounded. The website with code and videos is at https://chunleili.github.io/project-page-rlmuscle/.