Papers for 2026-11-07

10 papers

Terrain-Indexed Motion Priors and Geometric Support for Extreme Humanoid Parkour

5.0/5 很相关 裁判分 10.0 Top pick 当日相对
Chenghao Sun, Youhao Lyu, Zizheng Li, ... , Ziyang Dong, Wenhao Cui
总结: 分阶段模仿学习地形适配的人形跑酷,并自适应手脚支撑。
方法: 特权模仿生成参考,地形索引对抗先验学技能,几何引导再改接触。
证据: 仿真混合成功率96.34%,无真机微调完成30次中的29次。
为什么适合我: 贴合高动态跑酷、机载深度感知与仿真到现实迁移。
推荐理由: 直接对应高动态人形跑酷、对抗运动先验与机载深度感知,并用渐进模仿完成可部署的全身运动技能。
原摘要

We present a progressive imitation framework that learns terrain-appropriate humanoid parkour skills and subsequently adapts their hand--knee support. Human demonstrations provide traversal behaviors, but their contact patterns need not suit a robot's proportions or the available support surfaces. Our framework changes the role of demonstrations across three stages: privileged imitation produces physically executable robot references; terrain-indexed adversarial motion priors guide skill acquisition; and geometric support guidance with a reduced motion-prior weight enables contact adaptation. Candidate-based precontact attraction and actual-contact evaluation guide support without prescribing a contact sequence. The deployed policy selects behaviors from onboard depth, proprioception, and velocity commands without an external skill label. On the Noetix E1, the policy achieves 96.34\% hybrid success in simulation and transfers without real-world policy fine-tuning, completing 29 of 30 individual obstacle trials and four of five hybrid courses. Support-adaptation ablations show higher traversal success for the complete adaptation stage on the tested terrains, while qualitative observations show changes in contact placement. These results support acquiring skills from demonstrations before relaxing their influence to accommodate robot-specific support configurations.

Youhao Lyu, Chenghao Sun, Zhenxuan Ding, ... , Ziyang Dong, Wenhao Cui
总结: 单阶段同时监督跨模态关系及动作依赖的状态演化。
方法: 查询聚合深度与本体感觉,双向掩码预测约束地形身体关系。
证据: 摘要未给定量成功率,仅提出共享控制表征中的显式预测。
为什么适合我: 服务于深度感知人形行走及视觉与本体感觉融合。
推荐理由: 聚焦感知型人形运动,联合监督深度与本体感觉的时空演化,贴近感知型足式运动。
原摘要

For perceptive humanoid locomotion, the relevance of terrain geometry depends on the robot's body state and how both evolve under its actions. Existing work has advanced terrain-aware sensory fusion and cross-modal prediction, improving how visual and proprioceptive information is combined and related. However, feature aggregation alone does not specify which cross-modal relationships should be learned, while prediction between simultaneous observations does not explicitly constrain their action-dependent evolution. We present Spatiotemporal Training with Explicit Prediction (STEP), a one-stage framework that jointly supervises cross-modal relationships and action-dependent state evolution within a shared control representation. Multiple learned queries aggregate depth and proprioceptive tokens into a compact policy state. Bidirectional masked prediction uses each visible modality to predict latent features of the omitted modality, explicitly supervising terrain-body relationships. An action-conditioned predictor maps the current policy state and executed action to the next fused state, with an exponential-moving-average target defined directly in the policy representation space. Simulation comparisons show overall improvements in terrain traversal over evaluated fusion alternatives and ablations, while depth-dropout and delay experiments demonstrate the benefits of both predictive objectives under degraded visual input. Deployed on a 23-DoF Noetix E1 humanoid with onboard sensing, STEP succeeds in 98 of 100 hardware trials across stairs, a 40\,cm platform, a 70\,cm gap, stepping stones, and backward stair traversal without rearward terrain visibility.

Dario Valenzuela, Jeeseop Kim
总结: 将人形动力学拆成约束耦合子系统,实现预测式全身协作。
方法: 非线性模型预测控制联合优化轨迹、力矩、地面反力与扳手。
证据: 以动态双臂操作为例,在线设计肩部与末端交互扳手。
为什么适合我: 直接对应全身模型预测控制及操作中的动态耦合。
推荐理由: 用人形双臂协作操作展示分层全身非线性模型预测控制,直接对应全身模型预测控制与人形移动操作。
原摘要

This paper presents a layered control framework that recasts high-dimensional system dynamics as a composition of dynamically interconnected subsystems, with dynamic dual-arm manipulation on the humanoid serving as a representative application. The humanoid is decomposed into a single rigid body (SRB) representing the torso and floating-base dynamic models representing the articulated arms and lower-body. The rigid connections between arm and torso and the closed kinematic chain formed between the end-effectors during cooperative manipulation are represented as holonomic constraints. The corresponding Lagrange multipliers parametrize the interaction wrenches transmitted at the shoulders and between the end-effectors. The high-level nonlinear model predictive control (NMPC) jointly optimizes the subsystem state trajectories, arm torques, ground reaction forces (GRFs), and interaction wrenches, enabling online wrench design and optimization while preserving the dynamic coupling among the independently modeled subsystems. The optimized arm torques are applied directly for cooperative dual-arm manipulation, whereas the prescribed GRFs are mapped to lower-body joint torque for whole-body coordination. We validate the proposed framework through numerical simulations and hardware experiments on a G1 humanoid robot. The results demonstrate optimal interaction wrenches consistent with simulated contact measurement, cooperative dual-arm trajectory tracking, and stable whole-body coordination while executing the NMPC in online manner.

Social Humanoid: Socially Responsive Human-Humanoid Interaction

5.0/5 很相关 裁判分 6.0 Good 当日相对
Vongani H. Maluleke, James Ni, Kie Horiuchi, ... , Trevor Darrell, Angjoo Kanazawa
总结: 根据人类伙伴动作生成并执行社交响应的全身运动。
方法: 共享词元建模双人运动,低层跟踪,两阶段重定向保持接触。
证据: 应用于五个动捕数据集,并在29自由度宇树G1上执行。
为什么适合我: 紧扣人体到机器人重定向、接触保持与全身跟踪。
推荐理由: 包含交互中的自适应人体到人形重定向,以及执行反应式动作的低层全身跟踪控制器,最接近全身运动跟踪与动作重定向。
原摘要

Humanoid robots are typically trained as solitary agents, leaving them unable to coordinate physically with human partners who naturally anticipate and adjust to one another. To bridge this gap, we present Social Humanoid, a hierarchical system that generates and executes socially responsive whole-body motion on a 29-DoF Unitree G1 conditioned on a human partner's motion. Its high-level policy performs flexible sequence modeling of human and humanoid motion through a shared tokenized representation, generating reactive humanoid motion and anticipating the partner's motion when they are occluded, while a low-level whole-body tracking controller executes the generated motion on the robot. To train the system, we introduce a two-phase adaptive retargeting procedure that retargets one interaction partner to the robot while morphing the other partner to preserve contact dynamics, reachability, and relative body scale. We apply this procedure across five motion-capture datasets to produce a new Human-Humanoid Interaction (HHI) benchmark of 260 hours of human-humanoid interactions. Our results demonstrate that Social Humanoid produces coordinated, human-aligned behavior across diverse interaction types in simulation and the real world. Project page: https://von31.github.io/social-humanoid/

LMGN: A Unified Policy Learning Model for Last-Mile Grasp-Navigation

4.0/5 相关 裁判分 8.0 Strong 当日相对
Xuenmin Hu, Qingxue Qian, Wei Luo, ... , Bo Tang, Long Chen
总结: 用统一策略学习四足最后一米的落位、接近与抓取。
方法: 以距离方向可达性表示,配合分层混合专家建模阶段行为。
证据: 摘要未给成功率,强调不同操作阶段需要不同控制行为。
为什么适合我: 契合足式移动操作中基座落位与抓取的联合学习。
推荐理由: 属于足式移动操作,统一学习基座停靠、接近方向与抓取,但平台是四足而非人形全身控制。
原摘要

Legged robots equipped with manipulators have shown significant potential in the fields of daily life and industry in complex environments. However, existing methods mainly focus on global navigation or manipulation independently, while overlooking the last-mile navigation problem, where the robot must jointly determine the base placement, approach direction, and grasp execution to achieve reliable object grasping, and different manipulation stages require distinct control behaviors. To address these challenges, we propose Last-Mile Grasp-Navigation (LMGN), a unified policy learning model for Quadruped robot loco-manipulation. An Explicit Distance and Directional Reachability (ED-DR) representation is first introduced to explicitly represent the features of grasp-compatible base placement and approach direction. We then develop a semantically aligned Hierarchical Mixture-of-Experts (HMoE) policy that models stage-aware behaviors across the stages of base adjustment, pre-grasp alignment and grasp execution, and object lift. Finally, a new optimization method that jointly aligns navigation and manipulation, stage-dependent expert routing, and action-level consistency is proposed to optimize the HMoE policy. Extensive experiments on the Habitat and Isaac Gym simulators demonstrate that LMGN consistently outperforms state-of-the-art methods, denoting that integrating grasp-aware reachability with stage-aware policy learning significantly improves the success rates of last-mile navigation and target grasp for robots.

Qingzhou Lu, Chenxin Liu, Xinming Wang, ... , Mengdi Xu, Jianyu Chen
总结: 用全身挽具接口让视觉语言模型指挥并学习操作人形。
方法: 图像选点返回三维位置,学习跟踪器执行全身部位指令。
证据: 无专项轨迹时SIMPLE为51%,补轨迹后达95%,真机可搬箱。
为什么适合我: 关乎视觉驱动全身控制、动作跟踪与无真机数据迁移。
推荐理由: 通过全身接口和学习跟踪器把视觉语言选择转成人形全身动作,贴近人形全身控制与遥操作,但重点在VLM操作接口。
原摘要

Enabling vision-language models (VLMs) to control humanoid robots requires both reliable whole-body interfaces and knowledge of how to use the body that carries over across tasks. We present HITCH (Harness Interfaces To Command Humanoids), a harness in which perception returns the 3D position of any point the VLM selects in its image, and execution turns commands for the pelvis, torso, hands, walking, and gaze into whole-body motion with a learned tracker. We find that both measuring the scene and executing commands accurately raise task success. We then fine-tune a VLM on trajectories in which teacher policies solve five base tasks through HITCH, so that it learns how to operate the humanoid. Without task-specific trajectories, the fine-tuned model completes 51% of SIMPLE episodes and 42.5% of HumanoidArena episodes, and adding five trajectories per SIMPLE task raises its SIMPLE success to 95%. On a real G1, it carries a box and walks to a goal without real training data.

PhySA: Decoupling Aesthetic and Physical Rewards for Humanoid Locomotion

3.0/5 一般 裁判分 7.0 Strong 当日相对
Yunwan Fu, Sicheng He, Jiageng Mao, Yue Wang
总结: 把审美奖励与物理平滑分开,以抑制抖动高能耗步态。
方法: 异步视觉语言评分与相位触发通道分别调节两类奖励。
证据: 四款模型显示加速度与审美正相关,仿真加速度降26%至32%。
为什么适合我: 可用于强化学习人形步态塑形,但非全身跟踪核心。
推荐理由: 虽是人形行走强化学习,但核心是美学评分与物理平滑奖励解耦,未涉及运动跟踪、感知或跑酷。
原摘要

Humanoid locomotion policies trained without an explicit smoothness objective tend to produce jerky, high-energy gaits that stress actuators. Vision-language models (VLMs) offer an appealing reward source: given a rendered rollout, a VLM returns an aesthetic quality score on [0,10] without task-specific feature engineering. We find, however, that aesthetic preference and physical smoothness point in opposite directions: across four production VLMs, clips with higher joint acceleration receive higher aesthetic scores (mean Pearson r=+0.66). We study PhySA (Physical Smoothness and Aesthetics), which keeps the two on separate, independently removable reward channels: a sparse, asynchronous VLM score and a phase-triggered correction channel that adjusts physical penalty weights from simulator state. Ablations attribute the smoothness gain to the correction channel. On MuJoCo Humanoid-v5 at 2.6M steps, PhySA lowers joint acceleration by 26 to 32% relative to vanilla PPO, Text2Reward, and Eureka (three seeds), at the cost of shorter episodes. In a single extended run it lowers joint acceleration by 59.5% relative to Prompt2Policy, while a three-seed replication shows a smaller, non-significant reduction (7.2%). The gain does not transfer to IsaacLab or the Unitree G1, even after rescaling the smoothness term.

Nikita Kuzmin, Georgii Demianchuk, Mariya Lezina, ... , Miguel Altamirano Cabrera, Dzmitry Tsetserukou
总结: 意图门控的视觉语言动作流程,支持安全的人形社交接触。
方法: 双相机识别意图并提示策略,接触时调阻抗,语音触发急停。
证据: 约1300段遥操作数据微调四模型,实机以GR00T表现最好。
为什么适合我: 结合遥操作采集、接触合规控制与人形真机互动。
推荐理由: 在G1上用遥操作数据做社交接触VLA,只边缘触及人形遥操作,主题是意图识别与安全接触而非全身运动控制。
原摘要

Social touch interactions, such as handshakes, fist bumps, and hugs, require humanoid robots to infer human intent, generate contact-rich motions, and maintain safe physical contact. We present AffectiveVLA, an intent-gated Vision-Language-Action (VLA) pipeline deployed on a Unitree G1 humanoid with dexterous hands. A lightweight RGB-based model recognizes interaction intent from two onboard cameras and converts it into a language prompt for the VLA policy. Low-confidence predictions suppress motion execution, reducing unintended activations. During physical contact, interaction-specific compliance profiles adjust arm impedance, while an independent voice-triggered mechanism initiates a latched safe-stop procedure. Using VR teleoperation, we collected approximately 1,300 multimodal interaction episodes and fine-tuned four VLA models: GR00T, RLDX-1, Being-H0.5, and UnifoLM-VLA-0. On-robot evaluation identifies GR00T as the best-performing standalone model. Integrating intent recognition increases its macro-F1 score from 0.75 to 0.90, while the intent model achieves a p95 inference latency of 6.27 ms. These results demonstrate that explicit intent gating improves the reliability of VLA-based social interaction without compromising real-time operation.

Sateesh Kumar, Arnav Balaji, Arpit Bahety, ... , Georgios Pavlakos, Roberto Mart\'in-Mart\'in
总结: 从未见基座位姿合成包含动态物体交互的移动操作示范。
方法: 利用采集时多视角重建场景和物体,并补全接触后渲染。
证据: 摘要指出静态新视角方法无法重渲染接触后的被操作物体。
为什么适合我: 服务于移动操作模仿学习对视角变化和接触阶段的鲁棒性。
推荐理由: 通用移动操作的新视角演示增强,未明确人形或足式全身控制,偏离运动跟踪与跑酷主线。
原摘要

Visuomotor policies trained through imitation learning are highly sensitive to changes in camera viewpoint. This is particularly problematic for mobile manipulation (MoMa), where robots rarely execute manipulation from the exact same base pose used during demonstration collection due to navigation uncertainty, localization drift, and environmental constraints. Recent approaches use novel-view synthesis (NVS) to augment demonstrations with unseen viewpoints, but they assume static scenes and therefore fail to re-render the manipulated object during the post-contact interaction phase, leaving the most critical parts of the demonstration unaugmented. We present MOMASPLAT, a novel-view demonstration augmentation framework for mobile manipulation that synthesizes full manipulation trajectories from unseen robot base poses, including dynamic object interactions. Our key insight is that standard MoMa data collection naturally provides the multiview observations needed to reconstruct both the static scene and the manipulated object in 3D. MOMASPLAT combines 3D Gaussian Splatting, feed-forward 3D foundation models, and kinematics-based action retargeting to separately reconstruct the scene and manipulated object, render them from novel viewpoints, and generate geometrically consistent synthetic demonstrations paired with transformed actions. We evaluate MOMASPLAT on five real-world mobile manipulation tasks involving both rigid and articulated objects across three scenes, and validate the fidelity of its synthesized demonstrations against ground truth in simulation. MOMASPLAT substantially improves robustness to unseen base poses, more than doubling the average success rate of the strongest baseline and maintaining non-trivial success at large base displacements where all baselines fail, while reducing the need for largescale real-world data collection.

Guanren Qiao, Ruixiang Ouyang, Sheng Xu, ... , Kui Jia, Guiliang Liu
总结: 先做语义聚焦再做接触细化,提升接触操作视觉鲁棒性。
方法: 视觉语言模型粗定位,人在回路残差强化学习用力反馈精调。
证据: 物体中心视觉增强随机化背景,同时保留机器人与相关物体。
为什么适合我: 偏接触操作与真机强化学习,跟足式全身控制关联间接。
推荐理由: 面向接触丰富机械臂操作的视觉鲁棒性与残差强化学习,不涉及人形/足式全身控制。
原摘要

Contact-rich manipulation requires precise control under complex contact dynamics while remaining robust to diverse visual conditions. However, real-world reinforcement learning often overfits to the visual appearance of training scenes. We propose Semantics-to-Contact (S2C), a stagewise framework for visually robust contact-rich manipulation. The insight is to decompose the problem into two stages: semantic focusing and contact refinement. In the first stage, a vision-language model localizes task-relevant regions across visually diverse scenes, providing coarse spatial guidance and reducing the burden of exploration. In the second stage, a real-world human-in-the-loop residual reinforcement learning policy learns fine-grained contact behaviors within the localized region using force feedback. To further improve visual robustness, we introduce an object-centric visual augmentation strategy that randomizes background appearance while preserving the robot and task-relevant objects. Experiments on six contact-rich real-world manipulation tasks exhibit that S2C consistently improves success rates over strong baselines and maintains reliable performance under significant visual and positional variations. These results demonstrate that stagewise semantic focusing and contact refinement provide a practical path toward visually robust real-world contact-rich manipulation. Our demos can be seen in https://edem-ai.github.io/S2C/.