Papers for 2026-11-21

10 papers

WEVO: World Model-Enabled VLA Optimization for Whole-Body Loco-Manipulation

5.0/5 很相关 裁判分 10.0 Top pick 当日相对
Qianzhong Chen, Muyuan Lin, Yuyin Sun, ... , Cheng-Hao Kuo, Roberto Mart\'in-Mart\'in
总结: 在动作条件视频世界模型中后训练人形全身操作策略。
方法: 仿真玩耍数据改进接触与视角预测,再用真机回放对齐并闭环强化学习。
证据: 玩耍数据改善全部世界模型指标,视频模型强化学习成功率高于真机。
为什么适合我: 契合人形全身移动操作、免遥操作后训练和仿真到现实。
推荐理由: 直接对应人形移动操作与全身控制:在动作条件视频世界模型中用强化学习后训练全身VLA,并处理仿真到现实差距。
原摘要

Reinforcement learning (RL) can improve vision-language-action (VLA) policies beyond imitation, but humanoid exploration is unsafe and simulator RL can exploit residual sim-to-real gaps. We present WEVO, a teleoperation-free framework for post-training a whole-body humanoid VLA in an action-conditioned video world model (WM). Simulated manipulation and locomotion play data improve contact-dynamics and viewpoint-change prediction. A VLA trained on synthetic demonstrations collects 20 autonomous real robot rollouts per task to align the WM with deployment observations. The VLA, whole-body controller (WBC), and video WM then form a closed-loop RL environment with dense progress rewards. Across three real robot tasks, play data improves every evaluated WM metric and downstream policy success; video WM RL raises mean success from 23.3% to 75.0%, compared with 48.3% for real robot RL. We therefore leverage the complementary strengths of both environments: simulation provides policy initialization and broad coverage, while the video WM offers a deployment-aligned environment for policy improvement. Project website: https://qianzhong-chen.github.io/wevo.github.io/.

Bin Li, Xing Zhu, Jingya Wang
总结: 统一从文本、音频和视频生成可物理执行的人形反应。
方法: 离散连续混合变换器预测状态,历史动作变换器负责物理执行。
证据: 离散引导减轻反应平均化,连续状态预测避开纯运动学伪影。
为什么适合我: 关乎多模态线索下的人形物理全身反应控制。
推荐理由: 贴近物理角色控制:由文本、音频和视频驱动的物理可执行人形反应,含扩散生成与历史感知动作执行。
原摘要

Humanoid agents in everyday environments are expected to react to diverse cues as humans do, with semantically appropriate and physically executable behaviors. Existing cue-driven reaction generation methods support only a narrow range of cue modalities and are limited to purely kinematic generation, leaving the resulting reactions susceptible to physical artifacts. To this end, we present MixReact, the first unified framework for physics-based reactive humanoid control from text, audio, and video cues, integrating a Discrete--Continuous Mixture of Transformers (DisCo-MoT) for state prediction with a History-aware Action Transformer (HAT) for physical execution. DisCo-MoT couples a discrete masked-modeling branch with a continuous autoregressive diffusion branch via one-way mixture attention within a MoT architecture. Guidance from discrete context mitigates collapse into averaged behaviors in one-to-many reaction generation, while direct prediction in continuous state space bypasses the decoding bottleneck of discrete representations. HAT then maps the predicted next state and observed execution history to executable actions, enabling robust closed-loop control in physics-based simulation. Experiments across all three modalities demonstrate consistent improvements over the baselines in fidelity, diversity, and physical consistency, while ablations validate the complementary contributions of the components. Our code will be released to facilitate future research. Project page: https://binlee26.github.io/MixReact_Page/

PGX: Physically Grounded Execution for Long-Horizon Humanoid Loco-Manipulation

5.0/5 很相关 裁判分 10.0 Top pick 当日相对
Kai Lu, Chenyang Ma, Ioannis Havoutis
总结: 把长时程人形移动操作的时序与精度交给免训练模块。
方法: 谓词状态机管阶段与恢复,技能模型执行,世界坐标锁定保精度。
证据: 摘要说明共享谓词连接分段、执行与评估,但未给成功率。
为什么适合我: 贴合人形长时程移动操作和视觉语言动作技能执行。
推荐理由: 核心是人形长时程移动操作,用物理谓词状态机协调导航、等待与VLA操作技能。
原摘要

Robotic vision-language-action (VLA) models can execute diverse individual manipulation skills, yet they still struggle with long-horizon loco-manipulation tasks, where temporal structure, skills, and precision are tightly coupled within a monolithic model. We propose Physically Grounded Execution (PGX), a framework that offloads temporal and precision control to two classical training-free components, allowing the VLA to focus on stage-specific skill dynamics. PGX decomposes long-horizon loco-manipulation into three layers: (i) a predicate-based state machine governs task progression, using physical predicates to gate phase transitions, trigger recovery, and coordinate navigation and waiting. Shared predicate semantics connect offline demonstration segmentation, online execution, and stage-wise evaluation; (ii) VLA skills operate within active manipulation phases, with flat or stage-trained models concentrating on task-relevant actions; and (iii) world-coordinate locking (WCL) maintains spatial consistency through base station keeping, object-relative positioning, and world-anchored contact targets. We demonstrate the effectiveness of PGX with extensive experiments conducted on three simulated tasks and one real-world task using a Unitree G1 humanoid robot for long-horizon loco-manipulation, showing improvements in stage progression and task success rates. Project page is available at: https://kl-research.github.io/pgx

ECHO: In-Context Behavior foundation model for Humanoid Whole-Body Control

5.0/5 很相关 裁判分 10.0 Top pick 当日相对
Xingyi Wang, Bin Li, Jiyuan Shi, ... , Shiqiang Zhu, Chenjia Bai
总结: 用上下文经验把通用行为先验改成环境特定全身控制。
方法: 混合退火采样预训练先验,历史表征指导残差且测试时不更新。
证据: 摘要称大量实验证明其优势,但未给出具体成功率。
为什么适合我: 对应人形全身控制、大规模模仿强化学习与动力学适应。
推荐理由: 完全落在人形全身控制:大规模运动模仿与强化学习得到行为先验,再用交互上下文做动力学残差适应。
原摘要

Humanoid behavior foundation models (BFMs) acquire diverse skills through large-scale motion imitation and massively parallel reinforcement learning, yet broad behavioral capability does not ensure precise execution across physical conditions. Without distinguishing the underlying dynamics, policies may favor conservative compromises over environment-specific precision. We introduce ECHO, an in-context BFM that turns interaction experience into environment-specific control, separating general behavioral capability from dynamics-specific execution. Mixed and Annealed Sampling first establishes a behavior prior in a single pretraining stage by balancing targeted practice with broad motion coverage. Building on this prior, Context Representation and Residual Adaptation learns structured dynamics representations from interaction histories to guide residual corrections to the frozen prior, enabling adaptation without test-time parameter updates. Extensive experiments show that ECHO outperforms all baseline methods across diverse motions and physical conditions. Moreover, deployment on the Unitree G1 demonstrates whole-body teleoperation spanning extreme acrobatics and terrain interaction, alongside payload adaptation in manipulation and whole-body control. Page is available at https://mwondering.github.io/echo-bfm/

LocoMuJoCo: A Comprehensive Imitation Learning Benchmark for Whole-Body Control

5.0/5 很相关 裁判分 9.0 Top pick 当日相对
Firas Al-Hafez, Davide Tateo, Oleg Kaidanov, ... , Zhengyi Luo, Jan Peters
总结: 开源全身控制模仿学习基准,统一多种形态、数据与算法。
方法: 单接口连接多后端,支持动捕、重定向、可换模块和真机部署。
证据: 涵盖九个人形、三种骨骼和四种四足,并为每个人形提供动捕。
为什么适合我: 直接服务全身控制、动作模仿、运动重定向和仿真到现实。
推荐理由: 全身控制模仿学习基准,覆盖人形与四足、AMASS/LAFAN1重定向及仿真到现实部署。
原摘要

LocoMuJoCo is an open-source benchmark for imitation learning (IL) and reinforcement learning (RL) on whole-body control. The library couples MuJoCo (CPU) with MJX and the MjWarp backend (GPU) under a single API, exposes 9 humanoid, 3 (musculo-)skeletal, and 4 quadruped embodiments, and ships a curated native motion-capture dataset for every humanoid, to which it can also retarget AMASS and LAFAN1. A modular plug-in architecture makes observations, rewards, control modes, terrains, and domain randomization fully interchangeable, while compact JAX implementations of many RL and IL algorithms provide JIT-compiled end-to-end training, and the companion loco-mujoco-real repository provides sim-to-real deployment. We describe the design, the SMPL-to-robot and robot-to-robot retargeting pipeline, and a usage example. LocoMuJoCo is released under MIT at https://github.com/robfiras/loco-mujoco.

Awaken-HI: Scaling Humanoid Intelligence with Whole-Body Human Experience

5.0/5 很相关 裁判分 9.0 Top pick 当日相对
EngineAI
总结: 用海量第一视角全身人体经验缩放人形移动操作智能。
方法: 共享动作专家同时预测原始人体运动和重定向机器人动作。
证据: 人体轨迹直接监督更有效,增加数据降低误差并提高成功率。
为什么适合我: 高度契合人体到机器人重定向与人形移动操作学习。
推荐理由: 同时命中人体到机器人运动重定向和人形移动操作:用大规模全身人体经验监督共享动作专家。
原摘要

A central goal of humanoid robotics is for robots to perform the everyday work people do in homes and workplaces. Existing egocentric video datasets cover many of these activities but provide little synchronized whole-body motion data for learning loco-manipulation at scale. We build Awaken-Human, which pairs approximately 50,000 hours of egocentric stereo video with synchronized whole-body human motion and covers more than 100 skills. Building on this, we also introduce Awaken-HI (Humanoid Intelligence), a vision-language-action model for humanoid loco-manipulation that jointly predicts original human motion and retargeted robot actions through a shared action expert. We show that loco-manipulation skills transfer more effectively from people to humanoids when human trajectories directly supervise the shared action expert. Increasing human pretraining data reduces held-out robot action prediction error according to an empirical scaling law and raises the mean success rate on downstream real-robot tasks. After full-corpus pretraining and T800 adaptation, Awaken-HI achieves a mean success rate of 93.75% across eight individual tasks and outperforms GR00T N1.7 and $\pi_{0.5}$ in both individual and long-horizon evaluations. The same policy generalizes to objects unseen during robot post-training and to new pairings of familiar objects and destinations.

Ganeshvar Soenessardien, Michael Przystupa, Kevin Sebastian Luck
总结: 无示范模型无关强化学习使人形学会与滑板协调前进。
方法: 任务奖励配合终止塑造,限制离板、双脚上板和过度扭转。
证据: 摘要只描述框架与行为约束,未报告定量成功结果。
为什么适合我: 属于高动态物体介导全身运动,靠强化学习而非示范。
推荐理由: 属于高动态敏捷全身运动:无示范深度强化学习让人形协调身体与可移动滑板,但是单项技能而非跟踪或移动操作主线。
原摘要

Humanoid locomotion research has advanced rapidly on walking and running, but \emph {object-mediated} locomotion, in which a robot must coordinate its body with a passive external object, remains difficult. One challenging task is skateboarding, which requires simultaneous interaction with the ground and a passive but movable platform to achieve forward motion. Current methods for tackling skateboarding with humanoid robots either require human demonstrations or utilise analytic steering models that require system identification. In this paper, we introduce a deep reinforcement learning framework that does not rely on prior data such as demonstrations and enables robots such as the K1 humanoid to learn to skateboard using model-free learning. The method combines task rewards with exploit-blocking terminations. We introduce a number of constraints restricting behaviours such as losing the planted board foot, standing with both feet on the board, excessive twisting, separation from the board, and unstable board motion. Their changing prevalence during training produces an implicit progression from stance control towards sustained robot--board locomotion.

Takahiro Miki, Robin Schmid, Manthan Patel, ... , Matthias M\"uller, Marco Hutter
总结: 视觉语言推理加仿真强化学习,支撑足式机器人野外导航。
方法: 模型评估风险并出语义代价,分割成风险图后调节运动与导航策略。
证据: 摘要称系统可在高山和城市场景完成公里级导航。
为什么适合我: 对应足式视觉语义运动、强化学习控制和开放世界导航。
推荐理由: 只弱相关于感知型足式运动;主体是VLM语义导航与风险地图,不是全身跟踪、跑酷或移动操作。
原摘要

Autonomous navigation in diverse real-world environments remains one of the greatest challenges in robotics. It demands both robust low-level motion control to handle diverse terrain and high-level reasoning to interpret environmental cues. We present ROAM (Robust Open-world Autonomous Mobility), a system that enables legged robots to perform kilometer-scale navigation in complex alpine and urban settings. ROAM combines the world knowledge of a Vision-Language Model (VLM), pre-trained on internet-scale data, with robust locomotion and local planning policies learned in simulation via reinforcement learning (RL). The VLM provides common-sense reasoning and semantic interpretation of the scene, assessing risks, interpreting signs, and producing semantic labels with associated navigation costs. An open-vocabulary segmentation module converts these labels into a dense, real-time risk map, which conditions two pre-trained RL policies responsible for agile locomotion and local navigation, effectively coupling high-level VLM reasoning with low-level physical control. Real-world experiments showed the system's ability to autonomously navigate a wide range of complex real-world environments. The robot completes kilometer-long hikes with minimal intervention, interprets signs to avoid danger, and evaluates its environment to identify safe areas on which to step. This work represents a step toward the reliable deployment of systems in unstructured environments. However, it also underscores the challenges that remain in VLM reliability and spatial reasoning, indicating that foundational models are a crucial component but not yet a complete solution for navigation in the wild.

Guillaume Bellegarda, Auke Ijspeert
总结: 四足最优步态由地形、性能指标和速度共同决定。
方法: 强化学习与振荡器网络学习十一种步态及风格,再按准则选择。
证据: 在该步态库内,每种步态至少在一种组合下成为最优。
为什么适合我: 关乎足式地形自适应运动、步态风格与强化学习控制。
推荐理由: 四足地形步态与风格选择偏生物力学最优性,未进入运动跟踪、重定向或人形全身控制。
原摘要

Animals use many different gaits to locomote at different speeds across varying terrains. A longstanding question in biology and robotics is: which gait is best? This question is most often addressed from an energy-efficiency perspective on flat terrain, leaving open how the answer depends on terrain and on the criterion optimized. Here we show that gait optimality is determined jointly by terrain, performance metric, and velocity. We use deep reinforcement learning and networks of abstract oscillators, modeling spinal central pattern generators, to learn 11 quadrupedal gaits across diverse challenging terrains, together with the locomotion style (base height, foot ground clearance, foot offsets) that each requires. A second policy then selects among them to optimize a user-specified criterion such as cost of transport, base stability, or joint smoothness. Across these performance metrics and within this repertoire, every gait is optimal under at least one terrain-metric-velocity combination. The framework adapts zero-shot to gait couplings and styles never seen during training, to terrains far beyond the training distribution, and to larger quadrupeds five times the training mass. With a single additional pitch-tracking reward term, the same gait couplings also support bipedal-mode locomotion. Our method provides a general and systematic way to produce expressive locomotion and to select the gait best suited to a given terrain and criterion. Project page: https://guillaume10.github.io/terrain-gaits.

Fatima Atwi (LAAS-GEPETTO, Nantes Univ - ECN), Constant Roux (LAAS-GEPETTO), Christian Bergaud (LAAS-MEMS), Olivier Stasse (LAAS-GEPETTO)
总结: 比较不同生物合理程度的脉冲网络能否控制人形行走。
方法: 依次测试冻结储备池、奖励调制可塑性与时间反向传播。
证据: 三种方案都在H1-2上测试,摘要未给出性能高低。
为什么适合我: 仅边缘相关,重点是生物脉冲网络而非全身跟踪或操作。
推荐理由: 虽在H1-2上做行走,核心是生物可解释脉冲网络,而非强化学习全身控制或模仿。
原摘要

Artificial Spiking Neural Networks have gained remarkable attention in the last few years due to their proven ability in simulating biological neural cultures and producing similar activity and trends. Nevertheless, whether they can be trained for humanoid locomotion while retaining biological plausibility and robust performance at once remains an open question. We study this problem by proposing a biologically inspired spiking network, trained by different learning schemes and tested on the H1-2 Unitree Humanoid. These schemes differ in how recurrent weights are trained. Ordered from most to least biologically faithful, they are: a fixed reservoir mimicking an in-vitro neural culture, where recurrent weights are kept frozen and only a linear readout is trained, the same network but with its synapses trained by reward-modulated spike timing dependent plasticity and eligibility propagation, and finally backpropagation through time as a non biological baseline for comparison. All schemes relied on distillation of a reinforcement learning expert policy with dataset aggregation. We show that e-prop reached 100% survival rate matching backpropagation through time, whereas both the fixed reservoir and the reward-modulated STDP, though the closest to biological cultures, plateaued at a much lower performance (~43.7% and ~11% survival). We explore in further detail which trends a real culture could and could not produce until now. Finally we demonstrate brief bipedal stepping on a full size humanoid H1-2.