Papers for 2026-11-05

10 papers

HLO-AMP: Hierarchical Loco-Manipulation from Object-Aware Motion Priors

5.0/5 很相关 裁判分 10.0 Top pick 当日相对
Jean-Pierre Sleiman, Bilal Hammoud, He Li, ... , Jessica K. Hodgins, Farbod Farshidian
总结: 把低层控制器训成带物体的动作先验,使人形移动操作更自然且能承受真实载荷。
方法: 用重定向动捕做模仿学习,场景中放入被操作物体,覆盖下游任务的核心动作基元。
证据: 摘要仅说明物体在场的模仿可继承类人协调,未给出定量结果。
为什么适合我: 直接对应人形移动操作、动捕重定向与物理仿真中的动作模仿。
推荐理由: 直接对应人形移动操作、动作模仿与人体运动重定向:用含物体的重定向动捕训练物体感知运动先验,作为全身低层控制器。
原摘要

Hierarchical pipelines for humanoid loco-manipulation typically pair a task-agnostic whole-body controller (WBC) with a downstream task policy. Such WBCs are often trained through manual reward shaping, which yields visibly machine-like motions; more consequentially, they are trained without the manipulated object in the scene. Object dynamics are instead approximated through randomized end-effector force disturbances, a proxy that fails to reproduce the distributed multi-contact wrench a real payload applies across the arms and torso, with the gap becoming critical near the platform's payload limit. In this work, we train the low-level (LL) controller itself as an object-aware motion prior: a single policy trained via motion imitation on retargeted motion-capture (MoCap) references that span the core motion primitives of the downstream task, with the manipulated object physically present in the scene. This single stage inherits natural human-like coordination from the references and absorbs the realistic contact wrench at its source, while exposing the same compact task-space command interface used by command-tracking WBCs. A reference-free high-level (HL) policy is then trained on top, with its action channel anchored to the empirical envelope of the low-level training data so that it generalizes the priors to a long-horizon multi-stage task while structurally preventing queries that would push the low level off-distribution. We instantiate the framework on a Unitree G1 for bimanual box pick-and-place: the same deployed policy generalizes across object masses and dimensions, initial and goal poses, and reactively recovers from intentional drops mid-carry. Video: https://youtu.be/viR_ULnRDQQ.

Ju Dong, Xinyu Xiao, Yiming Zhou, ... , Ping Luo, Jianwei Zhang
总结: 用人形动作条件潜空间预见,在执行前筛选不易掉物或交互失败的指令序列。
方法: 动作生成器出关节与手部指令,终点预测器估视觉进展,时间先验评一致性后滚动重规划。
证据: 训练用动作一致性正则,从观测与预测特征变化重建首条示范指令,摘要未给成功率。
为什么适合我: 贴合人形移动操作与视觉条件下的全身协调,但偏潜空间规划而非跟踪控制。
推荐理由: 聚焦人形loco-manipulation中身体与手部的时序协调和执行前前瞻,最接近人形移动操作与全身控制。
原摘要

Humanoid loco-manipulation requires coordinated body motion and hand interaction over time, and a plausible-looking command sequence can still drop a held object or leave an interaction incomplete. We present action-conditioned latent foresight, a latent world-action model (WAM) that evaluates proposed command sequences before execution. An action generator proposes joint motion-hand command sequences from vision, language, and the robot state. An action-conditioned endpoint predictor estimates the visual features each sequence prefix reaches and measures progress toward a demonstration-derived subgoal, while a joint temporal prior scores temporal consistency given the robot state and preceding commands. The robot executes the first commands of the best-scoring sequence and replans from the next observation. During training, action consistency regularization reconstructs the first demonstrated command from observed and predicted feature changes, encouraging the predictor to retain action-relevant information. On ten tasks in the humanoid loco-manipulation simulation benchmark SIMPLE, our method achieves 89.7% average success, compared with 73.7% for Psi-Zero and 82.0% for DiT4DiT. On seven real-robot tasks, it achieves 85.7% under standard conditions and 65.7% under unseen object configurations, while GR00T N1.7 achieves 70.0% and 48.6%, respectively. Project page: https://ju6276.github.io/Action-Grounded-Foresight/

Online iLQR-MPC with Whole-Body Control for Robust Humanoid Locomotion

5.0/5 很相关 裁判分 9.0 Top pick 当日相对
Xiang Li, Linlin Ou, Ruihan Wu, Xinyi Yu
总结: 在线iLQR模型预测控制同时改脚步与接触力,再由全身控制器输出关节力矩。
方法: 单刚体模型保留姿态与完整接触扳手,增广拉格朗日与松弛障碍把约束纳入后向传递。
证据: 摘要说明全身控制以五百赫兹映射力矩并保持接触可行,扰动响应可由显式代价与约束解释。
为什么适合我: 正中人形全身模型预测控制与接触力分配,且无需平台特定策略训练。
推荐理由: 在线联合优化落足与接触力,再由全身控制器跟踪,直接对应人形运动与全身模型预测控制。
原摘要

Humanoid locomotion requires deciding where to step and how to distribute contact forces. However, a disturbance can invalidate the nominal plan, requiring coordinated revision under constraints that switch with every footfall. To solve this online, the proposed approach employs the single-rigid-body model, which retains attitude dynamics and the full contact wrench. An iterative linear-quadratic-regulator MPC (iLQR-MPC) then jointly optimizes contact forces and footstep locations in a single receding-horizon problem, without a separate step planner. Specifically, contact and motion constraints are incorporated into the backward pass through augmented-Lagrangian and relaxed-barrier penalties. A whole-body controller subsequently maps the plan to joint torques at 500 Hz while preserving contact feasibility. The model-based formulation makes disturbance responses interpretable through explicit costs and constraints, and requires no platform-specific policy training. In MuJoCo simulations, the iLQR-MPC replans at 100 Hz (mean solve time 8.9 ms); the Unitree G1 sustains steady-state walking and recovers from impulsive 245 N forward and 175 N lateral pushes.

Mingyu Wang, Si Li, Ziqing Chen, He Wang, Li Yi
总结: 用十毫秒内的全身触觉反射补上低频策略延迟,使人形在撞击前撤让并恢复。
方法: 反射通路由全身触觉求撤退姿态并直达电机,再经在环学习、透明对齐与自适应维持平衡。
证据: 摘要称已部署到宇树G1,针对五十赫兹策略延迟和误把撤退当作跟踪误差。
为什么适合我: 关乎人形全身控制与真实接触安全,但是触觉反射而非运动跟踪或跑酷。
推荐理由: 把瞬时触觉反射接入学习式人形全身控制,最接近人形机器人全身控制;侧重接触反射而非运动跟踪或移动操作。
原摘要

Humanoid robots operating in cluttered environments must react to unexpected contacts before the first impact peak, which arrives within tens of milliseconds. Learned whole-body policies running at 50 Hz introduce a policy-cycle delay and may suppress a protective withdrawal as a tracking error. To address these challenges, we propose ARC (Ascending Reflex Control), a bottom-up framework that integrates instantaneous tactile reflexes with learned whole-body control. ARC consists of two components: an instantaneous reflex pathway and bottom-up reflex integration. The reflex pathway detects hazardous contacts from full-body tactile sensing, solves a whole-body withdrawal pose, and delivers it directly to the motors within 10 ms. Bottom-up reflex integration combines reflex-in-the-loop policy learning, reflex-transparent policy alignment, and reflex-informed policy adaptation to sustain the withdrawal, maintain balance, and complete longer-horizon recovery. We deploy ARC on a Unitree G1 and evaluate three categories of hazardous contact. Compared with an otherwise identical reflex delayed by 20 ms, ARC reduces the first force peak by 34.1% and reduces the contact impulse from 0.2632 N s to 0.1225 N s (a 2.15x ratio). We further demonstrate protected motion tracking, teleoperation, and vision-free obstacle traversal guided by foot tactile sensing. Project page: https://minusmodulo.github.io/reflexarc-website/

Seungyong Lee, Suhyeok Jang, Minyoung Kim, Jinwoo Shin
总结: 把人类视频虚拟试穿成目标机器人视频,从而合成高自由度灵巧操作示范。
方法: 自回归翻译关键帧并引导视频合成,以人手骨架约束运动,再用逆动力学模型得到动作。
证据: 摘要强调保留原任务、物体与背景,并缩小高自由度机体的视觉和动作差距,无定量结果。
为什么适合我: 服务于人体到机器人的示范迁移,可减少遥操作,但不是全身运动跟踪本身。
推荐理由: 从人类视频合成灵巧机器人演示,只在人体到机器人迁移上弱相关,不是人形全身跟踪或运动重定向控制。
原摘要

Robot foundation models have shown remarkable performance in robot control, yet adapting them to a new task still relies heavily on teleoperated demonstrations. Egocentric human videos offer a scalable substitute, but overlaying rendered robots on human videos or retargeting human poses into robot actions leaves significant visual or action gaps for high-DoF robot embodiments. To address this limitation, we propose RoboTryOn, which treats human-to-robot translation as virtual try-on: try-on keeps the person and swaps the garment, RoboTryOn keeps the demonstrated motion and swaps the human for the robot. Specifically, RoboTryOn converts a human video into a video of the target robot specified by a reference image, while preserving the original task, objects, and background. To maintain visual consistency, we translate selected keyframes autoregressively and use them to guide video synthesis with motion conditioned on the human hand skeleton. Additionally, we use an inverse dynamics model (IDM) that incorporates depth features to infer action labels from synthesized videos for policy learning. Our experiments across multiple humanoid embodiments show that policies trained with our synthetic data improve the average success rate on novel tasks, including challenging bimanual manipulation, from 2% to 60%, without any real robot demonstration of those tasks. Moreover, training with our synthetic data matches the performance of policies trained solely on teleoperated demonstrations at approximately a third of the collection cost, demonstrating the cost efficiency of our approach.

Abhijeet M. Kulkarni, Saimouli Katragadda, Yuze Du, Zijun Sha, Guoquan Huang
总结: 用屏障认证的安全层约束足式视觉语言导航,补上端到端规划缺少的物理落地。
方法: 高层视觉语言动作规划器配合状态估计、现场三维感知和符号距离场,生成可执行轨迹。
证据: 摘要称在所述假设下导航规划具有可证明的避碰保证,未报告具体成功率。
为什么适合我: 涉及足式机器人与三维空间感知,但重点是导航安全而非全身控制或动作模仿。
推荐理由: 足式视觉语言导航的安全约束与空间感知,只擦边感知型足式运动,核心是导航安全而非运动控制。
原摘要

By harnessing the inherent commonsense capabilities of pretrained Large Language Models (LLMs) and Vision-Language Models (VLMs), current Vision-Language Action(VLA) models for legged robots excel at interpreting language-based navigation commands but lack the physical grounding and explicit safety assurance required for real-world operation. Therefore, this paper aims to cage Legged-Vision-Language Navigation (VLN) through a safety architecture that provides physical grounding and ensures navigation safety. The key idea is not to abandon the powerful capabilities of end-to-end VLA planning, but rather to augment and cage them. In particular, we integrate a high-level VLA planner with a low-level physical-grounding layer that leverages efficient state estimation, in-situ 3D spatial perception, and an Signed Distance Field (SDF) representation to generate executable trajectories. We provide a navigation planner with provable collision-avoidance guarantees under the stated assumptions by combining a low-dimensional Bezier-parameterized MPPI planner with Discrete Exponential Barrier Function (dEBF) constraints. The proposed Legged-VLN system is modular, making it compatible with various VLM/VLA models and agnostic to the robot platform. The proposed system is validated with extensive real-world experiments on humanoid and quadrupedal robots, demonstrating versatile, efficient, and safe vision-language driven navigation capabilities.

Zeyu Gao, Chengkai Hou, Chaoyue Li, ... , Li Zhang, Shanghang Zhang
总结: 在噪声空间学习保守方向残差,在线改进冻结的生成式机器人策略。
方法: 围绕高斯潜变量学习依赖状态和噪声的方向,用软保守约束调节适应并取消固定可行域。
证据: 摘要对比固定潜空间区域与动作空间残差,但未给出实验数字或具体机器人平台。
为什么适合我: 只一般地改进强化学习策略先验,与人形全身跟踪、重定向和移动操作关联较弱。
推荐理由: 冻结生成策略上的残差强化学习方法,未涉及人形/足式全身控制、运动跟踪或移动操作。
原摘要

Pretrained generative policies provide strong behavior priors for robot control, but adapting them efficiently with online reinforcement learning remains difficult. A central challenge is how to improve a frozen generative policy without destroying the structure learned from offline demonstrations. Noise-space residual methods address this by modifying the policy's input noise, but existing noise-space methods typically regulate adaptation through fixed latent feasible regions, which may exclude valuable noise edits. In contrast, action-space residual methods are more sample-efficient, but their efficiency can still be further improved. We propose Conservative Noise Directional Optimization (CNDO), a noise-space residual RL method that improves a frozen generative policy by learning state- and noise-conditioned directions around Gaussian latent samples. CNDO removes predefined latent feasible regions and instead regulates noise-space adaptation through the soft conservative constraint, which consists of conservative value estimation and gated execution. We further introduce Conservative Action-Noise Directional Optimization (CANDO), which adds an action-space residual, combining structured latent-space improvement with local action correction. Experiments show that, once CNDO narrows the gap between noise-space and action-space residual learning, CANDO further improves over single-space optimization, achieving 76.1 NAUC and a 91.4% final success rate, surpassing the best single-space results by 6.6 NAUC and 4.1% success rate. More demos and info are available on https://cando-rl.github.io/.

Yueh-Hua Wu, Kei Ota, Hanlong Li, ... , Yuma Shin, Zixuan Zhong
总结: 仅用手持夹爪示范训练双手操作,测试时才上机器人,完成长时程精细跨本体任务。
方法: 用开源手持夹爪随处采集,仅靠双腕相机,不做遥操作、强化学习和机上修正。
证据: 十项任务平均成功率百分之七十八,高于同样任务后训练的四种视觉语言动作模型。
为什么适合我: 可替代遥操作数据采集,但对象是固定双手操作,不是人形足式全身运动。
推荐理由: 手持数据学习的跨本体双臂长时程操作,属于通用操作,不是人形全身控制或足式运动。
原摘要

AIRoA-0 (A-0) is a bimanual manipulation system learned from 16,151 hours of handheld demonstrations, with no robot data collected at any stage. The demonstrations are recorded with YUBI, our open-source handheld gripper, which lets people record manipulation data anywhere with no robot present. From a single instruction, A-0 sorts more than a dozen small parts into their numbered cells and changes the mop pads and brush of a robot vacuum; among other tasks, it unzips a pencil case and folds a towel. It performs these tasks using only a camera on each wrist, with no head or scene camera, at a site where none of its training data was recorded. Across the ten tasks of our evaluation, it averages a success rate of 78%, against 27% for the strongest of four other vision-language-action models post-trained on the same task demonstrations. We collect no teleoperation data and use no reinforcement learning or on-robot corrections, and the robot appears only at test time. The system learns which regions of each wrist view matter for the task, predicts the stage the task has reached, and decodes actions as a continuous stream. A-0 is embodiment-agnostic: it runs on robots that mount the YUBI gripper, such as a bimanual Franka, TX-G2, OpenArm and UR. We are releasing the dataset, to our knowledge the largest open dataset of handheld-gripper demonstrations with task and subtask instructions, together with the pre-trained weights and the post-training recipe. Videos of A-0 on every task are on the project page, https://a-0.airoa.io. Anyone can build the gripper, record a task of their own and post-train the model on it in under four hours on a single node of eight H200 GPUs.

Jeremy Collins, Namra Patel, Ayush Agarwal, Shitij Govil, Animesh Garg
总结: 把三维场景流当作视频预训练到机器人动作的中间表示,提高动作预测样本效率。
方法: 五十亿视频生成模型微调以预测场景流和动作词元,并联合人类视频与遥操作数据训练。
证据: 摘要称可泛化到未见物体、动作基元和任务,改为视频预测会降低分布外泛化。
为什么适合我: 连接人类视频、遥操作与动作学习,但对人形全身跟踪和足式高动态运动不直接。
推荐理由: 用三维场景流把人类视频先验迁到通用机器人动作,不是人形运动重定向或全身运动跟踪。
原摘要

Pretrained video generation models distill internet-scale motion priors into robot policies, but full video is a redundant signal for action prediction. We argue that dense 3D motion is the medium through which video pretraining best transfers to robot actions, and that predicting motion rather than video affords better sample-efficiency for action prediction. FloMo is a 5B video-generation model fine-tuned to predict 3D scene-flow and action tokens. Rendering scene flow as RGB and encoding it into the same latent space as video enables a pretrained video generation backbone to efficiently adapt to predict 3D scene flow. Co-trained on action-free human video and action-labeled robot teleoperation data, our model benefits from human-video pretraining and generalizes to unseen objects, motion primitives, and tasks. Replacing motion prediction with a video prediction objective degrades out-of-distribution generalization, while adding a video prediction target alongside motion lowers policy success rather than improving it. When motion, video, and action are predicted jointly, action tokens attend far more to motion than to video, suggesting that predicting video wastes model capacity. Together, these results indicate that motion effectively mediates the transfer from video pretraining to action. More information can be found at flomo-wam.github.io.

Haitham El-Hussieny
总结: 用移动时域估计和带约束模型预测控制,在未知扰动下平衡轮腿机器人。
方法: 虚拟腿模型增广髋部水平力、俯仰力矩和陀螺零偏,冲击时加大过程噪声并做无偏置约束控制。
证据: 仿真含噪声、零偏、量化与延迟,并对比真值、互补滤波及同一估计下的线性二次调节。
为什么适合我: 符合足式平衡与模型预测控制兴趣,但是轮腿矢状平衡,不是全身运动模仿。
推荐理由: 轮腿机器人两轮平衡的估计与MPC,平台落在不感兴趣的轮式移动机器人,而非人形/足式全身控制。
原摘要

Wheel-legged robots balance on two wheels while using articulated legs to change height, absorb impacts, and cross uneven terrain. On hardware, the balance controller must rely on noisy inertial and encoder measurements, and it must reject disturbances that are not known in advance, such as payloads and wind, without exceeding motor torque limits. This paper presents a moving horizon estimator (MHE) combined with an offset-free, constrained model predictive controller (MPC) for the sagittal balance of a 9.95 kg wheel-legged robot. The estimator uses a virtual-leg model augmented with a horizontal hip force, a body pitch torque, and a gyroscope bias. Its accelerometer model removes the predicted hip acceleration, and its process noise is increased when an impact is detected. We evaluate the approach in MuJoCo with simulated sensor noise, bias, quantization, and delay, and compare it with LQR baselines that use ground-truth state, a complementary filter, or the same MHE. In a driving, squatting, reversing, and bowing sequence, MPC with MHE reduced the RMS speed error to 0.11 m/s, compared with 0.24 m/s for LQR with ground-truth state. Under an unknown 2 kg payload and a 10 N headwind, the mean position error in wind was 6 cm, compared with 81 cm for LQR with ground-truth state. Across a 3 $\times$ 4 grid of payloads and headwinds, LQR with the same estimator fell in 3 of 12 combinations, compared with 1 of 12 for MPC. These results are limited to simulation on flat ground, and experimental validation on hardware remains future work.