Papers for 2026-11-18

10 papers

MoViC: Real-Time Humanoid Whole-Body Control from Monocular Video Streams

5.0/5 很相关 裁判分 10.0 Top pick 当日相对
Yimin Han, Jiahui Zhang, Wanyue Li, ... , Annan Tang, Peng Lu
总结: 用实时单目视频驱动人形全身模仿与无设备遥操作。
方法: 结合人体重建、重定向和频域上下文的Transformer策略。
证据: 硬件展示高动态跟踪,以及双臂和单臂操作遥操作。
为什么适合我: 契合视觉全身跟踪、人体到机器人重定向与遥操作。
推荐理由: 直接对应人形全身控制:单目视频人体运动重建、机器人重定向与实时全身跟踪,并含动态硬件部署。
原摘要

Real-time whole-body imitation from monocular video requires both a robust universal tracker and high-quality real-time reconstruction. We present MoViC, a framework that uses live RGB video as a control signal for humanoid robots without wearable tracking devices. The framework combines monocular human motion reconstruction, robot retargeting, and a transformer-based behavior foundation model. The policy receives proprioceptive history, a single explicit next-step reference, and a compact frequency-domain motion representation that supplies temporal context without explicitly stacking future reference frames, which reduces the input dimension and network size. During deployment, our extrapolation method bridges the 10-Hz visual estimation with 50-Hz controller, enabling asynchronous visual processing and policy execution. Hardware experiments further demonstrate highly dynamic motion tracking and device-free teleoperation of bimanual and single-arm manipulation tasks from live monocular RGB input. Project page: https://movicproject.github.io/ .

Nico Bohlinger, Jan Peters
总结: 一分钟内学会人形动作跟踪与崎岖地形行走。
方法: 世界模型短程想象配合解析梯度,并多卡并行训练。
证据: 四卡快速跟踪全部片段,训练速度较既有方法大幅提升。
为什么适合我: 直接服务仿真强化学习下的全身跟踪与地形运动。
推荐理由: 核心是人形运动跟踪与粗糙地形运动学习,属于全身运动跟踪和足式运动。
原摘要

We present XMimic and XSoma, which learn humanoid motion tracking and locomotion on rough terrain in under a minute of training. Both follow SuperMimic, which learns a world model of the robot and optimizes the policy with analytic gradients through short rollouts imagined in this model. Existing implementations of this method do not make use of larger GPUs, as every training iteration is a long chain of small GPU operations that run one after another. XMimic and XSoma therefore shorten this chain and run the steps of every iteration at the same time on up to four GPUs. We compile the complete training loop with JAX, including the MuJoCo Warp simulator. For locomotion, XSoma leaves only the dynamics of the robot to the world model and computes everything else exactly, including the terrain height under the feet. It learns with a horizon of one second and without a critic. On four H100 GPUs, XMimic learns to track all 40 LAFAN1 clips with the Unitree G1 in 0.86 minutes, 3.7 times faster than BlinkMimic on one GPU. XSoma reaches the tracking target that Holosoma's tests set for FastSAC in 0.77 minutes, 12.3 times faster than FastSAC on one GPU, while needing only an eighth of its environment steps.

Murad Dawood, Oezge Drama Berkenkamp, Antonio Gonzales Marin, Nutan Chen
总结: 速度指令条件下复现示范风格的人形全身控制。
方法: 用全身跟踪奖励编码风格,执行器不接收参考关节轨迹。
证据: 风格编码进策略,运行时无需未来参考或教师蒸馏。
为什么适合我: 兼顾可部署全身控制与运动示范的风格模仿。
推荐理由: 由动作示范学习可部署的速度条件化人形全身控制,衔接对抗风格模仿与全身跟踪。
原摘要

Achieving human-like behavior on humanoid robots has advanced rapidly by leveraging motion-capture demonstrations rather than hand-crafted cost functions. Adversarial imitation supplies a style signal that lets a policy fulfill a task while adhering to a desired style, but is difficult to transfer to hardware. Whole-body tracking through reinforcement learning instead deploys reliably and imitates demonstrations faithfully, but the resulting policies are conditioned on the future reference, so they require a reference trajectory at runtime and can fail when the motion differs from the training data. We train a humanoid policy directly with whole-body tracking rewards while restricting its actor to compact locomotion commands and robot observations. Reference motion supervises the reward and privileged critic, without supplying reference joint trajectories to the actor or requiring teacher distillation. The demonstrated style is thereby encoded in the policy through the tracking reward, so the robot reproduces the whole-body style under commanded velocities. Trained on a single random-walk motion and commanded with velocity profiles from held-out motions, our policy reaches at least 98\% success in simulation and completes every trial on the Unitree G1. On held-out slow walking, whole-body tracking drops to 2\% success and command-conditioned distillation to 56\%, and both fail on the real robot. On the trained motions, the joint position error of our policy stays within 8\% of whole-body tracking, and it has a higher style coverage than adversarial motion priors (AMP), which fell on the real robot without a safety crane.

QTOP: Scaling Humanoid Motion Tracking with Quantile Critics

5.0/5 很相关 裁判分 10.0 Top pick 当日相对
Markus Peschl, Dana Kianfar, Daniel Dijkman
总结: 分位数评论家提升人形运动跟踪的样本效率。
方法: 按状态与参考估计分布优势,减轻多样动作学习干扰。
证据: 仿真和真机跟踪最多提升约一成,并呈现可预测缩放。
为什么适合我: 针对大规模动作模仿中的跟踪泛化与样本效率。
推荐理由: 针对人形运动跟踪策略的样本效率与扩展,直接命中全身运动跟踪。
原摘要

Learning human motion tracking policies through reinforcement learning (RL) has emerged as a core capability that underpins a majority of recent progress in humanoid whole-body control. However, achieving generalization to unseen motions at high fidelity requires jointly scaling training to thousands of GPU and dataset hours. This work hypothesizes that motion tracking is inherently compute-bound because training on a diverse set of motions introduces learning interference, similar to multi-task RL. To tackle this, we propose Quantile Tracking Optimization (QTOP), which improves sample efficiency by introducing a distributional advantage estimate conditioned on state and reference pairs. QTOP outperforms on and off-policy RL baselines both in simulation and real-world tracking scenarios on multiple datasets by up to 10\%. Furthermore, we establish a scaling law for QTOP and find that, aside from scaling predictably, dataset size has minimal impact on performance for moderately sized GPU hour regimes.

Chen Liu, Nikolaus Correll
总结: 把第一视角人类演示转成物理可行的机器人训练轨迹。
方法: 智能体重建场景与接触带,再规划抓取并适配机器人。
证据: 无需机器人遥操作或穿戴全身追踪,即可得到可行轨迹。
为什么适合我: 服务人体到机器人重定向及无机器人演示的移动操作。
推荐理由: 把自我中心人体示范转成可行的人形移动操作轨迹,覆盖运动适配、real-to-sim与无机器人示范学习。
原摘要

Imitation learning for visuomotor policies typically relies on teleoperated robot demonstrations, which are costly because every episode requires a trained operator controlling a physical robot in real time through teleoperation interfaces and wearable body-tracking hardware. Egocentric glasses capture human demonstrations without a robot or handheld grippers, yet they track only the head and in-view human hands, and directly copying human motion to a robot with different stature, arm reach and hand morphology fails to produce feasible contacts. \method{} converts egocentric glasses recordings into physically verified robot training trajectories. An agentic real-to-sim stage reconstructs collision and textured visual assets from the recorded RGB-D and head streams, completes unseen surfaces when needed, and predicts target contact bands on the manipulated objects. During motion adaptation, a grasp planner turns each contact band into a feasible robot wrist pose and finger closure, bilevel stance optimization plans collision-free standing postures that keep hand targets reachable, and a whole-body motion generator synthesizes lower-body locomotion and out-of-view arm motion. Two-stage motion correction then blends visibility transitions kinematically and trains a reinforcement-learning tracking policy in physics simulation under agent-designed semantic rewards, completing manipulation phases even when the object leaves the camera view. Rendering the reconstructed scene through the robot's onboard camera along the optimized rollout yields synchronized robot-view images, proprioception and verified action labels. Fine-tuned exclusively on this converted dataset, without real-robot demonstrations, GR00T N1.7 performed loco-manipulation tasks autonomously on a Unitree G1. Project page: https://robonana.github.io/ego2sim2humanoid/

Leaning Over to Reach Far-Away Objects

5.0/5 很相关 裁判分 8.0 Strong 当日相对
Allen Devaraj, William Xie, Nikolaus Correll
总结: 借助环境支撑扩大人形伸手范围并完成恢复。
方法: 局部感知下用采样式全身模型预测控制完成撑靠伸手。
证据: 笔记本实时迁移到真机,多数试验到达并完成恢复。
为什么适合我: 正好对应全身模型预测控制、移动操作和仿真到现实。
推荐理由: 用人形全身MPC完成支撑、前倾、伸够与恢复,并做仿真到现实,对应全身模型预测控制与移动操作。
原摘要

We study supported reaching to extend a humanoid robot's workspace by bracing against the environment for stabilization using a straightforward yet capable whole-body model predictive control (MPC) approach, exclusively with local sensing and fiducial markers. While typical solutions for whole body humanoid control aim at avoiding or accommodating contact, contact is explicitly required to reach objects placed on surfaces far out of normal reach. We employ sampling-based MPC for whole-body leaning, bracing, reaching, and recovery on a Unitree H1-2 humanoid. The controller runs in real-time at 34 Hz on a laptop, transfers sim-to-real, extends reach by about 50 cm beyond the unbraced standing workspace, and sustains the loaded posture for over four minutes, before pose estimation drifts. The controller reaches the commanded target in 89% of trials and completes the full reach-and-recovery cycle in 81%, across 54 trials spanning six targets and three table heights. We further characterize why such a small real-time sampling budget is sufficient, finding that temporal persistence of sampled actions and exploration scale strongly govern success, while CEM provides the widest operating region over planning rate and model mismatch. These results establish a real-world baseline for online, contact-rich humanoid manipulation. Code, videos, and full hardware details are available at https://h12leaner.github.io/

Yiting Chen, Kelin Yu, Sarthak Kaingade, ... , Nima Fazeli, X. Alice Wu
总结: 触觉驱动在线适应,实现零样本强力人形操作迁移。
方法: 以锥形接触表征和触觉历史调节上肢,并配合行走策略。
证据: 以旋钮开门检验摩擦抓取、力矩与全身稳定等要求。
为什么适合我: 关注接触力主导的人形移动操作与仿真到现实。
推荐理由: 面向人形全身力交互操作和零样本仿真到现实,贴近人形移动操作,但重点是触觉抓取而非运动跟踪。
原摘要

Recent humanoid loco-manipulation in articulated environments has largely focused on geometry-dominant interactions, where success is primarily determined by geometric alignment and motion execution. In contrast, forceful multi-finger manipulation requires the robot to adapt to real-world uncertainties in contact and interaction dynamics. We present Tactile-Driven Motor Adaptation (TMA) and an integrated whole-body system for zero-shot sim-to-real forceful humanoid manipulation, and study round-doorknob opening as a challenging benchmark requiring stable multi-finger frictional grasping, sufficient torque generation without slip, force-regulated pushing, and whole-body stability. TMA structures policy learning around interactions observable through tactile feedback, uses cone geometry as an inductive bias for multi-finger contact representation, and adapts motor behavior online from tactile history. An upper-body manipulation policy is trained alongside a pretrained general locomotion controller to coordinate dexterous interaction and whole-body stabilization. Trained entirely in simulation, TMA transfers zero-shot to a Unitree G1 humanoid with BrainCo Revo2 multi-fingered hands and substantially outperforms standard Rapid Motor Adaptation (RMA) and reactive tactile policy baselines across four commercial round doorknobs in the real world.

Lei Zheng, Shivam Sood, Mithun Vanniasinghe, Guillaume Sartoretti, Armin Lederer
总结: 恢复引导的短时域扩散规划提升高速足式安全性。
方法: 用终端恢复评论家评估最坏间隙,并保留可行精英。
证据: 成对随机避障试验零碰撞,严格间隙成功率很高。
为什么适合我: 服务高动态足式运动中的安全预见与在线规划。
推荐理由: 针对高速足式避障运动的恢复引导扩散规划,贴近高动态足式运动,但是四足安全规划而非人形全身跟踪。
原摘要

Safety-critical high-speed legged locomotion requires motion planners to balance safety foresight with online computational efficiency. Although short prediction horizons reduce rollout costs, they provide limited safety foresight, and strict safety constraints severely restrict the feasible trajectory space. We propose a recovery-guided model-based diffusion (MBD) framework for safety-critical legged locomotion. It incorporates a learned terminal recovery critic into short-horizon trajectory optimization. Specifically, the critic approximates the worst-case future clearance margin along rollouts of a designated recovery policy. We introduce a feasibility-guided refinement strategy that combines within-horizon constraints and terminal recoverability at each sampling stage, while retaining feasible elites to mitigate sampling-support loss. Across 50 paired randomized Unitree Go2 obstacle-avoidance trials, our approach achieves zero collisions and 98% strict-clearance success, while requiring substantially less computation than long-horizon MBD baselines. A project page is available at https://zack4417.github.io/rg-mbd/.

Matthias Jammot, Pembe Gizem \"Ozdil
总结: 把可学习阻抗作为动作空间,同时提高运动与力保真。
方法: 策略调节各关节刚度、阻尼和位置,推理时只需运动。
证据: 多名受试者六类活动中,运动与力均优于固定增益控制。
为什么适合我: 强化物理仿真中力准确的动作模仿与接触控制。
推荐理由: 用强化学习做力一致的物理动作模仿,接近物理角色控制与模仿学习,但更偏人体/角色而非机器人全身部署。
原摘要

Recovering the forces behind a motion is ill-posed: different controllers can reproduce the same movement while generating different contact forces. Physics-based motion imitation resolves this ambiguity through simulation, but relies on fixed-gain proportional-derivative (PD) control. We show that fixed gains impose a trade-off between motion and force tracking, as contact and swing place conflicting demands on joint stiffness and damping. We introduce ImpedanceMimic, a reinforcement-learning policy that modulates per-joint stiffness, damping, and position setpoints from motion and force measurements, but requires only motion at inference. Across 129 participants and six activities, ImpedanceMimic outperforms matched fixed-gain PD under every integration scheme, direct-torque, muscle-actuated and existing imitation controllers in kinematic and force fidelity, approaching a dedicated motion-to-force regressor. By expressing gains as multiples of inertia-scaled priors, a single policy controls subject-specific biomechanical models without explicit anthropometric input and transfers zero-shot to unseen humans and a humanoid robot. The learned impedance forms a low-dimensional representation of motor state, shared across bodies, that generalizes to withheld activities: stiffness scales with load, and damping anticipates unloading.

Hongyu Ma, Hairong Qu, Shiqi Zhao, ... , Xin You, Peng Yin
总结: 野外度量级手部重建,服务灵巧遥操作与策略学习。
方法: 结合几何时序标注、鱼眼立体网络、蒸馏与低成本采集。
证据: 提供千小时训练标注,并用留出人工与独立光学评测。
为什么适合我: 为灵巧遥操作和模仿学习提供大规模野外手部运动。
推荐理由: 服务于灵巧遥操作的手部重建,仅弱相关遥操作;核心是手部度量重建并使用视觉惯性SLAM,偏离全身控制。
原摘要

Human manipulation data are widely used to train robots for human-like dexterity. However, low-cost, large-scale collection in the wild remains challenging. Existing methods remain limited in geometrically grounded metric-scale reconstruction, real-world generalization and inference speed. We present WinDEX, integrating (1) geometric-temporal optimization for metric 3D hand annotation from in-the-wild egocentric stereo videos, combined with visual-inertial SLAM for world-frame reconstruction; (2) WinDEX-Net, an end-to-end, fisheye-native network combining calibrated stereo, local detail and temporal context, with a data-model flywheel of model-assisted annotation and retraining; (3) distilled models for offline reconstruction and causal streaming, supporting reinforcement learning and teleoperation; and (4) low-cost CyberCap stereo-inertial capture, forming the WinDEX-Hand dataset with 1,000 hours of training annotations and held-out human-annotated in-the-wild 2D and independent optical motion-capture evaluation sets. Offline reconstruction runs at 43.61 FPS and causal streaming at 171 FPS. On held-out data, offline reconstruction achieves state-of-the-art accuracy: 14.5 pixels in-the-wild reprojection error and, with visual-inertial odometry, 37.29 mm world-frame joint error. We achieve 7.6 and 3.6 times the task success rates of reinforcement learning policies trained with WiLoR and HaWoR motion references, respectively, and demonstrate real-time arm-hand teleoperation. Project website: https://fruitrabbit.github.io/WinDEX/.