Papers for 2026-10-12

10 papers
Ikjun Choi, Geonho Leem, Jungdam Won
总结: GPU原生系统,秒级学习物理角色技能。
方法: 统一流水线整合仿真、环境、策略推理与更新于GPU。
证据: 解决端到端训练硬件利用率低问题,加速模仿学习。
为什么适合我: 加速强化学习与模仿学习训练,助力敏捷全身运动策略。
推荐理由: 高度契合物理角色控制与模仿学习:面向DeepMimic类物理技能学习,打通仿真与策略训练,是运动先验与全身控制的核心上游。
原摘要

Physics-based character control is a long-standing challenge in computer graphics and robotics, requiring policies that satisfy complex dynamics while producing realistic motion. Recent Deep RL approaches, particularly imitation learning methods such as DeepMimic, have had broad impact beyond animation, influencing robotics by enabling agile and expressive behaviors. While these approaches achieve impressive results, they remain computationally inefficient to train in practice. Despite GPU-accelerated simulation, we find that end-to-end pipelines often underutilize hardware due to overheads outside the physics solver, caused by fragmented GPU kernels and CPU memory access in the critical path. We present InstantMimic, a system that addresses these inefficiencies by making the entire training loop GPU-native. Built on a GPU-native physics backend, our unified pipeline integrates simulation, environment computation, policy inference, and policy updates within a single execution flow. As a result, InstantMimic reduces training time for diverse physics-based skills to a few seconds and makes LLM-agent-driven hyperparameter search practical.

Skinned Motion Retargeting via Artifact-driven Kinematic Prior Refinement

4.0/5 相关 裁判分 10.0 Top pick 当日相对
Seokhyeon Hong, Chaelin Kim, Inseo Jang, Soojin Choi, Junyong Noh
总结: 几何感知运动重定向,显式连接姿态几何伪影与运动精炼。
方法: 学习运动嵌入后,用伪影驱动运动学先验精炼目标几何。
证据: 解决自穿透等目标侧几何伪影,泛化多样骨架与体型。
为什么适合我: 支持人体动作重定向到人形机器人,利于真机全身控制。
推荐理由: 直接对应人体动作重定向:跨骨架、体型的运动迁移,并用几何伪影驱动运动修正,接近可重定向全身动作,但偏蒙皮角色运动学而非真机全身跟踪。
原摘要

Motion retargeting aims to transfer a source motion to target characters with different skeletal structures, proportions, and body shapes. Although recent neural retargeting methods have improved flexibility across diverse skeletons, target-side geometric artifacts such as self-penetration remain difficult to resolve. Specifically, existing geometry-aware approaches often rely on fixed skeleton templates or implicit geometry-conditioned prediction, requiring a single network to account for target geometry deformation, detect target-side artifacts, and predict the corresponding correction from target geometry alone, which limits their ability to generalize across diverse skeleton structures and body shapes. In this paper, we present a geometry-aware motion retargeting framework that explicitly connects artifacts observed in the posed character geometry to motion refinement while preserving the flexibility of skeleton-agnostic neural retargeting. Our method first learns a motion embedding shared across different skeletons using a transformer-based retargeting autoencoder that transfers motion across arbitrary source--target skeleton pairs. Building on this kinematic motion prior, we introduce an artifact-driven refinement module that observes self-penetration on the posed target mesh and converts it into a corrective cue through a motion-to-vertex Jacobian. We further condition motion decoding on target geometry using skinning weight-based joint-aligned geometry features derived from the rest pose mesh. This design combines explicit target-side artifact reasoning with flexible geometry-aware decoding in a unified framework. Experiments on both fixed and arbitrary skeleton structure settings show that our method improves kinematic retargeting accuracy and reduces geometric artifacts, producing plausible motions across seen and unseen target characters.

ReMoMask-2: Latent Retrieval-Augmented Masked Motion Generation

4.0/5 相关 裁判分 3.0 Candidate 当日相对
Yiran Wang, Zeyu Zhang, Ling Shao, Hao Tang
总结: 潜在检索增强掩码运动生成,改进文本到运动。
方法: 层次对比学习对齐特征,拓扑感知融合与结构化掩码。
证据: 解决粗粒度检索与表示间隙,提升复杂描述生成。
为什么适合我: 生成可跟踪人体运动,支持重定向与机器人全身控制。
推荐理由: 文本条件人体运动生成与层次运动先验,接近可模仿的人体动作表征,虽未做到物理跟踪或上真机。
原摘要

Text-to-motion (T2M) generation maps natural language to human joint movements, aiding gaming, VR, and robotics. Retrieval-Augmented Text-to-Motion (RAG-T2M) improves generation on complex descriptions by conditioning on retrieved motion-text pairs. However, existing RAG-T2M models face two challenges: coarse-grained retrieval and fusion mechanisms overlook the hierarchical, spatial-temporal topology of human motion, and a representation gap exists because retrieved evidence resides in a semantic space separate from the generator's latents. To address the first, we present ReMoMask, a structure-aware RAG framework coupling Hierarchical Bidirectional Momentum (HBM) contrastive learning to align global and part-level features with text; Semantic Spatial-Temporal Attention (SSTA) for topology-aware fusion; and Topology Structured Masking (TSM) to force robust part-level grounding via adaptive masking. To address the second, we introduce ReMoMask-2, which rebuilds the retrieval database directly within the generator's pre-quantization latent space and aligns text queries via a distilled lightweight projector, allowing the generator to directly consume the retrieved motion's semantic content. Extensive experiments on HumanML3D, KIT-ML, and SnapMoGen demonstrate our retriever achieves state-of-the-art accuracy, while ReMoMask-2 attains the lowest FID on KIT-ML and SnapMoGen; notably, its single mask-transformer stage surpasses ReMoMask's full two-stage pipeline and delivers the fastest inference.

Takumi Kobayashi, Masato Kobayashi, Yuki Uranishi
总结: 双边控制模仿学习,调节执行速度与接触强度。
方法: 修饰符条件解码,经adaLN-Zero注入约束潜在到Transformer。
证据: 白板擦拭任务中物理指令跟随优于基线。
为什么适合我: 接触丰富场景下力位控制,契合loco-manipulation需求。
推荐理由: 接触丰富模仿学习与力/速度调制相关,但对象是双边控制下的擦拭操作,不是人形或腿足全身loco-manipulation。
原摘要

Bilateral control-based imitation learning captures both position and force information, making it well suited to contact-rich manipulation. However, existing approaches provide limited means for an operator to specify how a learned task should be executed at inference time, such as slowly or quickly, gently or firmly. We propose Bi-MoDe, a modifier-conditioned decoding framework that injects a constrained latent into every layer of the Transformer action decoder via adaLN-Zero, allowing behavioral directives to directly influence action-chunk generation. We evaluate the method on a real-world whiteboard wiping task with combinations of temporal and physical modifiers. Bi-MoDe improves physical directive following over the action-chunking baseline while maintaining comparable temporal control. An ablation further shows that decoder conditioning and latent-space composition interact, and that their combination is important for accurate physical directive following. Additional material is available at the https://mertcookimg.github.io/bi-mode/

Flexible Motion Generation from Language and Style References

3.0/5 一般 裁判分 2.0 Candidate 当日相对
Kai Weixian Lan, Bodie Criswell, Briana Fedkiw, ... , Joseph Teran, Daniel Holden
总结: 语言与风格参考条件的灵活人体运动合成。
方法: 无监督变分风格编码器,支持长时多风格生成。
证据: 保留语义内容同时忠实反映目标风格细节。
为什么适合我: 风格化人体动作生成,便于重定向到腿足机器人。
推荐理由: 语言与风格参考的人体动作生成,只弱相关于运动先验,未进入可跟踪、可重定向的物理全身控制。
原摘要

We introduce FlexMoGen, a novel framework for flexible human motion synthesis conditioned on both natural language descriptions and motion style references. Text prompts are effective at defining semantic content, but they are often limited in capturing fine-grained style details such as timing, limb articulation, and expressive dynamics. A style example clip supplements the text by conveying these nuanced motion characteristics directly, enabling the model to preserve high-level intent while reproducing the desired stylistic traits. Given a text prompt and a style example clip, FlexMoGen generates high-quality motions that preserve semantic content while faithfully reflecting the target style, offering users greater control over the animation generation process. Unlike prior methods that rely on discrete style labels and do not generalize to long or multi-style generation, FlexMoGen learns a variational style encoder without style supervision and supports long, time-varying, multi-style synthesis. Our framework jointly pre-trains the style encoder and a text-to-motion latent diffusion model within a unified architecture, modulating motion style through a lightweight adaptation module. It integrates an efficient relative positional encoding scheme and is trained on both stylized and non-stylized datasets, enabling strong generalization to unseen text-style combinations. Experiments show that FlexMoGen achieves the best balance between content fidelity and style reflection.

Lin He, Min Deng
总结: 约束接地强化学习,实现接触丰富插入的变阻抗控制。
方法: 策略输出残差运动与增益,控制器投影到可接受范围。
证据: 斜插入仿真中成功率85.8±7.7%,不违反力限。
为什么适合我: 接触丰富场景自适应控制,利于人形机器人操作。
推荐理由: 接触丰富插拔的变阻抗强化学习,属于机械臂装配控制,不涉及腿足感知运动或人形全身控制。
原摘要

In robotic insertion under uncertain contact, the axial force limit and the appropriate controller gain vary across tasks. As a result, a single fixed gain is unlikely to remain suitable across different task conditions, making conventional impedance controllers reliant on manual retuning. To eliminate manual retuning, we propose Constraint-Grounded Reinforcement Learning (CG-RL), a variable impedance framework for online gain adaptation. Conditioned on the force limit and contact feedback, the policy outputs a residual motion, an insertion rate, and a requested gain. The controller projects this gain into the admissible range without exposing the range itself to the policy. This separation allows a single policy to operate under different force limits without retraining or manual retuning. We evaluate CG-RL on simulated oblique insertion across five training seeds. CG-RL achieves an $85.8\pm7.7\%$ (mean $\pm$ SD) success rate of insertions without violating the force limit, while keeping the applied gain within the admissible range. As a comparison, a fixed-gain baseline using the midpoint gain achieves a success rate of $50.1\%$. The policy adapts its insertion rate continuously to the specified force limit and further generalizes to more permissive force limits above the training range. In contrast, the same actor without force-limit input does not exhibit this adaptation. The applied gain is guaranteed to remain within the admissible range, while force-limit satisfaction is validated empirically rather than guaranteed formally.

Yanan Zhou, Zhaoyan Qian, James Zhao, Weiming Zhi
总结: 补丁对齐RGB-深度融合,提升机器人操作空间感知。
方法: 非对称交叉注意力将深度绑定到RGB补丁特征。
证据: 六项真机任务中提供几何感知视觉表示。
为什么适合我: 增强复杂地形感知运动,支持接触丰富全身策略。
推荐理由: 面向通用机器人模仿的RGB-深度表征,不针对腿足感知运动、跑酷或全身控制。
原摘要

Recent advances in robot imitation learning have produced visuomotor policies that predict actions directly from visual observations. Yet visually similar scenes can require different actions as target position, object height, or contact geometry changes. Pretrained RGB features may map these geometrically distinct states to similar policy inputs, while simply adding depth requires the policy to learn RGB-depth correspondence from the same limited demonstrations used to learn control. We introduce StereoPatch, a patch-aligned RGB-depth representation that binds registered metric geometry directly to the RGB patches used for action prediction. On a shared 2-D patch grid, asymmetric cross-attention incorporates depth information into the corresponding RGB features before action decoding. The resulting StereoPatch Tokens provide a geometry-aware visual representation that can condition general visuomotor policies without changing their underlying learning objectives. Across six real-robot tasks, StereoPatch achieves higher closed-loop success than appearance-only, geometry-only, raw RGB-D, and late-fusion baselines. Additional experiments across three simulation suites evaluate compatibility across visuomotor policy architectures, spatial generalization, and operating limits. Results suggest that resolving control-relevant geometric ambiguity benefits from aligning depth directly with the visual features used for action prediction, rather than supplying it as an independent modality. Project page: https://aus.bot/research/stereopatch/.

Seeing the World and the Self from Egocentric Video

2.0/5 偏低 裁判分 3.0 Candidate 当日相对
Kai Guan, Minchao Jiang, Ruichen WangLi, Wentao Zhu, Lei Zhang
总结: 从第一人称视频联合重建场景与穿戴者全身运动。
方法: 确定性度量几何重建耦合几何条件运动生成。
证据: 适配外中心预训练模型到自我中心视频。
为什么适合我: 获取人体动作与场景,助力可跟踪全身控制与重定向。
推荐理由: 第一人称视频的场景重建与身体运动估计,偏可穿戴视觉几何,不是动作跟踪、重定向或全身运动控制。
原摘要

Complete 3D perception from egocentric video requires recovering the surrounding scene and the wearer's full-body motion in a shared metric frame. Existing methods typically address scene reconstruction and motion estimation separately: scene reconstruction methods ignore the wearer, whereas motion estimation methods lack explicit scene geometry and often depend on external trajectories. Joint recovery is challenging because the two tasks exhibit asymmetric visibility and require different prediction paradigms. The largely visible scene supports deterministic geometric regression, whereas the severely occluded body requires generative motion inference. We therefore propose RESELF (REconstructing the Scene and the sELF), a unified framework that couples deterministic metric geometry reconstruction with geometry-conditioned motion generation. RESELF adapts a geometry foundation model pre-trained on large-scale exocentric data to egocentric video using frame-wise scale and relative-pose consistency objectives. The resulting camera trajectory and latent geometric features condition a diffusion model that recovers the wearer's motion. A subsequent closed-loop kinematic feedback stage further refines the camera head while preserving the reconstructed scene geometry. To support training and evaluation, we curate EE4D-JSM from EgoExo4D by aligning egocentric video, sparse metric scene geometry, camera trajectories, and full-body motion annotations. Experiments show that RESELF outperforms state-of-the-art methods designed for the individual tasks across depth estimation, camera tracking, and full-body motion estimation. Code, models, and datasets will be available at https://ka1guan.github.io/RESELF/.

JumpStart Your Policy Learning with Lessons from 160,000 Training Runs

2.0/5 偏低 裁判分 2.0 Candidate 当日相对
Nabil Omi, Eric Bae, Chung Yik Edward Yeung, Siddhartha Sen, Ali Farhadi
总结: 大规模离线策略学习实证研究,超16万次训练。
方法: 跨114数据集系统评估超参调优与基准影响。
证据: 无算法主导,调优常重排排名,基准组成影响结论。
为什么适合我: 指导强化学习与模仿学习可靠训练,优化全身运动策略。
推荐理由: 大规模离线强化与模仿学习实证,仅方法层面相邻,不针对人形或腿足任务。
原摘要

Reliable progress in offline policy learning depends on careful reporting, well-tuned baselines, and evaluation across diverse conditions. Prior work has shown that results can be sensitive to reporting choices, hyperparameter tuning, and dataset properties, but these sources of variability have not been systematically investigated together at the scale needed to understand how they shape conclusions. To address this gap, we present a large-scale empirical study of offline reinforcement and imitation learning, training over 160,000 policies across 114 datasets. At this scale, no algorithm dominates: aggregate performance among the strongest methods is often close, but the leaders differ substantially across environments. We find that proper hyperparameter tuning frequently reshuffles perceived algorithm rankings and that benchmark composition can produce conflicting conclusions. We also study hyperparameter sensitivity and transfer across environments, identifying a simple strategy for deriving strong default configurations. We use our findings to develop a dataset-conditioned recommender that provides task-specific algorithm recommendations for practitioners. Finally, we release JumpStart: a resource suite containing every trained policy, per-model scores and hyperparameters, strong baselines across all environments, training and evaluation code, and an extensible website for retrieving, analyzing, and contributing results. Together, these resources aim to make offline policy-learning research more reliable and enable future work beyond the scope of this study.

Giansalvo Cirrincione, Adriano Fagiolini
总结: 运行时增量Transformer,用于强化学习自适应控制。
方法: 据上下文有效秩与头输出幅度动态增减注意力头。
证据: 双连杆摩擦任务全记忆范围达完全成功。
为什么适合我: 自适应处理非可观摩擦记忆,提升接触丰富控制鲁棒性。
推荐理由: 面向带摩擦记忆的机械臂自适应控制,与人形全身运动、跑酷或loco-manipulation无关。
原摘要

Learning-based adaptive control of robotic manipulators with non-observable friction memory has been addressed by attention- based meta-controllers whose number of attention heads is fixed before training and is tuned by costly offline search. At long memory horizons, such fixed-capacity controllers are prone to catastrophic failures on a sizeable fraction of training seeds. The present paper introduces a runtime mechanism that grows and prunes the heads of the attention block during reinforcement learning, governed by two signals: the effective rank of the on-policy context distribution, which triggers growth when representational capacity becomes insufficient, and the per-head output magnitude, which flags redundant heads for removal. Policy continuity at growth events and a quantitative bound at prune events are established analytically. On a two- link manipulator with Stribeck friction, the proposed mechanism attains full success across all memory regimes, eliminating the long-horizon failure mode and removing the need for offline tuning of the head count.