跳到正文
Science Robotics· Ilyes Gharbi, Miquel Kegeleirs, Guillermo Legarda Herranz, Jeanne Szpirer, Ken Hasselmann, Mauro Birattari·· 28 天前AI 评分45

结构偏置使机器人集群强化学习实现可靠的sim-to-real迁移

Structural bias enables reliable sim-to-real transfer in reinforcement learning for robot swarms

AI 导读

发表于Science Robotics的研究提出以结构偏置实现机器人集群强化学习的可靠sim-to-real迁移。该方法将结构约束嵌入学习过程,使仿真中训练的策略可迁移至真实机器人集群。

原文摘要

出版方登记 · Crossref

Reinforcement learning offers a powerful framework for automatically generating control policies for robot swarms, yet its application to physical multirobot systems has been limited by the simulation-to-reality gap. Despite substantial advances in multiagent reinforcement learning, systematic demonstrations of gradient-based sim-to-real transfer to physical robot swarms remain scarce. Most work remains confined to simulation, and the few real-robot demonstrations lack evaluation across multiple runs and tasks. We show that this barrier can be overcome by embedding structural constraints into policy learning. We introduce attention-based counterfactual policy gradients for behavior selection in robot swarms (SwarmACB), a multiagent actor-critic method in which robots arbitrate among predefined behavior modules—reactive sensorimotor routines such as phototaxis or aggregation—rather than learning low-level motor commands. This architectural choice reduces the variance of learned solutions, which we argue mitigates overfitting to simulation artifacts. Through systematic evaluation on five collective missions using 20 physical e-puck robots (250 real-robot runs), we demonstrate that SwarmACB transfers successfully from simulation to reality, substantially outperforming continuous-action baselines. By progressively constraining policy architecture, perception, and model capacity, we show that smaller, structured policies transfer more reliably than large, unconstrained ones. These findings challenge the assumption that scale is necessary for successful transfer, suggesting instead that principled structural design enables reinforcement learning to produce control software for robot swarms that is both learned automatically and directly deployable in the physical world.

页面未能读取全文,以上为出版方登记的摘要;方法与图表见原文

来源:Science Robotics · science.org