Skill Expansion and Composition in Parameter Space

• DOI: 10.48550/arxiv.2502.05932
• Publication Date: 2025-02-09
• Authors: Ting Liu; Jianxiong Li; Yinan Zheng; Haoyi Niu; Yixing Lan; Xinyue Xu; Xianyuan Zhan
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Computer Science - Robotics; Artificial Intelligence (cs.AI); Computer Science - Artificial Intelligence; Robotics (cs.RO); Machine Learning (cs.LG)

Humans excel at reusing prior knowledge to address new challenges and developing skills while solving problems. This paradigm becomes increasingly popular in the development of autonomous agents, as it develops systems that can self-evolve response like human beings. However, previous methods suffer from limited training efficiency when expanding fail fully leverage facilitate task learning. In this paper, we propose Parametric Skill Expansion Composition (PSEC), a framework designed iteratively evolve agents' capabilities efficiently by maintaining manageable skill library. library progressively integrate primitives plug-and-play Low-Rank Adaptation (LoRA) modules parameter-efficient finetuning, facilitating efficient flexible expansion. structure also enables direct compositions parameter space merging LoRA encode different skills, leveraging shared information across effectively program skills. Based on this, context-aware module dynamically activate collaboratively handle tasks. Empowering diverse applications including multi-objective composition, dynamics shift, continual policy results D4RL, DSRL benchmarks, DeepMind Control Suite show PSEC exhibits superior capacity tackle challenges, well expand its libraries capabilities. Project website: https://ltlhuuu.github.io/PSEC/.