Compositional Skill Grounding
CSG partitions each demonstration into temporally coherent segments using robot-state dynamics, then uses VLM-grounded induction to assign skill labels and update the skill base.
Embodied Continual Learning
SCE enables robots to continually acquire new tasks by decomposing demonstrations into reusable skills and composing them through dual execution-and-transition experts.
Embodied Continual Learning (ECL) aims to enable robots to continually acquire new manipulation tasks while retaining previously learned behaviors under closed-loop control. In ECL, feature drift can propagate through sequential decision-making under closed-loop control, turning representation changes into compounding behavioral deviations on previously learned tasks. A key challenge in ECL lies in structured skill reuse across continually evolving tasks, since existing methods primarily focus on skill learning without explicitly organizing them for coherent task execution.
To address this issue, we propose SCE, a Skill-Compositional Experts framework for ECL. SCE builds a skill base via Compositional Skill Grounding (CSG), which decomposes task demonstrations into reusable skills. Based on this, Dual Execution-and-Transition Experts (DETE) enable new task learning through skill composition, where one branch ensures skill execution and the other supports transitions between skills for coherent behavior. Experiments on LIBERO benchmarks and real-world manipulation tasks show that SCE improves retention and overall task performance. Further feature drift analyses and ablation studies verify the effectiveness of our method.
Method
CSG partitions each demonstration into temporally coherent segments using robot-state dynamics, then uses VLM-grounded induction to assign skill labels and update the skill base.
DETE routes each input to a skill-specific expert through a skill distribution, giving each skill dedicated capacity and reducing interference with unrelated skills.
A transition-aware router models cross-skill dependencies around skill boundaries, and adaptive fusion balances execution and transition outputs during closed-loop action generation.
Results
The reported metrics follow the paper protocol: FWT measures new-task performance, NBT measures forgetting on previously learned tasks, AUC summarizes performance across stages, and Final SR reports final-stage average success.
Final SR with 4.3 NBT and 82.9 AUC.
Final SR with 2.5 NBT and 70.2 AUC.
Final SR with 0.6 NBT and 79.1 AUC.
| Method | LIBERO-Goal | LIBERO-Long | ||||||
|---|---|---|---|---|---|---|---|---|
| FWT | NBT | AUC | Final SR | FWT | NBT | AUC | Final SR | |
| UniVLA† | 87.5 ± 0.1 | 65.7 ± 6.1 | 35.2 ± 4.7 | 20.5 ± 4.7 | 70.3 ± 5.6 | 31.3 ± 4.0 | 43.6 ± 2.4 | 17.1 ± 2.0 |
| MoDE† | 69.7 ± 1.8 | 23.2 ± 2.9 | 46.8 ± 2.9 | 41.6 ± 4.8 | 71.1 ± 1.5 | 46.4 ± 1.1 | 32.9 ± 1.4 | 19.9 ± 1.4 |
| T-Stellar† | 76.6 ± 3.4 | 19.1 ± 3.6 | 57.5 ± 1.1 | 57.0 ± 2.0 | 80.3 ± 1.6 | 38.0 ± 2.5 | 48.6 ± 2.6 | 32.2 ± 4.4 |
| TS-Stellar† | 71.8 ± 4.0 | 25.9 ± 2.7 | 47.5 ± 2.5 | 47.7 ± 1.4 | 84.7 ± 1.4 | 40.8 ± 0.6 | 51.4 ± 1.4 | 39.7 ± 1.6 |
| IsCiL† | 66.8 ± 1.3 | 7.8 ± 2.5 | 60.9 ± 2.8 | 48.5 ± 2.0 | 65.8 ± 0.3 | 31.5 ± 0.3 | 41.4 ± 0.1 | 17.3 ± 0.3 |
| LOTUS† | 46.8 ± 4.3 | 37.7 ± 3.9 | 18.5 ± 1.5 | 10.8 ± 1.1 | 32.3 ± 2.3 | 2.8 ± 2.7 | 29.4 ± 0.5 | 31.8 ± 2.3 |
| Sequential | 81.7 ± 0.7 | 74.1 ± 0.6 | 24.3 ± 0.4 | 7.5 ± 0.1 | 61.7 ± 0.7 | 58.1 ± 0.3 | 14.4 ± 0.7 | 3.7 ± 0.6 |
| ER | 74.7 ± 0.4 | 17.3 ± 0.3 | 60.5 ± 0.3 | 44.9 ± 0.1 | 73.3 ± 1.4 | 22.5 ± 0.4 | 55.2 ± 1.2 | 50.9 ± 0.9 |
| Task-MoE | 84.0 ± 0.5 | 14.4 ± 1.2 | 72.3 ± 0.4 | 66.0 ± 1.1 | 73.5 ± 0.8 | 12.1 ± 0.5 | 64.4 ± 0.6 | 61.1 ± 2.9 |
| SCE | 86.5 ± 0.3 | 4.3 ± 0.4 | 82.9 ± 0.4 | 83.9 ± 0.6 | 71.7 ± 1.9 | 2.5 ± 1.8 | 70.2 ± 0.2 | 73.9 ± 1.0 |
| Method | FWT | NBT | AUC | Final SR |
|---|---|---|---|---|
| Sequential | 55.6 | 45.6 | 28.7 | 10.0 |
| ER | 52.2 | 23.9 | 37.2 | 22.2 |
| Task-MoE | 64.4 | 8.3 | 59.8 | 58.9 |
| SCE | 80.0 | 0.6 | 79.1 | 80.0 |
Videos
The videos show successful SCE rollouts and representative failure cases from baseline methods. All videos are played at 2x speed.
SCE moves the block from the box to the wooden board.
SCE opens the lid and transfers the block from the board into the bowl.
SCE composes the learned manipulation skills to move the block from the box into the bowl.
The policy hesitates between gripper-state changes and upward motion, causing execution to remain stuck around these actions.
The policy moves the cover aside but does not proceed to transfer the block from the wooden board into the bowl.
Analysis
| Method | FWT | NBT | AUC | Final SR |
|---|---|---|---|---|
| SCE w/o CSG | 72.4 | 7.0 | 66.1 | 63.6 |
| SCE w/o TEB | 61.4 | -6.3 | 67.1 | 69.4 |
| SCE w/o EEB | 80.6 | 6.5 | 75.0 | 80.0 |
| SCE | 86.2 | 4.3 | 82.5 | 83.4 |
CSG organizes reusable skills, while the two expert branches provide complementary execution and transition modeling. Full SCE achieves 82.5 AUC and 83.4 Final SR in the ablation setting.