Hierarchical Neural Skill-Based Meta-Reinforcement Learning for Efficient Adaptability in Robotic Manipulation Tasks

Robotic manipulation tasks frequently share foundational structures. Meta-reinforcement learning aims to develop generalizable policies that leverage these shared structures. However, existing methods usually struggle to efficiently encode this task-specific knowledge into their policy networks: hierarchical policies depend on task-agnostic action-level skills or intrinsic rewards, while context-based paradigms suffer from Markov Decision Process ambiguity. To address these challenges, we propose a hierarchical neural skill-based meta-reinforcement learning framework. This framework includes neural skill generation, neural skill decoding, and policy network construction. A Transformer-based neural skill generation unit sequentially generates hierarchical neural skills conditioned on a task description. The decoding mechanism translates these neural skills into network parameters for a policy. Using these decoded parameters, the policy network is constructed layer by layer to process the state information. Unlike previous work, our method treats skills as abstractions of layer-wise network parameters, allowing task-specific knowledge embedded in neural skills to directly configure the policy network. Experimental results demonstrate that our method has enhanced flexibility and efficiency.