APF-Guided PPO for Obstacle Avoidance and Trajectory Planning of Manipulators

Trajectory planning for robotic manipulators in complex, dynamic environments presents a fundamental challenge. While Deep Reinforcement Learning (DRL) offers a promising solution for real-time decision-making, standard algorithms such as Proximal Policy Optimization (PPO) often struggle with sparse rewards and inefficient exploration in high-dimensional continuous action spaces. To address these limitations, this work proposes a novel Dynamic Artificial Potential Field-Guided PPO (DAPF-PPO). We introduce a Dynamic Potential-Based Reward Shaping (DPBRS) mechanism that integrates prior physical knowledge from Artificial Potential Fields (APF) into the RL reward structure. During the early training stages, dense APF-based rewards effectively guide the agent, substantially accelerating the initial learning phase. As training progresses, the guidance weight gradually attenuates, allowing the PPO agent to dominate decision-making and explore globally optimal policies without bias. Simulations utilizing a Franka Emika Panda manipulator within the MuJoCo physics engine demonstrate the effectiveness of the proposed framework. Compared with standard PPO, DAPF-PPO significantly suppresses policy oscillations during early exploration, mitigates the sparse reward problem, and achieves higher success rates as well as smoother trajectories in dynamic obstacle avoidance tasks.