Sequential Experimental Design for Complex Industrial Systems: An Efficient Proximal Policy Optimization Approach

This paper proposes a reinforcement learning (RL) framework to optimize sequential experimental design for complex industrial systems. The framework addresses efficient identification of unknown parameters in complex industrial systems under budget constraints. A deep neural network approximates the action-value function, while Proximal Policy Optimization (PPO) with a clipped surrogate objective optimizes the policy, represented by a separate deep neural network. The policy maps the posterior over unknown parameters to an optimal experimental input. Evaluated on a one-dimensional thermal conduction problem, the method strategically positions a temperature sensor to estimate unknown parameters of an instantaneous heat source. Simulation results demonstrate that the policy effectively guides experiments towards informative measurements, concentrating the posterior around true parameter values. Compared to a genetic algorithm, the RL approach achieves superior performance, highlighting its potential for optimizing sequential experimental design in complex, resource-constrained scenarios.