Multi-task learning enhanced deep neural networks for solving delay differential equations

Deep neural networks (DNNs) exhibit effectiveness in solving forward and inverse problems of nonlinear partial differential equations (PDEs). However, traditional DNNs generally struggle with the treatment of delay differential equations (DDEs) due to challenges such as low regularities and primary discontinuity points induced by delay. In this paper, a novel multi-task learning (MTL) enhanced DNN is proposed to solve both forward and inverse problems of DDEs. The core idea behind this approach is to capture information at primary discontinuity points by employing a MTL enhanced DNN that can seamlessly integrate the regularity at primary discontinuity points into the loss function. Subsequently, the network is trained task by task, and transfer learning is applied to task-specific parameters of the network across tasks to expedite network convergence, thereby reducing the complexity of network training and offering reference solutions for subsequent tasks. The MTL enhanced DNN with the sequential training scheme significantly improves approximation accuracy and computation efficiency, which is demonstrated by solving several problems involving time-dependent DDEs and spatio-temporal DDEs, contrasting its performance with that of the conventional DNN method.