Robust Physics-Informed Full Waveform Inversion via a Transformer-Based Autoencoder

Full waveform inversion (FWI) can produce high-resolution subsurface parameter models. However, due to its limitations in data acquisition, the observed data often lacks low-frequency information and contains noise, which often results in a local optimization method converging to a local minima. Recently, the methods based on physics-informed neural networks for FWI show good performance in mitigating the local minima problem. However, these approaches usually involve convolutional neural networks (CNNs), tend to focus on local features, and often ignore the global characteristics. In this article, we propose a physics-informed FWI method based on a transformer-based autoencoder network to consider both the local and global features in seismic data. Our network architecture is mainly comprised of a transformer-based encoder and a CNN-based decoder. The transformer-encoder based on the self-attention mechanism can capture the implicit spatial structure features from the observed data, and the CNN-based decoder serves as a sparse representation of the inverted models. To mitigate the quadratic complexity in time and memory for the traditional self-attention mechanisms, a cross-covariance attention mechanism with linear complexity is employed. One essential advantage of our proposed method is that it does not require pretraining and a labeled training dataset. Compared with other data-driven inversion frameworks, our method can effectively use the physics information to ensure the consistency of the model provided by the neural network. Several synthetic numerical experiments indicate that our inversion method is less sensitive to the presence of noise, missing low frequencies in the observed data. Additionally, we further introduce a method for the simultaneous inversion of source wavelets and model parameters, and validate its robustness and effectiveness using both synthetic data and the Chevron 2014 blind benchmark dataset. Furthermore, our method achieves higher accuracy with a lower network parameters compared to the CNN-based and Transformer-only methods.