STPformer: Mutation-Aware Spatial-Temporal Pivotal Attention Networks for Transformer-Based Traffic Forecasting

The primary challenge in traffic forecasting lies in effectively capturing the spatio-temporal patterns in traffic data. Recent studies have highlighted the importance of pivotal nodes in road networks, which exhibit dominant impacts due to their prominent role in flow distribution. However, existing methods focus solely on the pivotal properties of the spatial dimension, inevitably diminishing the synchronisation of spatio-temporal patterns. Additionally, nodes with critical spatial semantic attributes are often overlooked. Despite their limited capacity for traffic distribution, these nodes are equally influential due to their strategic geographic positioning or intricate location characteristics. To overcome those limitations, we introduce a novel Spatial-Temporal Pivotal Attention Networks (STPformer) for traffic forecasting. Specifically, our model incorporates a mutation-aware pivotal temporal attention mechanism, which is integrated with Hawkes process, ensuring precise attention to the transition patterns from historical to future sequences. Moreover, the pivotal spatial attention integrated with a probabilistic sparsification mechanism is proposed to adaptively capture the spatial heterogeneity of nodes with significant spatial semantic attributes. By integrating these two innovative components into a Transformer-based architecture, STPformer efficiently learns fine-grained and synchronised spatial-temporal dependencies through stacked layers. Comprehensive experiments have demonstrated the superiority of STPformer in precision, efficiency, scalability, and interpretability.