Ada-Matcher: A deep detector-based local feature matcher with adaptive weight sharing

Establishing point-to-point correspondence between image pairs through local feature matching is essential for many vision applications. Recently, detector-based feature matching methods leveraging deep learning have achieved a balance between accuracy and computational efficiency, gaining broad attention. To enhance matching performance or reduce storage requirements, these approaches focus on improving the Transformer structure for more effective feature aggregation. Unlike these studies, the present study explores the impact of Transformer block numbers on the matching performance. Theoretically, increasing the number of Transformer modules can enhance matching accuracy, but this also results in a proportional increase in the model size, making the matcher unsustainable with limited resources. To address this, this study introduces Ada-Matcher, a detector-based deep local feature matching framework that improves accuracy while maintaining model capacity. Ada-Matcher incorporates an adaptive weight sharing mechanism, allowing dynamic weight sharing among adjacent Transformer blocks, thus mitigating the storage overhead associated with deep network structures. Complementing this, lightweight feature transformations are applied to each Transformer block, enriching feature diversity and boosting matching performance. Furthermore, Ada-Matcher uses a novel mask-attention technology, focusing on critical task features and dynamically masking irrelevant information to enhance the model generalization ability. Rigorous empirical evaluations indicate that Ada-Matcher exhibits superior performance across various benchmark tests. The code and data related to this work are publicly available at https://github.com/zfj-mc/ada-matcher.