High-dimensional asymptotic analysis of adaptive radar detectors under Gaussian and compound-Gaussian clutter

This paper investigates the asymptotic performance of three widely used adaptive detectors: the adaptive matched filter (AMF), the generalized likelihood ratio test (GLRT), and the adaptive coherence estimator (ACE), under the high-dimensional regime, where both the dimension N and the sample size K grow to infinity whereas their ratio N/K converges to a constant c ∈ (0, 1). Both Gaussian and non-Gaussian clutter environments are considered. Under Gaussian clutter, closed-form expressions for asymptotic false alarm rate (P_fa) and detection probability (P_d) are derived. The theoretical results show that all three detectors (i) possess constant false alarm rate (CFAR) property with respect to the unknown covariance matrix and (ii) achieve asymptotically equivalent detection performance in high-dimensional Gaussian clutter. For non-Gaussian clutter, general analytical expressions for the asymptotic P_fa and P_d are derived under the compound-Gaussian model with arbitrary texture. These results are then specialized to the K-distributed clutter. The resulting simplified expressions reveal two main theoretical conclusions: (i) the three detectors maintain CFAR with respect to the speckle covariance matrix and the scale parameter, but not to the shape parameter; and (ii) the ACE is preferable in high-dimensional K-distributed clutter owing to its superior and robust performance across both weak and strong target scenarios.