ITRMD: A Dimensionality Reduction Framework for Accurate and Efficient Multivariate KPI Anomaly Detection

KPIs (key performance indicators) reflect the operational state of Cloud services and infrastructure. With the rapid development of cloud services, many dimensional KPIs are constantly generated during service operations. Recently, deep-learning-based Multivariate KPI Anomaly Detection (MAD) methods have become the mainstream trend to meet the requirements of effective anomaly detection. As the number of KPI dimensions grows, the redundant dimensions in anomaly detection increase. Data with high dimensional complexity increases computational costs and storage requirements. This phenomenon is called the “Curse of dimensionality", which prevents common MAD strategies from being efficient. Therefore, dimensionality reduction is important in MAD. However, there are few researches on dimensionality reduction methods specifically designed for MAD. In this paper, we propose a feature selection KPI dimensionality reduction framework ITRMD, which reduces multivariate KPI series from two aspects: inter-metric dimensionality selection and temporal dimensionality selection. ITRMD preserves features of original data by selecting KPI dimensions with effective information and discards redundant KPI dimensions. Experiments on real KPI datasets show that ITRMD can greatly improve the detection accuracy of the state-of-the-art MAD and reduce the computation cost of model training.