A Survey of Deep Model Compression and Acceleration

Recently, Deep neural networks (DNNs) have attained remarkable achievements across numerous visual recognition tasks. Nevertheless, the existing deep neural network models are characterized by high computational costs and substantial memory usage, which pose significant barriers to their deployment in devices with limited memory resources or applications with strict latency requirements. Consequently, model compression and acceleration for deep networks without causing notable degradation in model performance is in urgent need. This paper provides a comprehensive review of the recent techniques employed for compacting and accelerating DNN models. From the perspective of model architecture, the main approaches are compact structure design and neural architecture search. From an algorithmic dimension, methods are briefly categorized into static compression methods and dynamic acceleration methods, specifically covering implementation strategies such as model pruning, parameter quantization, low-rank factorization, and knowledge distillation. For each category, we demonstrate the development of mainstream methods as well as the characteristics and advantages of each method. We also provide insightful analysis about the integration of multiple methods, their advantages and drawbacks.