Spatial and channel level feature redundancy reduction for differentiable neural architecture search

Differentiable architecture search (DARTS), based on the continuous relaxation of the architectural representation and gradient descent, achieves effective results in Neural Architecture Search (NAS) field. Among the neural architectures, convolutional neural networks (CNNs) have achieved remarkable performance in various computer vision tasks. However, convolutional layers inevitably extract redundant features as the limitation of the weight-sharing property by convolutional kernels, thus slowing down the search efficiency of DARTS. In this paper, we propose a novel search approach named Slim-DARTS from the perspective of reducing feature redundancy, to further achieve high-speed and efficient neural architecture search. At the level of spatial redundancy, we design a spatial reconstruction module to eliminate spatial feature redundancy and facilitate representative feature learning. At the channel redundancy level, partial channel connection is applied to randomly sample a small subset of channels for operation selection to reduce unfair competition among candidate operations. And we introduce a group of channel parameters to automatically adjust the proportion of selected channels. The experimental results show that our research greatly improves search efficiency and memory utilization, achieving classification error rates of 2.39% and 16.78% on CIFAR-10 and CIFAR-100, respectively.