DeepSim: Deep similarity for image quality assessment

This paper studies one interesting problem: how does the deep neural network (DNN) architecture affect the image quality assessment (IQA) performance? In order to find the answer, we propose a novel full-reference IQA framework, codenamed deep similarity (DeepSim). In DeepSim, we first measure the local similarities between the features (produced by a DNN model) of the test image and those of the reference image; Afterwards, the local quality indices are gradually pooled together to estimate the overall quality score. In addition, various factors that may affect the IQA performance are investigated. Thorough experiments conducted on standard databases show that: (1) DeepSim can accurately predict human perceived image quality and outperforms previous state-of-the-art; (2) mid-level representations are most effective for quality prediction; and (3) preprocessing, the restricted linear units and max-pooling operations are beneficial for the IQA performance.