Single Image Super-Resolution via a Holistic Attention Network

Ben Niu; Weilei Wen; Wenqi Ren; Xiangde Zhang; Lianping Yang; Shuzhen Wang; Kaihao Zhang; Xiaochun Cao; Haifeng Shen

doi:10.1007/978-3-030-58610-2_12

Single Image Super-Resolution via a Holistic Attention Network

Ben Niu
, Weilei Wen
, Wenqi Ren
, Xiangde Zhang
, Lianping Yang^*
, Shuzhen Wang
, Kaihao Zhang
, Xiaochun Cao
, Haifeng Shen

^*Corresponding author for this work

Northeastern University
Xidian University
CAS - Institute of Information Engineering
Australian National University
Peng Cheng Laboratory
DiDi Chuxing

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

814 Scopus citations

Abstract

Informative features play a crucial role in the single image super-resolution task. Channel attention has been demonstrated to be effective for preserving information-rich features in each layer. However, channel attention treats each convolution layer as a separate process that misses the correlation among different layers. To address this problem, we propose a new holistic attention network (HAN), which consists of a layer attention module (LAM) and a channel-spatial attention module (CSAM), to model the holistic interdependencies among layers, channels, and positions. Specifically, the proposed LAM adaptively emphasizes hierarchical features by considering correlations among layers. Meanwhile, CSAM learns the confidence at all the positions of each channel to selectively capture more informative features. Extensive experiments demonstrate that the proposed HAN performs favorably against the state-of-the-art single image super-resolution approaches.

Original language	English
Title of host publication	Computer Vision – ECCV 2020 - 16th European Conference, Proceedings
Editors	Andrea Vedaldi, Horst Bischof, Thomas Brox, Jan-Michael Frahm
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	191-207
Number of pages	17
ISBN (Print)	9783030586096
DOIs	https://doi.org/10.1007/978-3-030-58610-2_12
State	Published - 2020
Externally published	Yes
Event	16th European Conference on Computer Vision, ECCV 2020 - Glasgow, United Kingdom Duration: 23 Aug 2020 → 28 Aug 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12357 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	16th European Conference on Computer Vision, ECCV 2020
Country/Territory	United Kingdom
City	Glasgow
Period	23/08/20 → 28/08/20

Keywords

Channel-spatial attention
Holistic attention
Layer attention
Super-resolution

Access to Document

10.1007/978-3-030-58610-2_12

Cite this

Niu, B., Wen, W., Ren, W., Zhang, X., Yang, L., Wang, S., Zhang, K., Cao, X., & Shen, H. (2020). Single Image Super-Resolution via a Holistic Attention Network. In A. Vedaldi, H. Bischof, T. Brox, & J.-M. Frahm (Eds.), Computer Vision – ECCV 2020 - 16th European Conference, Proceedings (pp. 191-207). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12357 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-58610-2_12

Niu, Ben ; Wen, Weilei ; Ren, Wenqi et al. / Single Image Super-Resolution via a Holistic Attention Network. Computer Vision – ECCV 2020 - 16th European Conference, Proceedings. editor / Andrea Vedaldi ; Horst Bischof ; Thomas Brox ; Jan-Michael Frahm. Springer Science and Business Media Deutschland GmbH, 2020. pp. 191-207 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{faf3def51c734ac5999e293662e83130,

title = "Single Image Super-Resolution via a Holistic Attention Network",

abstract = "Informative features play a crucial role in the single image super-resolution task. Channel attention has been demonstrated to be effective for preserving information-rich features in each layer. However, channel attention treats each convolution layer as a separate process that misses the correlation among different layers. To address this problem, we propose a new holistic attention network (HAN), which consists of a layer attention module (LAM) and a channel-spatial attention module (CSAM), to model the holistic interdependencies among layers, channels, and positions. Specifically, the proposed LAM adaptively emphasizes hierarchical features by considering correlations among layers. Meanwhile, CSAM learns the confidence at all the positions of each channel to selectively capture more informative features. Extensive experiments demonstrate that the proposed HAN performs favorably against the state-of-the-art single image super-resolution approaches.",

keywords = "Channel-spatial attention, Holistic attention, Layer attention, Super-resolution",

author = "Ben Niu and Weilei Wen and Wenqi Ren and Xiangde Zhang and Lianping Yang and Shuzhen Wang and Kaihao Zhang and Xiaochun Cao and Haifeng Shen",

note = "Publisher Copyright: {\textcopyright} 2020, Springer Nature Switzerland AG.; 16th European Conference on Computer Vision, ECCV 2020 ; Conference date: 23-08-2020 Through 28-08-2020",

year = "2020",

doi = "10.1007/978-3-030-58610-2\_12",

language = "英语",

isbn = "9783030586096",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "191--207",

editor = "Andrea Vedaldi and Horst Bischof and Thomas Brox and Jan-Michael Frahm",

booktitle = "Computer Vision – ECCV 2020 - 16th European Conference, Proceedings",

address = "德国",

}

Niu, B, Wen, W, Ren, W, Zhang, X, Yang, L, Wang, S, Zhang, K, Cao, X & Shen, H 2020, Single Image Super-Resolution via a Holistic Attention Network. in A Vedaldi, H Bischof, T Brox & J-M Frahm (eds), Computer Vision – ECCV 2020 - 16th European Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12357 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 191-207, 16th European Conference on Computer Vision, ECCV 2020, Glasgow, United Kingdom, 23/08/20. https://doi.org/10.1007/978-3-030-58610-2_12

Single Image Super-Resolution via a Holistic Attention Network. / Niu, Ben; Wen, Weilei; Ren, Wenqi et al.
Computer Vision – ECCV 2020 - 16th European Conference, Proceedings. ed. / Andrea Vedaldi; Horst Bischof; Thomas Brox; Jan-Michael Frahm. Springer Science and Business Media Deutschland GmbH, 2020. p. 191-207 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12357 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Single Image Super-Resolution via a Holistic Attention Network

AU - Niu, Ben

AU - Wen, Weilei

AU - Ren, Wenqi

AU - Zhang, Xiangde

AU - Yang, Lianping

AU - Wang, Shuzhen

AU - Zhang, Kaihao

AU - Cao, Xiaochun

AU - Shen, Haifeng

PY - 2020

Y1 - 2020

N2 - Informative features play a crucial role in the single image super-resolution task. Channel attention has been demonstrated to be effective for preserving information-rich features in each layer. However, channel attention treats each convolution layer as a separate process that misses the correlation among different layers. To address this problem, we propose a new holistic attention network (HAN), which consists of a layer attention module (LAM) and a channel-spatial attention module (CSAM), to model the holistic interdependencies among layers, channels, and positions. Specifically, the proposed LAM adaptively emphasizes hierarchical features by considering correlations among layers. Meanwhile, CSAM learns the confidence at all the positions of each channel to selectively capture more informative features. Extensive experiments demonstrate that the proposed HAN performs favorably against the state-of-the-art single image super-resolution approaches.

AB - Informative features play a crucial role in the single image super-resolution task. Channel attention has been demonstrated to be effective for preserving information-rich features in each layer. However, channel attention treats each convolution layer as a separate process that misses the correlation among different layers. To address this problem, we propose a new holistic attention network (HAN), which consists of a layer attention module (LAM) and a channel-spatial attention module (CSAM), to model the holistic interdependencies among layers, channels, and positions. Specifically, the proposed LAM adaptively emphasizes hierarchical features by considering correlations among layers. Meanwhile, CSAM learns the confidence at all the positions of each channel to selectively capture more informative features. Extensive experiments demonstrate that the proposed HAN performs favorably against the state-of-the-art single image super-resolution approaches.

KW - Channel-spatial attention

KW - Holistic attention

KW - Layer attention

KW - Super-resolution

UR - https://www.scopus.com/pages/publications/85093071875

U2 - 10.1007/978-3-030-58610-2_12

DO - 10.1007/978-3-030-58610-2_12

M3 - 会议稿件

AN - SCOPUS:85093071875

SN - 9783030586096

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 191

EP - 207

BT - Computer Vision – ECCV 2020 - 16th European Conference, Proceedings

A2 - Vedaldi, Andrea

A2 - Bischof, Horst

A2 - Brox, Thomas

A2 - Frahm, Jan-Michael

PB - Springer Science and Business Media Deutschland GmbH

T2 - 16th European Conference on Computer Vision, ECCV 2020

Y2 - 23 August 2020 through 28 August 2020

ER -

Niu B, Wen W, Ren W, Zhang X, Yang L, Wang S et al. Single Image Super-Resolution via a Holistic Attention Network. In Vedaldi A, Bischof H, Brox T, Frahm JM, editors, Computer Vision – ECCV 2020 - 16th European Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2020. p. 191-207. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-58610-2_12

Single Image Super-Resolution via a Holistic Attention Network

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this