Spatial-semantic fusion network for semantic segmentation in real-time

Yu Fang; Xuehe Zhang; He Zhang; Gangfeng Liu; Changle Li; Jie Zhao

doi:10.1109/AIM.2019.8868787

Spatial-semantic fusion network for semantic segmentation in real-time

Yu Fang
, Xuehe Zhang
, He Zhang
, Gangfeng Liu
, Changle Li
, Jie Zhao^*

^*Corresponding author for this work

School of Mechatronics Engineering

Harbin Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

In this paper, a real-time semantics segmentation algorithm for mobile terminals such as robots is proposed, which captures large-scale and small-scale features respectively by using convolution and dilation convolution to obtain spatial information. It can get semantic information by using decode-encode structure. It contains two branches, one extracts semantic information, the other extracts spatial information, and finally fuses the two branches together through self-attention method. Our algorithm achieves 71.3% mIoU on the Pascal VOC 2012 data sets, and it takes only 16ms to infer a 480*640 picture in GTX1080Ti. It can be applied to most mobile robot platforms.

Original language	English
Title of host publication	Proceedings of the 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	30-35
Number of pages	6
ISBN (Electronic)	9781728124933
DOIs	https://doi.org/10.1109/AIM.2019.8868787
State	Published - Jul 2019
Event	2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2019 - Hong Kong, China Duration: 8 Jul 2019 → 12 Jul 2019

Publication series

Name	IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM
Volume	2019-July

Conference

Conference	2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2019
Country/Territory	China
City	Hong Kong
Period	8/07/19 → 12/07/19

Keywords

Deep learning
Real-time
Semantic segmentation

Access to Document

10.1109/AIM.2019.8868787

Cite this

Fang, Y., Zhang, X., Zhang, H., Liu, G., Li, C., & Zhao, J. (2019). Spatial-semantic fusion network for semantic segmentation in real-time. In Proceedings of the 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2019 (pp. 30-35). Article 8868787 (IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM; Vol. 2019-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/AIM.2019.8868787

Fang, Yu ; Zhang, Xuehe ; Zhang, He et al. / Spatial-semantic fusion network for semantic segmentation in real-time. Proceedings of the 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 30-35 (IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM).

@inproceedings{f35acce5acc34cf5a89cb09c62cbed17,

title = "Spatial-semantic fusion network for semantic segmentation in real-time",

abstract = "In this paper, a real-time semantics segmentation algorithm for mobile terminals such as robots is proposed, which captures large-scale and small-scale features respectively by using convolution and dilation convolution to obtain spatial information. It can get semantic information by using decode-encode structure. It contains two branches, one extracts semantic information, the other extracts spatial information, and finally fuses the two branches together through self-attention method. Our algorithm achieves 71.3\% mIoU on the Pascal VOC 2012 data sets, and it takes only 16ms to infer a 480*640 picture in GTX1080Ti. It can be applied to most mobile robot platforms.",

keywords = "Deep learning, Real-time, Semantic segmentation",

author = "Yu Fang and Xuehe Zhang and He Zhang and Gangfeng Liu and Changle Li and Jie Zhao",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2019 ; Conference date: 08-07-2019 Through 12-07-2019",

year = "2019",

month = jul,

doi = "10.1109/AIM.2019.8868787",

language = "英语",

series = "IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "30--35",

booktitle = "Proceedings of the 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2019",

address = "美国",

}

Fang, Y, Zhang, X , Zhang, H , Liu, G , Li, C & Zhao, J 2019, Spatial-semantic fusion network for semantic segmentation in real-time. in Proceedings of the 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2019., 8868787, IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM, vol. 2019-July, Institute of Electrical and Electronics Engineers Inc., pp. 30-35, 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2019, Hong Kong, China, 8/07/19. https://doi.org/10.1109/AIM.2019.8868787

Spatial-semantic fusion network for semantic segmentation in real-time. / Fang, Yu; Zhang, Xuehe ; Zhang, He et al.
Proceedings of the 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 30-35 8868787 (IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM; Vol. 2019-July).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Spatial-semantic fusion network for semantic segmentation in real-time

AU - Fang, Yu

AU - Zhang, Xuehe

AU - Zhang, He

AU - Liu, Gangfeng

AU - Li, Changle

AU - Zhao, Jie

PY - 2019/7

Y1 - 2019/7

N2 - In this paper, a real-time semantics segmentation algorithm for mobile terminals such as robots is proposed, which captures large-scale and small-scale features respectively by using convolution and dilation convolution to obtain spatial information. It can get semantic information by using decode-encode structure. It contains two branches, one extracts semantic information, the other extracts spatial information, and finally fuses the two branches together through self-attention method. Our algorithm achieves 71.3% mIoU on the Pascal VOC 2012 data sets, and it takes only 16ms to infer a 480*640 picture in GTX1080Ti. It can be applied to most mobile robot platforms.

AB - In this paper, a real-time semantics segmentation algorithm for mobile terminals such as robots is proposed, which captures large-scale and small-scale features respectively by using convolution and dilation convolution to obtain spatial information. It can get semantic information by using decode-encode structure. It contains two branches, one extracts semantic information, the other extracts spatial information, and finally fuses the two branches together through self-attention method. Our algorithm achieves 71.3% mIoU on the Pascal VOC 2012 data sets, and it takes only 16ms to infer a 480*640 picture in GTX1080Ti. It can be applied to most mobile robot platforms.

KW - Deep learning

KW - Real-time

KW - Semantic segmentation

UR - https://www.scopus.com/pages/publications/85074246912

U2 - 10.1109/AIM.2019.8868787

DO - 10.1109/AIM.2019.8868787

M3 - 会议稿件

AN - SCOPUS:85074246912

T3 - IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM

SP - 30

EP - 35

BT - Proceedings of the 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2019

Y2 - 8 July 2019 through 12 July 2019

ER -

Fang Y, Zhang X , Zhang H , Liu G , Li C , Zhao J. Spatial-semantic fusion network for semantic segmentation in real-time. In Proceedings of the 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 30-35. 8868787. (IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM). doi: 10.1109/AIM.2019.8868787

Spatial-semantic fusion network for semantic segmentation in real-time

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this