Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features

Jiaqi Chen; Lujiao Shao; Haibin Zhou; Jianghong Ma; Weizhi Meng; Zenghui Wang; Haijun Zhang

doi:10.1007/978-981-19-6142-7_29

Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features

Jiaqi Chen
, Lujiao Shao
, Haibin Zhou
, Jianghong Ma
, Weizhi Meng
, Zenghui Wang
, Haijun Zhang^*

^*Corresponding author for this work

Harbin Institute of Technology Shenzhen
Technical University of Denmark
University of South Africa

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This research presents a new key information extraction algorithm from shopping receipts. Specifically, we train semantic, visual and structural features through three deep learning methods, respectively, and formulate rule features according to the characteristics of shopping receipts. Then we propose a multi-class text classification algorithm based on multi-modal features using Bayesian deep learning. After post-processing the output of the classification algorithm, the key information we seek for can be obtained. Our algorithm was trained on a self-labeled Chinese shopping receipt dataset and compared with several baseline methods. Extensive experimental results demonstrate that the proposed method achieves optimal results on our Chinese receipt dataset.

Original language	English
Title of host publication	Neural Computing for Advanced Applications - 3rd International Conference, NCAA 2022, Proceedings
Editors	Haijun Zhang, Yuehui Chen, Xianghua Chu, Zhao Zhang, Tianyong Hao, Zhou Wu, Yimin Yang
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	378-393
Number of pages	16
ISBN (Print)	9789811961410
DOIs	https://doi.org/10.1007/978-981-19-6142-7_29
State	Published - 2022
Externally published	Yes
Event	3rd International Conference on Neural Computing for Advanced Applications, NCAA 2022 - Jinan, China Duration: 8 Jul 2022 → 10 Jul 2022

Publication series

Name	Communications in Computer and Information Science
Volume	1637 CCIS
ISSN (Print)	1865-0929
ISSN (Electronic)	1865-0937

Conference

Conference	3rd International Conference on Neural Computing for Advanced Applications, NCAA 2022
Country/Territory	China
City	Jinan
Period	8/07/22 → 10/07/22

Keywords

Bayesian deep learning
Information extraction
Multi-modal feature fusion
Shopping receipt

Access to Document

10.1007/978-981-19-6142-7_29

Cite this

Chen, J., Shao, L., Zhou, H., Ma, J., Meng, W., Wang, Z., & Zhang, H. (2022). Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features. In H. Zhang, Y. Chen, X. Chu, Z. Zhang, T. Hao, Z. Wu, & Y. Yang (Eds.), Neural Computing for Advanced Applications - 3rd International Conference, NCAA 2022, Proceedings (pp. 378-393). (Communications in Computer and Information Science; Vol. 1637 CCIS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-19-6142-7_29

Chen, Jiaqi ; Shao, Lujiao ; Zhou, Haibin et al. / Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features. Neural Computing for Advanced Applications - 3rd International Conference, NCAA 2022, Proceedings. editor / Haijun Zhang ; Yuehui Chen ; Xianghua Chu ; Zhao Zhang ; Tianyong Hao ; Zhou Wu ; Yimin Yang. Springer Science and Business Media Deutschland GmbH, 2022. pp. 378-393 (Communications in Computer and Information Science).

@inproceedings{6e11da9d92f34d0fb1ae4700b54d1091,

title = "Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features",

abstract = "This research presents a new key information extraction algorithm from shopping receipts. Specifically, we train semantic, visual and structural features through three deep learning methods, respectively, and formulate rule features according to the characteristics of shopping receipts. Then we propose a multi-class text classification algorithm based on multi-modal features using Bayesian deep learning. After post-processing the output of the classification algorithm, the key information we seek for can be obtained. Our algorithm was trained on a self-labeled Chinese shopping receipt dataset and compared with several baseline methods. Extensive experimental results demonstrate that the proposed method achieves optimal results on our Chinese receipt dataset.",

keywords = "Bayesian deep learning, Information extraction, Multi-modal feature fusion, Shopping receipt",

author = "Jiaqi Chen and Lujiao Shao and Haibin Zhou and Jianghong Ma and Weizhi Meng and Zenghui Wang and Haijun Zhang",

note = "Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.; 3rd International Conference on Neural Computing for Advanced Applications, NCAA 2022 ; Conference date: 08-07-2022 Through 10-07-2022",

year = "2022",

doi = "10.1007/978-981-19-6142-7\_29",

language = "英语",

isbn = "9789811961410",

series = "Communications in Computer and Information Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "378--393",

editor = "Haijun Zhang and Yuehui Chen and Xianghua Chu and Zhao Zhang and Tianyong Hao and Zhou Wu and Yimin Yang",

booktitle = "Neural Computing for Advanced Applications - 3rd International Conference, NCAA 2022, Proceedings",

address = "德国",

}

Chen, J, Shao, L, Zhou, H, Ma, J, Meng, W, Wang, Z & Zhang, H 2022, Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features. in H Zhang, Y Chen, X Chu, Z Zhang, T Hao, Z Wu & Y Yang (eds), Neural Computing for Advanced Applications - 3rd International Conference, NCAA 2022, Proceedings. Communications in Computer and Information Science, vol. 1637 CCIS, Springer Science and Business Media Deutschland GmbH, pp. 378-393, 3rd International Conference on Neural Computing for Advanced Applications, NCAA 2022, Jinan, China, 8/07/22. https://doi.org/10.1007/978-981-19-6142-7_29

Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features. / Chen, Jiaqi; Shao, Lujiao; Zhou, Haibin et al.
Neural Computing for Advanced Applications - 3rd International Conference, NCAA 2022, Proceedings. ed. / Haijun Zhang; Yuehui Chen; Xianghua Chu; Zhao Zhang; Tianyong Hao; Zhou Wu; Yimin Yang. Springer Science and Business Media Deutschland GmbH, 2022. p. 378-393 (Communications in Computer and Information Science; Vol. 1637 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features

AU - Chen, Jiaqi

AU - Shao, Lujiao

AU - Zhou, Haibin

AU - Ma, Jianghong

AU - Meng, Weizhi

AU - Wang, Zenghui

AU - Zhang, Haijun

PY - 2022

Y1 - 2022

N2 - This research presents a new key information extraction algorithm from shopping receipts. Specifically, we train semantic, visual and structural features through three deep learning methods, respectively, and formulate rule features according to the characteristics of shopping receipts. Then we propose a multi-class text classification algorithm based on multi-modal features using Bayesian deep learning. After post-processing the output of the classification algorithm, the key information we seek for can be obtained. Our algorithm was trained on a self-labeled Chinese shopping receipt dataset and compared with several baseline methods. Extensive experimental results demonstrate that the proposed method achieves optimal results on our Chinese receipt dataset.

AB - This research presents a new key information extraction algorithm from shopping receipts. Specifically, we train semantic, visual and structural features through three deep learning methods, respectively, and formulate rule features according to the characteristics of shopping receipts. Then we propose a multi-class text classification algorithm based on multi-modal features using Bayesian deep learning. After post-processing the output of the classification algorithm, the key information we seek for can be obtained. Our algorithm was trained on a self-labeled Chinese shopping receipt dataset and compared with several baseline methods. Extensive experimental results demonstrate that the proposed method achieves optimal results on our Chinese receipt dataset.

KW - Bayesian deep learning

KW - Information extraction

KW - Multi-modal feature fusion

KW - Shopping receipt

UR - https://www.scopus.com/pages/publications/85142766959

U2 - 10.1007/978-981-19-6142-7_29

DO - 10.1007/978-981-19-6142-7_29

M3 - 会议稿件

AN - SCOPUS:85142766959

SN - 9789811961410

T3 - Communications in Computer and Information Science

SP - 378

EP - 393

BT - Neural Computing for Advanced Applications - 3rd International Conference, NCAA 2022, Proceedings

A2 - Zhang, Haijun

A2 - Chen, Yuehui

A2 - Chu, Xianghua

A2 - Zhang, Zhao

A2 - Hao, Tianyong

A2 - Wu, Zhou

A2 - Yang, Yimin

PB - Springer Science and Business Media Deutschland GmbH

T2 - 3rd International Conference on Neural Computing for Advanced Applications, NCAA 2022

Y2 - 8 July 2022 through 10 July 2022

ER -

Chen J, Shao L, Zhou H, Ma J, Meng W, Wang Z et al. Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features. In Zhang H, Chen Y, Chu X, Zhang Z, Hao T, Wu Z, Yang Y, editors, Neural Computing for Advanced Applications - 3rd International Conference, NCAA 2022, Proceedings. Springer Science and Business Media Deutschland GmbH. 2022. p. 378-393. (Communications in Computer and Information Science). doi: 10.1007/978-981-19-6142-7_29