Improving LLM-Based Health Information Extraction with In-Context Learning

Junkai Liu; Jiayi Wang; Hui Huang; Rui Zhang; Muyun Yang; Tiejun Zhao

doi:10.1007/978-981-97-1717-0_4

Improving LLM-Based Health Information Extraction with In-Context Learning

Junkai Liu
, Jiayi Wang
, Hui Huang
, Rui Zhang
, Muyun Yang^*
, Tiejun Zhao

^*Corresponding author for this work

School of Computer Science and Technology, Harbin Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

3 Scopus citations

Abstract

The Large Language Model (LLM) has received widespread attention in the industry. In the context of the popularity of LLM, almost all NLP tasks are transformed into prompt based language generation tasks. On the other hand, LLM can also achieve superior results on brand new tasks without fine-tuning, solely with a few in-context examples. This paper describes our participation in the China Health Information Processing Conference (CHIP 2023). We focused on in-context learning (ICL) and experimented with different combinations of demonstration retrieval strategies on the given task and tested the optimal strategy combination proposed by us. The experimental results show that our retrieval strategies based on Chinese-LlaMA2-13B-chat achieved a average score of 40.27, ranked the first place among five teams, confirmed the effectiveness of our method.

Original language	English
Title of host publication	Health Information Processing. Evaluation Track Papers - 9th China Conference, CHIP 2023, Proceedings
Editors	Hua Xu, Qingcai Chen, Hongfei Lin, Fei Wu, Lei Liu, Buzhou Tang, Tianyong Hao, Zhengxing Huang, Jianbo Lei, Zuofeng Li, Hui Zong
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	49-59
Number of pages	11
ISBN (Print)	9789819717163
DOIs	https://doi.org/10.1007/978-981-97-1717-0_4
State	Published - 2024
Externally published	Yes
Event	Evaluation track of the 9th China Health Information Processing Conference, CHIP 2023 - Hangzhou, China Duration: 27 Oct 2023 → 29 Oct 2023

Publication series

Name	Communications in Computer and Information Science
Volume	2080 CCIS
ISSN (Print)	1865-0929
ISSN (Electronic)	1865-0937

Conference

Conference	Evaluation track of the 9th China Health Information Processing Conference, CHIP 2023
Country/Territory	China
City	Hangzhou
Period	27/10/23 → 29/10/23

Keywords

Health Information Extraction
In-context Learning
Large Language Model

Access to Document

10.1007/978-981-97-1717-0_4

Cite this

Liu, J., Wang, J., Huang, H., Zhang, R., Yang, M., & Zhao, T. (2024). Improving LLM-Based Health Information Extraction with In-Context Learning. In H. Xu, Q. Chen, H. Lin, F. Wu, L. Liu, B. Tang, T. Hao, Z. Huang, J. Lei, Z. Li, & H. Zong (Eds.), Health Information Processing. Evaluation Track Papers - 9th China Conference, CHIP 2023, Proceedings (pp. 49-59). (Communications in Computer and Information Science; Vol. 2080 CCIS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-97-1717-0_4

Liu, Junkai ; Wang, Jiayi ; Huang, Hui et al. / Improving LLM-Based Health Information Extraction with In-Context Learning. Health Information Processing. Evaluation Track Papers - 9th China Conference, CHIP 2023, Proceedings. editor / Hua Xu ; Qingcai Chen ; Hongfei Lin ; Fei Wu ; Lei Liu ; Buzhou Tang ; Tianyong Hao ; Zhengxing Huang ; Jianbo Lei ; Zuofeng Li ; Hui Zong. Springer Science and Business Media Deutschland GmbH, 2024. pp. 49-59 (Communications in Computer and Information Science).

@inproceedings{788a8b05a444437a9e45ca1aa30c09c6,

title = "Improving LLM-Based Health Information Extraction with In-Context Learning",

abstract = "The Large Language Model (LLM) has received widespread attention in the industry. In the context of the popularity of LLM, almost all NLP tasks are transformed into prompt based language generation tasks. On the other hand, LLM can also achieve superior results on brand new tasks without fine-tuning, solely with a few in-context examples. This paper describes our participation in the China Health Information Processing Conference (CHIP 2023). We focused on in-context learning (ICL) and experimented with different combinations of demonstration retrieval strategies on the given task and tested the optimal strategy combination proposed by us. The experimental results show that our retrieval strategies based on Chinese-LlaMA2-13B-chat achieved a average score of 40.27, ranked the first place among five teams, confirmed the effectiveness of our method.",

keywords = "Health Information Extraction, In-context Learning, Large Language Model",

author = "Junkai Liu and Jiayi Wang and Hui Huang and Rui Zhang and Muyun Yang and Tiejun Zhao",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.; Evaluation track of the 9th China Health Information Processing Conference, CHIP 2023 ; Conference date: 27-10-2023 Through 29-10-2023",

year = "2024",

doi = "10.1007/978-981-97-1717-0\_4",

language = "英语",

isbn = "9789819717163",

series = "Communications in Computer and Information Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "49--59",

editor = "Hua Xu and Qingcai Chen and Hongfei Lin and Fei Wu and Lei Liu and Buzhou Tang and Tianyong Hao and Zhengxing Huang and Jianbo Lei and Zuofeng Li and Hui Zong",

booktitle = "Health Information Processing. Evaluation Track Papers - 9th China Conference, CHIP 2023, Proceedings",

address = "德国",

}

Liu, J, Wang, J, Huang, H, Zhang, R, Yang, M & Zhao, T 2024, Improving LLM-Based Health Information Extraction with In-Context Learning. in H Xu, Q Chen, H Lin, F Wu, L Liu, B Tang, T Hao, Z Huang, J Lei, Z Li & H Zong (eds), Health Information Processing. Evaluation Track Papers - 9th China Conference, CHIP 2023, Proceedings. Communications in Computer and Information Science, vol. 2080 CCIS, Springer Science and Business Media Deutschland GmbH, pp. 49-59, Evaluation track of the 9th China Health Information Processing Conference, CHIP 2023, Hangzhou, China, 27/10/23. https://doi.org/10.1007/978-981-97-1717-0_4

Improving LLM-Based Health Information Extraction with In-Context Learning. / Liu, Junkai; Wang, Jiayi; Huang, Hui et al.
Health Information Processing. Evaluation Track Papers - 9th China Conference, CHIP 2023, Proceedings. ed. / Hua Xu; Qingcai Chen; Hongfei Lin; Fei Wu; Lei Liu; Buzhou Tang; Tianyong Hao; Zhengxing Huang; Jianbo Lei; Zuofeng Li; Hui Zong. Springer Science and Business Media Deutschland GmbH, 2024. p. 49-59 (Communications in Computer and Information Science; Vol. 2080 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Improving LLM-Based Health Information Extraction with In-Context Learning

AU - Liu, Junkai

AU - Wang, Jiayi

AU - Huang, Hui

AU - Zhang, Rui

AU - Yang, Muyun

AU - Zhao, Tiejun

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.

PY - 2024

Y1 - 2024

N2 - The Large Language Model (LLM) has received widespread attention in the industry. In the context of the popularity of LLM, almost all NLP tasks are transformed into prompt based language generation tasks. On the other hand, LLM can also achieve superior results on brand new tasks without fine-tuning, solely with a few in-context examples. This paper describes our participation in the China Health Information Processing Conference (CHIP 2023). We focused on in-context learning (ICL) and experimented with different combinations of demonstration retrieval strategies on the given task and tested the optimal strategy combination proposed by us. The experimental results show that our retrieval strategies based on Chinese-LlaMA2-13B-chat achieved a average score of 40.27, ranked the first place among five teams, confirmed the effectiveness of our method.

AB - The Large Language Model (LLM) has received widespread attention in the industry. In the context of the popularity of LLM, almost all NLP tasks are transformed into prompt based language generation tasks. On the other hand, LLM can also achieve superior results on brand new tasks without fine-tuning, solely with a few in-context examples. This paper describes our participation in the China Health Information Processing Conference (CHIP 2023). We focused on in-context learning (ICL) and experimented with different combinations of demonstration retrieval strategies on the given task and tested the optimal strategy combination proposed by us. The experimental results show that our retrieval strategies based on Chinese-LlaMA2-13B-chat achieved a average score of 40.27, ranked the first place among five teams, confirmed the effectiveness of our method.

KW - Health Information Extraction

KW - In-context Learning

KW - Large Language Model

UR - https://www.scopus.com/pages/publications/85189633014

U2 - 10.1007/978-981-97-1717-0_4

DO - 10.1007/978-981-97-1717-0_4

M3 - 会议稿件

AN - SCOPUS:85189633014

SN - 9789819717163

T3 - Communications in Computer and Information Science

SP - 49

EP - 59

BT - Health Information Processing. Evaluation Track Papers - 9th China Conference, CHIP 2023, Proceedings

A2 - Xu, Hua

A2 - Chen, Qingcai

A2 - Lin, Hongfei

A2 - Wu, Fei

A2 - Liu, Lei

A2 - Tang, Buzhou

A2 - Hao, Tianyong

A2 - Huang, Zhengxing

A2 - Lei, Jianbo

A2 - Li, Zuofeng

A2 - Zong, Hui

PB - Springer Science and Business Media Deutschland GmbH

T2 - Evaluation track of the 9th China Health Information Processing Conference, CHIP 2023

Y2 - 27 October 2023 through 29 October 2023

ER -

Liu J, Wang J, Huang H, Zhang R, Yang M , Zhao T. Improving LLM-Based Health Information Extraction with In-Context Learning. In Xu H, Chen Q, Lin H, Wu F, Liu L, Tang B, Hao T, Huang Z, Lei J, Li Z, Zong H, editors, Health Information Processing. Evaluation Track Papers - 9th China Conference, CHIP 2023, Proceedings. Springer Science and Business Media Deutschland GmbH. 2024. p. 49-59. (Communications in Computer and Information Science). doi: 10.1007/978-981-97-1717-0_4

Improving LLM-Based Health Information Extraction with In-Context Learning

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this