Prompt-based Extraction of Social Determinants of Health Using Few-shot Learning

Giridhar Kaushik Ramachandran; Yujuan Fu; Bin Han; Kevin Lybarger; Nicholas J. Dobbins; Özlem Uzuner; Meliha Yetisgen

Prompt-based Extraction of Social Determinants of Health Using Few-shot Learning

Giridhar Kaushik Ramachandran, Yujuan Fu, Bin Han, Kevin Lybarger, Nicholas J. Dobbins, Özlem Uzuner, Meliha Yetisgen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Social determinants of health (SDOH) documented in the electronic health record through unstructured text are increasingly being studied to understand how SDOH impacts patient health outcomes. In this work, we utilize the Social History Annotation Corpus (SHAC), a multi-institutional corpus of de-identified social history sections annotated for SDOH, including substance use, employment, and living status information. We explore the automatic extraction of SDOH information with SHAC in both standoff and inline annotation formats using GPT-4 in a one-shot prompting setting. We compare GPT-4 extraction performance with a high-performing supervised approach and perform thorough error analyses. Our prompt-based GPT-4 method achieved an overall 0.652 F1 on the SHAC test set, similar to the 7th best-performing system among all teams in the n2c2 challenge with SHAC.

Original language	English (US)
Title of host publication	5th Workshop on Clinical Natural Language Processing, ClinicalNLP 2023 - Proceedings of the Workshop
Publisher	Association for Computational Linguistics (ACL)
Pages	385-393
Number of pages	9
ISBN (Electronic)	9781959429883
State	Published - 2023
Externally published	Yes
Event	5th Workshop on Clinical Natural Language Processing, ClinicalNLP 2023. held at ACL 2023 - Toronto, Canada Duration: Jul 14 2023 → …

Publication series

Name	Proceedings of the Annual Meeting of the Association for Computational Linguistics
ISSN (Print)	0736-587X

Conference

Conference	5th Workshop on Clinical Natural Language Processing, ClinicalNLP 2023. held at ACL 2023
Country/Territory	Canada
City	Toronto
Period	7/14/23 → …

ASJC Scopus subject areas

Computer Science Applications
Linguistics and Language
Language and Linguistics

Cite this

Ramachandran, G. K., Fu, Y., Han, B., Lybarger, K., Dobbins, N. J., Uzuner, Ö., & Yetisgen, M. (2023). Prompt-based Extraction of Social Determinants of Health Using Few-shot Learning. In 5th Workshop on Clinical Natural Language Processing, ClinicalNLP 2023 - Proceedings of the Workshop (pp. 385-393). (Proceedings of the Annual Meeting of the Association for Computational Linguistics). Association for Computational Linguistics (ACL).

Prompt-based Extraction of Social Determinants of Health Using Few-shot Learning. / Ramachandran, Giridhar Kaushik; Fu, Yujuan; Han, Bin et al.
5th Workshop on Clinical Natural Language Processing, ClinicalNLP 2023 - Proceedings of the Workshop. Association for Computational Linguistics (ACL), 2023. p. 385-393 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Ramachandran, GK, Fu, Y, Han, B, Lybarger, K, Dobbins, NJ, Uzuner, Ö & Yetisgen, M 2023, Prompt-based Extraction of Social Determinants of Health Using Few-shot Learning. in 5th Workshop on Clinical Natural Language Processing, ClinicalNLP 2023 - Proceedings of the Workshop. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), pp. 385-393, 5th Workshop on Clinical Natural Language Processing, ClinicalNLP 2023. held at ACL 2023, Toronto, Canada, 7/14/23.

Ramachandran GK, Fu Y, Han B, Lybarger K, Dobbins NJ, Uzuner Ö et al. Prompt-based Extraction of Social Determinants of Health Using Few-shot Learning. In 5th Workshop on Clinical Natural Language Processing, ClinicalNLP 2023 - Proceedings of the Workshop. Association for Computational Linguistics (ACL). 2023. p. 385-393. (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Ramachandran, Giridhar Kaushik ; Fu, Yujuan ; Han, Bin et al. / Prompt-based Extraction of Social Determinants of Health Using Few-shot Learning. 5th Workshop on Clinical Natural Language Processing, ClinicalNLP 2023 - Proceedings of the Workshop. Association for Computational Linguistics (ACL), 2023. pp. 385-393 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

@inproceedings{e738809b5b204c91b0da79d98fe450c7,

title = "Prompt-based Extraction of Social Determinants of Health Using Few-shot Learning",

abstract = "Social determinants of health (SDOH) documented in the electronic health record through unstructured text are increasingly being studied to understand how SDOH impacts patient health outcomes. In this work, we utilize the Social History Annotation Corpus (SHAC), a multi-institutional corpus of de-identified social history sections annotated for SDOH, including substance use, employment, and living status information. We explore the automatic extraction of SDOH information with SHAC in both standoff and inline annotation formats using GPT-4 in a one-shot prompting setting. We compare GPT-4 extraction performance with a high-performing supervised approach and perform thorough error analyses. Our prompt-based GPT-4 method achieved an overall 0.652 F1 on the SHAC test set, similar to the 7th best-performing system among all teams in the n2c2 challenge with SHAC.",

author = "Ramachandran, {Giridhar Kaushik} and Yujuan Fu and Bin Han and Kevin Lybarger and Dobbins, {Nicholas J.} and {\"O}zlem Uzuner and Meliha Yetisgen",

note = "Publisher Copyright: {\textcopyright} 2023 Association for Computational Linguistics.; 5th Workshop on Clinical Natural Language Processing, ClinicalNLP 2023. held at ACL 2023 ; Conference date: 14-07-2023",

year = "2023",

language = "English (US)",

series = "Proceedings of the Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

pages = "385--393",

booktitle = "5th Workshop on Clinical Natural Language Processing, ClinicalNLP 2023 - Proceedings of the Workshop",

address = "United States",

}

TY - GEN

T1 - Prompt-based Extraction of Social Determinants of Health Using Few-shot Learning

AU - Ramachandran, Giridhar Kaushik

AU - Fu, Yujuan

AU - Han, Bin

AU - Lybarger, Kevin

AU - Dobbins, Nicholas J.

AU - Uzuner, Özlem

AU - Yetisgen, Meliha

PY - 2023

Y1 - 2023

N2 - Social determinants of health (SDOH) documented in the electronic health record through unstructured text are increasingly being studied to understand how SDOH impacts patient health outcomes. In this work, we utilize the Social History Annotation Corpus (SHAC), a multi-institutional corpus of de-identified social history sections annotated for SDOH, including substance use, employment, and living status information. We explore the automatic extraction of SDOH information with SHAC in both standoff and inline annotation formats using GPT-4 in a one-shot prompting setting. We compare GPT-4 extraction performance with a high-performing supervised approach and perform thorough error analyses. Our prompt-based GPT-4 method achieved an overall 0.652 F1 on the SHAC test set, similar to the 7th best-performing system among all teams in the n2c2 challenge with SHAC.

AB - Social determinants of health (SDOH) documented in the electronic health record through unstructured text are increasingly being studied to understand how SDOH impacts patient health outcomes. In this work, we utilize the Social History Annotation Corpus (SHAC), a multi-institutional corpus of de-identified social history sections annotated for SDOH, including substance use, employment, and living status information. We explore the automatic extraction of SDOH information with SHAC in both standoff and inline annotation formats using GPT-4 in a one-shot prompting setting. We compare GPT-4 extraction performance with a high-performing supervised approach and perform thorough error analyses. Our prompt-based GPT-4 method achieved an overall 0.652 F1 on the SHAC test set, similar to the 7th best-performing system among all teams in the n2c2 challenge with SHAC.

UR - http://www.scopus.com/inward/record.url?scp=85175428586&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85175428586&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85175428586

T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics

SP - 385

EP - 393

BT - 5th Workshop on Clinical Natural Language Processing, ClinicalNLP 2023 - Proceedings of the Workshop

PB - Association for Computational Linguistics (ACL)

T2 - 5th Workshop on Clinical Natural Language Processing, ClinicalNLP 2023. held at ACL 2023

Y2 - 14 July 2023

ER -

Prompt-based Extraction of Social Determinants of Health Using Few-shot Learning

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this