Knowledge-centric Prompt Composition for Knowledge Base Construction from Pre-trained Language Models

Xue Li; Anthony Hughes; Majlinda Llugiqi; Fina Polat; Paul Groth; Fajar J. Ekaputra

Knowledge-centric Prompt Composition for Knowledge Base Construction from Pre-trained Language Models

Xue Li
, Anthony Hughes
, Majlinda Llugiqi
, Fina Polat
, Paul Groth
, Fajar J. Ekaputra

Institute for Data, Process and Knowledge Management (AE Sabou)

Publication: Chapter in book/Conference proceeding › Contribution to conference proceedings

Abstract

Pretrained language models (PLMs), exemplified by the GPT family of models, have exhibited remarkable proficiency across a spectrum of natural language processing tasks and have displayed potential for extracting knowledge from within the model itself. While numerous endeavors have delved into this
capability through probing or prompting methodologies, the potential for constructing comprehensive knowledge bases from PLMs remains relatively uncharted. The Knowledge Base Construction from Pre-trained Language Model Challenge (LM-KBC) [1] looks to bridge this gap. This paper presents the
system implementation from team thames to Track 2 of LM-KBC. Our methodology achieves 67 % F1 score on the test set provided by the organisers outperforming the baseline by over 40 points, which ranked 2nd place for Track 2. It does so through the use of additional prompt context derived from both
training data and the constraints and descriptions of the relations. All code and results can be found on GitHub.

Original language	English
Title of host publication	Joint proceedings of the KBC-LM workshop and the LM-KBC challenge @ ISWC 2023
Subtitle of host publication	Joint proceedings of the 1st workshop on Knowledge Base Construction from Pre-trained Language Models the 2nd challenge on Language Models for Knowledge Base Construction (LM-KBC) (KBC-LM + LM-KBC 2023) : Athens, Greece, November 6, 2023
Editors	Simon Razniewski, Jan-Christoph Kalo, Sneha Singhania, Jeff Z. Pan
Publisher	CEUR Workshop Proceedings
Number of pages	13
Publication status	Published - 2023

Publication series

Series	CEUR Workshop Proceedings
Volume	3577
ISSN	1613-0073

Access to Document

https://ceur-ws.org/Vol-3577/paper3.pdfLicence: CC BY

Cite this

Li, X., Hughes, A., Llugiqi, M., Polat, F., Groth, P., & Ekaputra, F. J. (2023). Knowledge-centric Prompt Composition for Knowledge Base Construction from Pre-trained Language Models. In S. Razniewski, J.-C. Kalo, S. Singhania, & J. Z. Pan (Eds.), Joint proceedings of the KBC-LM workshop and the LM-KBC challenge @ ISWC 2023: Joint proceedings of the 1st workshop on Knowledge Base Construction from Pre-trained Language Models the 2nd challenge on Language Models for Knowledge Base Construction (LM-KBC) (KBC-LM + LM-KBC 2023) : Athens, Greece, November 6, 2023 CEUR Workshop Proceedings. https://ceur-ws.org/Vol-3577/paper3.pdf

Li, Xue ; Hughes, Anthony ; Llugiqi, Majlinda et al. / Knowledge-centric Prompt Composition for Knowledge Base Construction from Pre-trained Language Models. Joint proceedings of the KBC-LM workshop and the LM-KBC challenge @ ISWC 2023: Joint proceedings of the 1st workshop on Knowledge Base Construction from Pre-trained Language Models the 2nd challenge on Language Models for Knowledge Base Construction (LM-KBC) (KBC-LM + LM-KBC 2023) : Athens, Greece, November 6, 2023. editor / Simon Razniewski ; Jan-Christoph Kalo ; Sneha Singhania ; Jeff Z. Pan. CEUR Workshop Proceedings, 2023. (CEUR Workshop Proceedings, Vol. 3577).

@inproceedings{486d8ffea97347d18d034adc0309edeb,

title = "Knowledge-centric Prompt Composition for Knowledge Base Construction from Pre-trained Language Models",

abstract = "Pretrained language models (PLMs), exemplified by the GPT family of models, have exhibited remarkable proficiency across a spectrum of natural language processing tasks and have displayed potential for extracting knowledge from within the model itself. While numerous endeavors have delved into this capability through probing or prompting methodologies, the potential for constructing comprehensive knowledge bases from PLMs remains relatively uncharted. The Knowledge Base Construction from Pre-trained Language Model Challenge (LM-KBC) [1] looks to bridge this gap. This paper presents the system implementation from team thames to Track 2 of LM-KBC. Our methodology achieves 67 \% F1 score on the test set provided by the organisers outperforming the baseline by over 40 points, which ranked 2nd place for Track 2. It does so through the use of additional prompt context derived from both training data and the constraints and descriptions of the relations. All code and results can be found on GitHub.",

author = "Xue Li and Anthony Hughes and Majlinda Llugiqi and Fina Polat and Paul Groth and Ekaputra, \{Fajar J.\}",

year = "2023",

language = "English",

series = "CEUR Workshop Proceedings",

publisher = "CEUR Workshop Proceedings",

editor = "Razniewski, \{Simon \} and Kalo, \{Jan-Christoph \} and Singhania, \{Sneha \} and Pan, \{Jeff Z. \}",

booktitle = "Joint proceedings of the KBC-LM workshop and the LM-KBC challenge @ ISWC 2023",

}

Li, X, Hughes, A, Llugiqi, M, Polat, F, Groth, P & Ekaputra, FJ 2023, Knowledge-centric Prompt Composition for Knowledge Base Construction from Pre-trained Language Models. in S Razniewski, J-C Kalo, S Singhania & JZ Pan (eds), Joint proceedings of the KBC-LM workshop and the LM-KBC challenge @ ISWC 2023: Joint proceedings of the 1st workshop on Knowledge Base Construction from Pre-trained Language Models the 2nd challenge on Language Models for Knowledge Base Construction (LM-KBC) (KBC-LM + LM-KBC 2023) : Athens, Greece, November 6, 2023. CEUR Workshop Proceedings, CEUR Workshop Proceedings, vol. 3577. <https://ceur-ws.org/Vol-3577/paper3.pdf>

Knowledge-centric Prompt Composition for Knowledge Base Construction from Pre-trained Language Models. / Li, Xue; Hughes, Anthony; Llugiqi, Majlinda et al.
Joint proceedings of the KBC-LM workshop and the LM-KBC challenge @ ISWC 2023: Joint proceedings of the 1st workshop on Knowledge Base Construction from Pre-trained Language Models the 2nd challenge on Language Models for Knowledge Base Construction (LM-KBC) (KBC-LM + LM-KBC 2023) : Athens, Greece, November 6, 2023. ed. / Simon Razniewski; Jan-Christoph Kalo; Sneha Singhania; Jeff Z. Pan. CEUR Workshop Proceedings, 2023. (CEUR Workshop Proceedings, Vol. 3577).

Publication: Chapter in book/Conference proceeding › Contribution to conference proceedings

TY - GEN

T1 - Knowledge-centric Prompt Composition for Knowledge Base Construction from Pre-trained Language Models

AU - Li, Xue

AU - Hughes, Anthony

AU - Llugiqi, Majlinda

AU - Polat, Fina

AU - Groth, Paul

AU - Ekaputra, Fajar J.

PY - 2023

Y1 - 2023

N2 - Pretrained language models (PLMs), exemplified by the GPT family of models, have exhibited remarkable proficiency across a spectrum of natural language processing tasks and have displayed potential for extracting knowledge from within the model itself. While numerous endeavors have delved into this capability through probing or prompting methodologies, the potential for constructing comprehensive knowledge bases from PLMs remains relatively uncharted. The Knowledge Base Construction from Pre-trained Language Model Challenge (LM-KBC) [1] looks to bridge this gap. This paper presents the system implementation from team thames to Track 2 of LM-KBC. Our methodology achieves 67 % F1 score on the test set provided by the organisers outperforming the baseline by over 40 points, which ranked 2nd place for Track 2. It does so through the use of additional prompt context derived from both training data and the constraints and descriptions of the relations. All code and results can be found on GitHub.

AB - Pretrained language models (PLMs), exemplified by the GPT family of models, have exhibited remarkable proficiency across a spectrum of natural language processing tasks and have displayed potential for extracting knowledge from within the model itself. While numerous endeavors have delved into this capability through probing or prompting methodologies, the potential for constructing comprehensive knowledge bases from PLMs remains relatively uncharted. The Knowledge Base Construction from Pre-trained Language Model Challenge (LM-KBC) [1] looks to bridge this gap. This paper presents the system implementation from team thames to Track 2 of LM-KBC. Our methodology achieves 67 % F1 score on the test set provided by the organisers outperforming the baseline by over 40 points, which ranked 2nd place for Track 2. It does so through the use of additional prompt context derived from both training data and the constraints and descriptions of the relations. All code and results can be found on GitHub.

M3 - Contribution to conference proceedings

T3 - CEUR Workshop Proceedings

BT - Joint proceedings of the KBC-LM workshop and the LM-KBC challenge @ ISWC 2023

A2 - Razniewski, Simon

A2 - Kalo, Jan-Christoph

A2 - Singhania, Sneha

A2 - Pan, Jeff Z.

PB - CEUR Workshop Proceedings

ER -

Li X, Hughes A, Llugiqi M, Polat F, Groth P, Ekaputra FJ. Knowledge-centric Prompt Composition for Knowledge Base Construction from Pre-trained Language Models. In Razniewski S, Kalo JC, Singhania S, Pan JZ, editors, Joint proceedings of the KBC-LM workshop and the LM-KBC challenge @ ISWC 2023: Joint proceedings of the 1st workshop on Knowledge Base Construction from Pre-trained Language Models the 2nd challenge on Language Models for Knowledge Base Construction (LM-KBC) (KBC-LM + LM-KBC 2023) : Athens, Greece, November 6, 2023. CEUR Workshop Proceedings. 2023. (CEUR Workshop Proceedings, Vol. 3577).