dc.date.accessioned | 2024-04-08T16:08:32Z | |
dc.date.available | 2024-04-08T16:08:32Z | |
dc.date.created | 2024-04-02T13:08:22Z | |
dc.date.issued | 2024 | |
dc.identifier.citation | Rønningstad, Egil Velldal, Erik Øvrelid, Lilja . A GPT among Annotators: LLM-based Entity-Level Sentiment Annotation. Proceedings of The 18th Linguistic Annotation Workshop (LAW-XVIII). 2024, 133-139 Association for Computational Linguistics | |
dc.identifier.uri | http://hdl.handle.net/10852/110500 | |
dc.description.abstract | We investigate annotator variation for the novel task of Entity-Level Sentiment Analysis (ELSA) which annotates the aggregated sentiment directed towards volitional entities in a text. More specifically, we analyze the annotations of a newly constructed Norwegian ELSA dataset and release additional data with each annotator’s labels for the 247 entities in the dataset’s test split. We also perform a number of experiments prompting ChatGPT for these sentiment labels regarding each entity in the text and compare the generated annotations with the human labels. Cohen’s Kappa for agreement between the best LLM-generated labels and curated gold was 0.425, which indicates that these labels would not have high quality. Our analyses further investigate the errors that ChatGPT outputs, and compare them with the variations that we find among the 5 trained annotators that all annotated the same test data. | |
dc.language | EN | |
dc.publisher | Association for Computational Linguistics | |
dc.rights | Attribution 4.0 International | |
dc.rights.uri | https://creativecommons.org/licenses/by/4.0/ | |
dc.title | A GPT among Annotators: LLM-based Entity-Level Sentiment Annotation | |
dc.title.alternative | ENEngelskEnglishA GPT among Annotators: LLM-based Entity-Level Sentiment Annotation | |
dc.type | Chapter | |
dc.creator.author | Rønningstad, Egil | |
dc.creator.author | Velldal, Erik | |
dc.creator.author | Øvrelid, Lilja | |
cristin.unitcode | 185,15,5,48 | |
cristin.unitname | Forskningsgruppen for språkteknologi | |
cristin.ispublished | true | |
cristin.fulltext | original | |
dc.identifier.cristin | 2258077 | |
dc.identifier.bibliographiccitation | info:ofi/fmt:kev:mtx:ctx&ctx_ver=Z39.88-2004&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.btitle=Proceedings of The 18th Linguistic Annotation Workshop (LAW-XVIII)&rft.spage=133&rft.date=2024 | |
dc.identifier.startpage | 133 | |
dc.identifier.endpage | 139 | |
dc.identifier.pagecount | 223 | |
dc.type.document | Bokkapittel | |
dc.type.peerreviewed | Peer reviewed | |
dc.source.isbn | 979-8-89176-073-8 | |
dc.type.version | PublishedVersion | |
cristin.btitle | Proceedings of The 18th Linguistic Annotation Workshop (LAW-XVIII) | |