MultiLexBATS: Multilingual Dataset of Lexical Semantic Relations

Dagmar Gromann (1) , Hugo Gonçalo Oliveira (2, 3) , Lucia Pitarch (4) , Elena-Simona Apostol (5, 6) , Jordi Bernad (4) , Eliot Bytyçi (7) , Chiara Cantone (8) , Sara Carvalho (9, 10) , Francesca Frontini (11) , Radovan Garabík (12) , Jorge Gracia (13) , Letizia Granata (14) , Anas Fahad Khan (11) , Timotej Knez (15) , Penny Labropoulou (16) , Chaya Liebeskind (17) , Maria Pia Di Buono (14) , Ana Ostroški Anić (18) , Sigita Rackevičienė (19) , Ricardo Rodrigues (20) , Gilles Serasset (21) , Linas Selmistraitis (19) , Mahammadou Sidibé (21) , Purificação Silvano (22) , Blerina Spahiu (23) , Enriketa Sogutlu (24) , Ranka Stanković (25, 26) , Ciprian-Octavian Truica (5) , Giedrė Valūnaitė Oleškevičienė (27) , Slavko Zitnik (15) , Katerina Zdravkova (28)

1 University of Vienna [Vienna]
2 Department of Informatics Engineering - DEI, University of Coimbra
3 Universidade de Coimbra [Coimbra]
4 University of Zaragoza - Universidad de Zaragoza [Zaragoza]
5 UPB - University Politehnica of Bucharest [Romania]
6 DIT-UPPSALA - Department of Information Technology
7 University of Prishtina
8 UNISI - Università degli Studi di Siena = University of Siena
9 CLUNL - Centro de Linguística da Universidade Nova de Lisboa
10 CLLC - Centro de Línguas, Literaturas e Culturas
11 CNR-ILC - Istituto di Linguistica Computazionale "Antonio Zampolli"
12 SAS - Slovak Academy of Sciences
13 I3A - Aragón Institute of Engineering Research [Zaragoza]
14 UniOr - Università di Napoli L'Orientale = University of Naples
15 University of Ljubljana
16 ILSP - Institute for Language and Speech Processing
17 JCT - Jerusalem College of Technology
18 Institute for the Croatian Language, Croatia
19 Mykolas Romeris University
20 CISUC - Centre for Informatics and Systems
21 GETALP - Groupe d’Étude en Traduction Automatique/Traitement Automatisé des Langues et de la Parole
22 Universidade do Porto = University of Porto
23 UNIMIB - Università degli Studi di Milano-Bicocca = University of Milano-Bicocca
24 University College Beder
25 University of Belgrade, Faculty of Mining and Geology
26 University of Belgrade [Belgrade]
27 Mykolas Romeris University
28 UKIM - Ss. Cyril and Methodius University in Skopje

Understanding the relation between the meanings of words is an important part of comprehending natural language. Prior work has either focused on analysing lexical semantic relations in word embeddings or probing pretrained language models (PLMs), with some exceptions. Given the rarity of highly multilingual benchmarks, it is unclear to what extent PLMs capture relational knowledge and are able to transfer it across languages. To start addressing this question, we propose MultiLexBATS, a multilingual parallel dataset of lexical semantic relations adapted from BATS in 15 languages including low-resource languages, such as Bambara, Lithuanian, and Albanian. As experiment on cross-lingual transfer of relational knowledge, we test the PLMs’ ability to (1) capture analogies across languages, and (2) predict translation targets. We find considerable differences across relation types and languages with a clear preference for hypernymy and antonymy as well as romance languages.

Mots clés: Lexical Semantic Relations Multilingual Benchmark BATS; Lexical Semantic Relations; Multilingual Benchmark; BATS

Domaines: Informatique et langage [cs.CL]

Dagmar Gromann, Hugo Gonçalo Oliveira, Lucia Pitarch, Elena-Simona Apostol, Jordi Bernad, et al.. MultiLexBATS: Multilingual Dataset of Lexical Semantic Relations. LREC-COLING 2024, ELDA, May 2024, Torino, Italy. ⟨hal-04539892⟩