Evaluation of Distributional Semantic Models of Ancient Greek: Preliminary Results and a Road Map for Future Work

TY - CHAP

T1 - Evaluation of Distributional Semantic Models of Ancient Greek

T2 - Preliminary Results and a Road Map for Future Work

AU - Stopponi, Silvia

AU - Pedrazzini, Nilo

AU - Peels, Saskia

AU - McGillivray, Barbara

AU - Nissim, Malvina

PY - 2023/9

Y1 - 2023/9

N2 - We evaluate four count-based and predictive distributional semantic models of Ancient Greek against AGREE, a composite benchmark of human judgements, to assess their ability to retrieve semantic relatedness. On the basis of the observations deriving from the analysis of the results, we design a procedure for a larger-scale intrinsic evaluation of count-based and predictive language models, including syntactic embeddings. We also propose possible ways of exploiting the different layers of the whole AGREE benchmark (including both human- and machine-generated data) and different evaluation metrics.

AB - We evaluate four count-based and predictive distributional semantic models of Ancient Greek against AGREE, a composite benchmark of human judgements, to assess their ability to retrieve semantic relatedness. On the basis of the observations deriving from the analysis of the results, we design a procedure for a larger-scale intrinsic evaluation of count-based and predictive language models, including syntactic embeddings. We also propose possible ways of exploiting the different layers of the whole AGREE benchmark (including both human- and machine-generated data) and different evaluation metrics.

M3 - Conference paper

SP - 49

EP - 58

BT - Proceedings of the 1st International Workshop on Ancient Language Processing (ALP) at RANLP

PB - INCOMA Ltd., Shoumen, Bulgaria

ER -