TY - CONF
T1 - Lon-eå at SemEval-2023 Task 11
T2 - 17th International Workshop on Semantic Evaluation, SemEval 2023, co-located with the 61st Annual Meeting of the Association for Computational Linguistics, ACL 2023
AU - Hosseini, Peyman
AU - Hosseini, Mehran
AU - Al-Azzawi, Sana Sabah
AU - Liwicki, Marcus
AU - Castro, Ignacio
AU - Purver, Matthew
N1 - Funding Information:
This work is partially supported by the UK EPSRC via the Centre for Doctoral Training in Intelligent Games and Game Intelligence (IGGI; EP/S022325/1) and the projects Sode-stream (EP/S033564/1), AP4L (EP/W032473/1), REPHRAIN (EP/V011189/1) and ARCIDUCA (EP/W001632/1); as well as the Slovenian Research Agency via research core funding for the programme Knowledge Technologies (P2-0103) and the project SOVRAG (Hate speech in contemporary conceptualizations of nationalism, racism, gender and migration, J5-3102). We also thank Zahraa Al Sahili for providing insightful comments during the early stages of this work.
Publisher Copyright:
© 2023 Association for Computational Linguistics.
PY - 2023/7/13
Y1 - 2023/7/13
N2 - We study the influence of different activation functions in the output layer of deep neural network models for soft and hard label prediction in the learning with disagreement task. In this task, the goal is to quantify the amount of disagreement via predicting soft labels. To predict the soft labels, we use BERT-based preprocessors and encoders and vary the activation function used in the output layer, while keeping other parameters constant. The soft labels are then used for the hard label prediction. The activation functions considered are sigmoid as well as a step-function that is added to the model post-training and a sinusoidal activation function, which is introduced for the first time in this paper.
AB - We study the influence of different activation functions in the output layer of deep neural network models for soft and hard label prediction in the learning with disagreement task. In this task, the goal is to quantify the amount of disagreement via predicting soft labels. To predict the soft labels, we use BERT-based preprocessors and encoders and vary the activation function used in the output layer, while keeping other parameters constant. The soft labels are then used for the hard label prediction. The activation functions considered are sigmoid as well as a step-function that is added to the model post-training and a sinusoidal activation function, which is introduced for the first time in this paper.
UR - http://www.scopus.com/inward/record.url?scp=85160934844&partnerID=8YFLogxK
M3 - Other
AN - SCOPUS:85160934844
SP - 1329
EP - 1334
Y2 - 13 July 2023 through 14 July 2023
ER -