DistilBERT with Adam Optimizer Tuning for Text-based Emotion Detection
Emotion detection (ED) refers to identifying individual emotions or feelings, such as happiness, sadness, disappointment, fear, etc. The classic machine learning technique still relies on feature engineering, which makes it difficult to convey the meaning of words. Deep learning-based algorithms have recently been shown to be beneficial for emotion detection because they require only a simple feature creation process. Transfer learning is an approach that uses data similarities, data distribution, models, tasks, and other factors to apply knowledge learnt in one domain to a new domain. This study is to shed light on the fine-tuned models' efficacy in detecting emotions from the International Survey on Emotion Antecedents and Reactions (ISEAR) dataset. In order to optimize the model, we conducted the hyperparameters tuning on Adam optimizer in DistilBERT. The experiment examined the moment estimators and learning rate of Adam optimizer. The effect of the parameters to training and validation accuracy were presented and analyzed.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlike International License (CC-BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
Copyright without Restrictions
The journal allows the author(s) to hold the copyright without restrictions and will retain publishing rights without restrictions.
The submitted papers are assumed to contain no proprietary material unprotected by patent or patent application; responsibility for technical content and for protection of proprietary material rests solely with the author(s) and their organizations and is not the responsibility of the IJNMT or its Editorial Staff. The main (first/corresponding) author is responsible for ensuring that the article has been seen and approved by all the other authors. It is the responsibility of the author to obtain all necessary copyright release permissions for the use of any copyrighted materials in the manuscript prior to the submission.