Text classification using improved bidirectional transformer

dc.authoridYILDIZ, Beytullah/0000-0001-7664-5145
dc.authoridTezgider, Murat/0000-0002-4918-5697
dc.authorscopusid57207471698
dc.authorscopusid14632851900
dc.authorscopusid8338657900
dc.contributor.authorTezgider, Murat
dc.contributor.authorYıldız, Beytullah
dc.contributor.authorYildiz, Beytullah
dc.contributor.authorAydin, Galip
dc.contributor.authorYıldız, Beytullah
dc.contributor.otherSoftware Engineering
dc.date.accessioned2024-07-05T15:21:19Z
dc.date.available2024-07-05T15:21:19Z
dc.date.issued2022
dc.departmentAtılım Universityen_US
dc.department-temp[Tezgider, Murat; Aydin, Galip] Firat Univ, Fac Engn, Dept Comp Engn, TR-23200 Elazig, Turkey; [Yildiz, Beytullah] Atilim Univ, Sch Engn, Dept Software Engn, Ankara, Turkeyen_US
dc.descriptionYILDIZ, Beytullah/0000-0001-7664-5145; Tezgider, Murat/0000-0002-4918-5697en_US
dc.description.abstractText data have an important place in our daily life. A huge amount of text data is generated everyday. As a result, automation becomes necessary to handle these large text data. Recently, we are witnessing important developments with the adaptation of new approaches in text processing. Attention mechanisms and transformers are emerging as methods with significant potential for text processing. In this study, we introduced a bidirectional transformer (BiTransformer) constructed using two transformer encoder blocks that utilize bidirectional position encoding to take into account the forward and backward position information of text data. We also created models to evaluate the contribution of attention mechanisms to the classification process. Four models, including long short term memory, attention, transformer, and BiTransformer, were used to conduct experiments on a large Turkish text dataset consisting of 30 categories. The effect of using pretrained embedding on models was also investigated. Experimental results show that the classification models using transformer and attention give promising results compared with classical deep learning models. We observed that the BiTransformer we proposed showed superior performance in text classification.en_US
dc.identifier.citation14
dc.identifier.doi10.1002/cpe.6486
dc.identifier.issn1532-0626
dc.identifier.issn1532-0634
dc.identifier.issue9en_US
dc.identifier.scopus2-s2.0-85110364023
dc.identifier.scopusqualityQ2
dc.identifier.urihttps://doi.org/10.1002/cpe.6486
dc.identifier.urihttps://hdl.handle.net/20.500.14411/2048
dc.identifier.volume34en_US
dc.identifier.wosWOS:000673965600001
dc.identifier.wosqualityQ3
dc.institutionauthorYıldız, Beytullah
dc.language.isoenen_US
dc.publisherWileyen_US
dc.relation.publicationcategoryMakale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanıen_US
dc.rightsinfo:eu-repo/semantics/closedAccessen_US
dc.subjectattentionen_US
dc.subjectdeep learningen_US
dc.subjectmachine learningen_US
dc.subjecttext classificationen_US
dc.subjecttext processingen_US
dc.subjecttransformeren_US
dc.titleText classification using improved bidirectional transformeren_US
dc.typeArticleen_US
dspace.entity.typePublication
relation.isAuthorOfPublication8eb144cb-95ff-4557-a99c-cd0ffa90749d
relation.isAuthorOfPublication8eb144cb-95ff-4557-a99c-cd0ffa90749d
relation.isAuthorOfPublication8eb144cb-95ff-4557-a99c-cd0ffa90749d
relation.isAuthorOfPublication.latestForDiscovery8eb144cb-95ff-4557-a99c-cd0ffa90749d
relation.isOrgUnitOfPublicationd86bbe4b-0f69-4303-a6de-c7ec0c515da5
relation.isOrgUnitOfPublication.latestForDiscoveryd86bbe4b-0f69-4303-a6de-c7ec0c515da5

Files

Collections