Text classification using improved bidirectional transformer
dc.authorid | YILDIZ, Beytullah/0000-0001-7664-5145 | |
dc.authorid | Tezgider, Murat/0000-0002-4918-5697 | |
dc.authorscopusid | 57207471698 | |
dc.authorscopusid | 14632851900 | |
dc.authorscopusid | 8338657900 | |
dc.contributor.author | Tezgider, Murat | |
dc.contributor.author | Yıldız, Beytullah | |
dc.contributor.author | Yildiz, Beytullah | |
dc.contributor.author | Aydin, Galip | |
dc.contributor.author | Yıldız, Beytullah | |
dc.contributor.other | Software Engineering | |
dc.date.accessioned | 2024-07-05T15:21:19Z | |
dc.date.available | 2024-07-05T15:21:19Z | |
dc.date.issued | 2022 | |
dc.department | Atılım University | en_US |
dc.department-temp | [Tezgider, Murat; Aydin, Galip] Firat Univ, Fac Engn, Dept Comp Engn, TR-23200 Elazig, Turkey; [Yildiz, Beytullah] Atilim Univ, Sch Engn, Dept Software Engn, Ankara, Turkey | en_US |
dc.description | YILDIZ, Beytullah/0000-0001-7664-5145; Tezgider, Murat/0000-0002-4918-5697 | en_US |
dc.description.abstract | Text data have an important place in our daily life. A huge amount of text data is generated everyday. As a result, automation becomes necessary to handle these large text data. Recently, we are witnessing important developments with the adaptation of new approaches in text processing. Attention mechanisms and transformers are emerging as methods with significant potential for text processing. In this study, we introduced a bidirectional transformer (BiTransformer) constructed using two transformer encoder blocks that utilize bidirectional position encoding to take into account the forward and backward position information of text data. We also created models to evaluate the contribution of attention mechanisms to the classification process. Four models, including long short term memory, attention, transformer, and BiTransformer, were used to conduct experiments on a large Turkish text dataset consisting of 30 categories. The effect of using pretrained embedding on models was also investigated. Experimental results show that the classification models using transformer and attention give promising results compared with classical deep learning models. We observed that the BiTransformer we proposed showed superior performance in text classification. | en_US |
dc.identifier.citation | 14 | |
dc.identifier.doi | 10.1002/cpe.6486 | |
dc.identifier.issn | 1532-0626 | |
dc.identifier.issn | 1532-0634 | |
dc.identifier.issue | 9 | en_US |
dc.identifier.scopus | 2-s2.0-85110364023 | |
dc.identifier.scopusquality | Q2 | |
dc.identifier.uri | https://doi.org/10.1002/cpe.6486 | |
dc.identifier.uri | https://hdl.handle.net/20.500.14411/2048 | |
dc.identifier.volume | 34 | en_US |
dc.identifier.wos | WOS:000673965600001 | |
dc.identifier.wosquality | Q3 | |
dc.institutionauthor | Yıldız, Beytullah | |
dc.language.iso | en | en_US |
dc.publisher | Wiley | en_US |
dc.relation.publicationcategory | Makale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanı | en_US |
dc.rights | info:eu-repo/semantics/closedAccess | en_US |
dc.subject | attention | en_US |
dc.subject | deep learning | en_US |
dc.subject | machine learning | en_US |
dc.subject | text classification | en_US |
dc.subject | text processing | en_US |
dc.subject | transformer | en_US |
dc.title | Text classification using improved bidirectional transformer | en_US |
dc.type | Article | en_US |
dspace.entity.type | Publication | |
relation.isAuthorOfPublication | 8eb144cb-95ff-4557-a99c-cd0ffa90749d | |
relation.isAuthorOfPublication | 8eb144cb-95ff-4557-a99c-cd0ffa90749d | |
relation.isAuthorOfPublication | 8eb144cb-95ff-4557-a99c-cd0ffa90749d | |
relation.isAuthorOfPublication.latestForDiscovery | 8eb144cb-95ff-4557-a99c-cd0ffa90749d | |
relation.isOrgUnitOfPublication | d86bbe4b-0f69-4303-a6de-c7ec0c515da5 | |
relation.isOrgUnitOfPublication.latestForDiscovery | d86bbe4b-0f69-4303-a6de-c7ec0c515da5 |