← Back to Publications List

Transformer Architectures for Romanized Bangla Dialect Understanding in Resource-Constrained Conversational Systems

Students & Supervisors

Student Authors
Hasin Almas Sifat
Bachelor of Science in Computer Science & Engineering, FST
Koushik Biswas Arko
Bachelor of Science in Computer Science & Engineering, FST
Foysal Ahmed Neloy
Bachelor of Science in Computer Science & Engineering, FST
Ashiqur Rahman Saron
Bachelor of Science in Computer Science & Engineering, FST
Supervisors
Dr. Abdus Salam
Associate Professor, Faculty, FST

Abstract

Multilingual societies deploy conversational systems to handle informal event-driven, and non-standard languages. Often, these conversational systems' user inputs will include Romanized and dialect-based user inputs. In the case of Bangla, many existing Natural Language Processing (NLP) tools have been developed for use with native scripts and therefore, present a challenge in dealing with real-world conversational data based on Romanized formats. This paper outlines our evaluation of several transformer-based models as a method to classify five different types of regional dialects of Romanized Bangla. The dataset we used, Vashantor, consists of several hundred locations that include users inputting Romanized Bangla against five different regional dialects. Each of the transformer models (multilingual, Bangla-specific, and compact) were run against a common experimental design. Evaluations were performed using agreementbased, error-sensitive, efficiency-aware metrics, and standard accuracy metrics. Our results demonstrate that multilingual and/or ‘script compatible’ models have greater success than models that have been specifically trained for Bangla with mBERT, yielding the best balance of predictive performance, robustness, and operational efficiency. In addition, training dynamics and confusion matrices substantiate the stability and generalization ability that multilingual pre-training has over Bangla-specific models. Therefore, we have concluded that in order to conduct NLP in Romanized Bangla, it is critical to select Transformer architecture(s) that have ‘script compatibility’ within the context of conversational applications with low resource requirements that generate real-world use cases.

Keywords

Romanization Dialects Transformers Bangla Multilingualism.

Publication Details

  • DOI: https://doi.org/10.1109/QPAIN69676.2026.11545854
  • Type of Publication:
  • Conference Name: 2026 IEEE 2nd International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN)
  • Date of Conference: 16/04/2026 - 16/04/2026
  • Venue: CUET, CHATTOGRAM, BANGLADESH
  • Organizer: IEEE Photonics Society Bangladesh Chapter
View on Publisher