BhaavNet: A Hybrid Transformer Framework Combining Transliteration Normalization and Lexicon Augmentation for Hinglish Code-Mixed Sentiment Analysis

This paper presented BhaavNet, a hybrid sentiment analysis framework for Hinglish code-mixed social media text that addresses the primary failure modes of vanilla mBERT: inconsistent subword tokenization due to orthographic variation, and inadequate representation of domain-specific Hindi sentiment vocabulary. BhaavNet achieves 82.3% accuracy and weighted F1 of 0.817 on the SemEval-2020 Task 9 benchmark, representing +8.2 percentage point improvements over vanilla mBERT and over +19 points over the SVM+TF-IDF baseline. Ablation analysis confirms both architectural components contribute independently and significantly. Future directions include: (1) expanding the transliteration map through automated social media corpus mining; (2) replacing rule-based lexicon with a learned lexicon from code-mixed training data; (3) adding sarcasm detection as an auxiliary task; and (4) extending the backbone to MuRIL or IndicBERT, which include Romanized Hindi in their pretraining corpora.

[1] T. Solorio and Y. Liu, “Learning to predict code-switching points,” in Proc. EMNLP, 2008, pp. 973–981. [2]  R. Sequeira et al., “Towards identifying the challenges of social media text in code-mixed scenarios,” in Proc. 1st Workshop on Language Technologies for Indian Social Media, 2015. [3] G. Aguilar, S. Kar, and T. Solorio, “LinCE: A centralized benchmark for linguistic code-switching evaluation,” in Proc. LREC, 2020, pp. 1–7. [4] S. Khanuja et al., “GLUECoS: An evaluation benchmark for code- switched NLP,” in Proc. ACL, 2020, pp. 3575–3585. [5] P. Patwa et al., “SemEval-2020 Task 9: Sentiment analysis for code- mixed social media text,” in Proc. SemEval-2020, 2020, pp. 1151– 1170. [6]  J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre- training of deep bidirectional transformers for language understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186. [7]   T. Pires, E. Schlinger, and D. Garrette, “How multilingual is multilingual BERT?” in Proc. ACL, 2019, pp. 4996–5001. [8] A. Kakwani et al., “IndicNLPSuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for Indian languages,” in Findings of EMNLP, 2020, pp. 4948–4961. [9]  S. Khanuja et al., “MuRIL: Multilingual representations for Indian languages,” arXiv preprint arXiv:2103.10730, 2021. [10]  Y. K. Lal et al., “De-mixing sentiment from code-mixed text,” in Proc. ACL Student Research Workshop, 2019, pp. 371–377.