A Web based Hybrid Cross-Language Plagiarism Detection System for Hindi and Punjabi Using Neural Machine Translation and Embedding

This paper discussed a hybrid semantic cross language plagiarism detection system for Hindi and Punjabi language. The proposed framework combines dictionary-based translation, Example-Based Machine Translation (EBMT), Neural Machine Translation (NMT) and sentence-level semantic similarity in a single Web application. This system is capable of uploading documents, management of the document corpus, keeping the document plagiarism history, and search results with the semantic approach which is an effective solution to the detection of translated plagiarism in Indian languages. Experimental results show that using semantic sentence embeddings is better than traditional keyword-based approaches for the detection of translated and paraphrased content.

[1]. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT) (pp. 4171–4186). [2]. Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP). [3]. Asghari, H., Fatemi, O., Mohtaj, S., Faili, H., & Rosso, P. (2019). On the use of word embeddings for cross-language plagiarism detection. Intelligent Data Analysis, 23(3), 661–680. [4]. Conneau, X., et al. (2020). Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL) (pp. 8440–8451). [5]. Goyal, V., Puri, R., Pubreja, J., & Singh, J. (2020). Plagiarism detection tool for Indian languages with special focus on Hindi and Punjabi. In Proceedings of the 17th International Conference on Natural Language Processing (ICON): Demonstration Papers (pp. 46–47). [6]. Hossain, A., & Shakery, A. (2020). Cross-lingual plagiarism detection using word embedding and IDF. In Proceedings of the 16th International Conference on Computer and Information Technology (ICCIT). [7]. Kakwani, D., Kunchukuttan, A., Golla, S., et al. (2020). IndicNLPSuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for Indian languages. In Findings of the Association for Computational Linguistics: EMNLP 2020. [8]. Khapra, M., et al. (2020). IndicGLUE: A benchmark for Indic language understanding. In Findings of the Association for Computational Linguistics: EMNLP 2020. [9]. Khanuja, S., et al. (2021). MuRIL: Multilingual representations for Indian languages. [10].              Aggarwal, D., Gupta, V., & Kunchukuttan, A. (2022). INDICXNLI: Evaluating multilingual inference for Indian languages. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP).