Hate Speech Detection in Hindi language using Machine Learning and Pre-Trained Models

Hate Speech Detection in Hindi language using Machine Learning and Pre-Trained Models

Apurva Pagadpalliwar1, Rahul Rathod2, Prabhit Chaugule3, Rasika Ransing4

1 Vidyalankar Institute Of Technology, Mumbai, India

2 Springer Heidelberg, Tiergartenstr. 17, 69121 Heidelberg, Germany
lncs@springer.com

Abstract:

Detecting hate speech is a crucial aspect of creating a safe and inclusive virtual space. As Hindi digital communication expands, it has become necessary to devise effective techniques for detecting hate speech in this language. This abstract presents a machine learning and natural language processing (NLP) approach for hate speech detection in Hindi texts. To this end, the research employs a dataset that consists of labeled Hindi text examples divided into two classes: hate speech and non-hate speech. This means that specific pre-processing techniques such as tokenizing, stemming and stop-word removal must be used to handle linguistic nuance in Hindi. For feature engineering- word embeddings, character level embeddings and TF-IDF vectors are extracted to represent the semantic meaning of the text. These features can then be used to train different machine learning algorithms including Support Vector Machines (SVM), Naïve Bayes and Random Forests. The models were trained using standard performance metrics like precision recall F1-score accuracy The proposed system for detecting hate-speech could be extended to social media platforms or online forums or content sharing websites with an aim of curbing hateful content spread to more tolerant online communities among people who speak Hindi.

Keywords: Hate speech detection, Machine learning, Natural language processing (NLP), Pre-processing techniques, Feature engineering, Word embeddings, Character level embeddings, TF-IDF vectors, Support Vector Machines (SVM), Naïve Bayes, Random Forests.