Hate Speech Detection in Twitter using Machine Learning and Deep Learning : A TFIDF and LSTM based Approach

The massive expansion of social networking websites has made it easier for people with various cultural and psychological backgrounds to communicate directly with one another. It has led to an increase in online conflicts between them. This paper proposes an approach to detect hate speech. A publicly available dataset of tweets in language is used. Data preprocessing includes the removal of stopwords , punctuations , emojis , numbers , URLs etc. and feature extraction is carried out using tokenization , lemmatization and POS tagging. The performances of XGBoost, Random Forest , Logistic Regression and SVM have been compared in this study for the detection of hate speech. XGBoost classifier provided highest accuracy of 74.93%.