Comparative analysis of SVM kernels for large-scale sentiment analysis on sentiment140 dataset

The performance of the SVM is studied in this experiment. This experiment was performed on the various data size of the sentiment140 dataset ranging from 5k to 1.6m(full dataset) and various kernel of SVM were used such as linear, RBF, polynomial and sigmoid. Also for this experiment TF-IDF vectorization is used and for evaluation matrices such as accuracy, precision, recall, f1 and training time. The result of an experiment shows that polynomial kernel achieve the highest accuracy meanwhile the computational cost increases in polynomial with increase in data size. Also the RBF kernel shows strong performance and can be used as an alternative with slight compromise in accuracy. However, when the size of the dataset is increased, the computation cost of non-linear kernel such as polynomial and RBF increases which make impractical for large scale dataset in real world. Where as in linear kernel, stable performance and scalability is provided with lower computational cost and it maintain consistent accuracy around 84% shows strong generalization for larger dataset. Another finding is that after some data size, increasing training sample produced slight improvement in accuracy thus showing saturation point in model learning. Overall the conclusion is that the kernel selection of the SVM should not only depend on the evaluation matrices but computational cost should also be kept in mind. As the polynomial kernel achieve the highest accuracy, with increase in data size it becomes impractical for computation in normal circumstances. Whereas the linear kernel is observed as the most suitable for real world sentiment application as it balances between the computational cost and accuracy and produced scalability.

[1]   B. Pang, L. Lee, and S. Vaithyanathan, “Thumbs up? Sentiment classification using machine learning techniques,” Proceedings of EMNLP, 2002. Link: https://aclanthology.org/W02-1011/ [2]   A. Go, R. Bhayani, and L. Huang, “Twitter sentiment classification using distant supervision,” Stanford University, 2009. Link: http://help.sentiment140.com/for-students/ [3]   C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, 1995. Link: https://link.springer.com/article/10.1007/BF00994018 [4]   https://www.geeksforgeeks.org/machine-learning/understanding-tf-idf-term-frequency-inverse-document-frequency/ [5]   T. Joachims, “Text categorization with support vector machines,” ECML, 1998. Link: https://link.springer.com/chapter/10.1007/BFb0026683 [6]   C. W. Hsu, C. C. Chang, and C. J. Lin, “A practical guide to support vector classification,” National Taiwan University, 2010. Link: https://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf [7]   A. Severyn and A. Moschitti, “Twitter sentiment analysis with deep convolutional neural networks,” Proceedings of SIGIR, 2015. Link: https://dl.acm.org/doi/10.1145/2766462.2767830 [8]   Y. Kim, “Convolutional neural networks for sentence classification,” Proceedings of EMNLP, 2014. Link: https://aclanthology.org/D14-1181/ [9]   J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” Proceedings of NAACL, 2019. Link: https://aclanthology.org/N19-1423/ [10] S. Minaee, N. Kalchbrenner, E. Cambria, N. Nikzad, M. Chenaghlu, and J. Gao, “Deep learning-based text classification: A comprehensive review,” ACM Computing Surveys, vol. 54, no. 3, 2021. Link: https://arxiv.org/abs/2004.03705