Measuring Political Bias in LLMs Using Fine-Tuned RoBERTa Model

The influence and power of LLMs over public opinion are huge. Therefore, both the public and researchers are increasingly becoming concerned with the implicit political bias in these models’ outputs. This concern has prompted the present study to use a fine-tuned RoBERTa-base model in a supervised learning setting, with the goal of detecting political bias in any given text. For training and testing, a large dataset of 37,554 news articles was compiled from AllSides.com, with each article labeled as left, center, or right. The model was implemented using the Transformers library of Hugging Face with GPU acceleration, yielding a validation accuracy score of 94.48%. Along with this, balanced measures were obtained for precision, recall, and F1- score, thus making the model particularly suitable for the detection of extreme bias. Besides accuracy, the system is interpretable; hence, it can be employed for bias auditing and support fair AI ethics. Further research will focus on multilingual datasets, on media evolution, and explainable AI for journalism, sentiment analysis, and ethical AI policy.