Development of LLM based Automatic research paper using evaluation mode and plagiarism detection using SBERT

This paper described the design and implementation of a modular hybrid NLP system for automated research-paper evaluation, combining PDF ingestion and validation, section-aware summarization, semantic novelty detection through Sentence-Transformer embeddings and FAISS retrieval, deterministic multi-dimensional evaluation scoring, rule-based reviewer-feedback generation, and independent segment-level plagiarism detection within a single FastAPI-based pipeline. The principal technical contributions are the section-aware construction of the novelty-detection query, the explicit architectural and conceptual separation of whole-document novelty assessment from passage-level plagiarism detection despite their shared retrieval infrastructure, and the traceability afforded by keeping evaluation scoring and review generation deterministic and rule-based rather than delegating them to opaque model inference. Qualitative evidence from an end-to-end run of the deployed system confirms that the pipeline executes successfully on a real manuscript and produces populated summary, review, scoring, novelty, and plagiarism outputs through its web interface; quantitative accuracy, agreement with human reviewers, and performance figures were not measured in this study and are identified explicitly as requiring controlled future evaluation rather than being estimated. The system's practical significance lies in its potential use as a preliminary, reviewer-support screening tool that can surface structural gaps, unsupported claims, and candidate similarity concerns before a manuscript reaches a human reviewer or editor, reducing the portion of routine checking that must be performed manually, while explicitly not functioning as a substitute for expert peer judgment.  

1.   Beltagy, I., Lo, K., & Cohan, A. (2019). SciBERT: A pretrained language model for scientific text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing  (EMNLP-IJCNLP)  (pp.  3615–3620). Association for Computational Linguistics. 2.  Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022. 3.   Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (pp. 4171–4186). Association for Computational Linguistics. 4.    Foltynek, T., Meuschke, N., & Gipp, B. (2019). Academic plagiarism detection: A systematic literature review. ACM Computing Surveys, 52(6), 1–42. 5.   Johnson, J., Douze, M., & Jegou, H. (2019). Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3), 535–547. 6.    Kang, D., Ammar, W., Dalvi, B., van Zuylen, M., Kohlmeier, S., Hovy, E., & Schwartz, R. (2018). A dataset of peer reviews (PeerRead): Collection, insights and NLP applications. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (pp. 1647–1661). Association for Computational Linguistics. 7.   Kusner, M., Sun, Y., Kolkin, N., & Weinberger, K. (2015). From word embeddings to document distances. In Proceedings of the 32nd International Conference on Machine Learning (pp. 957–966). PMLR. 8.  Liang, W., Zhang, Y., Cao, H., Wang, B., Ding, D. Y., Yang, X., Vodrahalli, K., He, S., Smith, D. S., Yin, Y., et al. (2024). Can large language models provide useful feedback on research papers? A large-scale empirical analysis. NEJM AI, 1(8), AIoa2400196. 9.  Lin, C.-Y. (2004). ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out (pp. 74–81). Association for Computational Linguistics. 10.   Liu, R., & Shah, N. B. (2023). ReviewerGPT? An exploratory study on using large language models for paper reviewing. arXiv preprint arXiv:2306.00622.