LearnLens - Video Summarization System

This paper has presented LearnLens, an intelligent video summarization and analysis system integrating automatic speech recognition, large language model inference via a unified multi-provider interface, and interactive visualization into a deployable web application. The system demonstrates that high-quality video comprehension tools can be built using freely available AI components, making them accessible to students and institutions without expensive cloud GPU infrastructure. The key innovation lies in the integration of multiple AI capabilities into a coherent user experience: from submitting a YouTube URL to receiving an abstractive summary in up to seven languages, sentiment and topic analysis, an interactive Canvas concept mind map, a prerequisite assessment table, downloadable Markdown study notes, and a real-time streaming transcript-grounded chatbot—all within a single web session. The adoption of LiteLLM reduced user-facing error rates from approximately 12% to under 2% through automatic retry and fallback across Groq, Gemini, and Ollama providers. Evaluation on a corpus of 20 videos demonstrates a mean summarization accuracy of 70.1% and end-to-end processing under three minutes for ten-minute videos on CPU-only hardware. Future enhancements will include multimodal frame analysis, an agentic AI architecture for multi-step learning path generation, vector-based RAG for long-form content, and quiz generation for interactive learning assessment.

[1] YouTube Press, “YouTube by the Numbers,” Google LLC, 2024. Available: https://www.youtube.com/about/press/ [2] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust Speech Recognition via Large-Scale Weak Supervision,” arXiv preprint arXiv:2212.04356, 2022. [3] G. Klein, F. Hernandez, T. Nguyen, and J. Senellart, “CTranslate2: Efficient Inference with Transformer Models,” arXiv preprint arXiv:2003.02030, 2020. [4] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension,” in Proc. ACL, 2020. [5] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer,” Journal of Machine Learning Research, vol. 21, no. 140, pp. 1–67, 2020. [6] OpenAI, “GPT-4 Technical Report,” arXiv preprint arXiv:2303.08774, 2023. [7] J. Lin, R. Ding, J. Liang, and X. Liu, “VideoXum: Cross-modal Visual and Textual Summarization of Videos,” IEEE Transactions on Multimedia, 2023. [8] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” in Proc. NeurIPS, 2020. [9] BerriAI, “LiteLLM: Call all LLM APIs using the OpenAI format,” GitHub repository, 2024. Available: https://github.com/BerriAI/litellm [10]  N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” in Proc. EMNLP, 2019.