Vision Stack AI Studio

This paper has described Vision Stack AI Studio, a web-based system that automates the generation of gameplay highlight montages through an integrated AI pipeline combining convolutional neural network-based visual frame scoring with OCR-derived textual event detection. The hybrid detection approach achieves an F1-score of 83.4% across a diverse evaluation corpus, representing a significant improvement over the CNN-only baseline attributable to the complementary coverage provided by textual signal detection. The modular pipeline architecture, FastAPI backend, and containerized deployment model together provide a practical foundation for real-world operation and iterative improvement. Future development directions identified from evaluation analysis and user feedback include the substitution of transformer-based video encoders for the ResNet-50 backbone to improve detection on games with complex temporal event patterns; extension of the OCR keyword system with user-uploadable custom vocabularies for rapid adaptation to new titles; development of real-time highlight detection operating on live streams rather than post-hoc recordings; and integration of audio-based event detection as a third complementary evidence channel. Each of these extensions is compatible with the existing pipeline interface design, facilitating their incremental introduction without architectural disruption.

1. Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436–444, 2015. 2. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, "Large-scale video classification with convolutional neural networks," in Proc. IEEE CVPR, pp. 1725–1732, 2014. 3. D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, "Learning spatiotemporal features with 3D convolutional networks," in Proc. IEEE ICCV, pp. 4489–4497, 2015. 4. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proc. IEEE CVPR, pp. 770–778, 2016. 5. R. Smith, "An overview of the Tesseract OCR engine," in Proc. 9th Int. Conf. Document Analysis and Recognition (ICDAR), pp. 629–633, 2007. 6. G. Bertasius, H. Wang, and L. Torresani, "Is space-time attention all you need for video understanding?" in Proc. ICML, 2021. 7. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proc. ICLR, 2015. 8. A. Dosovitskiy et al., "An image is worth 16×16 words: Transformers for image recognition at scale," in Proc. ICLR, 2021. 9. FFmpeg Project, "FFmpeg: A complete, cross-platform solution to record, convert and stream audio and video." [Online]. Available: https://ffmpeg.org. 10. FastAPI Documentation, "FastAPI: Modern, fast web framework for building APIs with Python." [Online]. Available: https://fastapi.tiangolo.com. 11. PaddleOCR, "Rich multilingual OCR toolkits based on PaddlePaddle." [Online]. Available: https://github.com/PaddlePaddle/PaddleOCR. 12. A. Paszke et al., "PyTorch: An imperative style, high-performance deep learning library," Advances in Neural Information Processing Systems, vol. 32, 2019.