Loading article...
Loading article...
Stream/Area:
-
Subject:
-
Paper ID:
IJSREM69012
Volume:
10 |
11 |
Nov 2026
Pages:
1-5
Publication Frequency:
Monthly (12 issues per year)
License:International Journal of Scientific Research in Engineering and Management follows the Creative Commons Attribution 4.0 (CC BY 4.0) License, allowing free use and distribution with proper attribution.
Computer Science & Engineering
Jawaharlal Nehru New College of Engineering
Computer Science & Engineering
Jawaharlal Nehru New College of Engineering
Computer Science & Engineering
Jawaharlal Nehru New College of Engineering
Computer Science & Engineering
Jawaharlal Nehru New College of Engineering
Computer Science & Engineering
Jawaharlal Nehru New College of Engineering
Learners who watch long educational videos on platforms such as YouTube often hit a concept they do not understand but keep playing the video, because stopping to look for help breaks their flow. This paper describes CogniLearn, a Chrome extension with a local backend that notices likely cognitive overload while a video is playing and explains the concept being taught at that moment. A webcam frame is sampled at 4 fps and processed entirely on the device by MediaPipe Face Landmarker, which yields facial blendshape scores and head pose. A per-user detector calibrates a personal baseline in roughly 30 seconds using an online mean and variance estimate, converts each frame into a weighted z-score of effort-related facial actions (brow lowering, eye squint, inner-brow raise, mouth press), and aggregates frames over a 10-second sliding window. Viewing behaviour (rewinds and pauses) can amplify the facial evidence but can never trigger an intervention alone. A trigger fires only when the score stays above a personal threshold for 3 seconds and a cooldown has expired. The threshold is then adapted from real user labels ("Helpful" and "I wasn't confused") and from manual help requests. On a trigger, the video is paused and a FastAPI server fetches the transcript around the current timestamp and asks Gemini 2.5 Flash for a short explanation in the speaker's language, which is also read aloud with browser speech synthesis. The same transcript and a memory of earlier explanations are used to build a three-question multiple-choice quiz and exportable Markdown study notes. Only numeric features and the video ID leave the browser; camera frames never do. The paper describes the design, the algorithms and the parameters in detail, states the limitations of the current prototype, and proposes an evaluation protocol.
KEYWORDS
CogniLearn shows that useful, timely help for video learners can be built without uploading camera images. A per-user calibrated facial detector, a conservative hold-and-cooldown trigger, behaviour used only as an amplifier, and a threshold that learns from real user labels together keep interruptions rare and personal. Grounding the LLM in the speaker's own transcript produces explanations that match both the moment and the language of the lesson, and the quiz and notes close the loop with retrieval practice. The prototype is modular and its labelled event log provides a path to data-driven improvement. A formal user study, as outlined in Section V, is the next step.
[1] J. Sweller, "Cognitive load during problem solving: Effects on learning," Cognitive Science, vol. 12, no. 2, pp. 257-285, 1988. [2] P. Ekman and W. V. Friesen, Facial Action Coding System. Palo Alto, CA, USA: Consulting Psychologists Press, 1978. [3] S. D'Mello and A. Graesser, "Dynamics of affective states during complex learning," Learning and Instruction, vol. 22, no. 2, pp. 145-157, 2012. [4] J. Whitehill, Z. Serpell, Y.-C. Lin, A. Foster, and J. R. Movellan, "The faces of engagement: Automatic recognition of student engagement from facial expressions," IEEE Trans. Affective Computing, vol. 5, no. 1, pp. 86-98, 2014. [5] C. Lugaresi et al., "MediaPipe: A framework for building perception pipelines," arXiv:1906.08172, 2019. [6] Y. Kartynnik, A. Ablavatski, I. Grishchenko, and M. Grundmann, "Real-time facial surface geometry from monocular video on mobile GPUs," arXiv:1907.06724, 2019. [7] Gemini Team, Google, "Gemini: A family of highly capable multimodal models," arXiv:2312.11805, 2023. [8] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA, USA: MIT Press, 2018. [9] H. L. Roediger III and J. D. Karpicke, "Test-enhanced learning: Taking memory tests improves long-term retention," Psychological Science, vol. 17, no. 3, pp. 249-255, 2006. [10] B. P. Welford, "Note on a method for calculating corrected sums of squares and products," Technometrics, vol. 4, no. 3, pp. 419-420, 1962.
Publication Date
11/16/2026
Peer Review Type
Double-Blind Peer Review
Review Rounds: 2
Documents
Article Views
0
Total views since publication
PDF Downloads
1
Total downloads since publication
Geographic Reach
5
Digital Preservation: This article is permanently archived on Zenodo and complies with all data preservation standards across major indexing platforms.
Plagiarism Screening
This manuscript has been checked for plagiarism using TurnitinPlagiarismChecker software.
Conflict of Interest
The authors declare no conflict of interest related to this research work.
Funding Information
No funding information available for this research.
Related research in the same subject area
Loading similar articles...