Crossref DOIMemberCiteFactoropen accessissn
SJIF Impact Factor 8.659 | ISSN: 2582-3930
IJSREM Logo

International Journal of Scientific Research in Engineering and Management

(Gateway to Global Scholars - An International, Scholarly, Multi-Disciplinary, Open Access Journal)

Peer-Reviewed Journal Designed to Meet UGC's Quality Parameters

HomeCurrent IssuePast IssuesPay OnlineConferencesEditorial BoardAbout UsContact Us
IJSREM

Loading article...

Follow Us

  • Google Scholar
  • ResearchGate
  • Academia.edu
  • Mendeley
  • LinkedIn
  • Facebook
  • Instagram

Journal Policies

  • Peer Review Policy
  • Publication Ethics
  • Copyright Infringement
  • Refund & Cancellation Policy
  • Terms and Conditions
  • Privacy Policy

Journal Identifiers & Licenses

  • ISSN Verified: 2582-3930ISSN Verified: 2582-3930
  • Crossref DOI: 10.55041Crossref DOI: 10.55041
  • SJIF Ranking: 08.659SJIF Ranking: 08.659
  • Licensed under CC BY 4.0Licensed under CC BY 4.0
  • ISO Certification: 9001:2015ISO Certification: 9001:2015
  • Indexed & Cited In: Google ScholarIndexed & Cited In: Google Scholar

Contact Us

International Journal of Scientific Research in Engineering and Management (IJSREM)

#62/1, New No 7, 1st Cross, 2nd Main, Ganganagar, R T Nagar, Bangalore North, Bangalore, Karnataka, India – 560032
[email protected]+91 93911 67991
logos
logos
Working Hours

Mon – Sat: 9:00 AM – 6:00 PM

Sunday: 10:00 AM – 4:00 PM

+91 93911 67991

© 2026 IJSREM – International Journal of Scientific Research in Engineering and Management. This is an open-access journal. Articles are published under the Creative Commons Attribution (CC BY 4.0) License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

DisclaimerPrivacy PolicyTerms and Conditions
ArticlePeer ReviewedOpen AccessPlagiarism Passed
Paper ID:IJSREM68837

Gesture2Speech: AI-Powered Sign Language Interpretation System

Stream/Area:

-

Subject:

-

Paper ID:

IJSREM68837

Volume:

10 |

11 |

Nov 2026

Pages:

1-6

Publication Frequency:

Monthly (12 issues per year)

DOI
Digital Object Identifier
10.55041/IJSREM68837

License:International Journal of Scientific Research in Engineering and Management follows the Creative Commons Attribution 4.0 (CC BY 4.0) License, allowing free use and distribution with proper attribution.

Share this article:

Authors

Hrittik M

Hrittik M

Department of Computer Science and Engineering

Don Bosco Institute of Technology

Download Author Certificate
Mahesh Kumar G

Mahesh Kumar G

Department of Computer Science and Engineering

Don Bosco Institute of Technology

Download Co-Author Certificate
Jayanth K

Jayanth K

Department of Computer Science and Engineering

Don Bosco Institute of Technology

Download Co-Author Certificate
Mithun N

Mithun N

Department of Computer Science and Engineering

Don Bosco Institute of Technology

Download Co-Author Certificate

Mentors

Hemalatha M

Hemalatha M

Department of Computer Science and Engineering

Don Bosco Institute of Technology

Download Mentor Certificate

Author Services & Downloads

Full ArticlesAcceptance Certificate

Abstract

Sign language is the primary mode of communication for many people with hearing and speech impairments, yet the lack of real-time interpretation tools limits their interaction with non-signers. This paper presents Gesture2Speech, a real-time, web-based sign language interpretation system that recognizes alphabets and word gestures and converts them into text and speech. The proposed pipeline captures webcam frames in the browser, streams them over a WebSocket to a FastAPI backend, and applies two complementary recognition paths: MediaPipe hand landmarks with a Random Forest classifier for alphabets, and a YOLOv11n detector trained on 58 classes (26 letters and 32 word gestures) for words. Temporal stabilization based on consecutive-frame agreement and sliding-window majority voting suppresses frame-level noise. In the video-call mode, confirmed words are assembled into grammatical sentences by a large language model (LLM) and spoken through text-to-speech, while a reverse Text-to-Sign module plays real ASL clips from the WLASL vocabulary. On held-out validation data, the Random Forest reached 94.2% accuracy for ASL letters, and YOLOv11n reached 93.4% [email protected], 78.6% [email protected]:0.95, 87.5% precision and 91.2% recall. The results indicate that the system is a practical baseline for bidirectional, real-time sign language communication.

KEYWORDS

sign language recognition MediaPipe Random Forest YOLOv11n real-time detection text-to-speech text-to-sign WebRTC assistive technology.

Cite This Article

Hrittik M, Mahesh Kumar G, Jayanth K, Mithun N, Hemalatha M, "Gesture2Speech: AI-Powered Sign Language Interpretation System," International Journal of Scientific Research in Engineering and Management, vol. 10, no. 11, pp. 1-6, Nov 2026. https://doi.org/10.55041/IJSREM68837

Full Article/Conclusion

This paper presented Gesture2Speech, a real-time web system that recognizes sign language alphabets and word gestures and converts them into text and speech, with a reverse Text-to-Sign path for replies. The pipeline combines MediaPipe landmarks and a Random Forest for letters with a YOLOv11n detector for 58 classes, stabilized by consecutive-frame and sliding-window voting, and adds an LLM and text-to-speech stage for video calls. The Random Forest reached 94.2% accuracy for ASL letters, and YOLOv11n reached 93.4% [email protected], 78.6% [email protected]:0.95, 87.5% precision and 91.2% recall on validation data at about 5 ms per image. The system is a practical baseline for bidirectional sign communication on a fixed vocabulary, and broader evaluation, a larger vocabulary and network-ready calls are the main directions for strengthening it.

References

[1]     Zhang, F., et al.: MediaPipe Hands: On-device Real-time Hand Tracking. arXiv:2006.10214 (2020). [2]     Breiman, L.: Random Forests. Machine Learning 45(1), 5–32 (2001). [3]     Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You Only Look Once: Unified, Real-Time Object Detection. IEEE CVPR, pp. 779–788 (2016). [4]     Ultralytics: YOLO11 Documentation (2024). Available: https://docs.ultralytics.com [5]     Li, D., Rodriguez, C., Yu, X., Li, H.: Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison. IEEE WACV, pp. 1459–1469 (2020). [6]     W3C: WebRTC 1.0: Real-Time Communication Between Browsers. W3C Recommendation (2021). [7]     Alsharif, B., et al.: American Sign Language Interpretation Using Deep Learning (2025). [8]     Rastogi, U., et al.: Indian Sign Language Recognition Using a Combination of YOLOv10 with Swin Transformer Model (2025). [9]     Shahin, N., Ismail, L.: ADAT: Time-Series-Aware Adaptive Transformer Architecture for Sign Language Translation (2025). [10]   Alkhorafi, A. A., et al.: Ensemble Transformer-Based Word-Level Sign Language Recognition with Multi-Modal Input Fusion (2026).

Editorial & Review Transparency

Publication Date

11/9/2026

Peer Review Type

Double-Blind Peer Review

Review Rounds: 2

Documents

Review Report (Confidential)
Copyright Form (Office)

Article Metrics & Impact

Article Views

0

Total views since publication

PDF Downloads

3

Total downloads since publication

Geographic Reach

5

Indexing, Archiving & Preservation

Google Scholar
Academia
ResearchGate
Crossref
SSRN
Semantic Scholar
Thomson Reuters
WorldCat
Mendeley
OpenAlex
EBSCO
Zenodo

Digital Preservation: This article is permanently archived on Zenodo and complies with all data preservation standards across major indexing platforms.

Ethics, Compliance & Transparency

Plagiarism Screening

This manuscript has been checked for plagiarism using TurnitinPlagiarismChecker software.

Conflict of Interest

The authors declare no conflict of interest related to this research work.

Funding Information

No funding information available for this research.

Similar & Recommended Articles

Similar Articles by Topic

Related research in the same subject area

Loading similar articles...