AI-Powered Sign Prediction and Detection System
编号:71
访问权限:仅限参会人
更新:2026-11-10 16:37:20 浏览:8次
Online
摘要
Sign language recognition systems bridge communication gaps for the hearing-impaired by translating gestures into text. In this work, we propose an AI-powered sign prediction and detection system that uses Google’s MediaPipe for hand landmark extraction and a Random Forest classifier for static gesture classification. We collected a dataset of 4,589 samples across 9 sign classes (A–I), extracting 63-dimensional feature vectors from 21 hand landmarks (each with x, y, and z coordinates). The model achieves an overall test accuracy of 99.78% and perfect class discrimination (AUC = 1.0000 for all classes). These results exceed previously reported performance on similar tasks. Key contributions include the end-to-end pipeline design (feature extraction and classification) and a thorough evaluation of performance metrics including accuracy, F1-score, and AUC. In particular, our system attains near-perfect agreement beyond chance (Cohen’s κ = 0.9975) and robust generalization (5-fold cross-validation accuracy 95.12% ± 3.64%). The results demonstrate that combining MediaPipe landmarks with a classical classifier can match or surpass more complex deep-learning models. Furthermore, to enhance real-world communication, the proposed system integrates the Gemini API for context-aware word prediction and sentence completion. Instead of recognizing only isolated letters, the system predicts complete words and grammatically meaningful sentences from sequential sign inputs, thereby reducing communication latency and improving user interaction. This hybrid framework combines efficient landmark-based gesture recognition with Large Language Model (LLM)-assisted language generation, providing a practical solution for assistive sign language communication. The study is limited to nine static classes and a random stratified split; the LLM module is demonstrated qualitatively, and quantitative latency, ablation, and signer-independent evaluations are left for future work.
关键词
American Sign Language (ASL),inclusive education,Sign Language Recognition,MediaPipe,computer vision,random forest classifier,Gemini API,Large Language Model,accessibility
稿件作者
R Uma
Sri Sairam Engineering College
Suraj Kumar K
Sri Sairam Engineering College
Roshan J
Sri Sairam Engineering College
Sanjeev M
Sri Sairam Engineering College
发表评论