Sign-Speak: Real-Time Continuous Indian Sign Language Translator

Rashmi N. D — 2026-09-15

Communication between deaf or hard-of-hearing individuals and people who are unfamiliar with Indian Sign Language (ISL) continues to present challenges in many everyday situations. Recent progress in artificial intelligence has made it possible to develop camera-based systems that recognize sign gestures without relying on wearable devices. This literature survey examines recent research on real-time ISL recognition, with particular attention to landmark-based feature extraction and sequence-learning models used for dynamic gesture interpretation. Four representative studies employing MediaPipe Holistic, Long Short-Term Memory (LSTM), Bidirectional LSTM (Bi-LSTM), Gated Recurrent Unit (GRU), and Transformer-based techniques are critically reviewed. Their methodologies, datasets, reported performance, strengths, and limitations are compared to identify current research trends and unresolved challenges. The analysis reveals that although existing approaches achieve promising recognition accuracy, many are constrained by limited vocabularies, small datasets, signer variability, and insufficient support for continuous sentence-level recognition. Based on these observations, this survey proposes a Sign-Speak framework that combines MediaPipe Holistic for extracting hand, facial, and body landmarks with a Bi-LSTM network for learning temporal gesture patterns. The recognized signs are intended to be converted into meaningful text and speech, enabling more accessible communication in real-world environments. The findings highlight the potential of vision-based deep learning systems to improve inclusive human-computer interaction while identifying opportunities for future research in multilingual translation, larger-scale datasets, and practical deployment.

Splat Search: Approach toward Real-Time Semantic Digital Twin

Chandana N — 2026-09-22

To meet the demand for high-fidelity spatial awareness in robotics and Augmented Reality (AR), this project introduces a unified framework for generating real-time, linguistically-aware digital twins. Traditional Simultaneous Localization and Mapping (SLAM) systems typically suffer from "semantic blindness"—they capture 3D geometry but lack object context. This system bridges the gap between spatial data and human language by integrating SplaTAM for dense Gaussian Splatting, LangSplat for open-vocabulary embedding, and Online LangSplat for high-speed encoding. The architecture leverages a robust client-server model where an iPhone streams synchronized RGB-D data directly to a GPU-accelerated backend. This setup enables real-time 3D reconstruction at over 45 frames per second. During this generation phase, the system dynamically assigns CLIP-based language features to millions of 3D Gaussians. The resulting digital twin is not only photorealistic but fully searchable via natural language queries. Users can seamlessly localize and identify specific objects within previously unknown environments with remarkable semantic accuracy. By eliminating the latency gap between mapping and understanding, this research delivers a scalable, highly efficient solution perfectly suited for advanced smart-home automation and assistive navigation technologies.

Journal of Data Mining and Management

Sign-Speak: Real-Time Continuous Indian Sign Language Translator

Splat Search: Approach toward Real-Time Semantic Digital Twin