Earlier web experiment / Computer vision
From movement to meaning.
An exploration of Indian Sign Language through video, pose estimation, and sequence models. This page documents the earlier OpenPose-based web demo.
The question
Indian Sign Language (ISL) is a crucial mode of communication for people with speaking and hearing impairments. Unlike American or British Sign Language, ISL has limited technological support. This project explores how computer vision and sequence models might help bridge that gap.
A look inside the pipeline
From video frames to landmarks, features, and classification. This is an illustrative animation: the displayed class probabilities are simulated, not model predictions.
Making movement visible
The INCLUDE dataset
The earlier demo uses INCLUDE videos of ISL signs recorded by deaf students from St. Louis School for the Deaf, Adyar, Chennai. These are the dataset characteristics documented for that demo.
| Characteristic | Details |
|---|---|
| Categories | 15 |
| Words | 263 |
| Videos | 4292 |
| Avg videos per class | 16.3 |
| Avg video length | 2.57s |
| Min video length | 1.28s |
| Max video length | 6.16s |
| Frame rate | 25 fps |
| Resolution | 1920×1080 |
Exploring the data
Loading the expression-count snapshot…
Video & frame preview
Choose a category and expression to inspect a clip and its extracted frames.
Loading the video catalogue…
Select a video to preview