Outstanding Graduation Project · AI · Computer Vision · Backend
An award-winning Turkish Sign Language platform combining real-time recognition, user-approved sentence assembly, optional speech output, persistent history, and interactive 3D sign visualization in one FastAPI application.
The Faculty of Engineering, Department of Computer Engineering recognized SignTurk as an outstanding graduation project for the Spring Semester. The award highlights the project’s engineering depth, product scope, and accessibility impact.
SignTurk was co-developed as a graduation-project team effort. I led the model design, training, and evaluation work and implemented nearly all of the FastAPI backend, including real-time inference, WebSocket communication, authentication, persistence, and operational APIs. Interface, data preparation, integration, and 3D animation work was shared across the team.
Authenticated webcam frames stream to /api/predict/live. The FastAPI backend extracts MediaPipe hand landmarks, builds 16-frame sequences, and returns ranked BiLSTM predictions with confidence and latency.
Approved signs flow into a deterministic Turkish sentence engine and optional gTTS/Piper speech output. Supported words can also be replayed through the Three.js avatar using local landmark and BVH assets.
/signs and /landmark/{word} avatar endpointsThese screens document the current product experience included in the public repository: live translation, approval-driven sentence building, avatar playback, dictionary, history, settings, and administrator views.
Authenticated webcam inference, ranked predictions, approval controls, and live diagnostics in one focused workspace.
The browser captures a compressed webcam frame every 250 ms and streams it to the protected /api/predict/live WebSocket endpoint.
FastAPI decodes each frame and MediaPipe Hands extracts 21 landmarks per detected hand. The service maintains an isolated temporal buffer for every session.
The selected model contract normalizes 126 hand-landmark coordinates relative to the wrists, combines them with 30 finger-angle features, and assembles 156 features across a 16-frame temporal sequence.
The 179-class BiLSTM returns Top-3 labels and confidence. A confident word is persisted and added to the Turkish sentence only after the user approves it.
| Path | Purpose |
|---|---|
| backend.py | FastAPI app, authentication, WebSockets, inference, approval flow, and APIs |
| frontend/sign_turk_ui.html | Responsive React product interface for translation, history, settings, and administration |
| database.py + models.py | Environment-based SQLAlchemy persistence with SQLite and PostgreSQL support |
| demo_assets_179/ | Live Keras model, label map, runtime config, and normalization statistics |
| text_processing/ | Turkish grammar, evaluation, optional ML adapters, and gTTS/Piper speech |
| extract_landmarks.py | AUTSL landmark extraction utility |
| dataset/landmarks/ | Local sign-animation landmark sequences |
Offline research track
Reached the higher offline score, but requires a different whole-body extractor, 32-frame input, and multi-stream runtime rather than a drop-in model swap.
Integrated runtime
Selected because its MediaPipe feature contract fits the authenticated webcam/WebSocket pipeline and the complete product flow reliably.
Engineering decision: ship the model that satisfies the complete product contract; migrate the higher-accuracy research model only with its extractor and regression pipeline.
