Real-time posture detection and scoring on edge hardware.
SpineSense watches your posture so you don't have to. Using a single webcam and an NVIDIA Jetson Orin Nano, it tracks your spine, neck, and shoulders in real time — no wearables, no cloud, no setup hassle. Just sit down and get a live score.
SpineSense runs a full computer vision pipeline on-device:
- A USB webcam captures your seated posture
- YOLOv11n Pose detects body keypoints (ears, shoulders, hips)
- Five virtual spine points are derived from those keypoints
- Four geometric features (neck angle, head forward ratio, shoulder difference, spine curvature) are computed and normalized
- A trained MLP classifier produces per-region fault labels (neck, spine, shoulder) and a continuous posture score from 0–100
- Everything is rendered as a live overlay on the camera feed
When your posture drops, the score drops and a fault indicator lights up. Sit back up, and it recovers. Immediate feedback, no delay.
| Metric | Value |
|---|---|
| Neck fault accuracy | 88.3% |
| Spine fault accuracy | 93.8% |
| Posture score MAE | 7.72 / 100 |
| Inference throughput | 155 FPS |
| Mean latency | 6.85 ms / frame |
| TensorRT engine size | 9.4 MB |
| Live demo FPS | 25–30 FPS |
Evaluated on a held-out test set of 240 images. The system runs entirely on the Jetson Orin Nano with FP16 TensorRT acceleration.
Camera Feed → YOLOv11n Pose (TensorRT) → Feature Extraction → MLP Classifier → Fault Labels + Posture Score
Pose Estimation — YOLOv11n Pose (nano variant) detects 17 body keypoints per frame. Chosen over MediaPipe BlazePose for better partial-occlusion handling (desks block the lower body) and native TensorRT export.
Virtual Spine — Five derived points (head, neck, upper back, lower back, hips) form a spine chain rendered on-screen. Green = good posture, orange = fault detected.
Geometric Features — Four measurements normalized by torso height:
- Neck angle (head → neck → upper back)
- Head forward ratio
- Shoulder level difference
- Spine curvature (back deviation from vertical)
MLP Classifier — Two-headed network: classification head (BCE loss) for independent fault detection per region, scoring head (MSE loss) for the continuous posture score.
Rule-Based Score — A parallel, interpretable scoring function applies penalties when features exceed thresholds. This is the score shown to the user.
| Component | Technology |
|---|---|
| Pose estimation | YOLOv11n Pose |
| Inference engine | TensorRT 10.3 (FP16) |
| Edge device | NVIDIA Jetson Orin Nano |
| SDK | JetPack 6.2, CUDA 12.6 |
| Classifier | PyTorch MLP |
| Camera | Logitech USB Webcam |
| Display | OpenCV overlay |
The training dataset (~1,595 images) was built from two sources:
- ~560 images captured by team members using the Logitech webcam, covering a range of posture positions (upright, forward lean, rounded back, and combinations) with intentional severity variation
- ~1,000 images from the Roboflow Sitting Posture dataset, adding subject and environment diversity
Team-captured images were manually labeled. Roboflow images were labeled using the rule-based scoring function applied to extracted geometric features.
Splits: 70% train (1,151) · 13% validation (204) · 15% test (240)
The TensorRT engine is compiled natively on the Jetson (takes ~12 min once, then loads instantly). A frame-skip strategy runs YOLO on every 3rd frame and caches results for in-between frames, keeping the display smooth at 25–30 FPS without burning the GPU on every single frame.
Everything runs locally. No internet. No server. Plug in the camera and go.
- Sitting vs. standing detection — Apply position-specific thresholds instead of one-size-fits-all scoring
- Multi-angle / 360° camera support — Enable front-facing laptop webcams so no side-mount is needed
- Shoulder fault training data — The classifier is implemented but needs a dataset with actual uneven-shoulder examples
Built by Bilal Tulek, Chase Buerger, Said Elsaadi, and Ubaid Mohammed at UT Dallas.