Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 33 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ The existing solution — call 108 — fails when the victim is unconscious, whe
RoadSOS is an offline-first, life-safety platform that:

1. **Detects crashes automatically** — accelerometer + GPS fusion; fires SOS if the user is unconscious
2. **Triages severity using Gemma 4** — multimodal: analyzes a crash-scene photo + voice description
3. **Dispatches emergency services** — SMS, BLE beacon, Supabase realtime, all in parallel
2. **Triages severity using Gemma 4** — text-first during auto-SOS, with bystander-assisted photo analysis when a scene image is available
3. **Dispatches parallel emergency signals** — SMS, BLE beacon, family link, and on-device incident logging in parallel
4. **Guides bystanders** — voice-assisted first aid in 6 Indian languages
5. **Works with no internet** — Gemma 4 E4B runs on-device for offline triage

Expand All @@ -38,8 +38,8 @@ RoadSOS is an offline-first, life-safety platform that:

Gemma 4 has three capabilities no previous model in its weight class provided:

**1. Multimodal vision (crash scene analysis)**
When SOS triggers, the phone silently captures one frame from the rear camera. Gemma 4 27B analyzes the image alongside the voice description: *fire visible? smoke? trapped occupants? vehicle count? road type?* This changes triage from "someone said it was bad" to "Gemma 4 confirmed fire and two trapped occupants."
**1. Multimodal vision (bystander scene analysis)**
When a bystander can safely frame the scene, RoadSOS can attach a crash photo for Gemma 4 27B to analyze alongside the voice description: *fire visible? smoke? trapped occupants? vehicle count? road type?* The automatic SOS path remains text-first because silent capture is unreliable when the phone is in a pocket, on a seat, or facing away from the crash.

**2. Multilingual for India**
Gemma 4 understands Hindi, Tamil, Bengali, Marathi, and Telugu natively — not via translation. An emergency description in mixed Hindi-English ("truck ne humari gaadi ko hit kiya, khoon aa raha hai, hospital kahan hai?") produces accurate triage, not garbled output.
Expand Down Expand Up @@ -78,7 +78,7 @@ The app automatically selects the highest-quality available tier at emergency ti
This JSON triggers:
- Automated SMS to 108/112 ERSS with GPS coordinates and severity
- BLE beacon broadcast for nearby RoadSOS users
- Real-time database record for emergency responders
- Local incident logging plus optional family-circle sharing
- Voice-guided first aid in the user's language

---
Expand All @@ -91,7 +91,7 @@ EmergencyOrchestrator
├── CrashDetectionService
│ └── accelerometer spike + GPS speed + stillness check (multi-stage)
├── CameraTriageService ← NEW: captures crash-scene photo for Gemma 4 vision
├── CameraTriageService ← bystander photo capture for Gemma 4 vision
├── AiTriageService — 4-tier Gemma 4 inference stack
│ ├── Tier 1: Gemma 4 27B + vision (Supabase Edge Function)
Expand All @@ -100,7 +100,7 @@ EmergencyOrchestrator
│ └── Tier 4: OfflineTriageClassifier (keyword fallback)
├── EmergencySmsDispatchService (Twilio, server-side — no key on device)
├── MeshNetworkService (BLE AES-GCM encrypted beacon)
├── MeshNetworkService (BLE SOS beacon for nearby RoadSOS phones)
└── VoiceAssistantService (TTS + STT, 6 Indian languages)
```

Expand All @@ -109,18 +109,35 @@ EmergencyOrchestrator
## Key Features

- **Crash auto-detection** — multi-stage accelerometer + GPS fusion; configurable thresholds; false-positive resistant
- **Gemma 4 vision triage** — crash-scene photo analyzed by Gemma 4 27B alongside voice description
- **Gemma 4 vision triage** — bystander-supplied crash-scene photo analyzed by Gemma 4 27B alongside voice description
- **4-tier inference** — seamless degradation from cloud to on-device to deterministic
- **Server-side SMS** — automated Twilio dispatch; works for unconscious victims; no API key on device
- **BLE encrypted mesh** — AES-GCM beacon so nearby users see an alert even with no server
- **First aid RAG** — 80+ entry SQLite FTS5 corpus; Gemma 4 E4B runs lookup on-device
- **6 Indian languages** — English, Hindi, Bengali, Marathi, Tamil, Telugu; full localization
- **BLE SOS beacon** — app-to-app broadcast so nearby RoadSOS users can detect an alert even with no server
- **First-aid guidance library** — 80+ entry SQLite FTS5 corpus with offline lookup and emergency disclaimer
- **6 Indian languages** — English, Hindi, Bengali, Marathi, Tamil, Telugu across the core emergency surfaces
- **Voice SOS** — TTS + STT for hands-busy emergencies
- **Offline maps** — PowerSync regional hospital/trauma center data; works offline
- **Good Samaritan guidance** — Indian law explained in-app so bystanders know they're protected

---

## Feature Status

RoadSOS is safer when its limits are explicit. Current status:

| Feature | Status | Reality check |
|---------|--------|---------------|
| Crash auto-detection | **Partial** | Requires motion sensors, permissions, and usable GPS speed context; not foolproof in tunnels, denied-permission cases, or cold-start GPS loss. |
| Gemma 4 auto-SOS triage | **Real** | Text-first triage runs in the emergency pipeline with cloud -> on-device -> heuristic -> keyword fallback. |
| Gemma 4 vision triage | **Partial** | Works only when a bystander supplies a scene photo; automatic silent camera capture is not used in the auto-SOS path. |
| SMS emergency dispatch | **Partial** | Real when the carrier/backend path is configured and reachable; request acceptance is not the same as confirmed ambulance arrival. |
| BLE SOS beacon | **Partial** | Nearby RoadSOS phones can detect the broadcast, but this is not a substitute for EMS dispatch and is not a multi-hop public mesh. |
| Nearby responder relay | **Planned / not configured** | The app shows this as skipped when no real responder relay is wired. |
| Offline first-aid guidance | **Real** | Uses the bundled SQLite/FTS guidance library with emergency disclaimers; it is guidance, not medical advice. |
| Family circle / tracking | **Partial** | Useful when the user already has a signed-in family circle and connectivity; not guaranteed in every emergency. |

---

## Why Gemma 4 Is Not Optional

This is the question that eliminates 90% of hackathon submissions: *"Could you replace Gemma 4 with GPT-4o or any other model?"*
Expand Down Expand Up @@ -152,7 +169,7 @@ RoadSOS is not a chatbot. It is an emergency response agent that takes real-worl
```
AGENT LOOP (fires within 10 seconds of crash detection):
┌─────────────────────────────────────────────────────────────┐
│ PERCEIVE → Accelerometer spike + GPS + camera frame
│ PERCEIVE → Accelerometer spike + GPS context
│ TRIAGE → Gemma 4 27B or E4B: structured severity JSON │
│ PLAN → Function calling: which services to dispatch │
│ ACT → dispatch_emergency() + lookup_trauma_center() │
Expand All @@ -162,7 +179,7 @@ AGENT LOOP (fires within 10 seconds of crash detection):
└─────────────────────────────────────────────────────────────┘
```

Gemma 4's function calling is what makes the PLAN → ACT step real. The model doesn't describe what should happen — it calls `dispatch_emergency(severity=5, services=["ambulance","fire_department","rescue"], gps="28.62,77.37", sms="RoadSOS SOS...")`. The Kaggle notebook (Cell 11) shows this live.
Gemma 4's structured output is what makes the PLAN → ACT step real. In the app, the model returns typed JSON (severity, services, first-aid focus), and the Dart dispatch pipeline executes the actual SMS / BLE / family-link actions. The Kaggle notebook (Cell 11) shows the structured triage flow.

---

Expand All @@ -172,7 +189,7 @@ Gemma 4's function calling is what makes the PLAN → ACT step real. The model d
|------------------|--------|-----------------|
| **Impact & Vision** | 40% | 170,000 deaths/year. 350M+ target users. MIT licensed for any state EMS. Deployable with zero custom infra. |
| **Video Storytelling** | 30% | Full 3-min script in `VIDEO_SCRIPT.md`. Emotional hook → live demo → wow moment → scale. Keyword vs Gemma split-screen. |
| **Technical Depth** | 30% | 4-tier inference routing. Real flutter_gemma LiteRT integration. Function calling agent (Cell 11). BLE AES-GCM mesh. Server-side Twilio SMS. 80-entry RAG corpus. |
| **Technical Depth** | 30% | 4-tier inference routing. Real flutter_gemma LiteRT integration. Structured triage demo (Cell 11). BLE SOS beacon. Server-side Twilio SMS. 80-entry RAG corpus. |

**Track alignment:**
- **Safety & Trust** — primary track; crash detection + dispatch + bystander guidance is pure safety infrastructure
Expand Down Expand Up @@ -270,13 +287,13 @@ RoadSOS exists because the difference between life and death on an Indian highwa
These are the five questions experienced hackathon judges ask every safety-AI project. Answered here so they're in the repo, not just in the video.

**"Is this just GPT-4 with a safety prompt?"**
No — GPT-4 doesn't run on a phone with no internet. Gemma 4 E4B does, via MediaPipe LiteRT. The offline tier is the entire point: 60% of fatal crashes in India happen where GPT-4 has no signal. See Cell 11 for function calling proof.
No — GPT-4 doesn't run on a phone with no internet. Gemma 4 E4B does, via MediaPipe LiteRT. The offline tier is the entire point: 60% of fatal crashes in India happen where GPT-4 has no signal. See Cell 11 for structured triage proof.

**"Does the offline mode actually work?"**
`gemma_local_service.dart` calls `FlutterGemmaPlugin.instance.init()` with the local model path. `gemma_model_manager.dart` handles the 2.4 GB download with resume support. Switch the phone to airplane mode — Tiers 3 and 4 always work, Tier 2 works once the model is downloaded.

**"Is the SMS dispatch real?"**
The Twilio relay is a Supabase Edge Function (`supabase/functions/triage-gemini/`). The API key lives server-side. The app sends no credentials. An unconscious victim's phone fires the SMS because the app detected the crash — not because they pressed anything.
The Twilio relay is a Supabase Edge Function (`supabase/functions/sms-dispatch/`). The API key lives server-side. The app sends no credentials. An unconscious victim's phone can request SMS dispatch because the app detected the crash — not because they pressed anything.

**"What about false positives — won't the accelerometer trigger while off-road?"**
Multi-stage detection: accelerometer spike AND sudden GPS velocity drop AND absence of deliberate phone movement afterward. Three independent signals must agree. False positive rate in testing: < 1 per 200 hours of driving.
Expand Down
7 changes: 6 additions & 1 deletion lib/l10n/app_bn.arb
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,17 @@
"sosButtonSub": "জরুরি শুরু করতে চাপুন",
"cancelSos": "বাতিল",
"orchestratorAcquiringLocation": "অবস্থান নেওয়া হচ্ছে…",
"orchestratorAiBrief": "ক্লাউড AI পরিস্থিতি যাচাই করছে…",
"orchestratorAiBrief": "AI ট্রায়েজ পরিস্থিতি যাচাই করছে…",
"orchestratorDispatching": "সব চ্যানেলে পাঠানো হচ্ছে…",
"orchestratorSosLive": "SOS সক্রিয়",
"triageResultTitle": "AI ট্রায়েজ ফল",
"triageDegradedTitle": "AI (অফলাইন)",
"firstAidGuidance": "প্রাথমিক চিকিৎসা",
"firstAidScreenTitle": "প্রাথমিক চিকিৎসা গাইড",
"firstAidSearchHint": "আঘাত বা উপসর্গ লিখুন…",
"firstAidLookupTitle": "জরুরি প্রাথমিক চিকিৎসা খোঁজ",
"firstAidLookupSubtitle": "অফলাইন নির্দেশিকা লাইব্রেরি খুঁজতে আঘাত লিখুন।\nএটি শুধু দিকনির্দেশনা — প্রকৃত চিকিৎসা সহায়তার জন্য 108/112 নম্বরে কল করুন।",
"firstAidLoadError": "এই ডিভাইসে প্রাথমিক চিকিৎসার নির্দেশনা লোড করা যায়নি।",
"settingsLanguage": "ভাষা",
"settingsLanguageSubtitle": "ইন্টারফেস ও ভয়েস",
"incidentVoiceHint": "যা দেখছেন বলুন…",
Expand Down
7 changes: 6 additions & 1 deletion lib/l10n/app_en.arb
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
"orchestratorLocationUnavailable": "Location unavailable — enable location services or move to open sky.",
"orchestratorManualActionRequired": "Manual action required — if automated dispatch fails, dial your emergency number now.",
"orchestratorSmsNoGpsPayload": "SOS (no GPS). Please call emergency services now. RoadSOS could not acquire location.",
"orchestratorAiBrief": "Cloud AI is assessing the situation…",
"orchestratorAiBrief": "AI triage is assessing the situation…",
"orchestratorTriageDone": "Triage complete — severity {level}",
"@orchestratorTriageDone": {
"placeholders": {
Expand Down Expand Up @@ -58,6 +58,11 @@
"severityUnknown": "UNKNOWN",
"dispatchedServices": "DISPATCHED SERVICES",
"firstAidGuidance": "FIRST AID GUIDANCE",
"firstAidScreenTitle": "First aid guide",
"firstAidSearchHint": "Describe injury or symptom…",
"firstAidLookupTitle": "Emergency first-aid lookup",
"firstAidLookupSubtitle": "Type an injury to search the offline guidance library.\nFor guidance only — call 108/112 for real medical help.",
"firstAidLoadError": "Could not load first-aid guidance on this device.",
"noAiBadge": "OFFLINE",
"settingsTitle": "SETTINGS",
"sectionConnectivity": "CONNECTIVITY",
Expand Down
7 changes: 6 additions & 1 deletion lib/l10n/app_hi.arb
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"orchestratorSelfSosStarted": "सेल्फ SOS शुरू",
"orchestratorAcquiringLocation": "स्थान प्राप्त कर रहे हैं…",
"orchestratorLocationSecured": "स्थान: {lat}, {lng}",
"orchestratorAiBrief": "क्लाउड AI स्थिति जांच रहा है…",
"orchestratorAiBrief": "AI ट्राइएज स्थिति जांच रहा है…",
"orchestratorTriageDone": "ट्राइएज पूर्ण — गंभीरता {level}",
"orchestratorDispatching": "सभी चैनलों पर अलर्ट भेज रहे हैं…",
"orchestratorSosLive": "SOS सक्रिय — सभी चैनल चालू",
Expand All @@ -37,6 +37,11 @@
"severityUnknown": "अज्ञात",
"dispatchedServices": "भेजी गई सेवाएँ",
"firstAidGuidance": "प्राथमिक उपचार मार्गदर्शन",
"firstAidScreenTitle": "प्राथमिक उपचार गाइड",
"firstAidSearchHint": "चोट या लक्षण लिखें…",
"firstAidLookupTitle": "आपातकालीन प्राथमिक उपचार खोज",
"firstAidLookupSubtitle": "ऑफ़लाइन मार्गदर्शन लाइब्रेरी खोजने के लिए चोट लिखें।\nयह केवल मार्गदर्शन है — वास्तविक चिकित्सा मदद के लिए 108/112 पर कॉल करें।",
"firstAidLoadError": "इस डिवाइस पर प्राथमिक उपचार मार्गदर्शन लोड नहीं हो सका।",
"noAiBadge": "ऑफ़लाइन",
"settingsTitle": "सेटिंग्स",
"sectionConnectivity": "कनेक्टिविटी",
Expand Down
32 changes: 31 additions & 1 deletion lib/l10n/app_localizations.dart
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ abstract class AppLocalizations {
/// No description provided for @orchestratorAiBrief.
///
/// In en, this message translates to:
/// **'Cloud AI is assessing the situation…'**
/// **'AI triage is assessing the situation…'**
String get orchestratorAiBrief;

/// No description provided for @orchestratorTriageDone.
Expand Down Expand Up @@ -352,6 +352,36 @@ abstract class AppLocalizations {
/// **'FIRST AID GUIDANCE'**
String get firstAidGuidance;

/// No description provided for @firstAidScreenTitle.
///
/// In en, this message translates to:
/// **'First aid guide'**
String get firstAidScreenTitle;

/// No description provided for @firstAidSearchHint.
///
/// In en, this message translates to:
/// **'Describe injury or symptom…'**
String get firstAidSearchHint;

/// No description provided for @firstAidLookupTitle.
///
/// In en, this message translates to:
/// **'Emergency first-aid lookup'**
String get firstAidLookupTitle;

/// No description provided for @firstAidLookupSubtitle.
///
/// In en, this message translates to:
/// **'Type an injury to search the offline guidance library.\nFor guidance only — call 108/112 for real medical help.'**
String get firstAidLookupSubtitle;

/// No description provided for @firstAidLoadError.
///
/// In en, this message translates to:
/// **'Could not load first-aid guidance on this device.'**
String get firstAidLoadError;

/// No description provided for @noAiBadge.
///
/// In en, this message translates to:
Expand Down
19 changes: 18 additions & 1 deletion lib/l10n/app_localizations_bn.dart
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ class AppLocalizationsBn extends AppLocalizations {
'SOS (no GPS). Please call emergency services now. RoadSOS could not acquire location.';

@override
String get orchestratorAiBrief => 'ক্লাউড AI পরিস্থিতি যাচাই করছে…';
String get orchestratorAiBrief => 'AI ট্রায়েজ পরিস্থিতি যাচাই করছে…';

@override
String orchestratorTriageDone(int level) {
Expand Down Expand Up @@ -141,6 +141,23 @@ class AppLocalizationsBn extends AppLocalizations {
@override
String get firstAidGuidance => 'প্রাথমিক চিকিৎসা';

@override
String get firstAidScreenTitle => 'প্রাথমিক চিকিৎসা গাইড';

@override
String get firstAidSearchHint => 'আঘাত বা উপসর্গ লিখুন…';

@override
String get firstAidLookupTitle => 'জরুরি প্রাথমিক চিকিৎসা খোঁজ';

@override
String get firstAidLookupSubtitle =>
'অফলাইন নির্দেশিকা লাইব্রেরি খুঁজতে আঘাত লিখুন।\nএটি শুধু দিকনির্দেশনা — প্রকৃত চিকিৎসা সহায়তার জন্য 108/112 নম্বরে কল করুন।';

@override
String get firstAidLoadError =>
'এই ডিভাইসে প্রাথমিক চিকিৎসার নির্দেশনা লোড করা যায়নি।';

@override
String get noAiBadge => 'OFFLINE';

Expand Down
Loading
Loading