Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
ea53ca7
feat: add platform field to Room and Meeting models
tito Sep 2, 2025
cf64e1a
feat: add database migration for platform field
tito Sep 2, 2025
d42380a
feat: add Jitsi configuration settings
tito Sep 2, 2025
d49fdcb
feat: create video platforms architecture with Jitsi directory structure
tito Sep 2, 2025
8e5ef5b
feat: implement JitsiClient with JWT authentication
tito Sep 2, 2025
3f4fc26
feat: register Jitsi platform in video platforms factory and registry
tito Sep 2, 2025
2b136ac
feat: create Jitsi webhook endpoints for event handling
tito Sep 2, 2025
f2bb6aa
feat: update rooms.py to use video platform abstraction
tito Sep 2, 2025
6d2092f
feat: create comprehensive Jitsi integration documentation
tito Sep 2, 2025
42a603d
feat: add PyJWT dependency and finalize Jitsi integration
tito Sep 2, 2025
2492342
feat: add comprehensive video platform test suite
tito Sep 2, 2025
24ff83a
docs: add comprehensive Whereby integration user guide
tito Sep 2, 2025
d861d92
docs: add comprehensive Jitsi Meet integration user guide
tito Sep 2, 2025
0acb9ca
Replace Literal with VideoPlatform StrEnum for platform field
tito Sep 2, 2025
2d2c23f
Create video_platforms/whereby structure and WherebyClient
tito Sep 2, 2025
51229a1
Fix Jitsi client issues and create typed meeting data
tito Sep 2, 2025
da70006
Add webhook events storage to meetings model
tito Sep 2, 2025
398be06
feat: add typing overloads and clean up platform client factory
tito Sep 3, 2025
7875ec3
feat: move platform routers to video_platforms folders
tito Sep 3, 2025
52eff2a
feat: clean up legacy code and remove excessive documentation
tito Sep 3, 2025
c26ce65
feat: update Jitsi documentation with webhook events storage system
tito Sep 3, 2025
fa559b1
feat: update and expand video platform tests
tito Sep 3, 2025
dd00899
fix: replace datetime.utcnow() with datetime.now(tz=timezone.utc) in …
tito Sep 3, 2025
41224a4
docs: move platform-jitsi.md to docs/ directory
tito Sep 3, 2025
293f7d4
feat: implement frontend video platform configuration and abstraction
tito Sep 4, 2025
95e8011
Merge main into jisti-integration branch
tito Sep 15, 2025
e91979a
feat: use jitsi file system
tito Sep 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
369 changes: 369 additions & 0 deletions docs/jitsi.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,369 @@
# Jitsi Integration for Reflector

This document contains research and planning notes for integrating Jitsi Meet as a replacement for Whereby in Reflector.

## Overview

Jitsi Meet is an open-source video conferencing solution that can replace Whereby in Reflector, providing:
- Cost reduction (no per-minute charges)
- Direct recording access via Jibri
- Real-time event webhooks
- Full customization and control

## Current Whereby Integration Analysis

### Architecture
1. **Room Creation**: User creates a "room" template in Reflector DB with settings
2. **Meeting Creation**: `/rooms/{room_name}/meeting` endpoint calls Whereby API to create meeting
3. **Recording**: Whereby handles recording automatically to S3 bucket
4. **Webhooks**: Whereby sends events for participant tracking

### Database Structure
```python
# Room = Template/Configuration
class Room:
id, name, user_id
recording_type, recording_trigger # cloud, automatic-2nd-participant
webhook_url, webhook_secret

# Meeting = Actual Whereby Meeting Instance
class Meeting:
id # Whereby meetingId
room_name # Generated by Whereby
room_url, host_room_url # Whereby URLs
num_clients # Updated via webhooks
```

## Jitsi Components

### Core Architecture
- **Jitsi Meet**: Web frontend (Next.js + React)
- **Prosody**: XMPP server for messaging/rooms
- **Jicofo**: Conference focus (orchestration)
- **JVB**: Videobridge (media routing)
- **Jibri**: Recording service
- **Jigasi**: SIP gateway (optional, for phone dial-in)

### Exposure Requirements
- **Web service**: 443/80 (frontend)
- **JVB**: 10000/UDP (media streams) - **MUST EXPOSE**
- **Prosody**: 5280 (BOSH/WebSocket) - can proxy via web
- **Jicofo, Jibri, Jigasi**: Internal only

## Recording with Jibri

### How Jibri Works
- Each Jibri instance handles **one recording at a time**
- Records mixed audio/video to MP4 format
- Uses Chrome headless + ffmpeg for capture
- Supports finalize scripts for post-processing

### Jibri Pool for Scaling
- Multiple Jibri instances join "jibribrewery" MUC
- Jicofo distributes recording requests to available instances
- Automatic load balancing and failover

```yaml
# Multiple Jibri instances
jibri1:
environment:
- JIBRI_INSTANCE_ID=jibri1
- JIBRI_BREWERY_MUC=jibribrewery

jibri2:
environment:
- JIBRI_INSTANCE_ID=jibri2
- JIBRI_BREWERY_MUC=jibribrewery
```

### Recording Automation Options
1. **Environment Variables**: `ENABLE_RECORDING=1`, `AUTO_RECORDING=1`
2. **URL Parameters**: `?config.autoRecord=true`
3. **JWT Token**: Include recording permissions in JWT
4. **API Control**: `api.executeCommand('startRecording')`

### Post-Processing Integration
```bash
#!/bin/bash
# finalize.sh - runs after recording completion
RECORDING_FILE=$1
MEETING_METADATA=$2
ROOM_NAME=$3

# Copy to Reflector-accessible location
cp "$RECORDING_FILE" /shared/reflector-uploads/

# Trigger Reflector processing
curl -X POST "http://reflector-api:8000/v1/transcripts/process" \
-H "Content-Type: application/json" \
-d "{
\"file_path\": \"/shared/reflector-uploads/$(basename $RECORDING_FILE)\",
\"room_name\": \"$ROOM_NAME\",
\"source\": \"jitsi\"
}"
```

## React Integration

### Official React SDK
```bash
npm i @jitsi/react-sdk
```

```jsx
import { JitsiMeeting } from '@jitsi/react-sdk'

<JitsiMeeting
room="meeting-room"
serverURL="https://your-jitsi.domain"
jwt="your-jwt-token"
config={{
startWithAudioMuted: true,
fileRecordingsEnabled: true,
autoRecord: true
}}
onParticipantJoined={(participant) => {
// Track participant events
}}
onRecordingStatusChanged={(status) => {
// Handle recording events
}}
/>
```

## Authentication & Room Control

### JWT-Based Access Control
```python
def generate_jitsi_jwt(payload):
return jwt.encode({
"aud": "jitsi",
"iss": "reflector",
"sub": "reflector-user",
"room": payload["room"],
"exp": int(payload["exp"].timestamp()),
"context": {
"user": {
"name": payload["user_name"],
"moderator": payload.get("moderator", False)
},
"features": {
"recording": payload.get("recording", True)
}
}
}, JITSI_JWT_SECRET)
```

### Prevent Anonymous Room Creation
```bash
# Environment configuration
ENABLE_AUTH=1
ENABLE_GUESTS=0
AUTH_TYPE=jwt
JWT_APP_ID=reflector
JWT_APP_SECRET=your-secret-key
```

## Webhook Integration

### Real-time Events via Prosody
Custom event-sync module can send webhooks for:
- Participant join/leave
- Recording start/stop
- Room creation/destruction
- Mute/unmute events

```lua
-- mod_event_sync.lua
module:hook("muc-occupant-joined", function(event)
send_event({
type = "participant_joined",
room = event.room.jid,
participant = {
nick = event.occupant.nick,
jid = event.occupant.jid,
},
timestamp = os.time(),
});
end);
```

### Jibri Recording Webhooks
```bash
# Environment variable
JIBRI_WEBHOOK_SUBSCRIBERS=https://your-reflector.com/webhooks/jibri
```

## Proposed Reflector Integration

### Modified Database Schema
```python
class Meeting(BaseModel):
id: str # Our generated meeting ID
room_name: str # Generated: reflector-{room.name}-{timestamp}
room_url: str # https://jitsi.domain/room_name?jwt=token
host_room_url: str # Same but with moderator JWT
# Add Jitsi-specific fields
jitsi_jwt: str # JWT token
jitsi_room_id: str # Internal room identifier
recording_status: str # pending, recording, completed
recording_file_path: Optional[str]
```

### API Replacement
```python
# Replace whereby.py with jitsi.py
async def create_meeting(room_name_prefix: str, end_date: datetime, room: Room):
# Generate unique room name
jitsi_room = f"reflector-{room.name}-{int(time.time())}"

# Generate JWT tokens
user_jwt = generate_jwt(room=jitsi_room, moderator=False, exp=end_date)
host_jwt = generate_jwt(room=jitsi_room, moderator=True, exp=end_date)

return {
"meetingId": generate_uuid4(), # Our ID
"roomName": jitsi_room,
"roomUrl": f"https://jitsi.domain/{jitsi_room}?jwt={user_jwt}",
"hostRoomUrl": f"https://jitsi.domain/{jitsi_room}?jwt={host_jwt}",
"startDate": datetime.now().isoformat(),
"endDate": end_date.isoformat(),
}
```

### Webhook Endpoints
```python
# Replace whereby webhook with jitsi webhooks
@router.post("/jitsi/events")
async def jitsi_events_webhook(event_data: dict):
event_type = event_data.get("event")
room_name = event_data.get("room", "").split("@")[0]

meeting = await Meeting.get_by_room(room_name)

if event_type == "muc-occupant-joined":
# Update participant count
meeting.num_clients += 1

elif event_type == "jibri-recording-on":
meeting.recording_status = "recording"

elif event_type == "jibri-recording-off":
meeting.recording_status = "processing"
await process_meeting_recording.delay(meeting.id)

@router.post("/jibri/recording-complete")
async def recording_complete(data: dict):
# Handle finalize script webhook
room_name = data.get("room_name")
file_path = data.get("file_path")

meeting = await Meeting.get_by_room(room_name)
meeting.recording_file_path = file_path
meeting.recording_status = "completed"

# Start Reflector processing
await process_recording_for_transcription(meeting.id, file_path)
```

## Deployment with Docker

### Official docker-jitsi-meet
```bash
# Download official release
wget $(wget -q -O - https://api.github.com/repos/jitsi/docker-jitsi-meet/releases/latest | grep zip | cut -d\" -f4)

# Setup
mkdir -p ~/.jitsi-meet-cfg/{web,transcripts,prosody/config,prosody/prosody-plugins-custom,jicofo,jvb,jigasi,jibri}
./gen-passwords.sh # Generate secure passwords
docker compose up -d
```

### Coolify Integration
```yaml
services:
web:
ports: ["80:80", "443:443"]
jvb:
ports: ["10000:10000/udp"] # Must expose for media
jibri1:
environment:
- JIBRI_INSTANCE_ID=jibri1
- JIBRI_FINALIZE_RECORDING_SCRIPT_PATH=/config/finalize.sh
jibri2:
environment:
- JIBRI_INSTANCE_ID=jibri2
```

## Benefits vs Whereby

### Cost & Control
✅ **No per-minute charges** - significant cost savings
✅ **Full recording control** - direct file access
✅ **Custom branding** - complete UI control
✅ **Self-hosted** - no vendor lock-in

### Technical Advantages
✅ **Real-time events** - immediate webhook notifications
✅ **Rich participant metadata** - detailed tracking
✅ **JWT security** - token-based access with expiration
✅ **Multiple recording formats** - audio-only options
✅ **Scalable architecture** - horizontal Jibri scaling

### Integration Benefits
✅ **Same API surface** - minimal changes to existing code
✅ **React SDK** - better frontend integration
✅ **Direct processing** - no S3 download delays
✅ **Event-driven architecture** - better real-time capabilities

## Implementation Plan

1. **Deploy Jitsi Stack** - Set up docker-jitsi-meet with multiple Jibri instances
2. **Create jitsi.py** - Replace whereby.py with Jitsi API functions
3. **Update Database** - Add Jitsi-specific fields to Meeting model
4. **Webhook Integration** - Replace Whereby webhooks with Jitsi events
5. **Frontend Updates** - Replace Whereby embed with Jitsi React SDK
6. **Testing & Migration** - Gradual rollout with fallback to Whereby

## Recording Limitations & Considerations

### Current Limitations
- **Mixed audio only** - Jibri doesn't separate participant tracks natively
- **One recording per Jibri** - requires multiple instances for concurrent recordings
- **Chrome dependency** - Jibri uses headless Chrome for recording

### Metadata Capabilities
✅ **Participant join/leave timestamps** - via webhooks
✅ **Speaking time tracking** - via audio level events
✅ **Meeting duration** - precise timing
✅ **Room-specific data** - custom metadata in JWT

### Alternative Recording Methods
- **Local recording** - browser-based, per-participant
- **Custom recording** - lib-jitsi-meet for individual streams
- **Third-party solutions** - Recall.ai, Otter.ai integrations

## Security Considerations

### JWT Configuration
- **Room-specific tokens** - limit access to specific rooms
- **Time-based expiration** - automatic cleanup
- **Feature permissions** - control recording, moderation rights
- **User identification** - embed user metadata in tokens

### Access Control
- **No anonymous rooms** - all rooms require valid JWT
- **API-only creation** - prevent direct room access
- **Webhook verification** - HMAC signature validation

## Next Steps

1. **Deploy test Jitsi instance** - validate recording pipeline
2. **Prototype jitsi.py** - create equivalent API functions
3. **Test webhook integration** - ensure event delivery works
4. **Performance testing** - validate multiple concurrent recordings
5. **Migration strategy** - plan gradual transition from Whereby

---

*This document serves as the comprehensive planning and research notes for Jitsi integration in Reflector. It should be updated as implementation progresses and new insights are discovered.*
Loading
Loading