Skip to content

Conversation

@Irakus
Copy link
Contributor

@Irakus Irakus commented Sep 3, 2025

📝 Description

Manager app:

  • Added WebRTC connection to retrieve video streams for processed and unprocessed camera feed
  • Removed live-view button from scene detail view
  • Added loading video to play before WebRTC connection is established
  • In calibration view, changed source of projection to be video from WebRTC stream
  • Removed handling of images with frames from MQTT
  • Adjusted calibration messages to use parameters from WebRTC video

DLS Pipeline Server:

  • Added frame publishing through WebRTC to Media Server(MediaMTX)
  • Added watermark to published frames
  • Removed Python code responsible for publishing frames to MQTT
  • Changed naming of streams for easy corellation between raw camera feed and dls-processed feed in Media Server
  • Removed no-rtsp pipelines as RTSP is needed for WebRTC
  • Adjusted all sample pipelines

Media Server (MediaMTX)

  • Added config file
  • Enabled WebRTC protocol

Coturn:

  • Added new component for establishing connection between Media Server and WebRTC receivers

Nginx:

  • Added new component to serve as reverse proxy to enable TLS connection between Web app and Media Server
  • DLS still uses insecure connection to Media Server
  • Added new certificates generation for nginx

Samples:

  • Changed mp4->ts conversion script to remove b-frames (WebRTC requirement) and add more keyframes (for WebRTC performance)

Docs:

  • Included guides on updating hostfile and CA certificates to enable accessing Scenescape through web.scenescape.intel.com instead of IP
  • Updated sample DLS pipelines code snippets

✨ Type of Change

Select the type of change your PR introduces:

  • 🐞 Bug fix – Non-breaking change which fixes an issue
  • 🚀 New feature – Non-breaking change which adds functionality
  • 🔨 Refactor – Non-breaking change which refactors the code base
  • 💥 Breaking change – Changes that break existing functionality
  • 📚 Documentation update
  • 🔒 Security update
  • 🧪 Tests
  • 🚂 CI

🧪 Testing Scenarios

Describe how the changes were tested and how reviewers can test them too:

  • [*] Ran demo
  • ✅ Tested manually
  • 🤖 Ran automated end-to-end tests

✅ Checklist

Before submitting the PR, ensure the following:

  • 🔍 PR title is clear and descriptive
  • 📝 For internal contributors: If applicable, include the JIRA ticket number (e.g., ITEP-123456) in the PR title. Do not include full URLs
  • 💬 I have commented my code, especially in hard-to-understand areas
  • 📄 I have made corresponding changes to the documentation
  • ✅ I have added tests that prove my fix is effective or my feature works

@Irakus Irakus changed the title Itep 68945 webrtc raw Itep 68945 webrtc Sep 12, 2025
@Irakus Irakus marked this pull request as ready for review September 17, 2025 12:31
@Irakus Irakus requested a review from saratpoluri as a code owner September 17, 2025 12:31
@@ -0,0 +1,21 @@
MIT License
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need individual license call out for each dependency. Those are done in third-party-programs.txt

@saratpoluri saratpoluri requested a review from Copilot September 17, 2025 21:53
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements WebRTC-based video streaming to replace MQTT-based frame publishing in the SceneScape system. The changes introduce real-time video streaming capabilities through MediaMTX and nginx proxy services, improving performance and user experience.

  • Replaced MQTT frame publishing with WebRTC streaming infrastructure
  • Added MediaMTX and nginx proxy services for secure TLS connections
  • Modified calibration and visualization components to use video streams instead of static images

Reviewed Changes

Copilot reviewed 28 out of 29 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tools/certificates/Makefile Added certificate generation for MediaMTX proxy service
scene_common/src/transform.py Added type checking in CameraPose constructor for intrinsics parameter
sample_data/nginx.conf New nginx configuration for MediaMTX proxy with SSL support
sample_data/mediamtx.yml MediaMTX configuration with WebRTC and TURN server settings
sample_data/docker-compose-dls-perf.yml Added MediaMTX, nginx, and coturn services to Docker composition
manager/src/templates/sscape/sceneDetail.html Replaced image elements with video elements for camera feeds
manager/src/templates/cam/cam_calibrate.html Updated calibration view to use video instead of images
manager/src/static/js/webrtc_reader.js New WebRTC reader implementation for video stream handling
manager/src/static/js/viewport.js Modified projection to use video texture instead of image loading
manager/src/static/js/sscape.js Removed MQTT image handling and added WebRTC stream initialization
dlstreamer-pipeline-server/user_scripts/gvapython/sscape/sscape_adapter.py Removed MQTT frame publishing functionality
dlstreamer-pipeline-server/retail-config.json Updated pipeline configurations with WebRTC destinations and watermarks
docs/user-guide/Getting-Started-Guide.md Added instructions for hosts file configuration and SSL certificate installation

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

"""

self.intrinsics = intrinsics
if type(intrinsics) != CameraIntrinsics:
Copy link

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use isinstance() instead of type() for type checking to support inheritance and follow Python best practices.

Suggested change
if type(intrinsics) != CameraIntrinsics:
if not isinstance(intrinsics, CameraIntrinsics):

Copilot uses AI. Check for mistakes.
Comment on lines +79 to +86
"destination": {
"frame": {
"type": "webrtc",
"peer-id": "camera2",
"bitrate": 5000,
"overlay": false
}
},
Copy link

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'destination' configuration is placed outside the 'payload' object, which is inconsistent with the structure in other configuration files where it appears inside 'payload'. This should be moved inside the 'payload' object for consistency.

Copilot uses AI. Check for mistakes.
@saratpoluri
Copy link
Contributor

Could you please add a "How to Verify" section in your description or Testing Scenarios section.

@Irakus
Copy link
Contributor Author

Irakus commented Sep 18, 2025

Could you please add a "How to Verify" section in your description or Testing Scenarios section.

So far, I only run demo. I've added new instructions in user guide, aside from that, deployment process looks the same.

Copy link
Contributor

@kblaszczak-intel kblaszczak-intel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doc looks good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants