[ITEP-73288] Computer vision pipeline API design document #494

rawatts10 · 2025-10-15T23:50:40Z

📝 Description

This PR introduces a comprehensive design document for the Vision Pipeline API, targeting domain experts who need to deploy computer vision capabilities without deep technical knowledge. The document defines a simple, API-first approach for connecting cameras, configuring composable analytics pipelines, and accessing object detection metadata.

What was changed:

Added vision-pipeline-overview.md with complete API specification
Defined API workflows with sequence diagrams for common deployment scenarios
Specified implementation considerations for coordinate systems, performance, and server architecture

Why it was needed:

Enable domain experts (ISV's and system integrators) to deploy vision analytics without computer vision expertise
Provide clear API specification for SceneScape integration and MQTT-based metadata publishing
Establish foundation for UI development and customer extensibility

How it addresses the need:

Focuses on traffic operations expert persona with smart intersection use case
Includes composable pipeline stages with hardware optimization targeting
Provides comprehensive workflows from camera setup through multi-camera analytics deployment

✨ Type of Change

Select the type of change your PR introduces:

🐞 Bug fix – Non-breaking change which fixes an issue
🚀 New feature – Non-breaking change which adds functionality
🔨 Refactor – Non-breaking change which refactors the code base
💥 Breaking change – Changes that break existing functionality
📚 Documentation update
🔒 Security update
🧪 Tests
🚂 CI

🧪 Testing Scenarios

Describe how the changes were tested and how reviewers can test them too:

✅ Tested manually – Document structure and markdown formatting validated
🤖 Ran automated end-to-end tests

Review checklist:

Verify mermaid diagrams render correctly in GitHub
Confirm API workflows align with SceneScape integration requirements
Validate technical accuracy of coordinate system and performance considerations

✅ Checklist

Before submitting the PR, ensure the following:

🔍 PR title is clear and descriptive
📝 For internal contributors: If applicable, include the JIRA ticket number (e.g., ITEP-123456) in the PR title. Do not include full URLs
💬 I have commented my code, especially in hard-to-understand areas
📄 I have made corresponding changes to the documentation
✅ I have added tests that prove my fix is effective or my feature works

Defines API workflows and implementation considerations for customer domain experts deploying vision analytics with SceneScape integration.

Keep only the vision pipeline API design document changes in this branch.

…ut details.

… into feature/vision-pipeline-api-design-doc

docs/design/vision-pipeline-overview.md

- Eliminate duplicate reference in MQTT Publishing bullet point - Keep detailed explanation in dedicated Topic Generation section - Improve document clarity and reduce redundancy

- Standardize sequence diagrams to use 'User' consistently - Add robust camera error handling with persistent reconnection capabilities - Add comprehensive system monitoring API for observability - Add multimodal input support section with audio, LiDAR, and radar - Add security considerations note referencing separate hardening guide - Improve document structure and readability for target audience

docs/design/vision-pipeline-overview.md

…ilience - Create core operating principles section - Strengthen separation of concerns language throughout Camera and Pipeline APIs - Add JSON configuration examples for a couple of camera types - Fix sequence diagram terminology and ensure consistency Incorporate review feedback from Vibhu and Sarat, emphasizing the importance of resilience in production environments and modular system design for operational manageability.

Specify that a stage operates on an array of outputs from a single previous stage, not outputs from multiple different stages, removing ambiguity about pipeline architecture.

Specify that pipelines must support configuring MQTT broker connection details including host, port, credentials, TLS settings, and optional topic namespace prefix via the REST API.

…e of the note container.

…ne-api-design-doc

… it breaks GitHub rendering of the diagrams. Will look for cross-support with Sphynx using some other method.

… feedback from Tomasz D.

…tion - Move misplaced items between Requirements and Considerations - Strengthen requirement language (should → must) for mandatory items - Reorganize Time Coordination, Performance Management, and Latency sections - Clarify system resource limits and frame drop strategy requirements - Separate actionable requirements from implementation guidance

…ion results" to "send pipeline results"

docs/design/vision-pipeline-overview.md

@gpu

…mposition per feedback from Tomasz D. - Add Advanced Pipeline Composition section with DAG architecture - Implement comprehensive DAG syntax with sequential, parallel, and branching patterns - Add visual Mermaid diagram showing metadata convergence architecture - Include practical JSON metadata examples with model tracking - Add hardware targeting syntax (@gpu, @NPU, @cpu) for per-stage optimization - Implement stage aliases (vehicle, vattrib, lpd, lpr, reid) with critical importance warnings - Add comprehensive model metadata tracking (name, version, hash) for reproducibility - Include complex DAG examples for traffic intersection and video analysis workflows - Add DAG validation, execution order, and performance optimization details - Update document date and clarify scope to include supporting functionality

docs/design/vision-pipeline-overview.md

…e, clarify model info is available via separate endpoint, and combine timestamp/camera ID bullets for clarity.

…ss to model info per stage, without specifying output structure and without requiring this information in pipeline metadata output

rawatts10 added 3 commits October 15, 2025 13:30

Add Vision Pipeline API design document for domain experts

47d3dfc

Defines API workflows and implementation considerations for customer domain experts deploying vision analytics with SceneScape integration.

Restored correct document overview and removed corrupted sections.

e928625

Remove unrelated sscape_adapter.py changes

4f12b1c

Keep only the vision pipeline API design document changes in this branch.

rawatts10 requested review from a team, GrzegorzWrobel, Irakus, MikolajKasprzak, daddo-intel, dmytroye, jdanieck, ltalarcz, saratpoluri, sarthakdeva-intel, sbelhaik and tdorauintc as code owners October 15, 2025 23:50

rawatts10 added 2 commits October 16, 2025 11:52

Added time synchronization section and clarified latency and throughp…

4234d67

…ut details.

Merge branch 'main' of https://github.com/open-edge-platform/scenescape…

6474302

… into feature/vision-pipeline-api-design-doc

rawatts10 force-pushed the feature/vision-pipeline-api-design-doc branch from d9d33b3 to 6474302 Compare October 16, 2025 18:57