Remote detection backend improvements: Roboflow fix, resize, CPU fallback, coralapi - #3
Merged
Merged
Conversation
…back, coralapi - Fix RoboflowInferenceDetector request format for Inference Server 1.x (model_id + image belong in the JSON body; the old query-param form gets HTTP 422, which previously surfaced as silent zero-detection results) - Add opt-in client-side downscaling (roboflow.resize_max_px / coralapi.resize_max_px): the server resizes to model input anyway, so smaller uploads cut per-frame latency 3-4x at identical accuracy; predictions are rescaled to original frame coordinates - Remote detectors now raise DetectorUnavailableError on transport failures instead of returning empty detections; new FallbackDetector optionally falls back to local CPU inference (detection_fallback: local) - immediate switch when the backend never succeeded this job, after 3 consecutive failures otherwise, sticky per job, local model loaded lazily - ProcessingResult now records detection_backend / detection_model / detection_fallback (what actually ran), surfaced in API responses and as backend/model/fallback tags on InfluxDB metrics for per-model dashboards and fallback-rate alerting - New coralapi backend (https://github.com/nathan-v/coralapi) for Coral Edge TPU inference: multipart /v1/vision/detect, normalized-box scaling, label mapping with COCO-90 index fallback for label-less models - Docs: backend comparison/config for coralapi, detection_fallback setting; gitignore local benchmark artifacts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Four related improvements to the remote detection backend layer:
model_idandimagein the JSON body. The old query-param format gets HTTP 422, which surfaced as clips "successfully" processing with zero detections.roboflow.resize_max_px,coralapi.resize_max_px, opt-in): the server resizes to model input anyway, so smaller uploads cut per-frame latency 3-4x at identical detection quality. Predictions are rescaled to original frame coordinates so calibration is unaffected.detection_fallback: local, opt-in): remote detectors now raiseDetectorUnavailableErroron transport failures instead of silently returning empty detections. AFallbackDetectorwrapper switches to local CPU inference (immediately if the backend never succeeded this job, after 3 consecutive failures otherwise; sticky per job; local model loaded lazily).ProcessingResultgainsdetection_backend/detection_model/detection_fallbackfields recording what actually ran, and these are emitted asbackend/model/fallbacktags on InfluxDB metrics.detection_backend: coralapi) for Coral Edge TPU inference via coralapi: multipart/v1/vision/detect, normalized-box scaling, label mapping with a COCO-90 index fallback for label-less models.Related Issue
N/A
Motivation and Context
The Roboflow format bug and the silent-empty-detection failure mode were both found in production. Resize was benchmarked at 3-4x per-frame speedup at every model size. Fallback protects a ~350 clip/day pipeline from remote host outages (laptop sleep and a desktop outage both occurred during development). The metric tags enable per-model dashboards and fallback-rate alerting.
How Has This Been Tested?
pytest); ruff check and format cleanTypes of changes
Checklist:
ruff checkandruff formatpass)