Skip to content

Remote detection backend improvements: Roboflow fix, resize, CPU fallback, coralapi - #3

Merged
nathan-v merged 1 commit into
mainfrom
feat/detection-backends
Jul 17, 2026
Merged

Remote detection backend improvements: Roboflow fix, resize, CPU fallback, coralapi#3
nathan-v merged 1 commit into
mainfrom
feat/detection-backends

Conversation

@nathan-v

Copy link
Copy Markdown
Owner

Description

Four related improvements to the remote detection backend layer:

  • Roboflow request-format fix: Inference Server 1.x requires model_id and image in the JSON body. The old query-param format gets HTTP 422, which surfaced as clips "successfully" processing with zero detections.
  • Client-side downscaling (roboflow.resize_max_px, coralapi.resize_max_px, opt-in): the server resizes to model input anyway, so smaller uploads cut per-frame latency 3-4x at identical detection quality. Predictions are rescaled to original frame coordinates so calibration is unaffected.
  • CPU fallback (detection_fallback: local, opt-in): remote detectors now raise DetectorUnavailableError on transport failures instead of silently returning empty detections. A FallbackDetector wrapper switches to local CPU inference (immediately if the backend never succeeded this job, after 3 consecutive failures otherwise; sticky per job; local model loaded lazily). ProcessingResult gains detection_backend/detection_model/detection_fallback fields recording what actually ran, and these are emitted as backend/model/fallback tags on InfluxDB metrics.
  • New coralapi backend (detection_backend: coralapi) for Coral Edge TPU inference via coralapi: multipart /v1/vision/detect, normalized-box scaling, label mapping with a COCO-90 index fallback for label-less models.

Related Issue

N/A

Motivation and Context

The Roboflow format bug and the silent-empty-detection failure mode were both found in production. Resize was benchmarked at 3-4x per-frame speedup at every model size. Fallback protects a ~350 clip/day pipeline from remote host outages (laptop sleep and a desktop outage both occurred during development). The metric tags enable per-model dashboards and fallback-rate alerting.

How Has This Been Tested?

  • 166 unit tests pass (pytest); ruff check and format clean
  • New tests: Roboflow wire format, resize/box-rescaling for both backends, fallback policy matrix (immediate/threshold/sticky/lazy/no-fallback), config validation, metric tag emission, coralapi wire format and COCO-90 index fallback
  • Live: deployed to a production cluster; Roboflow backend verified against GPU inference hosts; fallback exercised end-to-end against a dead backend (real detections via CPU, warning surfaced, tags correct); coralapi backend benchmarked against a Coral USB Accelerator across 8 models, including a byte-identical regression between label-path and index-fallback-path class mapping

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project (ruff check and ruff format pass)
  • My change requires a change to the documentation
  • I have updated the documentation accordingly
  • I have read the CONTRIBUTING document
  • I have added tests to cover my changes
  • All new and existing tests passed

…back, coralapi

- Fix RoboflowInferenceDetector request format for Inference Server 1.x
  (model_id + image belong in the JSON body; the old query-param form gets
  HTTP 422, which previously surfaced as silent zero-detection results)
- Add opt-in client-side downscaling (roboflow.resize_max_px /
  coralapi.resize_max_px): the server resizes to model input anyway, so
  smaller uploads cut per-frame latency 3-4x at identical accuracy;
  predictions are rescaled to original frame coordinates
- Remote detectors now raise DetectorUnavailableError on transport failures
  instead of returning empty detections; new FallbackDetector optionally
  falls back to local CPU inference (detection_fallback: local) - immediate
  switch when the backend never succeeded this job, after 3 consecutive
  failures otherwise, sticky per job, local model loaded lazily
- ProcessingResult now records detection_backend / detection_model /
  detection_fallback (what actually ran), surfaced in API responses and as
  backend/model/fallback tags on InfluxDB metrics for per-model dashboards
  and fallback-rate alerting
- New coralapi backend (https://github.com/nathan-v/coralapi) for Coral
  Edge TPU inference: multipart /v1/vision/detect, normalized-box scaling,
  label mapping with COCO-90 index fallback for label-less models
- Docs: backend comparison/config for coralapi, detection_fallback setting;
  gitignore local benchmark artifacts
@nathan-v
nathan-v enabled auto-merge (squash) July 17, 2026 03:16
@nathan-v nathan-v self-assigned this Jul 17, 2026
@nathan-v
nathan-v merged commit e831fb8 into main Jul 17, 2026
3 checks passed
@nathan-v
nathan-v deleted the feat/detection-backends branch July 17, 2026 03:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant