-
Notifications
You must be signed in to change notification settings - Fork 107
feat: add a production CPU-only model runtime profile #339
Copy link
Copy link
Open
Labels
backendFastAPI, database, storage, and API workFastAPI, database, storage, and API workenhancementNew feature, improvement, or enhancement request.New feature, improvement, or enhancement request.gssoc26Related to GirlScript Summer of Code 2026.Related to GirlScript Summer of Code 2026.infraDocker, compose, deployment, and runtime setupDocker, compose, deployment, and runtime setuplevel:criticalGSSoC difficulty level: critical. Base contributor points: 80.GSSoC difficulty level: critical. Base contributor points: 80.local-firstPrivacy-preserving local runtime and offline behaviorPrivacy-preserving local runtime and offline behaviormlModel inference, embeddings, OCR, captions, and search relevanceModel inference, embeddings, OCR, captions, and search relevanceperformanceSpeed, startup, memory, image size, and runtime efficiencySpeed, startup, memory, image size, and runtime efficiencypriority: highImportant issue that should be handled soonImportant issue that should be handled soonstatus: availableOpen for contributors to pick upOpen for contributors to pick up
Description
Metadata
Metadata
Assignees
Labels
backendFastAPI, database, storage, and API workFastAPI, database, storage, and API workenhancementNew feature, improvement, or enhancement request.New feature, improvement, or enhancement request.gssoc26Related to GirlScript Summer of Code 2026.Related to GirlScript Summer of Code 2026.infraDocker, compose, deployment, and runtime setupDocker, compose, deployment, and runtime setuplevel:criticalGSSoC difficulty level: critical. Base contributor points: 80.GSSoC difficulty level: critical. Base contributor points: 80.local-firstPrivacy-preserving local runtime and offline behaviorPrivacy-preserving local runtime and offline behaviormlModel inference, embeddings, OCR, captions, and search relevanceModel inference, embeddings, OCR, captions, and search relevanceperformanceSpeed, startup, memory, image size, and runtime efficiencySpeed, startup, memory, image size, and runtime efficiencypriority: highImportant issue that should be handled soonImportant issue that should be handled soonstatus: availableOpen for contributors to pick upOpen for contributors to pick up
Summary
Add a real CPU-only runtime between mock mode and the CUDA full stack. It should run local inference on ordinary laptops without installing CUDA wheels or requiring an NVIDIA device.
Why this is required
The light profile is useful for development but produces fake ML results; the full profile installs multiple large runtimes and requires GPU access. Users need a production-capable CPU profile with explicit quality and latency expectations.
Where to work
backend/pyproject.tomloptional dependency groupsbackend/Dockerfile.env.examplebackend/src/find_api/core/hardware.pyAcceptance criteria
Dependencies
Coordinate with #45 and the model-specific benchmark issues. Do not change default models without measurable quality comparisons.