Reducing Docker, model, and installer size without breaking local-first

[Abhash-Chakraborty]

Problem

The current Docker setup is too large for many contributors and impossible for normal users to think of as an installable app. Full ML mode needs GPU-oriented dependencies and large model downloads. Light mode helps contributors, but it is not the final packaging story.

What can be reduced

• Split API dependencies from ML dependencies.
• Use CPU-only or quantized model options where quality is acceptable.
• Download models on first use instead of bundling all weights in the installer.
• Cache models per user and show clear progress/retry UI.
• Avoid MinIO/PostgreSQL/Redis as required user-facing installer dependencies if embedded alternatives are viable.
• Keep Docker for development and server deployments, not necessarily for desktop app runtime.

Open questions

• What is the smallest useful offline mode?
• Should captions/OCR/search be separate optional model packs?
• Can we support a basic local index without every ML feature installed?
• How do we document disk usage honestly?
• Should GPU acceleration be an optional advanced mode?

Suggested outcome

Create an install/runtime matrix:

• Contributor light stack
• Full Docker ML stack
• Desktop app MVP local stack
• Optional advanced full-ML desktop stack
• Optional self-hosted server acceleration

This helps contributors avoid mixing unrelated goals in one PR.