FastembedTextEmbedder is reinstantiated and warmed up on every search request

[prquinlan]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

BODY:
get_embedder() in lettuce/components/embeddings.py (~lines 150-160):

def get_embedder(self) -> FastembedTextEmbedder:
    query_embedder = FastembedTextEmbedder(model=self._model.info.path, parallel=0)
    query_embedder.warm_up()
    return query_embedder

is called fresh on every request from:

• lettuce/routers/search_routes.py (vector_search, per request)
• lettuce/components/pipeline.py:141 (get_rag_assistant, called per ai_search request)
• lettuce-ui/src/search/__init__.py (per UI search click, and once per row in a batch upload —
  potentially hundreds of times per batch)

warm_up() loads the ONNX embedding model via fastembed/onnxruntime and initializes a new
inference session (native ONNX Runtime session objects + thread pools) each time.

Impact: high per-request latency/CPU from re-initializing the ONNX runtime session on
essentially every search that isn't a pure text search, plus repeated native
allocation/deallocation cycles for model weights and session threads. If these aren't fully
released between requests (common with native library bindings that Python's GC doesn't manage
precisely), memory and OS thread counts can creep upward under sustained load — worth confirming
with a load test (RSS/thread count over time against /search/vector-search/{term}).

Suggested fix direction: construct and warm up the embedder once at startup and reuse it
across requests, same as the LLM reload issue.

Expected Behavior

No response

Steps To Reproduce

No response

Environment

- OS:
- Other environment details:

I'm part of a Project Team

No response

Anything else?

No response

Are you willing to contribute to resolve this issue?

None