vMCP optimizer: optional cross-encoder rerank stage for find_tool #6852

Problem

The optimizer's find_tool ranks candidates with FTS5 and, optionally, with embeddings. Those methods are fast, but they score the query and each tool separately. A cross-encoder reads the query and the tool description together, which helps when many tools have similar names and descriptions, for example several GitHub, Grafana or Kubernetes servers behind one vMCP.

#1557 asked for semantic tool filtering and suggested a reranker (mxbai-rerank-v2). It was closed as shipped with FTS5 and embeddings only, so there is no rerank stage today.

Proposal

Add an optional second stage after the existing retrieval:

1. Retrieve the top rerankCandidates hits (default 20) as today.
2. Send the find_tool query and each candidate's name: description to a rerank endpoint.
3. Return the top results in the reranker's order.
4. On a timeout or any error, fall back to the current order.

Proposed config, next to the existing embedding* fields:

optimizer:
  rerankProvider: tei | openai     # tei: POST /rerank {query, texts}; openai: Cohere/Jina-style POST /v1/rerank {model, query, documents, top_n}
  rerankService: http://reranker:8080
  rerankModel: bge-reranker-v2-m3  # openai provider only
  rerankCandidates: 20
  rerankServiceTimeout: 5s

• tei fits next to the managed TEI embedding server: TEI serves rerankers such as BAAI/bge-reranker-base on /rerank.
• openai covers llama.cpp llama-server --rerank, vLLM, Infinity, Jina and Cohere.

Why it is cheap

• find_tool calls are infrequent and the candidate set is small: 20 short tool descriptions make a single rerank request. For scale, our llama.cpp bge-reranker-v2-m3 on an iGPU scores 12 passages of ~900 chars in 1.3-3.0 s, and tool descriptions are much shorter.
• It is off by default, and with nothing configured the behaviour is unchanged.

Environment

ToolHive operator 0.54.0, vMCP with the optimizer (embeddingProvider: openai) in front of 16 backends. We already run a bge-reranker-v2-m3 (/v1/rerank) in-cluster.