Support Rerank API

[zzzhouuu]

The rerank endpoints support popular re-rank models such as BAAI/bge-reranker-base and other models supporting the score task. Additionally, /v1/rerank endpoints are compatible with both Jina AI's re-rank API interface and Cohere's re-rank API interface to ensure compatibility with popular open-source tools.

models:
  bge-reranker-v2-m3:
    type: rerank
    providers:
      - provider: siliconflow
        model: BAAI/bge-reranker-v2-m3
  jina-reranker-v2-base-multilingual:
    type: rerank
    providers:
      - provider: jina
        model: jina-reranker-v2-base-multilingual

Create Rerank:

curl --request POST \
  --url https://api.siliconflow.cn/v1/rerank \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "model": "BAAI/bge-reranker-v2-m3",
  "query": "Apple",
  "documents": [
    "apple",
    "banana",
    "fruit",
    "vegetable"
  ]
}'

{
  "id": "0199b73abc2d7a3997ca40871523ae4c",
  "results": [
    {
      "index": 0,
      "relevance_score": 0.9909787178039551
    },
    {
      "index": 1,
      "relevance_score": 0.0030552432872354984
    },
    {
      "index": 2,
      "relevance_score": 0.0003349370090290904
    },
    {
      "index": 3,
      "relevance_score": 0.00001892922773549799
    }
  ],
  "meta": {
    "billed_units": {
      "input_tokens": 25,
      "output_tokens": 0,
      "search_units": 0,
      "classifications": 0
    },
    "tokens": {
      "input_tokens": 25,
      "output_tokens": 0
    }
  }
}

LMRouter

curl --request POST \
  --url http://127.0.0.1:3000/v1/rerank \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "model": "bge-reranker-v2-m3",
  "query": "Apple",
  "documents": [
    "apple",
    "banana",
    "fruit",
    "vegetable"
  ]
}'

{
  "id": "0199b738ff7279fabd0e5d731f38d7d4",
  "results": [
    {
      "index": 0,
      "relevance_score": 0.9911285638809204
    },
    {
      "index": 1,
      "relevance_score": 0.0033915601670742035
    },
    {
      "index": 2,
      "relevance_score": 0.00034934436553157866
    },
    {
      "index": 3,
      "relevance_score": 0.000019977886040578596
    }
  ],
  "meta": {
    "billed_units": {
      "input_tokens": 25,
      "output_tokens": 0,
      "search_units": 0,
      "classifications": 0
    },
    "tokens": {
      "input_tokens": 25,
      "output_tokens": 0
    }
  }
}

[yvbbrjdr]

Thanks for your contributions! Since openai doesn't support reranking, could you help me move the API to /v1 without /openai? Thanks!

[zzzhouuu]

Okay, it's been handled.

[yvbbrjdr]

@zzzhouuu Thanks for putting this together! From what I see, the rerank endpoint currently accepts both the Jina and Cohere formats and routes them transparently based on the model name, right?

Just to clarify, LMRouter’s philosophy is that each exposed endpoint should follow one consistent format, while handling any necessary conversions internally. That way, users can connect to multiple providers through one SDK without worrying about format differences.

Maybe we can standardize this endpoint to one format (e.g., Jina-style) and handle the other internally? What do you think? Which rerank API format do you think is the most widely used right now? We can make that the one LMRouter exposes by default.

[zzzhouuu]

On the v1 endpoint, Cohere and Jina differ by only one or two fields; since there’s no unified standard, sticking with the current implementation is the best approach—no code changes are needed to integrate via lmroute.