/predict does not work for batches for sequence classification

[vrdn-23]

System Info

It seems that whenever a set of inputs as batch is being passed in to /predict for sequence classification, only the first result is returned.

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

text-embeddings-router --model-id HuggingFaceFW/fineweb-edu-classifier --port 8080

~/dev/open-source-help/text-embeddings-inference main = *4 ?8 > curl -XPOST localhost:8080/predict -H 'Content-Type: application/json' -d '{"inputs": "Butterflies cute"}' | jq                 py 3.11  12:29:07
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    70  100    40  100    30    696    522 --:--:-- --:--:-- --:--:--  1228
[
  {
    "score": 0.50506574,
    "label": "LABEL_0"
  }
]

~/dev/open-source-help/text-embeddings-inference main = *4 ?8 > curl -XPOST localhost:8080/predict -H 'Content-Type: application/json' -d '{"inputs": ["Butterflies cute", "Where can I find the one piece?"]}' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   106  100    39  100    67   1324   2274 --:--:-- --:--:-- --:--:--  3655
[
  {
    "score": 0.5077051,
    "label": "LABEL_0"
  }
]

~/dev/open-source-help/text-embeddings-inference main = *4 ?8 > curl -XPOST localhost:8080/predict -H 'Content-Type: application/json' -d '{"inputs": ["Butterflies cute", "Where can I find the one piece?", "How about now"]}'     
Failed to deserialize the JSON body into the target type: inputs: invalid length 3, expected a string, a pair of strings [string, string] or a batch of mixed strings and pairs [[string], [string, string], ...] at line 1 column 83%   

Expected behavior

TEI should either throw an error /predict endpoint when size is 2 and model does not do pair classification.

[vrdn-23]

cc @alvarobartt @kozistr @Narsil

I can help raise a PR with the solution depending on whether we want to validate against the model or introduce a separate endpoint for pairwise classification