Skip to content

serve: an empty "model" is answered by an arbitrary workspace model #70

Description

@mzau

Symptom

POST /v1/chat/completions with "model": "" returns 200 and generates — with a model the
client never selected. The response echoes the empty string back, so the client cannot tell which
model answered:

{"object":"chat.completion","model":"","choices":[{"index":0,
 "message":{"role":"assistant","content":"Hello! How"},"finish_reason":"length"}]}

Every neighbouring case is handled correctly, which is what makes this one easy to miss:

request response
model absent 400 validation_errorbody.model: Field required
"model": " " 404 model_not_found
"model": "no-such-model" 404 model_not_found
"model": "" 200, arbitrary model, "model": "" echoed back

Cause

Model resolution matches a workspace directory by case-insensitive substring:

for subdir in workspace_home.iterdir():
    if subdir.is_dir() and model_spec.lower() in subdir.name.lower():
        return (str(subdir), None, None)

"" in name is true for every name, so the first entry iterdir() yields wins — in filesystem
order, so which model answers is not even stable across machines. The cache branch of the same
function treats a multi-match as ambiguous and returns the candidate list; the workspace branch
has no ambiguity concept and returns the first hit.

Reachable only when a workspace home is configured. Without one the empty spec falls through to
cache resolution, comes back ambiguous, and the server answers 404 — the correct answer.

The resolver is not server-specific: run, show, pull, rm, health, embed and serve
all call it, so an empty spec takes the same branch there.

Repro

# with a workspace home configured
curl -sS -w '\nhttp=%{http_code}\n' localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"","messages":[{"role":"user","content":"hi"}],"max_tokens":3}'

# -> http=200, "model":"", answered by whichever workspace directory iterdir() returns first

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingserverServer-related issues and functionality

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions