Skip to content

serve: a stop sequence split across two tokens is not seen in a stream #68

Description

@headl-fs

Symptom

On POST /v1/chat/completions and POST /v1/completions with "stream": true, a stop
sequence only ends the answer when it happens to fall inside a single token. A sequence that
spans a token boundary is never seen, and the generation runs on to its budget.

When it does match, the token carrying the match has already been sent. The answer therefore
ends one token late: the stop sequence itself, and whatever the model put after it inside that
same token, reach the client. OpenAI cuts exactly at the sequence and never emits it.

Cause

The check looks at each token on its own, and nothing trims what was already emitted:

if any(s in token for s in stop_sequences):
    break

No text is accumulated across tokens, so a sequence can only ever match within one.

Repro

curl -sN localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"<model>","stream":true,
       "messages":[{"role":"user","content":"Repeat exactly, nothing else: alpha beta gamma delta"}],
       "stop":["beta gamma"]}'

The two-word sequence is tokenized apart, so no single chunk contains it and the stream runs to
the end. With a sequence that is one token — "stop":["gamma"] — the stream does end, but the
chunk containing gamma has already been delivered.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingserverServer-related issues and functionality

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions