Problem
nanotune chat waits for the entire response before rendering anything. chatCompletion() in src/lib/llama-cpp.ts does a plain fetch and awaits the full JSON body.
On a long reply the UI sits on a "Generating..." spinner for many seconds with no indication of progress. Small local models are exactly the case where you want to see tokens land — it's how you tell the model is working versus stuck.
llama-server already supports SSE streaming on /v1/chat/completions with "stream": true.
Proposal
- Add a streaming variant alongside
chatCompletion (keep the non-streaming one — benchmark doesn't need streaming and relies on the final timings block)
- Render tokens into the chat transcript as they arrive
- Keep the existing per-turn stats line; SSE responses still carry timings in the final chunk
Acceptance criteria
Problem
nanotune chatwaits for the entire response before rendering anything.chatCompletion()insrc/lib/llama-cpp.tsdoes a plainfetchand awaits the full JSON body.On a long reply the UI sits on a "Generating..." spinner for many seconds with no indication of progress. Small local models are exactly the case where you want to see tokens land — it's how you tell the model is working versus stuck.
llama-serveralready supports SSE streaming on/v1/chat/completionswith"stream": true.Proposal
chatCompletion(keep the non-streaming one —benchmarkdoesn't need streaming and relies on the finaltimingsblock)Acceptance criteria
nanotune chatas they're generatedbenchmarkbehaviour is unchanged