🐛 Bug Report — workers cannot streamed response to web client

[Whyjsee]

i make a openai streaming service in fetch in workers, but the response in web client will always be not-streaming response , it will always return response after openai’s request totally finished. how can i make it return in stream ?

here’s my test code in workers:

export default {
  async fetch(request: Request, env: any, ctx: ExecutionContext) {
    const openai = new OpenAI({
      apiKey: env.OPENAI_API_KEY,
      baseURL: env.OPENAI_API,
    });

    const stream = await openai.chat.completions.create({
      model: 'gpt-3.5-turbo',
      messages: [{ role: 'user', content: (await request.json()) || '' }],
      stream: true,
    });

    return new Response(stream.toReadableStream());
  },
};

and here’s my web client code:

export const fetchTest = async () => {
  const streamApiUrl = '/xxx/query'
  const response = await fetch(streamApiUrl, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      content: 'who are you'
    })
  })

  const bodyReader = response.body.getReader()
  while (true) {
    const { value, done } = await bodyReader.read()    // it won't return like stream , it will return all data once and for all after request finished
    if (done) break
    console.log(value)
  }
}

fetchTest()

here's my chrome devtools screenshot: