Skip to content

server: emit GPU ttft/generation/decode_tps in stream usage - #570

Open
ai-hpc wants to merge 1 commit into
mainfrom
feat/stream-usage-gpu-timings
Open

server: emit GPU ttft/generation/decode_tps in stream usage#570
ai-hpc wants to merge 1 commit into
mainfrom
feat/stream-usage-gpu-timings

server: emit GPU ttft_ms / generation_ms / decode_tps in stream usage

c10c9a9
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar