[WIP] Scheduler experimentation: baseline scheduler on main by yannicks1 · Pull Request #907 · torch-spyre/sendnn-inference

yannicks1 · 2026-04-09T14:31:25Z

Description

Do not merge, ongoing experiment.

Related Issues

#746

Test Plan

Checklist

I have read the contributing guidelines
My code follows the project's code style (run bash format.sh)
I have added tests for my changes (if applicable)
I have updated the documentation (if applicable)
My commits include a Signed-off-by: line (DCO compliance)

Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>

github-actions · 2026-04-09T14:32:42Z

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, run ./format.sh.
Now you are good to go 🚀.

We also recommend installing prek and configuring it to check your code before every local commit.

yannicks1 · 2026-04-09T14:32:56Z

bot:bench
MAX_CONCURRENT=8

waleedqk · 2026-04-09T16:04:42Z

bot:bench
NUM_PROMPTS=3
VLLM_DT_CHUNK_LEN=512

yannicks1 · 2026-04-10T09:35:12Z

bot:bench
MAX_CONCURRENT=4
VLLM_DT_CHUNK_LEN=512

yannicks1 · 2026-04-10T09:35:20Z

bot:bench
MAX_CONCURRENT=8
VLLM_DT_CHUNK_LEN=512

yannicks1 · 2026-04-10T09:35:26Z

bot:bench
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4
VLLM_DT_CHUNK_LEN=512

yannicks1 · 2026-04-10T09:35:33Z

bot:bench
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8
VLLM_DT_CHUNK_LEN=512

waleedqk · 2026-04-10T13:47:32Z

bot:bench
NUM_PROMPTS=3
MAX_CONCURRENT=4

waleedqk · 2026-04-10T14:09:47Z

bot:bench
NUM_PROMPTS=3
MAX_CONCURRENT=4
VLLM_DT_CHUNK_LEN=512

waleedqk · 2026-04-10T14:12:52Z

bot:bench
NUM_PROMPTS=3
MAX_CONCURRENT=4
MAX_NUM_BATCHED_TOKENS=512

waleedqk · 2026-04-14T14:02:51Z

bot:bench NUM_PROMPTS=3 MAX_CONCURRENT=4

waleedqk · 2026-04-14T14:25:54Z

bot:bench
NUM_PROMPTS=3
MAX_CONCURRENT=4
MAX_NUM_BATCHED_TOKENS=512

waleedqk · 2026-04-14T14:29:59Z

bot:bench
NUM_PROMPTS=3
MAX_CONCURRENT=4
MAX_NUM_BATCHED_TOKENS=512

sducouedic · 2026-04-14T14:39:21Z

bot:bench
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8
NUM_PROMPT=3

waleedqk · 2026-04-14T15:31:15Z

bot:bench NUM_PROMPTS=3 MAX_CONCURRENT=4 MAX_NUM_BATCHED_TOKENS=1024

yannicks1 · 2026-04-15T09:47:39Z

bot:bench
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4
NUM_PROMPTS=4545

waleedqk · 2026-04-15T14:07:43Z

bot:bench
NUM_PROMPTS=3
TIMELINE_ITL_THRESHOLDS=100,1000

waleedqk · 2026-04-15T14:17:13Z

bot:bench
NUM_PROMPTS=10
TIMELINE_ITL_THRESHOLDS="70 400"

waleedqk · 2026-04-15T14:42:01Z

bot:bench
NUM_PROMPTS=10
TIMELINE_ITL_THRESHOLDS=70,400

waleedqk · 2026-04-15T15:01:21Z

bot:bench
NUM_PROMPTS=10

waleedqk · 2026-04-15T15:17:34Z

bot:bench
NUM_PROMPTS=10

waleedqk · 2026-04-15T15:48:56Z

bot:bench
NUM_PROMPTS=10

yannicks1 · 2026-04-15T22:29:06Z

bot:bench
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4
NUM_PROMPTS=600

yannicks1 · 2026-04-15T22:38:02Z

bot:bench
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4
NUM_PROMPTS=600

waleedqk · 2026-04-16T13:19:34Z

bot:bench
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4
NUM_PROMPTS=600

waleedqk · 2026-04-16T16:18:36Z

bot:bench
NUM_PROMPTS=5000
MAX_RUN_TIME=36000

waleedqk · 2026-04-16T16:23:36Z

bot:bench
NUM_PROMPTS=5000
MAX_RUN_TIME=36000

yannicks1 · 2026-04-19T14:09:08Z

bot:bench
NUM_PROMPTS=4545
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4

yannicks1 · 2026-04-21T11:40:54Z

bot:bench
NUM_PROMPTS=4545
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8

yannicks1 · 2026-04-21T15:45:20Z

bot:bench
NUM_PROMPTS=4545
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8

yannicks1 · 2026-04-30T22:19:28Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4

yannicks1 · 2026-05-01T18:16:47Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4

yannicks1 · 2026-05-04T08:06:37Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4

yannicks1 · 2026-05-04T08:10:43Z

bot:test

yannicks1 · 2026-05-04T08:16:27Z

bot:test

yannicks1 · 2026-05-04T08:22:03Z

bot:test

yannicks1 · 2026-05-08T09:02:02Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4

yannicks1 · 2026-05-08T11:08:20Z

bot:bench
NUM_PROMPTS=100
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4

yannicks1 · 2026-05-08T13:02:46Z

bot:bench
NUM_PROMPTS=100
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8

yannicks1 · 2026-05-08T13:21:54Z

bot:bench
NUM_PROMPTS=10
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8

yannicks1 · 2026-05-08T13:30:00Z

bot:bench
NUM_PROMPTS=10
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8

yannicks1 · 2026-05-08T15:24:14Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8

yannicks1 · 2026-05-20T13:25:00Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8
ENABLE_PC=0

yannicks1 · 2026-05-20T13:39:21Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8
ENABLE_PC=0

yannicks1 · 2026-05-20T13:40:23Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8
ENABLE_PC=0

yannicks1 · 2026-05-20T20:28:35Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8
ENABLE_PC=0

yannicks1 · 2026-05-20T21:30:25Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8
ENABLE_PC=0

sducouedic · 2026-05-22T20:25:17Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=32
ENABLE_PC=0

sducouedic · 2026-05-22T22:00:27Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4
AGENT=IMS

sducouedic · 2026-05-27T09:38:40Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=32
ENABLE_PC=0

sducouedic · 2026-05-27T12:55:00Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4

yannicks1 · 2026-05-27T13:48:20Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8
ENABLE_PC=0

yannicks1 · 2026-05-27T13:48:47Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4
ENABLE_PC=0

sducouedic · 2026-06-09T09:24:25Z

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=4

Just a dummy commit

8290aa3

Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>

yannicks1 mentioned this pull request Apr 9, 2026

Investigate chunked prefill scheduler optimizations #746

Closed

Merge branch 'torch-spyre:main' into main_scheduler_test

f337f5f

Merge branch 'main' into main_scheduler_test

9ef518a

Merge branch 'torch-spyre:main' into main_scheduler_test

5fd7079

Merge branch 'torch-spyre:main' into main_scheduler_test

fe1eae5

Uh oh!

Conversation

yannicks1 commented Apr 9, 2026

Description

Related Issues

Test Plan

Checklist

Uh oh!

github-actions Bot commented Apr 9, 2026

Uh oh!

yannicks1 commented Apr 9, 2026

Uh oh!

waleedqk commented Apr 9, 2026

Uh oh!

yannicks1 commented Apr 10, 2026

Uh oh!

yannicks1 commented Apr 10, 2026

Uh oh!

yannicks1 commented Apr 10, 2026

Uh oh!

yannicks1 commented Apr 10, 2026

Uh oh!

waleedqk commented Apr 10, 2026

Uh oh!

waleedqk commented Apr 10, 2026

Uh oh!

waleedqk commented Apr 10, 2026

Uh oh!

waleedqk commented Apr 14, 2026

Uh oh!

waleedqk commented Apr 14, 2026

Uh oh!

waleedqk commented Apr 14, 2026

Uh oh!

sducouedic commented Apr 14, 2026

Uh oh!

waleedqk commented Apr 14, 2026

Uh oh!

yannicks1 commented Apr 15, 2026

Uh oh!

waleedqk commented Apr 15, 2026

Uh oh!

waleedqk commented Apr 15, 2026

Uh oh!

waleedqk commented Apr 15, 2026

Uh oh!

waleedqk commented Apr 15, 2026

Uh oh!

waleedqk commented Apr 15, 2026

Uh oh!

waleedqk commented Apr 15, 2026

Uh oh!

yannicks1 commented Apr 15, 2026

Uh oh!

yannicks1 commented Apr 15, 2026

Uh oh!

waleedqk commented Apr 16, 2026

Uh oh!

waleedqk commented Apr 16, 2026

Uh oh!

waleedqk commented Apr 16, 2026

Uh oh!

yannicks1 commented Apr 19, 2026

Uh oh!

yannicks1 commented Apr 21, 2026

Uh oh!

yannicks1 commented Apr 21, 2026

Uh oh!

yannicks1 commented Apr 30, 2026

Uh oh!

yannicks1 commented May 1, 2026

Uh oh!

yannicks1 commented May 4, 2026

Uh oh!

yannicks1 commented May 4, 2026

Uh oh!

yannicks1 commented May 4, 2026

Uh oh!

yannicks1 commented May 4, 2026

Uh oh!