[WIP] Scheduler experimentation: don't shift tkv for new sequence

[yannicks1]

Do not merge, ongoing experiment

[github-actions]

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

[yannicks1]

bot:bench

[yannicks1]

bot triggered on main: #692

[sducouedic]

bot:bench
AGENT=CICS

[sducouedic]

bot:bench
AGENT=CICS

[sducouedic]

bot:bench
AGENT=TLS

[sducouedic]

bot:bench
AGENT=IMS

[sducouedic]

bot:bench
AGENT=DB2

[sducouedic]

bot:bench
AGENT=AIOPS

[sducouedic]

bot:bench

[sducouedic]

bot:bench
AGENT=ALL

[yannicks1]

bot:bench
AGENT=AIOPS

[sducouedic]

bot:bench

[sducouedic]

bot:bench

[waleedqk]

bot:bench
AGENT=AIOPS
NUM_PROMPTS=10

[waleedqk]

bot:bench
AGENT=AIOPS
NUM_PROMPTS=10

[yannicks1]

reopened here: #797