Describe the bug
test_real_metadata_dispatch_does_not_grow_the_cache failing due to below mentioned error :
`=================================== FAILURES ===================================
_____ TestRecordGraphs.test_real_metadata_dispatch_does_not_grow_the_cache _____
self = <test_spyre_attn_recorder.TestRecordGraphs object at 0x3fe3cc12a80>
impl = <spyre_inference.v1.attention.backends.spyre_attn.SpyreAttentionImpl object at 0x3fd23d72660>
kv_cache = <[TypeError("_patch_tensor_for_spyre..spyre_aware_repr() got an unexpected keyword argument 'tensor_contents'") raised in repr()] SpyrePagedKVCache object at 0x3fd229477c0>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x3fd23d143b0>
def test_real_metadata_dispatch_does_not_grow_the_cache(self, impl, kv_cache, monkeypatch):
"""The acceptance criterion, driven from real builder metadata.
Unlike ``test_dispatch_after_recording_does_not_grow_the_cache``, this
builds metadata for unbucketed kv_lens through
``SpyreAttentionMetadataBuilder`` and dispatches on the block counts
``build()`` actually produced.
"""
from vllm.config import get_current_vllm_config
from tests.attention.test_spyre_attn import _padded_mask_metadata
E ModuleNotFoundError: No module named 'tests.attention'
tests/attention/test_spyre_attn_recorder.py:139: ModuleNotFoundError
---------------------------- Captured stdout setup -----------------------------
INFO 09-08 04:02:36 [model.py:672] Resolved architecture: Qwen3ForCausalLM
WARNING 09-08 04:02:36 [model.py:2299] Casting torch.bfloat16 to torch.float16.
INFO 09-08 04:02:36 [model.py:1965] Using max model len 2048
WARNING 09-08 04:02:36 [platform.py:291] Capping max_num_batched_tokens to 512
INFO 09-08 04:02:36 [kernel.py:308] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
�[91m▄█▀▀█▄�[0m �[38;5;208m█▀▀▀█▄�[0m �[93m█ █�[0m �[92m█▀▀▀█▄�[0m �[94m█▀▀▀▀�[0m �[97;1m█ █▄ █ █▀▀▀▀ █▀▀▀▀ █▀▀▀█▄ █▀▀▀▀ █▄ █ ▄█▀▀█▄ █▀▀▀▀�[0m
�[91m▀▀▄▄▄�[0m �[38;5;208m█▄▄▄█▀�[0m �[93m▀▄ ▄▀�[0m �[92m█▄▄▄█▀�[0m �[94m█▄▄▄�[0m �[97;1m█ █ █ █ █▄▄▄ █▄▄▄ █▄▄▄█▀ █▄▄▄ █ █ █ █ █▄▄▄�[0m
�[91m█�[0m �[38;5;208m█�[0m �[93m▀█▀�[0m �[92m█ ▀█▄�[0m �[94m█�[0m �[97;1m█ █ █ █ █ █ █ ▀█▄ █ █ █ █ █ █�[0m
�[91m▀▄▄▄█▀�[0m �[38;5;208m█�[0m �[93m█�[0m �[92m█ ▀█�[0m �[94m█▄▄▄▄�[0m �[97;1m█ █ ▀█ █ █▄▄▄▄ █ ▀█ █▄▄▄▄ █ ▀█ ▀█▄▄█▀ █▄▄▄▄�[0m
version �[97;1m0.1.dev1�[0m model �[97;1mQwen/Qwen3-0.6B�[0m
INFO 09-08 04:02:36 [threading_config.py:92] Initial threading configuration: OMP_NUM_THREADS=16 OPENBLAS_NUM_THREADS=16 MKL_NUM_THREADS=16 NUMEXPR_NUM_THREADS=16 VECLIB_MAXIMUM_THREADS=16
INFO 09-08 04:02:36 [threading_config.py:119] Detected 16.0 physical CPUs from psutil. Setting each threading configuration to 16 for 1 worker(s) (SPYRE_UPDATE_THREAD_CONFIG enabled).
INFO 09-08 04:02:36 [platform.py:522] Loading worker from: spyre_inference.v1.worker.spyre_worker.TorchSpyreWorker
INFO 09-08 04:02:36 [platform.py:531] Loading scheduler from: vllm.v1.core.sched.scheduler.Scheduler
INFO 09-08 04:02:36 [platform.py:573] Setting num_gpu_blocks_override=2049 (128 seqs × 16 blocks/seq + 1 null block)
------------------------------ Captured log setup ------------------------------
INFO vllm.config.model:model.py:672 Resolved architecture: Qwen3ForCausalLM
WARNING vllm.config.model:model.py:2299 Casting torch.bfloat16 to torch.float16.
INFO vllm.config.model:model.py:1965 Using max model len 2048
WARNING spyre_inference.platform:platform.py:291 Capping max_num_batched_tokens to 512
INFO vllm.config.kernel:kernel.py:308 Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
INFO spyre_inference.threading_config:threading_config.py:92 Initial threading configuration: OMP_NUM_THREADS=16 OPENBLAS_NUM_THREADS=16 MKL_NUM_THREADS=16 NUMEXPR_NUM_THREADS=16 VECLIB_MAXIMUM_THREADS=16
INFO spyre_inference.threading_config:threading_config.py:119 Detected 16.0 physical CPUs from psutil. Setting each threading configuration to 16 for 1 worker(s) (SPYRE_UPDATE_THREAD_CONFIG enabled).
INFO spyre_inference.platform:platform.py:522 Loading worker from: spyre_inference.v1.worker.spyre_worker.TorchSpyreWorker
INFO spyre_inference.platform:platform.py:531 Loading scheduler from: vllm.v1.core.sched.scheduler.Scheduler
INFO spyre_inference.platform:platform.py:573 Setting num_gpu_blocks_override=2049 (128 seqs × 16 blocks/seq + 1 null block)`
How to reproduce
pytest -v -m "not distributed"
pytest -v -m "upstream"
Additional context
env details :
`Package Version Editable project location
absl-py 2.5.0
accelerate 1.14.0
agent-detector 2.0.0
aiohappyeyeballs 2.7.1
aiohttp 3.14.3
aiosignal 1.4.0
annotated-doc 0.0.5
annotated-types 0.8.0
anthropic 1.3.0
anyio 4.15.0
apache-tvm-ffi 0.1.13.post3
ast-serialize 0.9.0
astor 0.8.1
attrs 26.1.0
backports-zstd 1.7.0
black 26.5.1
blake3 1.0.9
build 1.5.0
buildkite-test-collector 0.1.9
cachetools 7.1.8
cbor2 6.1.4
certifi 2026.7.22
cffi 2.1.1
cfgv 3.5.0
charset-normalizer 3.5.1
click 8.5.0
clickhouse-connect 1.8.0
clickhouse-driver 0.2.11
cloudpickle 3.1.2
cmake 3.31.10
compressed-tensors 0.17.0
coverage 7.16.0
cryptography 50.0.1
cython 3.3.0
datasets 3.6.0
depyf 0.20.0
detect-installer 0.2.1
dill 0.3.8
distlib 0.4.3
dnspython 2.8.0
docstring-parser 0.18.0
einops 0.8.2
email-validator 2.3.0
execnet 2.1.2
expecttest 0.3.0
fastapi 0.136.3
fastapi-cli 0.0.32
fastapi-cloud-cli 0.25.0
fastar 0.12.0
filelock 3.32.5
frozenlist 1.8.0
fsspec 2025.3.0
googleapis-common-protos 1.75.3
grpcio 1.83.0
h11 0.16.0
hf-adapters-spyre 0.1.0 /home/senuser/hf-adapters
httpcore 1.0.9
httpcore2 2.12.0
httptools 0.8.0
httpx 0.28.1
httpx2 2.12.0
huggingface-hub 1.30.0
identify 2.6.19
idna 3.19
ijson 3.5.1
iniconfig 2.3.0
interegular 0.3.3
jinja2 3.1.6
jiter 0.16.0
jmespath 1.1.0
joblib 1.6.0
jsonschema 4.26.0
jsonschema-specifications 2025.9.1
lark 1.2.2
librt 0.15.0
llguidance 1.7.6
lm-format-enforcer 0.11.3
loguru 0.7.3
lxml 6.1.3
lz4 4.4.5
markdown-it-py 4.2.0
markupsafe 3.0.3
maturin 1.15.0
mcp 2.1.1
mcp-types 2.1.1
mdurl 0.1.2
meson 1.12.0
meson-python 0.21.0
mistral-common 1.11.7
model-hosting-container-standards 0.1.16
mpmath 1.3.0
msgspec 0.21.1
multidict 6.7.1
multiprocess 0.70.16
mypy 2.3.1
mypy-extensions 1.1.0
mypy-protobuf 5.1.0
narwhals 2.25.0
networkx 3.6.1
ninja 1.13.2
nodeenv 1.10.0
numpy 2.3.5
openai 3.8.0
openai-harmony 0.0.8
opencv-python-headless 5.0.0.93
opentelemetry-api 1.44.0
opentelemetry-exporter-otlp 1.44.0
opentelemetry-exporter-otlp-proto-common 1.44.0
opentelemetry-exporter-otlp-proto-grpc 1.44.0
opentelemetry-exporter-otlp-proto-http 1.44.0
opentelemetry-proto 1.44.0
opentelemetry-sdk 1.44.0
opentelemetry-semantic-conventions 0.65b0
opentelemetry-semantic-conventions-ai 0.5.1
ortools 9.11.9999
outlines-core 0.2.14
packaging 26.3
pandas 3.0.5
partial-json-parser 0.2.1.1.post7
pathspec 1.1.1
pillow 12.3.0
pip 26.2.1
platformdirs 4.11.7
pluggy 1.6.0
pre-commit 4.6.2
prometheus-client 0.26.0
prometheus-fastapi-instrumentator 8.1.0
propcache 0.5.2
protobuf 7.36.1
psutil 7.2.2
py-cpuinfo 9.0.0
pyarrow 25.0.0
pybase64 1.5.0
pybind11 3.1.0
pycountry 26.2.16
pycparser 3.0
pydantic 2.13.5
pydantic-core 2.46.5
pydantic-extra-types 2.11.1
pydantic-settings 2.15.0
pygments 2.21.0
pyjwt 2.13.0
pyproject-hooks 1.2.0
pyproject-metadata 0.12.1
pytest 9.1.1
pytest-asyncio 1.4.0
pytest-cov 7.1.0
pytest-forked 1.7.5
pytest-rerunfailures 16.6.1
pytest-shard 0.1.2
pytest-timeout 2.4.0
pytest-xdist 3.8.0
python-dateutil 2.9.0.post0
python-discovery 1.6.0
python-dotenv 1.2.3
python-json-logger 4.2.0
python-multipart 0.0.32
pytokens 0.4.1
pytz 2026.3.post1
pyyaml 6.0.3
pyzmq 27.1.0
referencing 0.37.0
regex 2026.9.3
requests 2.34.2
rich 15.0.0
rich-toolkit 0.20.4
rignore 0.8.1
rpds-py 2026.6.3
ruff 0.16.6
safetensors 0.8.0
scikit-build-core 1.0.3
scikit-learn 1.9.0
scipy 1.18.1
semantic-version 2.10.0
sentence-transformers 6.0.1
sentencepiece 0.2.2
sentry-sdk 2.68.1
setproctitle 1.3.7
setuptools 84.0.0
setuptools-rust 1.13.0
setuptools-scm 10.2.3
shellingham 1.5.4
six 1.17.0
sniffio 1.3.1
spyre-inference 0.1.dev1 /home/senuser/spyre-inference
spyre-testing-plugin 0.1.dev1 /home/senuser/spyre-inference/tests/plugin
sse-starlette 3.4.10
starlette 1.6.0
supervisor 4.3.0
sympy 1.14.0
tblib 3.2.2
threadpoolctl 3.6.0
tiktoken 0.14.0
tokenizers 0.23.2
torch 2.13.0+cpu
torch-spyre 0.0.1
torchvision 0.28.0
tqdm 4.70.0
transformers 5.16.1
truststore 0.10.4
typer 0.27.2
types-protobuf 7.35.1.20260906
types-pyyaml 6.0.12.20260724
typing-extensions 4.16.0
typing-inspection 0.4.4
tzlocal 5.4.4
urllib3 2.7.0
uv 0.11.33
uvicorn 0.52.0
uvloop 0.22.1
vcs-versioning 2.3.4
virtualenv 21.7.8
vllm 0.28.0+empty
watchfiles 1.2.0
websockets 17.1
wheel 0.47.0
xgrammar 0.2.3
xxhash 4.0.1
yarl 1.24.5`
Checklist
Describe the bug
test_real_metadata_dispatch_does_not_grow_the_cache failing due to below mentioned error :
`=================================== FAILURES ===================================
_____ TestRecordGraphs.test_real_metadata_dispatch_does_not_grow_the_cache _____
self = <test_spyre_attn_recorder.TestRecordGraphs object at 0x3fe3cc12a80>
impl = <spyre_inference.v1.attention.backends.spyre_attn.SpyreAttentionImpl object at 0x3fd23d72660>
kv_cache = <[TypeError("_patch_tensor_for_spyre..spyre_aware_repr() got an unexpected keyword argument 'tensor_contents'") raised in repr()] SpyrePagedKVCache object at 0x3fd229477c0>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x3fd23d143b0>
E ModuleNotFoundError: No module named 'tests.attention'
tests/attention/test_spyre_attn_recorder.py:139: ModuleNotFoundError
---------------------------- Captured stdout setup -----------------------------
INFO 09-08 04:02:36 [model.py:672] Resolved architecture: Qwen3ForCausalLM
WARNING 09-08 04:02:36 [model.py:2299] Casting torch.bfloat16 to torch.float16.
INFO 09-08 04:02:36 [model.py:1965] Using max model len 2048
WARNING 09-08 04:02:36 [platform.py:291] Capping max_num_batched_tokens to 512
INFO 09-08 04:02:36 [kernel.py:308] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
INFO 09-08 04:02:36 [threading_config.py:92] Initial threading configuration: OMP_NUM_THREADS=16 OPENBLAS_NUM_THREADS=16 MKL_NUM_THREADS=16 NUMEXPR_NUM_THREADS=16 VECLIB_MAXIMUM_THREADS=16
INFO 09-08 04:02:36 [threading_config.py:119] Detected 16.0 physical CPUs from psutil. Setting each threading configuration to 16 for 1 worker(s) (SPYRE_UPDATE_THREAD_CONFIG enabled).
INFO 09-08 04:02:36 [platform.py:522] Loading worker from: spyre_inference.v1.worker.spyre_worker.TorchSpyreWorker
INFO 09-08 04:02:36 [platform.py:531] Loading scheduler from: vllm.v1.core.sched.scheduler.Scheduler
INFO 09-08 04:02:36 [platform.py:573] Setting num_gpu_blocks_override=2049 (128 seqs × 16 blocks/seq + 1 null block)
------------------------------ Captured log setup ------------------------------
INFO vllm.config.model:model.py:672 Resolved architecture: Qwen3ForCausalLM
WARNING vllm.config.model:model.py:2299 Casting torch.bfloat16 to torch.float16.
INFO vllm.config.model:model.py:1965 Using max model len 2048
WARNING spyre_inference.platform:platform.py:291 Capping max_num_batched_tokens to 512
INFO vllm.config.kernel:kernel.py:308 Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
INFO spyre_inference.threading_config:threading_config.py:92 Initial threading configuration: OMP_NUM_THREADS=16 OPENBLAS_NUM_THREADS=16 MKL_NUM_THREADS=16 NUMEXPR_NUM_THREADS=16 VECLIB_MAXIMUM_THREADS=16
INFO spyre_inference.threading_config:threading_config.py:119 Detected 16.0 physical CPUs from psutil. Setting each threading configuration to 16 for 1 worker(s) (SPYRE_UPDATE_THREAD_CONFIG enabled).
INFO spyre_inference.platform:platform.py:522 Loading worker from: spyre_inference.v1.worker.spyre_worker.TorchSpyreWorker
INFO spyre_inference.platform:platform.py:531 Loading scheduler from: vllm.v1.core.sched.scheduler.Scheduler
INFO spyre_inference.platform:platform.py:573 Setting num_gpu_blocks_override=2049 (128 seqs × 16 blocks/seq + 1 null block)`
How to reproduce
pytest -v -m "not distributed"
pytest -v -m "upstream"
Additional context
env details :
`Package Version Editable project location
absl-py 2.5.0
accelerate 1.14.0
agent-detector 2.0.0
aiohappyeyeballs 2.7.1
aiohttp 3.14.3
aiosignal 1.4.0
annotated-doc 0.0.5
annotated-types 0.8.0
anthropic 1.3.0
anyio 4.15.0
apache-tvm-ffi 0.1.13.post3
ast-serialize 0.9.0
astor 0.8.1
attrs 26.1.0
backports-zstd 1.7.0
black 26.5.1
blake3 1.0.9
build 1.5.0
buildkite-test-collector 0.1.9
cachetools 7.1.8
cbor2 6.1.4
certifi 2026.7.22
cffi 2.1.1
cfgv 3.5.0
charset-normalizer 3.5.1
click 8.5.0
clickhouse-connect 1.8.0
clickhouse-driver 0.2.11
cloudpickle 3.1.2
cmake 3.31.10
compressed-tensors 0.17.0
coverage 7.16.0
cryptography 50.0.1
cython 3.3.0
datasets 3.6.0
depyf 0.20.0
detect-installer 0.2.1
dill 0.3.8
distlib 0.4.3
dnspython 2.8.0
docstring-parser 0.18.0
einops 0.8.2
email-validator 2.3.0
execnet 2.1.2
expecttest 0.3.0
fastapi 0.136.3
fastapi-cli 0.0.32
fastapi-cloud-cli 0.25.0
fastar 0.12.0
filelock 3.32.5
frozenlist 1.8.0
fsspec 2025.3.0
googleapis-common-protos 1.75.3
grpcio 1.83.0
h11 0.16.0
hf-adapters-spyre 0.1.0 /home/senuser/hf-adapters
httpcore 1.0.9
httpcore2 2.12.0
httptools 0.8.0
httpx 0.28.1
httpx2 2.12.0
huggingface-hub 1.30.0
identify 2.6.19
idna 3.19
ijson 3.5.1
iniconfig 2.3.0
interegular 0.3.3
jinja2 3.1.6
jiter 0.16.0
jmespath 1.1.0
joblib 1.6.0
jsonschema 4.26.0
jsonschema-specifications 2025.9.1
lark 1.2.2
librt 0.15.0
llguidance 1.7.6
lm-format-enforcer 0.11.3
loguru 0.7.3
lxml 6.1.3
lz4 4.4.5
markdown-it-py 4.2.0
markupsafe 3.0.3
maturin 1.15.0
mcp 2.1.1
mcp-types 2.1.1
mdurl 0.1.2
meson 1.12.0
meson-python 0.21.0
mistral-common 1.11.7
model-hosting-container-standards 0.1.16
mpmath 1.3.0
msgspec 0.21.1
multidict 6.7.1
multiprocess 0.70.16
mypy 2.3.1
mypy-extensions 1.1.0
mypy-protobuf 5.1.0
narwhals 2.25.0
networkx 3.6.1
ninja 1.13.2
nodeenv 1.10.0
numpy 2.3.5
openai 3.8.0
openai-harmony 0.0.8
opencv-python-headless 5.0.0.93
opentelemetry-api 1.44.0
opentelemetry-exporter-otlp 1.44.0
opentelemetry-exporter-otlp-proto-common 1.44.0
opentelemetry-exporter-otlp-proto-grpc 1.44.0
opentelemetry-exporter-otlp-proto-http 1.44.0
opentelemetry-proto 1.44.0
opentelemetry-sdk 1.44.0
opentelemetry-semantic-conventions 0.65b0
opentelemetry-semantic-conventions-ai 0.5.1
ortools 9.11.9999
outlines-core 0.2.14
packaging 26.3
pandas 3.0.5
partial-json-parser 0.2.1.1.post7
pathspec 1.1.1
pillow 12.3.0
pip 26.2.1
platformdirs 4.11.7
pluggy 1.6.0
pre-commit 4.6.2
prometheus-client 0.26.0
prometheus-fastapi-instrumentator 8.1.0
propcache 0.5.2
protobuf 7.36.1
psutil 7.2.2
py-cpuinfo 9.0.0
pyarrow 25.0.0
pybase64 1.5.0
pybind11 3.1.0
pycountry 26.2.16
pycparser 3.0
pydantic 2.13.5
pydantic-core 2.46.5
pydantic-extra-types 2.11.1
pydantic-settings 2.15.0
pygments 2.21.0
pyjwt 2.13.0
pyproject-hooks 1.2.0
pyproject-metadata 0.12.1
pytest 9.1.1
pytest-asyncio 1.4.0
pytest-cov 7.1.0
pytest-forked 1.7.5
pytest-rerunfailures 16.6.1
pytest-shard 0.1.2
pytest-timeout 2.4.0
pytest-xdist 3.8.0
python-dateutil 2.9.0.post0
python-discovery 1.6.0
python-dotenv 1.2.3
python-json-logger 4.2.0
python-multipart 0.0.32
pytokens 0.4.1
pytz 2026.3.post1
pyyaml 6.0.3
pyzmq 27.1.0
referencing 0.37.0
regex 2026.9.3
requests 2.34.2
rich 15.0.0
rich-toolkit 0.20.4
rignore 0.8.1
rpds-py 2026.6.3
ruff 0.16.6
safetensors 0.8.0
scikit-build-core 1.0.3
scikit-learn 1.9.0
scipy 1.18.1
semantic-version 2.10.0
sentence-transformers 6.0.1
sentencepiece 0.2.2
sentry-sdk 2.68.1
setproctitle 1.3.7
setuptools 84.0.0
setuptools-rust 1.13.0
setuptools-scm 10.2.3
shellingham 1.5.4
six 1.17.0
sniffio 1.3.1
spyre-inference 0.1.dev1 /home/senuser/spyre-inference
spyre-testing-plugin 0.1.dev1 /home/senuser/spyre-inference/tests/plugin
sse-starlette 3.4.10
starlette 1.6.0
supervisor 4.3.0
sympy 1.14.0
tblib 3.2.2
threadpoolctl 3.6.0
tiktoken 0.14.0
tokenizers 0.23.2
torch 2.13.0+cpu
torch-spyre 0.0.1
torchvision 0.28.0
tqdm 4.70.0
transformers 5.16.1
truststore 0.10.4
typer 0.27.2
types-protobuf 7.35.1.20260906
types-pyyaml 6.0.12.20260724
typing-extensions 4.16.0
typing-inspection 0.4.4
tzlocal 5.4.4
urllib3 2.7.0
uv 0.11.33
uvicorn 0.52.0
uvloop 0.22.1
vcs-versioning 2.3.4
virtualenv 21.7.8
vllm 0.28.0+empty
watchfiles 1.2.0
websockets 17.1
wheel 0.47.0
xgrammar 0.2.3
xxhash 4.0.1
yarl 1.24.5`
Checklist