Skip to content

fix(ci): bump pinned Spyre RPMs to unbreak fp8 _scaled_mm compile - #818

Closed
sducouedic wants to merge 1 commit into
torch-spyre:mainfrom
sducouedic:fix/bump-spyre-rpms-fp8-scaled-mm
Closed

fix(ci): bump pinned Spyre RPMs to unbreak fp8 _scaled_mm compile#818
sducouedic wants to merge 1 commit into
torch-spyre:mainfrom
sducouedic:fix/bump-spyre-rpms-fp8-scaled-mm

Conversation

@sducouedic

Copy link
Copy Markdown
Contributor

Description

tests/test_fp8_linear.py::TestSpyreFp8LinearKernel::test_scaled_mm_apply*[1] fails on main with dxp_standalone exiting 1 on the fused _scaled_mm + quantize_fp8_with_scale kernel. It is not a code regression: the same commit (d97de1f) passed on 09-08 evening and started failing on 09-09 morning, and the CI runner image changed across that boundary. This bumps spyre-rpms.lock to the RPM set installed on my dev pod, which compiles the kernel. ibm-flex and ibm-spyre-comms are the only packages moving commit; the rest are build-number-only bumps of the same commits.

Which package carries the fix is not isolated — every pin here is newer than the current lock, so this shows the failure is gone in the newer set rather than pinpointing the cause.

Related Issues

Relates to the shared failure on #812, #814, #815.

Test Plan

  • pytest tests/test_fp8_linear.py -m "not upstream" with SPYRE_KERNEL_CACHE=1 and a cleared inductor cache, against the pinned torch-spyre rev 7697811 — 17 passed, including the two cases failing in CI.
  • bash format.sh — no changes to this diff.
  • Lock validated as parseable, with every arch override matching its [packages] commit and no commit-count downgrades.

Checklist

  • I have read the contributing guidelines
  • My code follows the project's code style (run bash format.sh)
  • I have added tests for my changes (if applicable)
  • I have updated the documentation (if applicable)
  • My commits include a Signed-off-by: line (DCO compliance)

The fused _scaled_mm fp8 kernel fails to compile under dxp_standalone on the
current pins. The RPM set here compiles it; ibm-flex and ibm-spyre-comms are the
only packages moving commit.

Signed-off-by: Sophie du Couédic <sducouedic@users.noreply.github.com>
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

👋 Hi! Thank you for contributing.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, run ./format.sh.
Now you are good to go 🚀.

We also recommend installing prek and configuring it to check your code before every local commit.

@sducouedic sducouedic closed this Sep 9, 2026
HarikrishnanBalagopal added a commit to HarikrishnanBalagopal/spyre-inference that referenced this pull request Sep 9, 2026
HarikrishnanBalagopal added a commit to HarikrishnanBalagopal/spyre-inference that referenced this pull request Sep 9, 2026
…ply torch-spyre#818)

Signed-off-by: Harikrishnan Balagopal <harikrishmenon@gmail.com>
@sducouedic sducouedic reopened this Sep 9, 2026
@sducouedic sducouedic closed this Sep 9, 2026
joerunde added a commit that referenced this pull request Sep 10, 2026
## Description

Bumps `torch-spyre` from `7697811` to `c3d949a` — all 13 upstream
commits since the previous pin. The tip of `main` compiled cleanly
against the currently-installed RPMs (no bisect needed). Also refreshes
the full `uv.lock` (`uv lock --upgrade`); `vllm` is held at its pinned
git tag (`v0.28.0`) and intentionally **not** upgraded.

NB: We had a prior bump PR in today (#818) that was attempting to clear
out a regression. That didn't work so we instead disabled tests (#821),
but we still do need to keep upgrading

### Notable upstream changes in this range

- **Enable sliding window attention** (#3405) — new attention path.
- **Lx dump restore** (#3879) — dump/restore support.
- **Demote non-critical warnings to info/debug level** (#4374) and
**remove `_get_env_bool` / legacy `SPYRE_INDUCTOR_LOG` helpers** (#4241)
— logging cleanup and removal of legacy env-var helpers.
- **fix attention view error** (#4377) and **codegen: point
`maskingConstId_` at samv-maskvalue, not constant 0** (#4400) —
attention/masking correctness fixes.
- **fix(inductor): preserve offsets in coarse read copies** (#4376) —
inductor lowering fix (tip).

### Transitive dependency refresh

5 transitive packages moved via `uv lock --upgrade`: transformers 5.16.1
→ 5.17.0, openai 3.9.0 → 3.11.0, xgrammar 0.2.3 → 0.2.6, multidict 6.7.1
→ 6.8.0, filelock 3.32.5 → 3.32.6. `vllm` (v0.28.0) and `torch-spyre`
unchanged (pinned).

### Installed `ibm-*` packages on the build host

```
ibm-aiu-toolbox-e2e-2.0.0-0.main.1+28.47d9b91_101.el10.x86_64
ibm-deeptools-2.0.0-0.main.1+2429.561f418_335.el10.x86_64
ibm-deeptools-devel-2.0.0-0.main.1+2429.561f418_335.el10.x86_64
ibm-flex-2.0.0-0.main.1+553.8581a91_384.el10.x86_64
ibm-flex-devel-2.0.0-0.main.1+553.8581a91_384.el10.x86_64
ibm-libaiupti-2.0.0-0.main.1+27.e289f7d_13.el10.x86_64
ibm-senlib-core-2.0.0-0.main.1+268.c58bbc0_251.el10.x86_64
ibm-senlib-dd2-2.0.0-0.main.1+268.c58bbc0_251.el10.x86_64
ibm-senlib-headers-2.0.0-0.main.1+268.c58bbc0_251.el10.x86_64
ibm-spyre-comms-1.0.0-0.main.1+146.a445334_182.el10.x86_64
ibm-spyre-comms-devel-1.0.0-0.main.1+146.a445334_182.el10.x86_64
```

`spyre-rpms.lock` bumped to match: flex `7c58aa7` → `8581a91` and
spyre-comms `d3875d2` → `a445334` (new commits);
deeptools/senlib/aiu-toolbox/libaiupti advanced to newer builds of the
same commit. No downgrades.

## Test Plan

- [x] `uv lock` resolves cleanly to `c3d949a`
- [x] `uv lock --upgrade` refreshes all transitives (vllm held at its
git tag)
- [x] `uv sync --frozen` builds the torch-spyre C++ extension
successfully
- [x] Smoke test (`test_basic_model_load`) passes locally
- [ ] Full CI suite passes (pushed for CI validation)
- [x] `spyre-rpms.lock` updated — no downgrades (commit-count check
passed); resolves + downloads on all arches

**Reviewer note:** when pulling this branch onto an existing checkout,
`rm -rf /tmp/torchinductor_*` before running tests — the cache bakes in
references to internals that were renamed/removed across the bump.

---------

Signed-off-by: Joe Runde <joe@joerun.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant