Skip to content

Pin cmake>=4.0 and enable stubgen in CI - #59

Merged
fabianlim merged 2 commits into
torch-spyre:mainfrom
fabianlim:fix-stubgen-name-too-long
Aug 17, 2026
Merged

Pin cmake>=4.0 and enable stubgen in CI#59
fabianlim merged 2 commits into
torch-spyre:mainfrom
fabianlim:fix-stubgen-name-too-long

Conversation

@fabianlim

@fabianlim fabianlim commented Aug 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

MLIR_PYTHON_STUBGEN_ENABLED was never set in CI — pyproject.toml defines it under [tool.scikit-build.cmake.define], which only scikit-build-core reads, and ci.yml invokes bare cmake. So stub generation was only exercised on the wheel path, where it was failing.

Root cause. mlir_generate_type_stubs() names its custom target by joining every DEPENDS_TARGETS entry with ., giving a 283-char target name. CMake 3.x writes that full name into the file-API reply filename (316 chars total), overflowing NAME_MAX (255), so scikit-build-core dies with ENAMETOOLONG reading the reply. CMake 4.0.0 truncates and relies on the trailing hash.

Bisected 3.31.6 → 4.3.1:

CMake reply filename result
3.31.6, 3.31.10 316 chars fails (ENAMETOOLONG)
4.0.0 → 4.3.1 140 chars OK

The boundary is the 3.31→4.0 major bump, so there's no 3.x release to pin to. Fixing it in MLIR would mean forking mlir_generate_type_stubs to work around an already-fixed CMake bug, so a version floor seemed preferable.

Changes. Pin cmake>=4.0 in build-system.requires and in the llvm-build group; enable -DMLIR_PYTHON_STUBGEN_ENABLED=ON in CI.

Why build-system.requires is the right place. That field is resolved fresh into an isolated build env, ignoring uv.lock (which already had cmake 4.3.1 — hence CI's Resolved 2 packages, with cmake 3.31.6 arriving transitively via scikit-build-core). This also fixes the observed failure in torch-spyre/triton#108, whose install-ktdp-mlir-bindings.sh is a plain uv pip install <path> with no cmake provisioning of its own — so constraining this field is sufficient, with no change needed on the triton side.

ninja>=1.11 / sccache>=0.8 are the oldest versions that resolve, not fixes for known bugs — happy to drop them and pin cmake only.

Verified locally (macOS, NAME_MAX=255): both the bare-cmake and uv pip install . paths build with stubgen on, and all .pyi files generate. check-ktir is 26/26 with and without stubgen — this adds no test coverage, it guards stub generation from silently breaking.

🤖 Generated with Claude Code

@fabianlim
fabianlim force-pushed the fix-stubgen-name-too-long branch from d8c9e5b to 71c355d Compare August 14, 2026 23:30
@fabianlim @claude
Pin cmake>=4.0 and enable stubgen in CI
233d851 
MLIR's mlir_generate_type_stubs() names its custom target by joining every
DEPENDS_TARGETS entry with '.', which for our stubgen target yields a
283-char name. CMake 3.x writes that full name into the file-API reply
filename, overflowing NAME_MAX (255), so scikit-build-core dies with
ENAMETOOLONG while reading the reply. CMake 4.0.0 truncates the name and
relies on the trailing hash instead.

Bisected across cmake 3.31.6 -> 4.3.1: 3.31.6 and 3.31.10 emit a 316-char
filename and fail; 4.0.0 onward emit 140 chars and succeed. The boundary is
the 3.31->4.0 major bump, so there is no 3.x release to pin to.

Pin the floor in build-system.requires, which is resolved fresh into an
isolated build env and ignores uv.lock (cmake 3.31.6 was arriving there
transitively via scikit-build-core), and in the llvm-build group. The
ninja/sccache floors are the oldest versions we resolve against, not fixes
for known bugs.

With the pin in place, stubgen can be enabled in CI, so stub generation is
covered by the bare-cmake job instead of only on the wheel path. check-ktir
stays at 26/26; this adds no test coverage, it guards stubgen from silently
breaking.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
@fabianlim
fabianlim force-pushed the fix-stubgen-name-too-long branch from 71c355d to 233d851 Compare August 14, 2026 23:32
fabianlim added a commit to fabianlim/triton that referenced this pull request Aug 14, 2026
@fabianlim @claude
Bump ktir-mlir-frontend to the stubgen cmake>=4.0 fix
9ca2012 
Advances the submodule ed069fb6 -> 233d851, picking up
torch-spyre/ktir-mlir-frontend#59 ("Pin cmake>=4.0 and enable stubgen in CI").

This is what unblocks CI on this PR. `05e7a0f` (already in this branch's range)
added `MLIR_PYTHON_STUBGEN_ENABLED = "ON"` to the submodule's pyproject.toml,
which activated a latent overflow: MLIR's `mlir_generate_type_stubs()` builds a
283-char target name by joining DEPENDS_TARGETS, and CMake 3.x writes that full
name into its file-API reply filename, overflowing NAME_MAX (255). The bindings
install then died with ENAMETOOLONG — errno 63 locally on macOS, errno 36 in CI
on Linux.

torch-spyre#59 pins cmake>=4.0 in build-system.requires; CMake 4.0 truncates the name and
relies on the trailing hash, so stubgen can stay enabled rather than being
worked around here.

NOTE: 233d851 is the head of an unmerged PR branch, not on
ktir-mlir-frontend main. Re-pin to the squashed commit on main once torch-spyre#59 lands,
before merging this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
fabianlim added a commit to fabianlim/triton that referenced this pull request Aug 15, 2026
@fabianlim @claude
Bump ktir-mlir-frontend to the stubgen cmake>=4.0 fix
9ff1f30 
Advances the submodule ed069fb6 -> 233d851, picking up
torch-spyre/ktir-mlir-frontend#59 ("Pin cmake>=4.0 and enable stubgen in CI").

This is what unblocks CI on this PR. `05e7a0f` (already in this branch's range)
added `MLIR_PYTHON_STUBGEN_ENABLED = "ON"` to the submodule's pyproject.toml,
which activated a latent overflow: MLIR's `mlir_generate_type_stubs()` builds a
283-char target name by joining DEPENDS_TARGETS, and CMake 3.x writes that full
name into its file-API reply filename, overflowing NAME_MAX (255). The bindings
install then died with ENAMETOOLONG — errno 63 locally on macOS, errno 36 in CI
on Linux.

torch-spyre#59 pins cmake>=4.0 in build-system.requires; CMake 4.0 truncates the name and
relies on the trailing hash, so stubgen can stay enabled rather than being
worked around here.

NOTE: 233d851 is the head of an unmerged PR branch, not on
ktir-mlir-frontend main. Re-pin to the squashed commit on main once torch-spyre#59 lands,
before merging this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
This was referenced Aug 15, 2026
Merged
Merged
@fabianlim

fabianlim commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

@KFAFSP it seems that upgrading CMake to 4.0 will solve the long filename issue.

@fabianlim @claude
Regenerate uv.lock for the cmake>=4.0 pin
8de0314 
The llvm-build dependency groups gained >= specifiers, but uv.lock still
recorded the unconstrained requirements in
[package.metadata.requires-dev]. Metadata-only: cmake 4.3.1, ninja 1.13.0
and sccache 0.14.0 were already locked and all satisfy the new floors, so
no resolved version changes.

CI did not catch this because it runs plain `uv sync`, which silently
re-resolves and rewrites a drifted lock rather than failing; only
`uv sync --locked` treats drift as an error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
@fabianlim
fabianlim force-pushed the fix-stubgen-name-too-long branch from 2010902 to 8de0314 Compare August 17, 2026 01:03
@KFAFSP

KFAFSP commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

@KFAFSP it seems that upgrading CMake to 4.0 will solve the long filename issue.

Very good! I love it when we don't have to do anything. I'm not sure whether all downstream users can/want to go to 4.x yet though...

@fabianlim

fabianlim commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator Author

@KFAFSP

Very good! I love it when we don't have to do anything. I'm not sure whether all downstream users can/want to go to 4.x yet though...

for users that do the cmake build manually (which is the only way to bypass the stubgen flag) are free to use their system cmake , this requrement is only for users who go through the uv scikit build, and also via the uv machineary, these deps are pulled only in the uv build which is ephemeral into a /tmp/build

@fabianlim
fabianlim enabled auto-merge (squash) August 17, 2026 20:55
@fabianlim
fabianlim merged commit ecfb9ed into torch-spyre:main Aug 17, 2026
4 checks passed
fabianlim added a commit to fabianlim/triton that referenced this pull request Aug 18, 2026
@fabianlim
Bump ktir-mlir-frontend to the merged stubgen cmake>=4.0 fix
eb84e00 
Advances the submodule ed069fb6 -> ecfb9ed, picking up
torch-spyre/ktir-mlir-frontend#59 ("Pin cmake>=4.0 and enable stubgen in CI").

This is what unblocks CI. `05e7a0f` (already in this branch's range) added
`MLIR_PYTHON_STUBGEN_ENABLED = "ON"` to the submodule's pyproject.toml, which
activated a latent overflow: MLIR's `mlir_generate_type_stubs()` builds a
283-char target name by joining DEPENDS_TARGETS, and CMake 3.x writes that full
name into its file-API reply filename, overflowing NAME_MAX (255). The bindings
install then died with ENAMETOOLONG — errno 63 locally on macOS, errno 36 in CI
on Linux.

torch-spyre#59 pins cmake>=4.0 in build-system.requires; CMake 4.0 truncates the name and
relies on the trailing hash, so stubgen stays enabled rather than being worked
around here. Note that pin only applies to an isolated PEP 517 build — a
consumer running cmake directly is unaffected (the project itself declares
cmake_minimum_required(VERSION 3.21)), and the cmake it installs lives only in
the ephemeral build environment.

ecfb9ed is the squashed commit on ktir-mlir-frontend main, so this is no longer
a pin to an unmerged branch head.

Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
fabianlim added a commit to torch-spyre/triton that referenced this pull request Aug 18, 2026
@fabianlim @claude
Bump ktir-mlir-frontend past the ktdp.memory_space rename (#108)
33a694c 
* Bump ktir-mlir-frontend to latest main; adapt to source restructuring

Moves the submodule from 973d67e to 05e7a0f (4 commits), whose tip is
"Source structure refactoring (issue #37) (#46)". That refactor relocates
the KTDP public headers, so every consumer here needs its includes
rewritten:

  Ktdp/KtdpOps.hpp            -> ktir/Dialect/KTDP/KTDP.h
  Ktdp/KtdpDialect.hpp        -> ktir/Dialect/KTDP/KTDPDialect.h
  Ktdp/KtdpAttrs.hpp          -> ktir/Dialect/KTDP/KTDPAttrs.h
  Ktdp/KtdpTypes.hpp          -> ktir/Dialect/KTDP/KTDPTypes.h

No CMake change is needed: add_ktir_library still sets
EXPORT_NAME KtdpDialect and _install_ktir_library still creates the
KTIR::KtdpDialect alias, so existing link lines resolve unchanged. The
LLVM pin (cmake/llvm-hash-spyre.txt, e9846648) is identical on both
sides, so no LLVM rebuild is implied.

The `groups`-folded-into-!ktdp.tile_future syntax change was already
present at the previous pin and the lit expectations here already match
it, so no test updates were required.

Verified: spyre-triton-opt builds clean against the Spyre LLVM, and the
lit suite passes 34/34.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>

* Fix remaining KTDP include in triton_spyre.cc after submodule restructuring

triton_spyre.cc was missed by the include rewrite in 9c3084c: it is a
.cc file, and the sweep that found the other seven consumers only
matched .cpp/.h. It is compiled into libtriton rather than
spyre-triton-opt, so the lit-only verification in that commit did not
cover it — the failure surfaces on `uv pip install -e .` as:

  triton_spyre.cc:10:10: fatal error: 'Ktdp/KtdpDialect.hpp' file not found

Verified: the editable install now builds libtriton clean and
`import triton` resolves to this checkout.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>

* Bump ktir-mlir-frontend to ed069fb6; migrate off ktdp.spyre_memory_space

Moves the submodule 05e7a0f -> ed069fb6, picking up three commits:

  3da565b  [Ktdp] Add KtirCheckLegality pass stub
  a3d6bd1  Remove KtdpMemorySpaceAttr interface, replace
           ktdp.spyre_memory_space with ktdp.memory_space
  ed069fb  Add the `spyreop` dialect (#57)

a3d6bd1 is a breaking change and the reason for this bump. It is not just a
rename — the attribute's enum changed meaning:

  old: #ktdp.spyre_memory_space<HBM>     kind enum {unspecified, LX, HBM}
  new: #ktdp.memory_space<global>        kind enum {global, ct_local}

and the C++ parameter `value` became `kind`, with `core` becoming the optional
`ct_id`. Following upstream's own migration of test/Dialect/KTDP/add.mlir,
HBM maps to `global`.

Changes here:
  - two C++ construction sites (LowerDescriptorMemory, LowerScalarLoad):
    SpyreMemorySpaceAttr/SpyreMemorySpaceKind::HBM -> MemorySpaceAttr/
    MemorySpaceKind::global
  - 18 .mlir test files, 187 occurrences, all the single form
    spyre_memory_space<HBM> -> memory_space<global>

Nothing here referenced the deleted KTDPAttrInterfaces.h, so no include
changes were needed beyond those already made for the source restructuring.

Verified: spyre-triton-opt builds clean; lit 34/34; pytest 1146 passed,
47 skipped, 0 failed (the 3 inter_tile_reduce numerical failures present at
the previous pin are also resolved).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>

* Bump ktir-mlir-frontend to the merged stubgen cmake>=4.0 fix

Advances the submodule ed069fb6 -> ecfb9ed, picking up
torch-spyre/ktir-mlir-frontend#59 ("Pin cmake>=4.0 and enable stubgen in CI").

This is what unblocks CI. `05e7a0f` (already in this branch's range) added
`MLIR_PYTHON_STUBGEN_ENABLED = "ON"` to the submodule's pyproject.toml, which
activated a latent overflow: MLIR's `mlir_generate_type_stubs()` builds a
283-char target name by joining DEPENDS_TARGETS, and CMake 3.x writes that full
name into its file-API reply filename, overflowing NAME_MAX (255). The bindings
install then died with ENAMETOOLONG — errno 63 locally on macOS, errno 36 in CI
on Linux.

#59 pins cmake>=4.0 in build-system.requires; CMake 4.0 truncates the name and
relies on the trailing hash, so stubgen stays enabled rather than being worked
around here. Note that pin only applies to an isolated PEP 517 build — a
consumer running cmake directly is unaffected (the project itself declares
cmake_minimum_required(VERSION 3.21)), and the cmake it installs lives only in
the ephemeral build environment.

ecfb9ed is the squashed commit on ktir-mlir-frontend main, so this is no longer
a pin to an unmerged branch head.

Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>

---------

Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vswagath1989 vswagath1989 vswagath1989 approved these changes
@KFAFSP KFAFSP Awaiting requested review from KFAFSP

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@fabianlim @KFAFSP @vswagath1989