Skip to content

[Bug] check-pto does not cover test/basic lit cases #357

@Zhendong404

Description

@Zhendong404

Description

check-pto currently does not cover test/basic/**/*.pto lit-style cases.

This creates a blind spot in CI:

  • tests under test/basic/ can be added or drift out of date,
  • ninja -C build check-pto still passes,
  • but a broader/local lit run that explicitly includes test/basic fails after rebase or when someone manually runs those cases.

We hit this while rebasing onto mainline and investigating the tilebuf_invalid_* basic tests. Those tests could be merged earlier because CI did not exercise them, but they fail once test/basic is run explicitly.

Reproduction (minimal)

Repository state where this was observed:

  • current local HEAD: 0c9b8e48546167acd49fbb52278dc4eede36c613
  • original introducing commit for the tilebuf basic tests: c6974340ebc772c895c5237455b90b8757f3d8c5 (add tilebuf static check)

Commands:

ninja -C build check-pto

This passes, because check-pto only runs lit from build/test/lit and loads test/lit/lit.cfg.py.

But explicitly running test/basic reproduces uncovered failures:

llvm-lit -sv test/basic/tilebuf_invalid_32byte_align_colmajor.pto

More broadly:

llvm-lit -sv test/basic/acc_store_gm.pto \
  test/basic/acc_store_ub.pto \
  test/basic/cube_store_dma_copy.pto \
  test/basic/expand_tile_op_tilelang_tcmp.pto \
  test/basic/expand_tile_op_tilelang_tfmod.pto \
  test/basic/expand_tile_op_tilelang_trem.pto \
  test/basic/expand_tile_op_tprelu_tilelang.pto \
  test/basic/tcolargmax.pto \
  test/basic/tcolargmin.pto \
  test/basic/tilebuf_invalid_32byte_align_colmajor.pto \
  test/basic/tilebuf_invalid_32byte_align_rowmajor.pto \
  test/basic/tilebuf_invalid_boxed_cols_divisibility.pto \
  test/basic/tilebuf_invalid_boxed_rows_divisibility.pto \
  test/basic/tilebuf_invalid_valid_exceeds_shape.pto \
  test/basic/tilebuf_invalid_zero_shape.pto \
  test/basic/tilebuf_valid_positive.pto

Expected behavior

One of the following should happen:

  1. check-pto should cover test/basic lit cases, so stale/basic regressions are caught in CI.
  2. Or test/basic should be clearly split into a separate lit target that is also run in CI.
  3. Or if test/basic is intentionally not part of CI, we should document that explicitly and avoid treating it as protected regression coverage.

Actual behavior / evidence

Current check-pto wiring runs lit from build/test/lit:

build/build.ninja:
COMMAND = cd .../build/test/lit && .../lit.py -sv .../build/test/lit

And the generated site config loads only:

build/test/lit/lit.site.cfg.py -> test/lit/lit.cfg.py

That means discovery is rooted in test/lit/, not the repo test/ root, so test/basic is skipped by CI.

This is why tests such as test/basic/tilebuf_invalid_32byte_align_colmajor.pto could be merged and still pass CI, but fail later when test/basic is actually exercised.

Commit

0c9b8e48546167acd49fbb52278dc4eede36c613

Host platform

Linux (x86_64)

Target Ascend arch

N/A

PTO level

N/A

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions