fix(tests): make -k <registry-key> work in smoke test - #500
Open
ahsi5086 wants to merge 2 commits into
Open
Conversation
CAUSAL_PATHS used raw HF paths as pytest IDs, so -k gemma4_moe couldn't match google/gemma-4-26B-A4B-it. Introduce CAUSAL_PARAMS as a list of pytest.param(path, id=key) objects so test IDs are registry key names. Update xfail_non_blocking to handle both plain strings and pytest.param inputs (backward compat with all other callers that still use CAUSAL_PATHS). Add scripts/run_smoke_by_key.py for models excluded from CAUSAL_PATHS by the smallest-per-adapter selection (e.g. granite8b) — runs the smoke test for any registry key directly without touching the default collection. Fixes: -k gemma4_moe now collects 1 test (was 0) Default run unchanged: 23 tests, same models as before Signed-off-by: Ahmed Siddiqui <Ahmed.S@ibm.com>
ahsi5086
force-pushed
the
fix/smoke-test-k-filter-by-registry-key
branch
from
September 9, 2026 20:50
4d56e63 to
bc1e3b8
Compare
ahsi5086
marked this pull request as ready for review
September 10, 2026 17:12
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
pytest -k gemma4_moecollected 0 tests despitegemma4_moebeing avalid registry entry. The smoke test was parametrized with raw HF paths
as test IDs (e.g.
google/gemma-4-26B-A4B-it), so-k gemma4_moehadnothing to match against. Only models whose HF path happened to contain
the key name as a substring (e.g.
qwen3→Qwen/Qwen3-0.6B) workedby accident.
Fix
Introduce
CAUSAL_PARAMS— a list ofpytest.param(path, id=key)objects where the registry key name is the test ID. Switch
test_e2e_smoke_spyre.pyto useCAUSAL_PARAMSinstead ofCAUSAL_PATHS. Updatexfail_non_blockingto handle both plain strings(all existing callers are unchanged) and
pytest.paraminputs.CAUSAL_PATHSis untouched — every other test file that usesids=CAUSAL_PATHScontinues to work exactly as before.Add
scripts/run_smoke_by_key.pyfor models intentionally excluded fromthe default collection by the smallest-per-adapter selection rule (e.g.
granite8bloses togranite2bforhf_granite.py). These can't bereached via
-ksince they're never collected — the script runs thesmoke test for any registry key directly.
Usage