Add co-located test CI job; fix, guard, and prune co-located tests#21
Merged
Conversation
lfengad
force-pushed
the
colocated-tests-ci
branch
4 times, most recently
from
9ccfe0a to
ec8871f
Compare
xlu451
previously approved these changes
Jun 5, 2026
foreverlms
previously approved these changes
Jun 8, 2026
Adds a `unittest` job to gpu-tests.yml, parallel with the four existing GPU jobs (all gated on the single pre-commit lint). It runs every *_test.py under cosmos_framework/ (CPU + GPU together) in one pytest invocation on the self-hosted GPU runner, plus two torchrun steps for the distributed tests that hardcode their world size (cfgp_ar @2 ranks, context_parallel @4 ranks). conftest.py: - Guard the GPU-only torch.cuda.set_per_process_memory_fraction call so the suite runs on a GPU-less box (no-op when a GPU is present). - Reset autograd per test (inference/ray/serve.py disables it at import time, which otherwise breaks later backward() tests). - Whitelist TRITON_CACHE_DIR (set during flex-attention compilation). Fix four tests that had drifted from the code: - args_test: drop the obsolete s3_uri<->checkpoint.json cross-check. - common/config_test, configs/base/base_config_test: use a current experiment (vision_sft_nano) and set DATASET_PATH; fix an unimportable config path. - prompt_upsampling_test: remove tests of the stripped dense-upsampler path. Delete tests that cannot run in open-source / CI: - Modules uncollectable due to dropped helpers/files: inference/_test.py (missing eval.sh), the four tokenizer tests (missing tokenizers.unittest_utils / RunIf helper), easy_io_test.py. - Credential-gated tests (need internal credentials/pretrained.secret): the eight test_integration_nemotron_* (kept nemotron's 23 CPU unit tests), and cosmos3_vfm_qwen3_vl_network_test.py in full (all its tests were gated). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Transformer Engine sets NVTE_CUDA_INCLUDE_DIR at import time (common/__init__.py) to locate CUDA headers for its NVRTC kernels. Under torchrun, the cfgp_ar tests import omni_mot_model (which pulls in TE) inside the last test, so the var first appears mid-test and detect_env_modifications fails the teardown. It is a third-party library side-effect, not a test bug -- same case as TRITON_CACHE_DIR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two co-located tests (now run in CI) failed after the transfer control-CFG feature merge: * args_test::test_sample_args -- _build_transfer_data concretizes control_guidance to its no-op default (1.0) on every built sample, but _validate_transfer_hints rejected any non-None transfer-only field when no control hint was set. A built non-transfer sample then failed to re-validate from its own dump (idempotent re-build / config round-trip). Exempt the default control_guidance value (1.0) from the guard while still erroring on an explicit non-default control_guidance without a hint. * export_schemas_test::test_all_sample_args_have_descriptions -- the new transfer fields (edge/blur/depth/seg/wsm and the per-chunk frame controls) lacked field docstrings. Added them. No change to the normal inference path or generated output: the validator runs before control_guidance is set to 1.0, so single-shot inference is unaffected; the misconfiguration guard for explicit non-default values is preserved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lfengad
force-pushed
the
colocated-tests-ci
branch
from
31d265d to
8500df9
Compare
Dinghow
approved these changes
Jun 8, 2026
foreverlms
approved these changes
Jun 8, 2026
rahul-steiger-nv
pushed a commit
to rahul-steiger-nv/cosmos-framework
that referenced
this pull request
Jun 15, 2026
…VIDIA#21) Adds a `colocated-tests` job to gpu-tests.yml that runs every *_test.py under cosmos_framework/ (CPU + GPU together) in one pytest invocation on the self-hosted GPU runner, plus two torchrun steps for the distributed tests that hardcode their world size (cfgp_ar @2 ranks, context_parallel @4 ranks). conftest.py: - Guard the GPU-only torch.cuda.set_per_process_memory_fraction call so the suite runs on a GPU-less box (no-op when a GPU is present). - Reset autograd per test (inference/ray/serve.py disables it at import time, which otherwise breaks later backward() tests). - Whitelist TRITON_CACHE_DIR (set during flex-attention compilation). Restore cosmos_framework/utils/helper_test.py (the RunIf skip helper) so credential/asset-gated tests skip cleanly instead of failing at import. Skip-guard the S3-credential tests in nemotron_3_dense_vl_test.py and cosmos3_vfm_qwen3_vl_network_test.py so they skip (not error) when credentials/pretrained.secret is absent (open-source / CI). Fix four tests that had drifted from the code: - args_test: drop the obsolete s3_uri<->checkpoint.json cross-check. - common/config_test, configs/base/base_config_test: use a current experiment (vision_sft_nano) and set DATASET_PATH; fix an unimportable config path. - prompt_upsampling_test: remove tests of the stripped dense-upsampler path. Delete test files that cannot be collected on the open-source tree (missing eval.sh / tokenizers.unittest_utils), and remove easy_io_test.py. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a
colocated-testsjob to gpu-tests.yml that runs every *_test.py under cosmos_framework/ (CPU + GPU together) in one pytest invocation on the self-hosted GPU runner, plus two torchrun steps for the distributed tests that hardcode their world size (cfgp_ar @2 ranks, context_parallel @4 ranks).conftest.py:
Restore cosmos_framework/utils/helper_test.py (the RunIf skip helper) so credential/asset-gated tests skip cleanly instead of failing at import.
Skip-guard the S3-credential tests in nemotron_3_dense_vl_test.py and cosmos3_vfm_qwen3_vl_network_test.py so they skip (not error) when credentials/pretrained.secret is absent (open-source / CI).
Fix four tests that had drifted from the code:
Delete test files that cannot be collected on the open-source tree (missing eval.sh / tokenizers.unittest_utils), and remove easy_io_test.py.