[tests] test: add CPU coverage for MixGRPO and GRPO-Guard

[SamitHuang]

What does this PR do?

Add cheap CPU unit tests for MixGRPO window positioning/registry dispatch and extend GRPO-Guard coverage (loss registry, config validation, rollout correction integration) without expensive end-to-end runs.

Checklist Before Starting

• Search for similar PRs. Paste at least one query link here: ...
• Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
  • {modules} include fsdp, vllm_omni, rollout, trainer, ci, training_utils, recipe, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data, cfg, reward, diffusion, omni, tests, docker
  • If this PR involves multiple modules, separate them with , like [diffusion, doc]
  • {type} is in feat, fix, refactor, chore, test
  • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
  • Example: [BREAKING][diffusion, fsdp] feat: new rollout scheduler

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc.

API and Usage Example

Demonstrate how the API changes if any, and provide usage example(s) if possible.

# Add code snippet or script demonstrating how to use this

Design & Code Changes

Demonstrate the high-level design if this PR is complex, and list the specific changes.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

• Read the Contribute Guide.
• Apply pre-commit checks: pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always
• Add / Update the documentation.
• Add unit or end-to-end test(s) to the CI workflow to cover all the code. If not feasible, explain why: ...

[gemini-code-assist]

Code Review

This pull request introduces CPU tests for the MixGRPO pipeline registration and SDE window positioning, registers and tests the new grpo_guard loss mode, and adds integration tests for the grpo_guard loss. The feedback suggests standardizing the instantiation of DiffusionModelConfig to avoid bypassing validation, increasing the range in the random window positioning test to prevent flaky test failures, and resolving the configuration directory path relative to the package directory to make the test suite more robust.

[gemini-code-assist]

Using object.__new__ and object.__setattr__ to bypass the standard constructor of DiffusionModelConfig is fragile and bypasses validation, default values, and type checks. If DiffusionModelConfig is a standard dataclass, it should be instantiated normally to ensure maintainability and correctness.

Suggested change

	def _make_model_config(algorithm: str = "mix_grpo") -> DiffusionModelConfig:
	cfg = object.__new__(DiffusionModelConfig)
	object.__setattr__(cfg, "architecture", "QwenImagePipeline")
	object.__setattr__(cfg, "algorithm", algorithm)
	object.__setattr__(cfg, "external_lib", None)
	return cfg
	def _make_model_config(algorithm: str = "mix_grpo") -> DiffusionModelConfig:
	return DiffusionModelConfig(
	architecture="QwenImagePipeline",
	algorithm=algorithm,
	external_lib=None,
	)

References

1. Adhere to standard object instantiation and PEP 8 best practices to avoid bypassing class constructors and type-safety mechanisms. ^(link)

[gemini-code-assist]

The current test uses a small sde_window_range of [0, 10]. With sde_window_size = 2, the maximum start index is 8, meaning there are only 9 possible start positions. Since the test compares two random draws with different seeds (global_steps = 0 vs global_steps = 1), there is a 1 in 9 (~11.1%) chance that both seeds will randomly map to the same start position, causing the test to fail flakily.

Increasing the range to [0, 100000] reduces the collision probability to 1 in 100,000, making the test robust and deterministic.

Suggested change

	"sde_window_range": [0, 10],
	"sde_window_range": [0, 100000],

[gemini-code-assist]

Using os.path.abspath with a hardcoded relative path like "verl_omni/trainer/config/diffusion/actor" makes the test suite fragile because it depends on the current working directory from which pytest is executed. If pytest is run from a subdirectory (e.g., tests/), the path resolution will fail.

Instead, resolve the path relative to the verl_omni package directory using os.path.dirname(verl_omni.__file__), which is more robust and consistent with other tests in the codebase.

        import verl_omni
        config_dir = os.path.join(os.path.dirname(verl_omni.__file__), "trainer/config/diffusion/actor")

[zhtmike]

add mix_grpo test under tests/trainer/diffusion/test_diffusion_core_algos_on_cpu.py instead of creating a new file