tests: port upstream kernel tests and adapt test infrastructure for MACA compat#255
tests: port upstream kernel tests and adapt test infrastructure for MACA compat#255Dayuxiaoshui wants to merge 1 commit into
Conversation
…ACA compat
1. Port missing upstream vLLM kernel tests to tests/kernels/core/:
- test_apply_rotary_emb.py
- test_cpu_activation.py
- test_fused_qk_norm_rope.py
- test_fused_rms_norm_gated.py
- test_fused_silu_mul_block_quant.py
- test_fused_q_kv_rmsnorm.py
- test_minimax_reduce_rms.py
- test_rotary_embedding_mla_cache_fused.py
- test_vit_bilinear_pos_embed.py
- test_vit_fp8_attn.py
- test_vit_fp8_quant.py
- test_vit_fp8_scaling.py
2. Adapt test infrastructure for upstream API changes:
- tests/conftest.py: add default_vllm_config fixture; compat imports for
ModelDType, RunnerOption, ConvertOption, Logprob, InputContext, etc.;
inject infer_schema monkey-patch for PyTorch 2.6+ list[int] compat
- tests/utils.py: compat imports for FlexibleArgumentParser, GB_bytes,
cuda_device_count_stateless, get_open_port; add ensure_current_vllm_config
- tests/kernels/utils.py: compat imports for AttentionBackend,
AttentionMetadata, AttentionType; fallback constants for _Backend,
STR_BACKEND_ENV_VAR, etc.
- tests/kernels/quant_utils.py: compat import for round_up
- tests/models/utils.py: compat imports for InputContext, Logprob
- tests/models/registry.py: compat imports for ModelDType, TokenizerMode
|
cc @ILikeIneine |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive set of new kernel tests covering CPU activations, rotary embeddings, fused RMS norms, block quantization, and Vision Transformer (ViT) specific kernels like bilinear position embedding and FP8 attention. It also updates test configurations and utilities with compatibility shims and robust import handling to ensure the test suite works across different vLLM versions. A critical security vulnerability was noted in tests/conftest.py due to the inclusion of /tmp in the system path, which could lead to arbitrary code execution.
| import sys | ||
| sys.path.insert(0, '/tmp') | ||
| import patch_infer_schema |
There was a problem hiding this comment.
Modifying sys.path to include a world-writable directory like /tmp is a critical security vulnerability. This allows for arbitrary code execution if a malicious patch_infer_schema.py file is placed in /tmp. The test suite would then import and execute this malicious code, potentially compromising the execution environment.
Instead of modifying sys.path with a hardcoded, insecure path, consider placing the patch_infer_schema.py file within the test directory structure and importing it using a relative path. For example, if it's a test utility, it could live in a tests/utils directory.
|
@Dayuxiaoshui Have you ever run pytest on metax backend devices? |
|
@ILikeIneine We have run pytest on the MetaX C500 backend. The test environment was MACA SDK 3.3.0.2 with PyTorch 2.6.0+metax3.3.0.2, using upstream vLLM source via PYTHONPATH. Among the 12 ported kernel test files, 8 execute normally (pass/skip). Two files have substantive failures: test_fused_silu_mul_block_quant.py fails 178 cases due to numerical mismatch in the silu_and_mul_per_block_quant kernel output against the CPU reference, and test_rotary_embedding_mla_cache_fused.py fails 322 cases because the concat_and_cache_mla_rope_fused op is not registered in the current MACA environment. The remaining 4 files are skipped entirely as they depend on upstream modules (e.g., deepseek_v4_ops, qwen3_vl, triton fp8) not available in this environment. |
|
cc @ILikeIneine |
|
What your vllm-metax version? We'd like to re-run it with the latest vllm-metax release v0.18.0 in this docker image. Or following the build guild with the latest v0.20.0-dev. |
no problem,I will do this work |
2 participants
Port missing upstream vLLM kernel tests to tests/kernels/core/:
Adapt test infrastructure for upstream API changes:
PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.
Purpose
Test Plan
Test Result
(Optional) Documentation Update
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.