Skip to content

[rollout] chore: bump up trtllm version to 1.3.0rc10#5841

Open
Superjomn wants to merge 17 commits intoverl-project:mainfrom
Superjomn:upgrade-megatron-0-16
Open

[rollout] chore: bump up trtllm version to 1.3.0rc10#5841
Superjomn wants to merge 17 commits intoverl-project:mainfrom
Superjomn:upgrade-megatron-0-16

Conversation

@Superjomn
Copy link
Copy Markdown
Collaborator

@Superjomn Superjomn commented Apr 1, 2026

What does this PR do?

This PR bump up the trtllm docker image to v1.3.0rc10.

Checklist Before Starting

  • Search for similar PRs. Paste at least one query link here: ...
  • Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
    • {modules} include fsdp, megatron, veomni, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data, cfg, reward, fully_async, one_step_off
    • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
    • {type} is in feat, fix, refactor, chore, test
    • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
    • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc.

API and Usage Example

Demonstrate how the API changes if any, and provide usage example(s) if possible.

# Add code snippet or script demonstrating how to use this

Design & Code Changes

Demonstrate the high-level design if this PR is complex, and list the specific changes.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

Superjomn and others added 4 commits April 1, 2026 14:00
Update DeepEP branch from v1.2.1 to hybrid-ep (removing the now-unnecessary
patch) and add CCCL CPATH for build compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>

cleanup
Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the TRT-LLM rollout implementation and Docker environment, including upgrading Megatron-LM to v0.16.0 and transitioning to SleepConfig for server management. It also removes the single-node restriction for TRT-LLM replicas. Review feedback points out that using a branch name for the DeepEP dependency in the Dockerfile compromises build reproducibility and identifies a potential IndexError in the placement group indexing logic that requires a bounds check.

@wuxibin89
Copy link
Copy Markdown
Collaborator

Should we also bump ci image?

Superjomn and others added 5 commits April 1, 2026 15:32
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…d import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown
Collaborator

@hchings hchings left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you cherry-pick this commit as well? 4d36c21

Other than that, LGTM.

Superjomn and others added 2 commits April 3, 2026 09:54
ExecutorMemoryType is not yet available in trtllm v1.3.0rc10. Use
try/except fallback so sleep mode gracefully degrades on older versions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Superjomn Superjomn changed the title [rollout,trtllm] bump up trtllm version to 1.3.0rc10 [rollout] bump up trtllm version to 1.3.0rc10 Apr 3, 2026
@Superjomn Superjomn changed the title [rollout] bump up trtllm version to 1.3.0rc10 [rollout, trtllm] bump up trtllm version to 1.3.0rc10 Apr 3, 2026
@Superjomn Superjomn changed the title [rollout, trtllm] bump up trtllm version to 1.3.0rc10 [rollout] chore: bump up trtllm version to 1.3.0rc10 Apr 3, 2026
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@Superjomn Superjomn requested a review from hchings April 3, 2026 04:30
Copy link
Copy Markdown
Collaborator

@hchings hchings left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address the cherry-pick comment. Other parts LGTM.

Superjomn and others added 3 commits April 3, 2026 16:26
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The job was renamed on main from e2e_grpo_trainer_fsdp-vlm to
e2e_grpo_trainer_megatron-vlm. Align our branch to avoid merge
validation error in cleanup job's needs list.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The TRT-LLM 1.3.0rc10 base image ships with a newer CUDA toolkit
that requires a newer NVIDIA driver than the CI runners have,
causing "No CUDA GPUs are available" errors. Adding the cuda-compat
package and setting LD_LIBRARY_PATH enables the container to run
on hosts with older drivers (>= R535).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Superjomn Superjomn force-pushed the upgrade-megatron-0-16 branch from 440c5f3 to 029b394 Compare April 3, 2026 12:59
"model_extra",
"executor_extra",
"model",
"model_weights",
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Superjomn Is this section for backward compatibility (for older trtllm version before we have the fine-grained labels)? If yes, then it should be exactly the same as the old tags at https://github.com/verl-project/verl/pull/5841/changes#diff-4d19b99d5dc8054a16c391ce00301671727c4c3549ecb6d904d33c2aa1f552beL263 (aka without model_weights and draft_model_weights). Otherwise I think it'll error out.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, let me update it.

…allback _WEIGHTS_TAGS

The fallback else branch (for older trtllm without ExecutorMemoryType)
should match the original hard-coded list, which never included these tags.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants