Add DR.Kernel recipe: Triton kernel-generation RL on NPU

[HwCARI]

[recipe] feat: add Dr. Kernel asynchronous RL recipe for Triton kernel generation on Ascend NPU

What does this PR do?

This PR adds a new drkernel recipe that brings the Dr. Kernel approach (RL Done Right for Triton kernel generation) to verl-ascend-recipe, adapted to run on Ascend NPU using fully asynchronous RL training. The model is given a Torch reference implementation and learns to produce optimized kernel code, with execution-based verification.

Background

Dr. Kernel targets two failure modes common to RL for kernel generation — reward hacking and lazy optimization — using the KernelGym execution environment for correctness/performance checks, profiling-based rewards, and unbiased multi-turn RL. This recipe ports that workflow onto the verl + Ascend stack.

To address this, the recipe is built on verl's fully asynchronous training path:

• The trainer and rollout/evaluation workers are fully decoupled and run concurrently.
• Rollouts and kernel verification stream results back through an async queue, so the trainer consumes samples as they complete instead of waiting on the slowest kernel.
• Training proceeds off-policy with controlled staleness, keeping device utilization high during long-horizon, multi-turn rollouts.

This asynchronous design is the key enabler for scaling Dr. Kernel-style RL efficiently on Ascend NPU.

Changes

• New drkernel/ recipe directory containing:
  • README.md — setup, data prep, and async training instructions for Ascend NPU
  • REQUIRED_VERL.txt — pinned/rolling verl version per repo convention
  • Fully asynchronous training scripts and config for [model, e.g. Qwen3-8B] multi-turn GRPO RL (decoupled trainer / rollout, off-policy with staleness control)
  • Reward functions covering compilation, correctness, and performance
  • Integration with the kernel evaluation environment through the async rollout/reward pipeline
• Ascend-specific adaptations (triton-ascend, NPU device handling, async-mode env setup)

[gemini-code-assist]

Code Review

This pull request ports the DR.Kernel framework to verl, introducing a multi-turn agent loop, a TRLOO advantage estimator, a multi-turn rejection sampling batch filter, and a distributed KernelGYM evaluation server. The review identified several critical issues: a runtime TypeError when iterating over the BatchFilterConfig dataclass, a blocking psutil call that halts the FastAPI event loop, unhandled exceptions leaving tasks dangling in Redis, and broad exception handling that swallows HTTPExceptions. Additionally, the agent loop's truncation logic is inverted, the two-gate filter contains redundant tensor allocations, and the subprocess pool's error handling is overly broad, leading to unnecessary worker restarts.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The BatchFilterConfig class is a standard Python dataclass and is not iterable by default. Attempting to iterate over omega_cfg using for k in omega_cfg will raise a TypeError at runtime. Use dataclasses.asdict to safely convert the dataclass to a dictionary.

Suggested change

	if isinstance(omega_cfg, BatchFilterConfig):
	as_dict = {k: getattr(omega_cfg, k) for k in omega_cfg}
	if isinstance(omega_cfg, BatchFilterConfig):
	from dataclasses import asdict
	as_dict = asdict(omega_cfg)

[gemini-code-assist]

Calling psutil.cpu_percent(interval=1) is a blocking operation that halts the entire FastAPI event loop for 1 full second. In an asynchronous server, this will block all concurrent request handling and task processing. Use interval=None to retrieve the CPU utilization non-blockingly.

Suggested change

	async def get_system_health() -> dict[str, Any]:
	try:
	cpu_percent = psutil.cpu_percent(interval=None)

[gemini-code-assist]

If controller.handle_request raises an exception, the task is never marked as FAILED in Redis because task_mgr.fail_task is not called. This leaves the task permanently dangling in PENDING or PROCESSING status, causing clients polling for results to hang indefinitely. Wrap the execution in a try...except block to ensure the task is failed gracefully.

Suggested change

	try:
	controller = get_workflow_controller(workflow_name or "kernelbench")
	except KeyError as exc:
	raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc
	try:
	result = await controller.handle_request(payload, scheduler)
	if isinstance(result, dict):
	result.setdefault("task_id", task_id)
	await task_mgr.complete_task(task_id, result)
	return task_id, result, _result_status(result)
	except Exception as e:
	error_code = classify_error(str(e), "system")
	await task_mgr.fail_task(task_id, str(e), error_code)
	raise

[gemini-code-assist]

The broad except Exception: block catches and silently swallows the HTTPException raised for a device mismatch. This prevents the validation error from being propagated to the client, causing mismatched heartbeats to succeed silently. Explicitly catch and re-raise HTTPException first.

Suggested change

	raise HTTPException(
	status_code=status.HTTP_409_CONFLICT, detail="Device mismatch; please re-register"
	)
	except Exception:
	pass
	raise HTTPException(
	status_code=status.HTTP_409_CONFLICT, detail="Device mismatch; please re-register"
	)
	except HTTPException:
	raise
	except Exception:
	pass

[gemini-code-assist]

The truncation logic for left and right sides is inverted. When tool_response_truncate_side is 'left', it should truncate the left side (keeping the right side of the text), but the current implementation returns the left side (text[:n]) and truncates the right side. Conversely, when it is 'right', it should truncate the right side (keeping the left side), but it returns the right side (text[-n:]).

Suggested change

	if self.tool_response_truncate_side == "left":
	return text[:n] + "...(truncated)"
	if self.tool_response_truncate_side == "right":
	return "(truncated)..." + text[-n:]
	if self.tool_response_truncate_side == "left":
	return "...(truncated)" + text[-n:]
	if self.tool_response_truncate_side == "right":
	return text[:n] + "...(truncated)"

[gemini-code-assist]

The intermediate tensor masked_diffs and its associated torch.zeros_like allocation are redundant. Since min_diffs_per_seq replaces masked positions with inf anyway, you can compute it directly from logprob_diffs using a single torch.where call. This avoids multiple unnecessary tensor allocations and improves efficiency.

        # Check stability: find minimum difference among valid tokens
        min_diffs_per_seq = (
            torch.where(response_mask.bool(), logprob_diffs, float("inf"))
            .min(dim=-1)
            .values
        )  # [bs]

[gemini-code-assist]

Treating any RuntimeError as a CUDA error is overly broad. Standard model execution errors (such as shape mismatches or value errors) will raise a RuntimeError and unnecessarily trigger a worker subprocess termination and restart, introducing significant overhead. Use the already imported is_device_fatal_error helper to only restart on actual device-fatal errors.

Suggested change

	is_cuda_error = (
	"CUDA" in error_type
	or "CUDA" in error_message
	or "cuda" in error_message.lower()
	or error_type in ["RuntimeError", "CudaError"]
	)
	is_profiler_error = "PROFILER_NO_CUDA_EVENTS" in error_message
	is_cuda_error = (
	"CUDA" in error_type
	or "CUDA" in error_message
	or "cuda" in error_message.lower()
	or error_type == "CudaError"
	or is_device_fatal_error(error_message)
	)