Real World ATOM Kimi K2.7-Code Production Inferencing Enablement Request

[functionstackx]

Suggestion Description

hi @chunfangamd @andyluo7

After an thoughtful discussion with AMD senior leader, Here is the features that need to be enablement & all the features need to have composability with each other in order to use ATOM in real world production inferencing. Happy to help beta test once the features are enabled & composability has been tested

Here is the use case that all need to be composed with each other:

• Kimi K2.7-Code
• with MTP support (EAGLE3 or EAGLE3.1) Adapt EAGLE 3.1 (fc_norm + post-norm feedback) and add Kimi K2.6 MLA draft #943
• claude code /v1/messages API support & composability [Parity with vLLM, SGLang]: Anthropic Messages API endpoint (/v1/messages) support to use with claude code [functional enablement + docs] #552
• With Prompt Caching enablement & composability (need to strip the claude claude billing headers) [Parity with vLLM, SGLang]: Anthropic Messages API endpoint (/v1/messages) support to use with claude code [functional enablement + docs] #552 (comment)
• With CPU kvcache offloading & composability https://github.com/ROCm/ATOM/pull/997/changes
• With multinode distributed inferencing support & composability
• With production kubernetes deployment enablement & composability [Parity with CUDA vLLM, CUDA SGLang]: native ATOM llm-d Kubernetes Distributed Inferencing Orchestrator for Production LLM Inferencing [Functional Enablement + Guides + Nightly CI] #1187
• With Verified Function Calling Evals CI enablement [Parity with CUDA vLLM]: ATOM Nightly CI and/or per PR CI Function Calling Eval #1186
• and not crash under agentic traffic Two crashes when serving Kimi-K2.5-MXFP4 under agent traffic (gather_kv_b_proj None scale, scheduler assert on oversize prompt) #680
• and ideally documentation on how to deploy it with k8s yaml

We are happy to retry out ATOM internally once this use case is supported.

Operating System

No response

GPU

No response

ROCm Component

No response

[functionstackx]

changing to Kimi K2.7 Code but fundmentally nothing much should changed since it is same model arch as K2.6