fix: Fix Kimi K2.5 function calling and structured outputs#1430

Open

gongwei-130 wants to merge 27 commits into

mainfrom

wei/kimi-k25-consolidate

Commits on May 1, 2026

fix(grpc): skip reasoning parser when constrained decoding is active

vschandramourya
authored and
Wei Gong
committed
fix(grpc): register raise_exception in chat templates and coerce tool argument types

ConnorLi96
authored and
Wei Gong
committed
fix(tool_parser): fix function call parsing for models with native tool-call tokens

ConnorLi96
authored and
Wei Gong
committed
fix(gateway): comprehensive func call and response quality fixes

ConnorLi96
authored and
Wei Gong
committed
fix(tokenizer): load merged EOS token IDs from config.json + generation_config.json

ConnorLi96
authored and
Wei Gong
committed
fix(tokenizer): reduce tiktoken partial UTF-8 decode log from warn to debug

ConnorLi96
authored and
Wei Gong
committed
feat(protocol): add thinking param to Chat API and support bare string image_url

ConnorLi96
authored and
Wei Gong
committed
fix(reasoning): run reasoning parser before JSON/tool post-processing when thinking is active

ConnorLi96
authored and
Wei Gong
committed
style: fix formatting, clippy warnings, and merge artifacts from cherry-pick

ConnorLi96
authored and
Wei Gong
committed
fix(streaming): enable reasoning parser for constrained outputs

ConnorLi96
authored and
Wei Gong
committed
fix(kimik2): rewrite tool_call IDs and fix cross-chunk fence stripping

ConnorLi96
authored and
Wei Gong
committed
feat(health): make /health_generate issue a real backend probe with logging

ConnorLi96
authored and
Wei Gong
committed
feat(logging): pass through user-supplied request_id to engine

ConnorLi96
authored and
Wei Gong
committed
feat(logging): compute per-message SHA-256 hashes for session reconstruction

ConnorLi96
authored and
Wei Gong
committed
fix(streaming): replace fence_buffer with simple cross-chunk fence stripping

ConnorLi96
authored and
Wei Gong
committed
refactor(streaming): extract strip_json_fence helper with unit tests

ConnorLi96
authored and
Wei Gong
committed
feat(grpc): pass message_hashes through gRPC proto to TRT-LLM

ConnorLi96
authored and
Wei Gong
committed
fix(health): increase health_generate probe timeout from 3s to 60s

ConnorLi96
authored and
Wei Gong
committed
fix(reasoning): skip reasoning parsing for structured output requests

ConnorLi96
authored and
Wei Gong
committed
feat(health): skip inference probe when tokens forwarded recently

ConnorLi96
authored and
Wei Gong
committed
fix(multimodal): use 1 placeholder per image for Kimi-K2.5

ConnorLi96
authored and
Wei Gong
committed
fix(multimodal): collapse media placeholders for TRT-LLM only

ConnorLi96
authored and
Wei Gong
committed
fix(logging): downgrade message_hash log from info to debug

ConnorLi96
authored and
Wei Gong
committed
feat(protocols): add response_format.type=regex support

vschandramourya
authored and
Wei Gong
committed

Commits on May 2, 2026

Fix regex structured output reasoning parsing
Wei Gong
committed
Fix lint formatting
Wei Gong
committed
Make health_generate probe timeout configurable
Wei Gong
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Fix Kimi K2.5 function calling and structured outputs#1430

fix: Fix Kimi K2.5 function calling and structured outputs#1430
gongwei-130 wants to merge 27 commits into
mainfrom
wei/kimi-k25-consolidate

Commits on May 1, 2026

fix(grpc): skip reasoning parser when constrained decoding is active

fix(grpc): register raise_exception in chat templates and coerce tool argument types

fix(tool_parser): fix function call parsing for models with native tool-call tokens

fix(gateway): comprehensive func call and response quality fixes

fix(tokenizer): load merged EOS token IDs from config.json + generation_config.json

fix(tokenizer): reduce tiktoken partial UTF-8 decode log from warn to debug

feat(protocol): add thinking param to Chat API and support bare string image_url

fix(reasoning): run reasoning parser before JSON/tool post-processing when thinking is active

style: fix formatting, clippy warnings, and merge artifacts from cherry-pick

fix(streaming): enable reasoning parser for constrained outputs

fix(kimik2): rewrite tool_call IDs and fix cross-chunk fence stripping

feat(health): make /health_generate issue a real backend probe with logging

feat(logging): pass through user-supplied request_id to engine

feat(logging): compute per-message SHA-256 hashes for session reconstruction

fix(streaming): replace fence_buffer with simple cross-chunk fence stripping

refactor(streaming): extract strip_json_fence helper with unit tests

feat(grpc): pass message_hashes through gRPC proto to TRT-LLM

fix(health): increase health_generate probe timeout from 3s to 60s

fix(reasoning): skip reasoning parsing for structured output requests

feat(health): skip inference probe when tokens forwarded recently

fix(multimodal): use 1 placeholder per image for Kimi-K2.5

fix(multimodal): collapse media placeholders for TRT-LLM only

fix(logging): downgrade message_hash log from info to debug

feat(protocols): add response_format.type=regex support

Commits on May 2, 2026

Fix regex structured output reasoning parsing

Fix lint formatting

Make health_generate probe timeout configurable