Skip to content

docs(faq): wiring omp and Claude Code to models, every route tested - #190

Merged
aarora79 merged 1 commit into
mainfrom
docs/faq-omp-claude-wiring
Sep 11, 2026
Merged

aarora79 merged 1 commit into
mainfrom
docs/faq-omp-claude-wiring

Conversation

@aarora79

Copy link
Copy Markdown
Owner

The codex FAQ answers three questions with plain agent commands. omp and Claude Code now have the same page, in the same format, and all six routes were run on this box before shipping. Each returned OK.

omp (omp/18.1.15)

Route Verified against
vLLM live qwen3.6-35b-fp8 at 262,144 tokens, via a models.yml provider at :8000
Open-weight on Bedrock qwen.qwen3-coder-30b-a3b-instruct through the mantle proxy
Bedrock directly claude-haiku-4-5 via --model amazon-bedrock/<inference-profile>

Its two structural facts get stated up front: the config is YAML under PI_CODING_AGENT_DIR (models.yml for providers, config.yml for compaction), and a model is named provider/model. Plus the two that bite: it hangs on an inherited stdin, and the prompt needs -- before it or a SKILL.md's --- frontmatter is read as a flag.

Claude Code (2.1.266)

Route Verified against
vLLM live qwen3.6-35b-fp8, ANTHROPIC_BASE_URL at :8000
Open-weight on Bedrock qwen.qwen3-coder-30b-a3b-instruct through the mantle proxy
Bedrock directly claude-haiku-4-5 with CLAUDE_CODE_USE_BEDROCK=1

vLLM answers /v1/messages, so no bridge is needed for a self-hosted model, and the page includes the curl that proves it. Bedrock's non-Anthropic models do not, and sent through Converse they return tool-call tokens as plain text, so the mantle proxy is what makes tool calls survive.

Three findings from actually running them

Each is recorded next to its example rather than in a footnote.

omp prices a run from the cost block in models.yml. The mantle run reported total: 0.002826114 for 18,284 input and 2 output tokens, which is exactly the Qwen Standard rates applied. The block is load-bearing, not decoration: leave zeros in for a self-hosted model, put real rates in for a metered one.

Claude Code's total_cost_usd is fiction against a self-hosted endpoint. It claimed 0.1138 for a model that costs nothing per token, and reported contextWindow: 200000 while the server was serving 262,144. This is why the repo derives self-hosted cost from GPU-seconds.

DISABLE_NON_ESSENTIAL_MODEL_CALLS=1 is load-bearing on the mantle route. Without it the same command emitted [claude-code:unrecognized_model] {"query_source":"generate_session_title"} before the answer, because Claude Code makes side calls that fail noisily on an unrecognized model id. With it: "result":"OK","subtype":"success","num_turns":1.

Verification

  • Six agent invocations, all returning OK; the vLLM server ran the guide's own flags.
  • Links resolve, tables well formed, both files indexed in docs/faq/README.md.
  • The arithmetic in the omp page was checked against what the runs reported: the 237952 compaction threshold, and the 0.002826114 cost figure.
  • Both proxies I started for verification are stopped again.

…oute tested

The codex FAQ answered three questions -- open-weight on Bedrock, open-weight on
vLLM, a model on Bedrock directly -- with plain agent commands. omp and Claude
Code now have the same page, and all six routes were run on this box before
shipping. Each returned OK.

omp (omp/18.1.15):
- vLLM: a models.yml provider at :8000, api openai-completions, verified against
  a live qwen3.6-35b-fp8 server at 262,144 tokens
- open-weight on Bedrock: the same provider shape pointed at the mantle proxy,
  verified against qwen.qwen3-coder-30b-a3b-instruct
- Bedrock directly: --model amazon-bedrock/<inference-profile>, verified against
  claude-haiku-4-5

Claude Code (2.1.266):
- vLLM: ANTHROPIC_BASE_URL at :8000. vLLM answers /v1/messages, so no bridge is
  needed; the page includes the curl that proves it
- open-weight on Bedrock: ANTHROPIC_BASE_URL at the mantle proxy, which
  translates Messages to chat completions so tool calls survive
- Bedrock directly: CLAUDE_CODE_USE_BEDROCK=1 plus the inference profile

Three findings from running them, each recorded next to its example:

- omp prices a run from the cost block in models.yml. The mantle run reported
  total 0.002826114 for 18,284 input and 2 output tokens, which is exactly the
  Qwen Standard rates applied, so the block is load-bearing rather than
  decoration.
- Claude Code's total_cost_usd is fiction against a self-hosted endpoint. It
  claimed 0.1138 for a model that costs nothing per token, and reported a
  200,000 context window while the server served 262,144.
- DISABLE_NON_ESSENTIAL_MODEL_CALLS=1 is load-bearing on the mantle route.
  Without it the run emitted [claude-code:unrecognized_model] from a
  generate_session_title side call before answering.

Links resolve, tables are well formed, both files appear in the index, and the
arithmetic in the omp page (the 237952 compaction threshold, the cost figure)
was checked against the numbers the runs reported.
@aarora79
aarora79 merged commit d8b12d5 into main Sep 11, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant