Repository navigation
docs(faq): wiring omp and Claude Code to models, every route tested - #190
Merged
Merged
Conversation
…oute tested The codex FAQ answered three questions -- open-weight on Bedrock, open-weight on vLLM, a model on Bedrock directly -- with plain agent commands. omp and Claude Code now have the same page, and all six routes were run on this box before shipping. Each returned OK. omp (omp/18.1.15): - vLLM: a models.yml provider at :8000, api openai-completions, verified against a live qwen3.6-35b-fp8 server at 262,144 tokens - open-weight on Bedrock: the same provider shape pointed at the mantle proxy, verified against qwen.qwen3-coder-30b-a3b-instruct - Bedrock directly: --model amazon-bedrock/<inference-profile>, verified against claude-haiku-4-5 Claude Code (2.1.266): - vLLM: ANTHROPIC_BASE_URL at :8000. vLLM answers /v1/messages, so no bridge is needed; the page includes the curl that proves it - open-weight on Bedrock: ANTHROPIC_BASE_URL at the mantle proxy, which translates Messages to chat completions so tool calls survive - Bedrock directly: CLAUDE_CODE_USE_BEDROCK=1 plus the inference profile Three findings from running them, each recorded next to its example: - omp prices a run from the cost block in models.yml. The mantle run reported total 0.002826114 for 18,284 input and 2 output tokens, which is exactly the Qwen Standard rates applied, so the block is load-bearing rather than decoration. - Claude Code's total_cost_usd is fiction against a self-hosted endpoint. It claimed 0.1138 for a model that costs nothing per token, and reported a 200,000 context window while the server served 262,144. - DISABLE_NON_ESSENTIAL_MODEL_CALLS=1 is load-bearing on the mantle route. Without it the run emitted [claude-code:unrecognized_model] from a generate_session_title side call before answering. Links resolve, tables are well formed, both files appear in the index, and the arithmetic in the omp page (the 237952 compaction threshold, the cost figure) was checked against the numbers the runs reported.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The codex FAQ answers three questions with plain agent commands. omp and Claude Code now have the same page, in the same format, and all six routes were run on this box before shipping. Each returned
OK.omp (
omp/18.1.15)qwen3.6-35b-fp8at 262,144 tokens, via amodels.ymlprovider at:8000qwen.qwen3-coder-30b-a3b-instructthrough the mantle proxyclaude-haiku-4-5via--model amazon-bedrock/<inference-profile>Its two structural facts get stated up front: the config is YAML under
PI_CODING_AGENT_DIR(models.ymlfor providers,config.ymlfor compaction), and a model is namedprovider/model. Plus the two that bite: it hangs on an inherited stdin, and the prompt needs--before it or aSKILL.md's---frontmatter is read as a flag.Claude Code (
2.1.266)qwen3.6-35b-fp8,ANTHROPIC_BASE_URLat:8000qwen.qwen3-coder-30b-a3b-instructthrough the mantle proxyclaude-haiku-4-5withCLAUDE_CODE_USE_BEDROCK=1vLLM answers
/v1/messages, so no bridge is needed for a self-hosted model, and the page includes thecurlthat proves it. Bedrock's non-Anthropic models do not, and sent through Converse they return tool-call tokens as plain text, so the mantle proxy is what makes tool calls survive.Three findings from actually running them
Each is recorded next to its example rather than in a footnote.
omp prices a run from the
costblock inmodels.yml. The mantle run reportedtotal: 0.002826114for 18,284 input and 2 output tokens, which is exactly the Qwen Standard rates applied. The block is load-bearing, not decoration: leave zeros in for a self-hosted model, put real rates in for a metered one.Claude Code's
total_cost_usdis fiction against a self-hosted endpoint. It claimed0.1138for a model that costs nothing per token, and reportedcontextWindow: 200000while the server was serving 262,144. This is why the repo derives self-hosted cost from GPU-seconds.DISABLE_NON_ESSENTIAL_MODEL_CALLS=1is load-bearing on the mantle route. Without it the same command emitted[claude-code:unrecognized_model] {"query_source":"generate_session_title"}before the answer, because Claude Code makes side calls that fail noisily on an unrecognized model id. With it:"result":"OK","subtype":"success","num_turns":1.Verification
OK; the vLLM server ran the guide's own flags.docs/faq/README.md.237952compaction threshold, and the0.002826114cost figure.