[NOT-780] feat(llm): add OrcaRouter as a named LLM provider - #893
[NOT-780] feat(llm): add OrcaRouter as a named LLM provider#893XiaoHuo888-hue wants to merge 1 commit into
Conversation
Mirror the existing OpenRouter wiring so OrcaRouter is selectable as a provider behind the ENABLE_ORCAROUTER env toggle: - LlmProvider.orcarouter + LlmModel.orcarouter + enable_orcarouter() - LlmModel.get_orcarouter_model() maps Notte provider prefixes to the namespaced ids OrcaRouter serves - LLMEngine routes through litellm's OpenAI-compatible path with base_url https://api.orcarouter.ai/v1 and ORCAROUTER_API_KEY - structured_completion uses the OpenAI json_schema wrapper for all OrcaRouter-routed upstreams - tests: config mapping, engine routing, agent model smoke list - .env.example: ORCAROUTER_API_KEY + ENABLE_ORCAROUTER
|
PR author is not in the allowed authors list. |
|
🚨 Contributor flagged. Click here for more info: Superagent Dashboard |
WalkthroughAdds OrcaRouter environment configuration, provider and model definitions, API-key resolution, and model normalization. Updates Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![superagent-security[bot]](https://avatars.githubusercontent.com/in/3287076?s=60&v=4){kind=small}
| temperature = LlmModel.get_temperature(model, temperature) | ||
| completion_kwargs: dict[str, Any] = {} | ||
| if enable_orcarouter(): | ||
| completion_kwargs["base_url"] = ORCAROUTER_BASE_URL |
There was a problem hiding this comment.
P1: Missing ORCAROUTER_API_KEY causes OpenAI API key leakage to third-party endpoint
ENABLE_ORCAROUTER=true without ORCAROUTER_API_KEY leaks OPENAI_API_KEY to api.orcarouter.ai via LiteLLM fallback.
Require ORCAROUTER_API_KEY before setting base_url, or explicitly pass api_key to prevent LiteLLM provider fallback.
AI prompt
Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior.
<file name="packages/notte-llm/src/notte_llm/engine.py">
<violation number="1" location="packages/notte-llm/src/notte_llm/engine.py:586">
<priority>P1</priority>
<title>Missing ORCAROUTER_API_KEY causes OpenAI API key leakage to third-party endpoint</title>
<evidence>When `ENABLE_ORCAROUTER=true` is set but `ORCAROUTER_API_KEY` is missing, the code unconditionally configures `base_url=https://api.orcarouter.ai/v1` while only conditionally adding `api_key` to `completion_kwargs`. Because `_get_model()` prefixes the model with `openai/` for LiteLLM compatibility, LiteLLM's OpenAI provider falls back to the `OPENAI_API_KEY` environment variable when no explicit `api_key` is passed. This causes the user's OpenAI API key to be transmitted to the OrcaRouter endpoint without explicit consent.</evidence>
<recommendation>Validate that `ORCAROUTER_API_KEY` is present before setting `base_url`. Raise a clear `ValueError` or configuration error when `ENABLE_ORCAROUTER=true` but the required API key is missing. For example:
```python
if enable_orcarouter():
orcarouter_api_key = os.environ.get('ORCAROUTER_API_KEY')
if not orcarouter_api_key:
raise ValueError('ORCAROUTER_API_KEY must be set when ENABLE_ORCAROUTER=true')
completion_kwargs['base_url'] = ORCAROUTER_BASE_URL
completion_kwargs['api_key'] = orcarouter_api_key
```</recommendation>
</violation>
</file>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
tests/config/test_orcarouter_provider.py (1)
94-110: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winAssert the supported model ID.
The namespace assertion accepts model IDs that OrcaRouter does not serve. Use
ORCAROUTER_MODELSto assert that each converted result is a supported model ID.Proposed test change
result = LlmModel.get_orcarouter_model(model.value) - assert result.startswith( - ( - "openai/", - "anthropic/", - "google/", - "deepseek/", - "minimax/", - "kimi/", - "grok/", - "z-ai/", - "orcarouter/", - ) - ) + assert result in ORCAROUTER_MODELS🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/config/test_orcarouter_provider.py` around lines 94 - 110, Update test_all_models_can_be_converted_to_orcarouter to assert that each result from LlmModel.get_orcarouter_model(model.value) is contained in ORCAROUTER_MODELS, replacing the broad namespace-prefix assertion while preserving the parameterization over every LlmModel value.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/notte-core/src/notte_core/common/config.py`:
- Around line 163-166: The default LlmModel.orcarouter configuration must not be
advertised when only OpenRouter is enabled, because it is incorrectly routed as
openrouter/orcarouter/auto. Update the model availability or routing logic
around LlmModel.orcarouter and LLMEngine._get_model so it is included only when
ENABLE_ORCAROUTER is enabled, or explicitly route orcarouter/ models through
ORCAROUTER_BASE_URL regardless of the global toggle.
In `@tests/llms/test_engine.py`:
- Around line 63-67: Wrap the completion call and any related assertions in a
try/finally within the ENABLE_ORCAROUTER environment patch, and reset
notte_config._enable_orcarouter to None in the finally block. Remove the
existing trailing reset so cleanup occurs even when the test fails.
In `@tests/llms/test_orcarouter_models.py`:
- Around line 42-47: Update check_orcarouter_available to read
ORCAROUTER_API_KEY and return True only when its value contains non-whitespace
characters; treat missing, empty, and whitespace-only values as unavailable.
---
Nitpick comments:
In `@tests/config/test_orcarouter_provider.py`:
- Around line 94-110: Update test_all_models_can_be_converted_to_orcarouter to
assert that each result from LlmModel.get_orcarouter_model(model.value) is
contained in ORCAROUTER_MODELS, replacing the broad namespace-prefix assertion
while preserving the parameterization over every LlmModel value.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 1b1b0271-361a-4d9c-bdc9-6024295b6764
📒 Files selected for processing (6)
.env.examplepackages/notte-core/src/notte_core/common/config.pypackages/notte-llm/src/notte_llm/engine.pytests/config/test_orcarouter_provider.pytests/llms/test_engine.pytests/llms/test_orcarouter_models.py
| # Auto-routing OrcaRouter model. Prefer a fixed model (e.g. openai/gpt-4o) | ||
| # when using strict structured output, as orcarouter/auto does not | ||
| # guarantee json_schema compliance across all upstreams. | ||
| orcarouter = "orcarouter/auto" |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Do not advertise orcarouter/auto for an OpenRouter-only configuration.
When ENABLE_OPENROUTER=true and ENABLE_ORCAROUTER=false, Line 103 makes LlmProvider.orcarouter.apikey_name return OPENROUTER_API_KEY. LlmModel.valid() then includes this model. LLMEngine._get_model() sends it as openrouter/orcarouter/auto instead of using ORCAROUTER_BASE_URL.
Exclude LlmModel.orcarouter unless OrcaRouter routing is enabled, or route explicit orcarouter/ models through OrcaRouter independent of the global toggle.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/notte-core/src/notte_core/common/config.py` around lines 163 - 166,
The default LlmModel.orcarouter configuration must not be advertised when only
OpenRouter is enabled, because it is incorrectly routed as
openrouter/orcarouter/auto. Update the model availability or routing logic
around LlmModel.orcarouter and LLMEngine._get_model so it is included only when
ENABLE_ORCAROUTER is enabled, or explicitly route orcarouter/ models through
ORCAROUTER_BASE_URL regardless of the global toggle.
| with patch.dict(os.environ, {"ENABLE_ORCAROUTER": "true", "ORCAROUTER_API_KEY": "sk-orca-test"}): | ||
| notte_config._enable_orcarouter = None # Reset cached value | ||
| with patch("litellm.acompletion", return_value=mock_response) as mock_acompletion: | ||
| response = await llm_engine.completion(messages=messages, model=model) | ||
| notte_config._enable_orcarouter = None # Reset cached value |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Reset _enable_orcarouter in a finally block.
If llm_engine.completion() or a later assertion fails, Line 67 does not run. patch.dict restores the environment, but the cached value remains True. Later tests can then route through OrcaRouter unexpectedly.
Proposed fix
with patch.dict(os.environ, {"ENABLE_ORCAROUTER": "true", "ORCAROUTER_API_KEY": "sk-orca-test"}):
notte_config._enable_orcarouter = None # Reset cached value
- with patch("litellm.acompletion", return_value=mock_response) as mock_acompletion:
- response = await llm_engine.completion(messages=messages, model=model)
- notte_config._enable_orcarouter = None # Reset cached value
+ try:
+ with patch("litellm.acompletion", return_value=mock_response) as mock_acompletion:
+ response = await llm_engine.completion(messages=messages, model=model)
+ finally:
+ notte_config._enable_orcarouter = None # Reset cached value📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| with patch.dict(os.environ, {"ENABLE_ORCAROUTER": "true", "ORCAROUTER_API_KEY": "sk-orca-test"}): | |
| notte_config._enable_orcarouter = None # Reset cached value | |
| with patch("litellm.acompletion", return_value=mock_response) as mock_acompletion: | |
| response = await llm_engine.completion(messages=messages, model=model) | |
| notte_config._enable_orcarouter = None # Reset cached value | |
| with patch.dict(os.environ, {"ENABLE_ORCAROUTER": "true", "ORCAROUTER_API_KEY": "sk-orca-test"}): | |
| notte_config._enable_orcarouter = None # Reset cached value | |
| try: | |
| with patch("litellm.acompletion", return_value=mock_response) as mock_acompletion: | |
| response = await llm_engine.completion(messages=messages, model=model) | |
| finally: | |
| notte_config._enable_orcarouter = None # Reset cached value |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/llms/test_engine.py` around lines 63 - 67, Wrap the completion call and
any related assertions in a try/finally within the ENABLE_ORCAROUTER environment
patch, and reset notte_config._enable_orcarouter to None in the finally block.
Remove the existing trailing reset so cleanup occurs even when the test fails.
| def check_orcarouter_available() -> bool: | ||
| """Check if OrcaRouter API key is available. | ||
|
|
||
| Note: Relies on load_dotenv() having been called at module import time. | ||
| """ | ||
| return os.getenv("ORCAROUTER_API_KEY") is not None |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Reject empty OrcaRouter API keys.
An empty ORCAROUTER_API_KEY is not None, so the live test suite runs without usable credentials. Return False for empty or whitespace-only values.
Proposed fix
def check_orcarouter_available() -> bool:
"""Check if OrcaRouter API key is available.
Note: Relies on load_dotenv() having been called at module import time.
"""
- return os.getenv("ORCAROUTER_API_KEY") is not None
+ return bool(os.getenv("ORCAROUTER_API_KEY", "").strip())📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def check_orcarouter_available() -> bool: | |
| """Check if OrcaRouter API key is available. | |
| Note: Relies on load_dotenv() having been called at module import time. | |
| """ | |
| return os.getenv("ORCAROUTER_API_KEY") is not None | |
| def check_orcarouter_available() -> bool: | |
| """Check if OrcaRouter API key is available. | |
| Note: Relies on load_dotenv() having been called at module import time. | |
| """ | |
| return bool(os.getenv("ORCAROUTER_API_KEY", "").strip()) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/llms/test_orcarouter_models.py` around lines 42 - 47, Update
check_orcarouter_available to read ORCAROUTER_API_KEY and return True only when
its value contains non-whitespace characters; treat missing, empty, and
whitespace-only values as unavailable.
giordano-lucas
changed the title
2 participants
Summary
notte-core:LlmProvider.orcarouterenum member plus anenable_orcarouter()toggle gated by theENABLE_ORCAROUTERenv var.LlmModel.orcarouter = "orcarouter/auto"auto-routing model (a fixed model is preferable when strict structured output is required).LlmModel.get_orcarouter_model()that maps Notte's internal provider prefixes to the namespaced ids OrcaRouter serves (e.g.gemini/gemini-2.5-flash→google/gemini-2.5-flash,moonshot/kimi-k2.5→kimi/kimi-k2.5).notte-llm:LLMEngine._get_model()prefixes the mapped id withopenai/so litellm uses its OpenAI-compatible path and forwards the full namespaced id tohttps://api.orcarouter.ai/v1.LLMEngine.completion()passesbase_urlandORCAROUTER_API_KEYwhen OrcaRouter mode is enabled.structured_completion()uses the OpenAIjson_schemawrapper for OrcaRouter-routed models (works across OpenAI, Anthropic, Google and DeepSeek backends)..env.example: addedORCAROUTER_API_KEY=andENABLE_ORCAROUTER=false.tests/config/test_orcarouter_provider.py,tests/llms/test_orcarouter_models.py, and an engine routing test intests/llms/test_engine.py.OrcaRouter
This registers OrcaRouter the same way the existing OpenRouter provider is wired, so the model registry, config toggle and docs stay consistent. OrcaRouter is an OpenAI-compatible gateway: one
ORCAROUTER_API_KEY(keys start withsk-orca-) unlocks 150+ models from OpenAI, Anthropic, Google, DeepSeek, Qwen, MiniMax and xAI behind a singlehttps://api.orcarouter.ai/v1endpoint. It also runs gateway-level, zero-trust security for AI agents on the same endpoint - screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.I'm an engineer on the OrcaRouter team.
Test plan
pytest tests/config/test_orcarouter_provider.py tests/llms/test_engine.py- all passruff checkon the touched filesopenai/gpt-4o,google/gemini-2.5-flash,anthropic/claude-sonnet-4.6,orcarouter/autoall return 200, and structured output (OpenAIjson_schema) succeeds on both OpenAI and Anthropic backends.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Summary by CodeRabbit
New Features
Tests