Skip to content

feat: add env var support for custom endpoint default models #592

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
carbap wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: add env var support for custom endpoint default models #592

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions content/docs/configuration/librechat_yaml/object_structure/custom_endpoint.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ endpoints:
apiKey: "${YOUR_ENV_VAR_KEY}"
baseURL: "https://api.mistral.ai/v1"
models:
default: ["mistral-tiny", "mistral-small", "mistral-medium", "mistral-large-latest"]
# NOTE: a single model or a comma-separated list of models can be specified in YOUR_MISTRAL_MODELS_ENV_VAR
default: ["mistral-tiny", "mistral-small", "mistral-medium", "${YOUR_MISTRAL_MODELS_ENV_VAR}"]
titleConvo: true
titleModel: "mistral-tiny"
modelDisplayLabel: "Mistral"
Expand Down Expand Up @@ -152,7 +153,7 @@ iconURL: https://github.com/danny-avila/LibreChat/raw/main/docs/assets/LibreChat
**Key:**
<OptionTable
options={[
['default', 'Array of Strings', 'An array of strings indicating the default models to use.', 'If fetching models fails, these defaults are used as a fallback.'],
['default', 'Array of Strings', 'An array of strings indicating the default models to use. Entries may be literal model names or ${ENV_VAR} references.', 'If fetching models fails, these defaults are used as a fallback. Env vars that resolve to comma-separated values are expanded into multiple models.'],
]}
/>

Expand All @@ -164,6 +165,7 @@ default:
- "mistral-tiny"
- "mistral-small"
- "mistral-medium"
- "${YOUR_MISTRAL_MODELS_ENV_VAR}"
```

### fetch
Expand Down
8 changes: 8 additions & 0 deletions content/docs/quick_start/custom_endpoints.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,14 @@ endpoints:
titleModel: "current_model"
```

<Callout type="info" title="Models from Environment Variables">

`models.default` can include `${ENV_VAR}` references in addition to literal model names. If an env var resolves to a comma-separated string, LibreChat expands it into multiple model names.

Example: `default: ["mistral-small", "${MISTRAL_MODELS}"]` with `MISTRAL_MODELS="mistral-medium, mistral-large-latest"`.

</Callout>

Browse all compatible providers in the [AI Endpoints](/docs/configuration/librechat_yaml/ai_endpoints) section. For the full field reference, see [Custom Endpoint Object Structure](/docs/configuration/librechat_yaml/object_structure/custom_endpoint).

<Callout type="warning" title="API Key Configuration">
Expand Down