@@ -7,7 +7,7 @@ ALL Bedrock models (Anthropic, Meta, Deepseek, Mistral, Amazon, etc.) are Suppor
77| Property | Details |
88| -------| -------|
99| Description | Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs). |
10- | Provider Route on LiteLLM | ` bedrock/ ` , [ ` bedrock/converse/ ` ] ( #set-converse--invoke-route ) , [ ` bedrock/invoke/ ` ] ( #set-invoke-route ) , [ ` bedrock/converse_like/ ` ] ( #calling-via-internal-proxy ) , [ ` bedrock/llama/ ` ] ( #deepseek-not-r1 ) , [ ` bedrock/deepseek_r1/ ` ] ( #deepseek-r1 ) , [ ` bedrock/qwen3/ ` ] ( #qwen3-imported-models ) |
10+ | Provider Route on LiteLLM | ` bedrock/ ` , [ ` bedrock/converse/ ` ] ( #set-converse--invoke-route ) , [ ` bedrock/invoke/ ` ] ( #set-invoke-route ) , [ ` bedrock/converse_like/ ` ] ( #calling-via-internal-proxy ) , [ ` bedrock/llama/ ` ] ( #deepseek-not-r1 ) , [ ` bedrock/deepseek_r1/ ` ] ( #deepseek-r1 ) , [ ` bedrock/qwen3/ ` ] ( #qwen3-imported-models ) , [ ` bedrock/openai/ ` ] ( ./bedrock_imported.md#openai-compatible-imported-models-qwen-25-vl-etc ) |
1111| Provider Doc | [ Amazon Bedrock ↗] ( https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html ) |
1212| Supported OpenAI Endpoints | ` /chat/completions ` , ` /completions ` , ` /embeddings ` , ` /images/generations ` |
1313| Rerank Endpoint | ` /rerank ` |
@@ -1598,206 +1598,6 @@ curl -X POST 'http://0.0.0.0:4000/chat/completions' \
15981598</Tabs>
15991599
16001600
1601- # # Bedrock Imported Models (Deepseek, Deepseek R1)
1602-
1603- # ## Deepseek R1
1604-
1605- This is a separate route, as the chat template is different.
1606-
1607- | Property | Details |
1608- |----------|---------|
1609- | Provider Route | `bedrock/deepseek_r1/{model_arn}` |
1610- | Provider Documentation | [Bedrock Imported Models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-import-model.html), [Deepseek Bedrock Imported Model](https://aws.amazon.com/blogs/machine-learning/deploy-deepseek-r1-distilled-llama-models-with-amazon-bedrock-custom-model-import/) |
1611-
1612- <Tabs>
1613- <TabItem value="sdk" label="SDK">
1614-
1615- ` ` ` python
1616- from litellm import completion
1617- import os
1618-
1619- response = completion(
1620- model="bedrock/deepseek_r1/arn:aws:bedrock:us-east-1:086734376398:imported-model/r4c4kewx2s0n", # bedrock/deepseek_r1/{your-model-arn}
1621- messages=[{"role": "user", "content": "Tell me a joke"}],
1622- )
1623- ` ` `
1624-
1625- </TabItem>
1626-
1627- <TabItem value="proxy" label="Proxy">
1628-
1629-
1630- **1. Add to config**
1631-
1632- ` ` ` yaml
1633- model_list:
1634- - model_name: DeepSeek-R1-Distill-Llama-70B
1635- litellm_params:
1636- model: bedrock/deepseek_r1/arn:aws:bedrock:us-east-1:086734376398:imported-model/r4c4kewx2s0n
1637-
1638- ` ` `
1639-
1640- **2. Start proxy**
1641-
1642- ` ` ` bash
1643- litellm --config /path/to/config.yaml
1644-
1645- # RUNNING at http://0.0.0.0:4000
1646- ` ` `
1647-
1648- **3. Test it!**
1649-
1650- ` ` ` bash
1651- curl --location 'http://0.0.0.0:4000/chat/completions' \
1652- --header 'Authorization: Bearer sk-1234' \
1653- --header 'Content-Type: application/json' \
1654- --data '{
1655- "model": "DeepSeek-R1-Distill-Llama-70B", # 👈 the 'model_name' in config
1656- "messages": [
1657- {
1658- "role": "user",
1659- "content": "what llm are you"
1660- }
1661- ],
1662- }'
1663- ` ` `
1664-
1665- </TabItem>
1666- </Tabs>
1667-
1668-
1669- # ## Deepseek (not R1)
1670-
1671- | Property | Details |
1672- |----------|---------|
1673- | Provider Route | `bedrock/llama/{model_arn}` |
1674- | Provider Documentation | [Bedrock Imported Models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-import-model.html), [Deepseek Bedrock Imported Model](https://aws.amazon.com/blogs/machine-learning/deploy-deepseek-r1-distilled-llama-models-with-amazon-bedrock-custom-model-import/) |
1675-
1676-
1677-
1678- Use this route to call Bedrock Imported Models that follow the `llama` Invoke Request / Response spec
1679-
1680-
1681- <Tabs>
1682- <TabItem value="sdk" label="SDK">
1683-
1684- ` ` ` python
1685- from litellm import completion
1686- import os
1687-
1688- response = completion(
1689- model="bedrock/llama/arn:aws:bedrock:us-east-1:086734376398:imported-model/r4c4kewx2s0n", # bedrock/llama/{your-model-arn}
1690- messages=[{"role": "user", "content": "Tell me a joke"}],
1691- )
1692- ` ` `
1693-
1694- </TabItem>
1695-
1696- <TabItem value="proxy" label="Proxy">
1697-
1698-
1699- **1. Add to config**
1700-
1701- ` ` ` yaml
1702- model_list:
1703- - model_name: DeepSeek-R1-Distill-Llama-70B
1704- litellm_params:
1705- model: bedrock/llama/arn:aws:bedrock:us-east-1:086734376398:imported-model/r4c4kewx2s0n
1706-
1707- ` ` `
1708-
1709- **2. Start proxy**
1710-
1711- ` ` ` bash
1712- litellm --config /path/to/config.yaml
1713-
1714- # RUNNING at http://0.0.0.0:4000
1715- ` ` `
1716-
1717- **3. Test it!**
1718-
1719- ` ` ` bash
1720- curl --location 'http://0.0.0.0:4000/chat/completions' \
1721- --header 'Authorization: Bearer sk-1234' \
1722- --header 'Content-Type: application/json' \
1723- --data '{
1724- "model": "DeepSeek-R1-Distill-Llama-70B", # 👈 the 'model_name' in config
1725- "messages": [
1726- {
1727- "role": "user",
1728- "content": "what llm are you"
1729- }
1730- ],
1731- }'
1732- ` ` `
1733-
1734- </TabItem>
1735- </Tabs>
1736-
1737- # ## Qwen3 Imported Models
1738-
1739- | Property | Details |
1740- |----------|---------|
1741- | Provider Route | `bedrock/qwen3/{model_arn}` |
1742- | Provider Documentation | [Bedrock Imported Models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-import-model.html), [Qwen3 Models](https://aws.amazon.com/about-aws/whats-new/2025/09/qwen3-models-fully-managed-amazon-bedrock/) |
1743-
1744- <Tabs>
1745- <TabItem value="sdk" label="SDK">
1746-
1747- ` ` ` python
1748- from litellm import completion
1749- import os
1750-
1751- response = completion(
1752- model="bedrock/qwen3/arn:aws:bedrock:us-east-1:086734376398:imported-model/your-qwen3-model", # bedrock/qwen3/{your-model-arn}
1753- messages=[{"role": "user", "content": "Tell me a joke"}],
1754- max_tokens=100,
1755- temperature=0.7
1756- )
1757- ` ` `
1758-
1759- </TabItem>
1760-
1761- <TabItem value="proxy" label="Proxy">
1762-
1763- **1. Add to config**
1764-
1765- ` ` ` yaml
1766- model_list:
1767- - model_name: Qwen3-32B
1768- litellm_params:
1769- model: bedrock/qwen3/arn:aws:bedrock:us-east-1:086734376398:imported-model/your-qwen3-model
1770-
1771- ` ` `
1772-
1773- **2. Start proxy**
1774-
1775- ` ` ` bash
1776- litellm --config /path/to/config.yaml
1777-
1778- # RUNNING at http://0.0.0.0:4000
1779- ` ` `
1780-
1781- **3. Test it!**
1782-
1783- ` ` ` bash
1784- curl --location 'http://0.0.0.0:4000/chat/completions' \
1785- --header 'Authorization: Bearer sk-1234' \
1786- --header 'Content-Type: application/json' \
1787- --data '{
1788- "model": "Qwen3-32B", # 👈 the 'model_name' in config
1789- "messages": [
1790- {
1791- "role": "user",
1792- "content": "what llm are you"
1793- }
1794- ],
1795- }'
1796- ` ` `
1797-
1798- </TabItem>
1799- </Tabs>
1800-
18011601# ## OpenAI GPT OSS
18021602
18031603| Property | Details |
0 commit comments