AWS Bedrock LLM Client support

[MrCsabaToth]

It'd be great to have an AWS Bedrock LLM provider support. This would mean instead of an API Key the framework needs to accept a few AWS environment variables.

Bedrock can provide Open AI API for Open AI OSS models out of the box, and for other models via Lambda or Fargate layers. The best would be native Bedrock support via boto.

[pbelcak]

You can achieve this by customizing the function create_lm_client. For example, using LiteLLM

from litellm import completion

def create_lm_client(model_config: ModelConfig | None = None):
    """
    Create a LiteLLM client wrapper with the specified configuration.

    This function creates a wrapper for the configured LLM provider
    using LiteLLM. The wrapper can be customized with specific model
    configurations or will use the default model from configuration.

    Args:
        model_config: Optional model configuration to override defaults.
                     If None, uses the default model from configuration.

    Returns:
        function: A callable `chat_completion(messages, **kwargs)` 
                  that uses LiteLLM under the hood.

    Example:
        >>> client = create_lm_client()
        >>> response = client([{"role": "user", "content": "Hello!"}])
        >>> print(response["choices"][0]["message"]["content"])
    """
    model_config = model_config or MODEL_CONFIGS[DEFAULT_MODEL]
    api_key = get_api_key(model_config["api_type"])

    # set env var for LiteLLM
    import os
    os.environ[f"{model_config['api_type'].upper()}_API_KEY"] = api_key

    model = model_config["model"]

    def chat_completion(messages, **kwargs):
        return completion(
            model=model,
            messages=messages,
            **kwargs
        )

    return chat_completion

Now, anything that you have an API key for will work through an OpenAI-client-like interface.

[MrCsabaToth]

Actually I realized that currently the OpenAI-client-like models on Bedrock (OSS models) do not fully support tool calling yet. Need to double check, but even in that case the best would be if we could use Claude 3.7 or 4 for plenty of context window and intelligence.
https://dev.to/aws-builders/aws-launches-openai-compatible-api-for-bedrock-and-i-did-some-tests-49cd