Add LiteLLM - support for Sambanova, Vertex AI, Gemini, Anthropic, Bedrock (100+LLMs)

[ishaan-jaff]

This PR adds support for the above mentioned LLMs using LiteLLM https://github.com/BerriAI/litellm/
LiteLLM is a lightweight package to simplify LLM API calls - use any llm as a drop in replacement for gpt-4o.

• Resolves [FEATURE] SambaNova Models Native Support #482

Example

from litellm import completion
import os

## set ENV variables
os.environ["OPENAI_API_KEY"] = "your-openai-key"
os.environ["ANTHROPIC_API_KEY"] = "your-cohere-key"

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="openai/gpt-4o", messages=messages)

# anthropic call
response = completion(model="anthropic/claude-3-sonnet-20240229", messages=messages)
print(response)

Response (OpenAI Format)

{
    "id": "chatcmpl-565d891b-a42e-4c39-8d14-82a1f5208885",
    "created": 1734366691,
    "model": "claude-3-sonnet-20240229",
    "object": "chat.completion",
    "system_fingerprint": null,
    "choices": [
        {
            "finish_reason": "stop",
            "index": 0,
            "message": {
                "content": "Hello! As an AI language model, I don't have feelings, but I'm operating properly and ready to assist you with any questions or tasks you may have. How can I help you today?",
                "role": "assistant",
                "tool_calls": null,
                "function_call": null
            }
        }
    ],
    "usage": {
        "completion_tokens": 43,
        "prompt_tokens": 13,
        "total_tokens": 56,
        "completion_tokens_details": null,
        "prompt_tokens_details": {
            "audio_tokens": null,
            "cached_tokens": 0
        },
        "cache_creation_input_tokens": 0,
        "cache_read_input_tokens": 0
    }
}

[plutoless]

@ishaan-jaff thank you for the PR!
We love and are excited to see litellm added part of our llm extension support list, while we would like to keep the original openai extension, so i would suggest to create a separate extension for litellm.
You may achieve this via 2 approaches,

1. copy the whole openai extension folder, and rename the addon/extension class name and rename the extension registry name in
   

   TEN-Agent/agents/ten_packages/extension/openai_chatgpt_python/addon.py

   Line 15 in 6c873a3

   @register_addon_as_extension("openai_chatgpt_python")

   
   and
   

   TEN-Agent/agents/ten_packages/extension/openai_chatgpt_python/manifest.json

   Line 3 in 6c873a3

   "name": "openai_chatgpt_python",

2. or you can follow the guide in doc to create an extension from scratch.

According to your pr your change is mostly identical to what we have in openai_chat_python. so maybe approach 1 is easier and quicker.