(feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo

docs/my-website/docs/adding_provider/generic_guardrail_api.md

@@ -0,0 +1,160 @@
+# Generic Guardrail API - Integrate Without a PR
+
+## The Problem
+
+As a guardrail provider, integrating with LiteLLM traditionally requires:
+- Making a PR to the LiteLLM repository
+- Waiting for review and merge
+- Maintaining provider-specific code in LiteLLM's codebase
+- Updating the integration for changes to your API
+
+## The Solution
+
+The **Generic Guardrail API** lets you integrate with LiteLLM **instantly** by implementing a simple API endpoint. No PR required.
+
+### Key Benefits
+
+1. **No PR Needed** - Deploy and integrate immediately
+2. **Universal Support** - Works across ALL LiteLLM endpoints (chat, embeddings, image generation, etc.)
+3. **Simple Contract** - One endpoint, three response types
+4. **Custom Parameters** - Pass provider-specific params via config
+5. **Full Control** - You own and maintain your guardrail API
+
+## How It Works
+
+1. LiteLLM extracts text from any request (chat messages, embeddings, image prompts, etc.)
+2. Sends extracted text + original request to your API endpoint
+3. Your API responds with: `BLOCKED`, `NONE`, or `GUARDRAIL_INTERVENED`
+4. LiteLLM enforces the decision
+
+## API Contract
+
+### Endpoint
+
+Implement `POST /beta/litellm_basic_guardrail_api`
+
+### Request Format
+
+```json
+{
+  "text": "extracted text from the request",
+  "request_body": {},  // full original request for context
+  "additional_provider_specific_params": {
+    // your custom params from config
+  }
+}
+```
+
+### Response Format
+
+```json
+{
+  "action": "BLOCKED" | "NONE" | "GUARDRAIL_INTERVENED",
+  "blocked_reason": "why content was blocked",  // required if action=BLOCKED
+  "text": "modified text"  // required if action=GUARDRAIL_INTERVENED
+}
+```
+
+**Actions:**
+- `BLOCKED` - LiteLLM raises error and blocks request
+- `NONE` - Request proceeds unchanged  
+- `GUARDRAIL_INTERVENED` - Request proceeds with modified text
+
+## LiteLLM Configuration
+
+Add to `config.yaml`:
+
+```yaml
+litellm_settings:
+  guardrails:
+    - guardrail_name: "my-guardrail"
+      litellm_params:
+        guardrail: generic_guardrail_api
+        mode: pre_call  # or post_call, during_call
+        api_base: https://your-guardrail-api.com
+        api_key: os.environ/YOUR_GUARDRAIL_API_KEY  # optional
+        additional_provider_specific_params:
+          # your custom parameters
+          threshold: 0.8
+          language: "en"
+```
+
+## Usage
+
+Users apply your guardrail by name:
+
+```python
+response = client.chat.completions.create(
+    model="gpt-4",
+    messages=[{"role": "user", "content": "hello"}],
+    guardrails=["my-guardrail"]
+)
+```
+
+Or with dynamic parameters:
+
+```python
+response = client.chat.completions.create(
+    model="gpt-4",
+    messages=[{"role": "user", "content": "hello"}],
+    guardrails=[{
+        "my-guardrail": {
+            "extra_body": {
+                "custom_threshold": 0.9
+            }
+        }
+    }]
+)
+```
+
+## Implementation Example
+
+See [mock_bedrock_guardrail_server.py](https://github.com/BerriAI/litellm/blob/main/cookbook/mock_guardrail_server/mock_bedrock_guardrail_server.py) for a complete reference implementation.
+
+**Minimal FastAPI example:**
+
+```python
+from fastapi import FastAPI
+from pydantic import BaseModel
+
+app = FastAPI()
+
+class GuardrailRequest(BaseModel):
+    text: str
+    request_body: dict
+    additional_provider_specific_params: dict
+
+class GuardrailResponse(BaseModel):
+    action: str  # BLOCKED, NONE, or GUARDRAIL_INTERVENED
+    blocked_reason: str | None = None
+    text: str | None = None
+
+@app.post("/beta/litellm_basic_guardrail_api")
+async def apply_guardrail(request: GuardrailRequest):
+    # Your guardrail logic here
+    if "badword" in request.text.lower():
+        return GuardrailResponse(
+            action="BLOCKED",
+            blocked_reason="Content contains prohibited terms"
+        )
+
+    return GuardrailResponse(action="NONE")
+```
+
+## When to Use This
+
+✅ **Use Generic Guardrail API when:**
+- You want instant integration without waiting for PRs
+- You maintain your own guardrail service
+- You need full control over updates and features
+- You want to support all LiteLLM endpoints automatically
+
+❌ **Make a PR when:**
+- You want deeper integration with LiteLLM internals
+- Your guardrail requires complex LiteLLM-specific logic
+- You want to be featured as a built-in provider
+
+## Questions?
+
+This is a **beta API**. We're actively improving it based on feedback. Open an issue or PR if you need additional capabilities.
+

docs/my-website/sidebars.js

@@ -35,6 +35,7 @@ const sidebars = {
           type: "category",
           "label": "Contributing to Guardrails",
           items: [
+            "adding_provider/generic_guardrail_api",
             "adding_provider/simple_guardrail_tutorial",
             "adding_provider/adding_guardrail_support",
           ]

litellm/proxy/_new_secret_config.yaml

@@ -5,17 +5,22 @@ model_list:
       api_key: os.environ/OPENAI_API_KEY
 
 guardrails:
-  - guardrail_name: model-armor-shield
+  # - guardrail_name: model-armor-shield
+  #   litellm_params:
+  #     guardrail: model_armor
+  #     mode: "post_call"  # Run on both input and output
+  #     template_id: "test-prompt-template"  # Required: Your Model Armor template ID
+  #     project_id: "test-vector-store-db"    # Your GCP project ID
+  #     location: "us"          # GCP location (default: us-central1)
+  #     mask_request_content: true       # Enable request content masking
+  #     mask_response_content: true      # Enable response content masking
+  #     fail_on_error: true             # Fail request if Model Armor errors (default: true)
+  #     default_on: true                # Run by default for all requests
+  - guardrail_name: generic-guardrail
     litellm_params:
-      guardrail: model_armor
-      mode: "post_call"  # Run on both input and output
-      template_id: "test-prompt-template"  # Required: Your Model Armor template ID
-      project_id: "test-vector-store-db"    # Your GCP project ID
-      location: "us"          # GCP location (default: us-central1)
-      mask_request_content: true       # Enable request content masking
-      mask_response_content: true      # Enable response content masking
-      fail_on_error: true             # Fail request if Model Armor errors (default: true)
-      default_on: true                # Run by default for all requests
-
-litellm_settings:
-  callbacks: ["arize_phoenix"]
+      guardrail: generic_guardrail_api
+      mode: pre_call
+      headers:
+        Authorization: Bearer mock-bedrock-token-12345
+      api_base: http://localhost:8080
+      default_on: true

litellm/proxy/guardrails/guardrail_hooks/generic_guardrail_api/__init__.py

@@ -0,0 +1,37 @@
+from typing import TYPE_CHECKING
+
+from litellm.types.guardrails import SupportedGuardrailIntegrations
+
+from .generic_guardrail_api import GenericGuardrailAPI
+
+if TYPE_CHECKING:
+    from litellm.types.guardrails import Guardrail, LitellmParams
+
+
+def initialize_guardrail(litellm_params: "LitellmParams", guardrail: "Guardrail"):
+    import litellm
+
+    _generic_guardrail_api_callback = GenericGuardrailAPI(
+        api_base=litellm_params.api_base,
+        api_key=litellm_params.api_key,
+        additional_provider_specific_params=getattr(
+            litellm_params, "additional_provider_specific_params", {}
+        ),
+        guardrail_name=guardrail.get("guardrail_name", ""),
+        event_hook=litellm_params.mode,
+        default_on=litellm_params.default_on,
+    )
+
+    litellm.logging_callback_manager.add_litellm_callback(
+        _generic_guardrail_api_callback
+    )
+    return _generic_guardrail_api_callback
+
+
+guardrail_initializer_registry = {
+    SupportedGuardrailIntegrations.GENERIC_GUARDRAIL_API.value: initialize_guardrail,
+}
+
+guardrail_class_registry = {
+    SupportedGuardrailIntegrations.GENERIC_GUARDRAIL_API.value: GenericGuardrailAPI,
+}

litellm/proxy/guardrails/guardrail_hooks/generic_guardrail_api/example_config.yaml

@@ -0,0 +1,52 @@
+# Example configuration for Generic Guardrail API
+
+model_list:
+  - model_name: gpt-4
+    litellm_params:
+      model: gpt-4
+      api_key: os.environ/OPENAI_API_KEY
+
+litellm_settings:
+  guardrails:
+    - guardrail_name: "my-generic-guardrail"
+      litellm_params:
+        guardrail: generic_guardrail_api
+        mode: pre_call  # Options: pre_call, post_call, during_call, [pre_call, post_call]
+        api_key: os.environ/GENERIC_GUARDRAIL_API_KEY  # Optional if using Bearer auth
+        api_base: http://localhost:8080  # Required. Endpoint /beta/litellm_basic_guardrail_api is automatically appended
+        default_on: false  # Set to true to apply to all requests by default
+        additional_provider_specific_params:
+          # Any additional parameters your guardrail API needs
+          api_version: "v1"
+          custom_param: "value"
+
+# Usage examples:
+
+# 1. Apply guardrail to a specific request:
+# curl --location 'http://localhost:4000/chat/completions' \
+#   --header 'Authorization: Bearer sk-1234' \
+#   --header 'Content-Type: application/json' \
+#   --data '{
+#     "model": "gpt-4",
+#     "messages": [{"role": "user", "content": "Test message"}],
+#     "guardrails": ["my-generic-guardrail"]
+#   }'
+
+# 2. Apply guardrail with dynamic parameters:
+# curl --location 'http://localhost:4000/chat/completions' \
+#   --header 'Authorization: Bearer sk-1234' \
+#   --header 'Content-Type: application/json' \
+#   --data '{
+#     "model": "gpt-4",
+#     "messages": [{"role": "user", "content": "Test message"}],
+#     "guardrails": [
+#       {
+#         "my-generic-guardrail": {
+#           "extra_body": {
+#             "custom_threshold": 0.8
+#           }
+#         }
+#       }
+#     ]
+#   }'
+