(feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo #17175
Conversation
Allows guardrail providers to work with litellm for guardrails without needing to make a PR to LiteLLM
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
This is the final PR Bugbot will review for you during this billing cycle
Your free Bugbot reviews will reset on December 25
Details
You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.
To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.
| guardrail_name=guardrail.get("guardrail_name", ""), | ||
| event_hook=litellm_params.mode, | ||
| default_on=litellm_params.default_on, | ||
| ) |
There was a problem hiding this comment.
Bug: Headers from config not passed to GenericGuardrailAPI
The initialize_guardrail function creates a GenericGuardrailAPI instance without passing the headers parameter from the configuration. The GenericGuardrailAPI.__init__ expects headers as a parameter (used for authentication), but only api_base, api_key, and additional_provider_specific_params are passed. Since headers isn't forwarded, any authentication headers (like Authorization: Bearer ...) configured in the YAML will be ignored, and API calls to the external guardrail will fail authentication. The api_key being passed is also unused since GenericGuardrailAPI doesn't have an api_key parameter—it expects authentication via headers.
| text=text, | ||
| request_body={}, | ||
| additional_provider_specific_params=additional_params, | ||
| ) |
There was a problem hiding this comment.
Bug: Request body always empty despite documented context
The apply_guardrail method always passes an empty dict {} for request_body when creating GenericGuardrailAPIRequest, even though request_data is available as a parameter and is already being used to extract dynamic parameters. The documentation states that request_body should contain the "full original request for context", but the external guardrail API will always receive an empty object, preventing it from making context-aware decisions based on the original request data.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| action = "ANONYMIZED" if GUARDRAIL_CONFIG.anonymize_pii else "BLOCKED" | ||
|
|
||
| for pii_type, pattern in GUARDRAIL_CONFIG.pii_patterns.items(): | ||
| matches = re.finditer(pattern, text) |
Check failure
Code scanning / CodeQL
Regular expression injection High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 3 days ago
To fix this vulnerability, you must sanitize all user-supplied regular expression patterns before using them in re.finditer. The recommended approach is to escape the patterns using re.escape, which converts user input into literal strings safe for use in regular expressions (preventing regex metacharacters from having special meaning). The sanitation should occur right before pattern usage, in the loop over GUARDRAIL_CONFIG.pii_patterns.items() in the check_pii function. The change should only affect the usage of pattern so that the functionality (detecting substrings as PII) remains the same, but only literal matches are performed.
What to change:
- In
check_pii, update the assignment ofpatternso thatre.finditerusesre.escape(pattern)instead of the raw user-supplied pattern.
Methods/Imports/Definitions needed:
- Since
reis already imported, no additional imports are needed.
| @@ -249,7 +249,8 @@ | ||
| action = "ANONYMIZED" if GUARDRAIL_CONFIG.anonymize_pii else "BLOCKED" | ||
|
|
||
| for pii_type, pattern in GUARDRAIL_CONFIG.pii_patterns.items(): | ||
| matches = re.finditer(pattern, text) | ||
| safe_pattern = re.escape(pattern) | ||
| matches = re.finditer(safe_pattern, text) | ||
| for match in matches: | ||
| matched_text = match.group() | ||
| pii_entities.append( |
Title
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes
Note
Adds a Generic Guardrail API hook enabling external providers via a simple HTTP endpoint, plus a mock server, docs, and example configs.
generic_guardrail_apihook with request/response models and async HTTP client (generic_guardrail_api.py,__init__.py).SupportedGuardrailIntegrationsand extendsLitellmParamswithadditional_provider_specific_params.POST /beta/litellm_basic_guardrail_apifor the Generic Guardrail API, plus config and health endpoints.example_config.yaml) and updates_new_secret_config.yamlto demonstrate usinggeneric_guardrail_apiwith headers andapi_base.adding_provider/generic_guardrail_api.mdand sidebar entry explaining the API contract, usage, and integration steps.Written by Cursor Bugbot for commit f076e98. This will update automatically on new commits. Configure here.