Description
ResponseValidator claims to "catch common hallucination signals and enforce quality gates", but hedging matches only produce a warning log and the response is still returned to the user. In an HR/Finance assistant, this means unreliable tool-less answers can still be delivered after being flagged internally.
Component
Steps to Reproduce
- Call
ResponseValidator.validate("I think you probably have 12 days left").
- Observe that the validator logs a warning but still returns the original text.
Expected Behavior
Once the validator identifies strong hallucination/hedging indicators, it should either:
- replace the response with the safe fallback, or
- mark the answer for a retry/tool-only recovery path.
Actual Behavior
The validator passes the risky answer through unchanged.
Environment
- OS: Any
- Python version: Source analysis
- Docker version: N/A
- Chat platform: Any
- Flowise version: Any
Logs
Relevant code paths:
- chat-connector/app/response_validator.py:29-74
Additional Context
This is particularly visible because the class docstring promises stronger protection than the implementation actually provides.
Evidence
Suggested fix
- Convert hedging detection into a fail-closed behavior for tool-grounded intents
- Consider returning the existing safe fallback or triggering a second-pass tool retry
Description
ResponseValidatorclaims to "catch common hallucination signals and enforce quality gates", but hedging matches only produce a warning log and the response is still returned to the user. In an HR/Finance assistant, this means unreliable tool-less answers can still be delivered after being flagged internally.Component
Steps to Reproduce
ResponseValidator.validate("I think you probably have 12 days left").Expected Behavior
Once the validator identifies strong hallucination/hedging indicators, it should either:
Actual Behavior
The validator passes the risky answer through unchanged.
Environment
Logs
Additional Context
This is particularly visible because the class docstring promises stronger protection than the implementation actually provides.
Evidence
Suggested fix