[RizulChauhan]

Summary

This PR hardens perplexity_tools.py and web_tools.py in backend/utils/retrieval/tools/ against unhandled runtime crashes:

1. perplexity_tools.py:

   • _extract_citations IndexError defense: When a model gateway response contains 'choices': [], result.get('choices', [{}])[0] previously raised IndexError: list index out of range. Added defensive length and isinstance checks to return an empty citations list cleanly.
   • AttributeError on None message: When choices[0]['message'] is None or malformed, .get('message', {}).get('citations') raised AttributeError: 'NoneType' object has no attribute 'get'.
   • Input Validation: Added early guard in perplexity_web_search_tool against empty, None, and whitespace-only queries to prevent wasted LLM gateway compute and downstream string concatenation TypeError in hop-2 retry logic (query + WEB_SEARCH_RETRIEVAL_APPENDIX).
   • _model_text & _format_perplexity_response: Defensive unpacking of choices list and message dictionaries.

2. web_tools.py:

   • fetch_url_tool: Added input validation against url=None, non-string, or empty/whitespace URLs, and trims leading/trailing whitespace before protocol verification (startswith(('http://', 'https://'))).

3. Hermetic Test Suite:

   • Added backend/tests/unit/test_search_and_web_tools_resilience.py with 5 focused unit tests covering empty/None queries, malformed choices, nested citations, and URL whitespace trimming.
   • 5/5 tests pass in 0.001s.

Verification

python3 -m unittest backend/tests/unit/test_search_and_web_tools_resilience.py
# Ran 5 tests in 0.001s
# OK

[Git-on-my-level]

The hardening here is correct and well-scoped — nice first contribution; one small non-blocking gap noted at the end.

Per-file findings (all verified on head d46b8c8):

• backend/utils/retrieval/tools/perplexity_tools.py — perplexity_web_search_tool (line ~100): the guard is ordered correctly (None / non-string / whitespace-only all short-circuit before any gateway call) and query.strip() also prevents the downstream query + WEB_SEARCH_RETRIEVAL_APPENDIX TypeError on None queries.
• perplexity_tools.py — _extract_citations (line ~246): the old result.get('choices', [{}])[0].get('message', {}) ... genuinely raised IndexError on "choices": [] and AttributeError on message: None; the new isinstance ladder fixes both while keeping top-level citations/search_results precedence.
• perplexity_tools.py — _model_text / _format_perplexity_response: defensive unpacking is correct; minor note that the try/except in _model_text is now vestigial (nothing inside can raise) — harmless.
• backend/utils/retrieval/tools/web_tools.py — fetch_url_tool (line ~297): same guard plus whitespace trim before the startswith(('http://', 'https://')) check, so padded URLs now fetch cleanly instead of erroring.
• backend/tests/unit/test_search_and_web_tools_resilience.py: 5/5 tests pass — reproduced in a clean sandbox (0.001s, matching your verification). Coverage genuinely exercises the fixed paths (None/empty/whitespace/non-string queries, empty choices, None message, non-dict choice, nested citations, URL trim with mocked _fetch_page), and the sys.modules stubbing matches the established hermetic convention in backend/tests/unit.

Cost: empty queries no longer burn a gateway LLM call (small positive). Security: untrusted web content still wrapped via untrusted_block — unchanged. Blast radius: backend retrieval tools; the pre-fix failure mode was an unhandled exception in the search-tool path.

USER IMPACT: none — internal hardening; users only see fewer failed/empty web-search answers when the search backend returns a malformed response.

Still open (non-blocking): in _format_perplexity_response, a message dict with "content": None renders a literal "None" via str(content) — _model_text guards this case but this path doesn't. A follow-up first_choice['message'].get('content') or '' (or an isinstance check) would close it. Preexisting behavior, not a regression — fine to address in this PR or a follow-up.

---

by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.

[RizulChauhan]

Thanks @Git-on-my-level for the thorough review and approval!

Addressed the non-blocking edge case in _format_perplexity_response: guarded raw_content = first_choice['message'].get('content') with an isinstance(raw_content, str) check so a message with "content": None cleanly resolves to empty string "" rather than the literal string "None". Added corresponding test coverage to backend/tests/unit/test_search_and_web_tools_resilience.py (5/5 tests passing in 0.001s).