Upgrade legacy Maintenance Copilot offline retriever

[hoang12122]

Scope clarification

This PR upgrades only the legacy MaintenanceCopilot offline retrieval path (askCopilot -> src/lib/ai/rag_engine.py) and its document-preparation script.

It does not replace or upgrade the primary AI stack. The primary runtime remains:

• NemoClawClient as the first-priority local inference gateway;
• the configured stable model nemotron-3-ultra unless overridden by environment/runtime configuration;
• TrustGraph for DocumentRAG, GraphRAG and agent retrieval;
• the Central AI Manager and governance/runtime guards around all main AI requests.

The word “Copilot” here is only the product/UI name of the maintenance widget. It is not GitHub Copilot.

What changed

• Replaced the four-record exact-token demo retriever used by the maintenance widget with a standard-library hybrid BM25 retriever.
• Added Vietnamese accent normalization, metadata-aware ranking, top-k retrieval, bounded confidence, low-confidence refusal and source provenance.
• Added loading from RAG_KNOWLEDGE_PATH, data/vector_db/latest.json and versioned indexes while preserving the existing answer, confidence and engine fields used by askCopilot.
• Upgraded the related document-preparation script with chunking, overlap, stable IDs, SHA-256 hashes, duplicate removal, source metadata, atomic version publishing and a latest.json pointer.
• Added unit tests for retrieval quality, abstention, provenance, chunking, classification, deduplication and invalid documents.
• Added docs/rag_upgrade_audit.md describing weaknesses and recommended improvements for the separate NemoClaw + TrustGraph core path.

Why

The maintenance widget previously counted overlapping words against four hard-coded sample records. Its ingestion helper wrote whole documents to JSON but described the result as a vector index. This caused weak retrieval, misleading confidence, missing citations and poor index lifecycle control in that specific legacy path.

Compatibility

The existing askCopilot server action can continue parsing the response because the historical fields remain present. The response now also includes sources and knowledge_source for auditability.

Validation

python src/lib/ai/test_rag_engine.py
python infra/ai-server/rag/test_document_ingestion_v2.py

Result recorded during implementation: 5 tests passed.

Core RAG follow-up

A separate PR should target the actual NemoClaw + TrustGraph path, including:

• preserving both memory context and retrieved grounding context;
• propagating typed citations and source metadata through askWithRAG;
• treating retrieved content as untrusted evidence to reduce prompt-injection risk;
• adding dynamic retrieval budgets, reranking and a golden RAG evaluation dataset.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6d051043-456c-4f8d-80e3-bf159adcc327

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch agent/rag-quality-upgrade

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}