fix(rag): fix document ID Python compatibility and respect defaultConfig limit#1362
fix(rag): fix document ID Python compatibility and respect defaultConfig limit#1362Fruank4 wants to merge 1 commit into
Conversation
…ltConfig limit - Document.generateDocumentId: use getContentText() instead of getContent() so the JSON key contains a plain string matching the Python implementation's _map_text_to_uuid, ensuring Java and Python generate identical UUIDs for the same document content when sharing a vector store. - KnowledgeRetrievalTools.retrieveKnowledge: fall back to defaultConfig.getLimit() instead of hardcoded 5 when the LLM omits the limit parameter, so the limit configured at construction time is honoured. Add tests: DocumentTest.testDocumentIdUsesTextNotContentBlockObject and KnowledgeRetrievalToolsTest covering both limit fallback and explicit override.
24df01e to
e14d687
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
LearningGp
left a comment
There was a problem hiding this comment.
Currently, DocumentMetadata.getContentText() calls content.toString() for non-TextBlock elements. Because these subclasses do not override toString(), this falls back to Object.toString(), which appends the identity hash code (e.g., io.agentscope.core.message.ImageBlock@1f23a4b).
The Impact:
This means that instantiating the same image twice will result in different IDs. This directly violates the Document class Javadoc's promise of a "deterministic UUID for the same content," rendering cross-process and cross-instance deduplication impossible for multimodal documents.
Suggestion:
Restrict the plain-text path exclusively to TextBlock to maintain compatibility with Python. For all other ContentBlock types, we should use Jackson structured serialization. Since these subclasses already have @JsonProperty annotations, they are inherently stable and safe to serialize.
Summary
Document.generateDocumentId: 将getContent()(返回ContentBlock对象)替换为getContentText()(返回纯文本字符串)。原来 Jackson 序列化后的 JSON key 是{"content":{"text":"...","type":"text"}},而 Python 端的_map_text_to_uuid用的是{"content":"..."},导致同一份文档 Java/Python 生成的 UUID 不同,跨语言共用向量库时文档无法正确匹配。KnowledgeRetrievalTools.retrieveKnowledge: LLM 未传limit参数时,从硬编码的5改为使用defaultConfig.getLimit(),使构建时配置的 limit 生效。Test plan
DocumentTest#testDocumentIdUsesTextNotContentBlockObject:验证 ID 基于文本字符串而非 ContentBlock 对象结构生成KnowledgeRetrievalToolsTest#testNullLimitFallsBackToDefaultConfig:验证 null limit 时使用 defaultConfig.getLimit()KnowledgeRetrievalToolsTest#testExplicitLimitOverridesDefault:验证显式传入的 limit 优先于 defaultConfigDocumentTest、KnowledgeTest、ReActAgentRAGConfigTest全部通过🤖 Generated with Claude Code