Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions document/en/modules/chat.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ The stream terminates with `data: [DONE]`. Example frames:
```
data: {"type":"tool_call","tool_name":"internet_search","tool_display_name":"Web Search","tool_args":{"query":"Beijing IC industry"},"tool_id":"call_abc"}

data: {"type":"tool_result","tool_name":"internet_search","result":{...},"tool_id":"call_abc","citations":[{"id":"internet_search-1","title":"...","url":"...","snippet":"...","source_type":"internet"}]}
data: {"type":"tool_result","tool_name":"internet_search","result":{...},"tool_id":"call_abc","citations":[{"id":"e1","title":"...","url":"...","snippet":"...","source_type":"internet","item_index":0}]}

data: {"type":"content","event":"ai_message","delta":"Based on the search results…","chat_id":"chat_x"}

Expand All @@ -104,13 +104,16 @@ event, the frontend replaces the body in place, and the database stores only
the reviewed final answer. It persists `ontology_governance` with the assistant
message so the module remains available after a history refresh.

## Citation system
## Citation system (Evidence Anchors)

Citations make every fact in the answer traceable back to a specific tool result. The chain has three segments:
Citations make every fact in the answer traceable back to a specific tool result. Numbering authority belongs to a single backend source of truth — the model only **copies** ids, never computes them. The chain has four segments:

1. **Prompt contract**: the system prompt (fallback file `prompts/prompt_text/default/system/40_format.system.md`; the active DB version is authoritative at runtime) instructs the model to emit `[ref:tool_name-N]` markers when citing tool data, e.g. `[ref:internet_search-1]`, or `[ref:tool1-N][ref:tool2-M]` for multiple sources.
2. **Backend extraction**: every `tool_result` is normalized by `orchestration/citations.py` into `CitationItem` objects (`id` / `tool_name` / `tool_id` / `title` / `url` / `snippet` / `source_type`). When the same tool is called multiple times in one turn, `extract_citations_with_offset()` keeps ids unique via a per-turn offset table. CE's `_SOURCE_TYPE_MAP` only defines `internet`, `knowledge_base`, and `database`. CE does not infer an industry-specific shape for unknown remote MCP results; the frontend renders them as generic JSON. EE adds its industry citation types separately.
3. **Frontend rendering**: citations ride on `tool_result` and `meta` events and are persisted with the message; `src/frontend/src/utils/citations.ts` parses inline markers with `/\[ref:([\w]+-\d+)\]/g`, `components/citation/CitationBadge.tsx` renders clickable badges, and `CitationMarkdownBlock` / `CitationHtmlBlock` handle in-body display.
1. **Anchor allocation & injection (backend middleware)**: `core/llm/middlewares.py::CitationAnchorMiddleware` hooks AgentScope 2.0's `on_acting` and, before a tool result reaches the model, calls `orchestration/citation_anchor.py` to extract → allocate → inject: each citable item gets a session-monotonic anchor id (`e1`, `e2`, … — unique across tools, calls, and turns; a new turn continues from the max anchor found in the chat's persisted messages), and `"cite_id": "e7"` is written into the result JSON in place (plain-text results get a trailing `[cite_id: e7]` line). **The allocator is bound to the agent instance** (`attach_allocator()` / `resolve_allocator()`), which is how the orchestrator and the middleware share one counter; the ContextVar is only a fallback for sub-agent chains, because `astream_chat_workflow` is an async generator whose context does not reach the task the agent actually runs in. Extraction degrades through four layers: tool-declared `__citations__` → the tool spec registry (`TOOL_SPECS` config: list paths + CN/EN field aliases) → a generic heuristic (unique dict-array field) → the whole result as one anchor. Operational tools (file writes, pin, etc. — `SKIP_TOOLS`) pass through untouched. Any exception passes the original result through — citations degrade, the conversation never breaks.
2. **Prompt contract**: the system prompt (fallback file `prompts/prompt_text/default/system/40_format.system.md`; the active DB version is authoritative at runtime) needs only one tool-count-independent rule: copy the `cite_id` annotated in the result verbatim into `[anchor text](cite:e7)` (or `[来源](cite:e7)` at sentence end); never self-number.
3. **Orchestration consumption**: each `tool_result` event calls `collect_citation_dicts()`, which fetches `CitationItem`s (`id` / `tool_name` / `tool_id` / `title` / `url` / `snippet` / `source_type` / `item_index`) from the allocator registry keyed by `tool_id`; when no allocator is installed (legacy replay paths) it falls back to the old offset extraction in `orchestration/citations.py`. `source_type` values: CE ships `internet`, `knowledge_base`, and `database`; industry citation types such as `industry_news`, `ai_news`, `chain_info`, and `company_profile` are added by the Enterprise Edition (EE).
4. **Frontend rendering**: citations ride on `tool_result` and `meta` events and are persisted with the message (what is persisted is the annotated result, so replay/share shows the same numbering as generation). `components/citation/CitationMarkdownBlock.tsx` recognizes three marker forms in parallel — `[anchor text](cite:eN)` (rendered as a text link with a hover source card), `[[eN]]` (obsidian-style tolerance), and the legacy `[ref:tool_name-N]` (historical messages, rendered as a superscript badge). Tool cards show a matching `cite_id` chip on each item (`jx-tr-citeTag`), and tools without a dedicated renderer fall back to a generic list-card renderer.

**Tool development convention**: a tool (in-house or MCP) that wants precise citation granularity should return a `__citations__` field in its JSON — `[{"title": "...", "url": "...", "snippet": "...", "source_type": "..."}, …]`, entries ordered to match the result body; the middleware adopts it verbatim and injects `cite_id` in place. Tools without the field fall back to registry config or heuristics — at worst the whole result becomes one anchor, so **every tool is citable by default**. See the citation-declaration section in [MCP tools](mcp-tools.md).

## Plan Mode

Expand Down
43 changes: 43 additions & 0 deletions document/en/modules/mcp-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,49 @@ The shared layer (root of `mcp_servers/`):

Two iron rules: **stdout is reserved for the MCP protocol** (business logs go to stderr; server.py wraps calls in `contextlib.redirect_stdout` as a backstop), and **be tolerant of malformed LLM-generated arguments** (e.g. auto-unpacking when a dict lands in a string parameter).

## Declaring citations from a tool (`__citations__`)

The platform's [citation system](chat.md) (evidence anchors) automatically extracts citable
items from every tool result before it reaches the model, allocates session-unique anchors
(`e1`, `e2`, …) and injects `cite_id` back into the result — **every tool is citable by
default**, with zero citation code. Extraction granularity, however, depends on the backend
recognizing your return shape, so when developing a new tool (in-house or MCP) follow this
priority order:

1. **Result should be citable with precise granularity → return a `__citations__` field
in your JSON (recommended)**:

```json
{
"result": "…business payload…",
"__citations__": [
{"title": "Source title", "url": "https://…", "snippet": "key excerpt", "source_type": "internet"},
{"title": "Second source"}
]
}
```

- Entry order matches the result body (`item_index` records the declared order);
- `title` is strongly recommended (falls back to the tool display name); `url` /
`snippet` / `source_type` are optional;
- the middleware (`CitationAnchorMiddleware`) **adopts the declaration verbatim** and
injects `"cite_id": "eN"` into each entry in place; the model copies the id as-is.
2. **Standard list shape → one registry line**: list-style tools returning
`{"items": [{"title": …, "content": …}]}` just need an `items_paths` + field-alias
entry in `orchestration/citation_anchor.py::TOOL_SPECS` — per-item numbering with no
tool code change.
3. **Do nothing → automatic fallback**: a generic heuristic finds the unique dict-array
field at the top level (or one level under `result`) and numbers items; when nothing is
recognizable the whole result becomes a single anchor.
4. **Operational tools (file writes / publish / CRUD receipts) → add to `SKIP_TOOLS`**:
such results have no citation value; registering them avoids pointless anchor noise.

Note: `__citations__` and `cite_id` are **platform-level conventions**, not MCP protocol
fields; third-party MCP servers can use them too (just return JSON). Injection happens
before the result enters the model context and before persistence; the frontend tool cards
render `cite_id` as per-item chips that match the `[anchor text](cite:eN)` references in
the answer body.

## Backend client: connection pool & bare-name restoration

The backend connects through AgentScope 2.0's `MCPClient`, centred on two files:
Expand Down
15 changes: 9 additions & 6 deletions document/zh-CN/modules/chat.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ SSE follower:chat_run_executor.follow_run_as_sse()
```
data: {"type":"tool_call","tool_name":"internet_search","tool_display_name":"联网搜索","tool_args":{"query":"北京 集成电路 产业"},"tool_id":"call_abc"}

data: {"type":"tool_result","tool_name":"internet_search","result":{...},"tool_id":"call_abc","citations":[{"id":"internet_search-1","title":"...","url":"...","snippet":"...","source_type":"internet"}]}
data: {"type":"tool_result","tool_name":"internet_search","result":{...},"tool_id":"call_abc","citations":[{"id":"e1","title":"...","url":"...","snippet":"...","source_type":"internet","item_index":0}]}

data: {"type":"content","event":"ai_message","delta":"根据检索结果……","chat_id":"chat_x"}

Expand All @@ -94,13 +94,16 @@ data: [DONE]

`meta` 之后,`chat_run_executor.py` 持久化助手消息、回填 artifact,并起后台任务生成追问问题(`orchestration/followups.py`,结果写进消息 `extra_data.follow_up_questions`,前端经 `GET /v1/chats/{chat_id}/messages/{message_id}/followups` 拉取)。本体事件在前端汇总为独立的“领域本体治理”模块,不再写入或显示在“思考过程”中。模型草稿保持逐 token 流式展示;委员会仅在实际修订答案时发送一次 `content_replace`,前端原位替换正文,数据库只保存评审后的最终答案。`ontology_governance` 随助手消息持久化,刷新历史会话后仍可回显。

## 引用系统(Citations)
## 引用系统(Citations · 证据锚点

引用让回答里的每个事实可溯源到具体工具结果,链路分三段
引用让回答里的每个事实可溯源到具体工具结果。编号权收归后端唯一真源——模型只**复制**编号、不做任何计算,链路分四段

1. **提示词约定**:系统提示词(`prompts/prompt_text/default/system/40_format.system.md` 的兜底版本,运行时以 DB 激活版本为准)要求模型引用工具数据时输出 `[ref:工具名-序号]` 标记,如 `[ref:internet_search-1]`、多来源并列 `[ref:tool1-N][ref:tool2-M]`。
2. **后端抽取**:每个 `tool_result` 事件经 `orchestration/citations.py` 归一化为 `CitationItem`(`id` / `tool_name` / `tool_id` / `title` / `url` / `snippet` / `source_type`)。同一回合内同一工具被多次调用时,`extract_citations_with_offset()` 用 per-turn 偏移表保证 id 不重复。CE 的 `_SOURCE_TYPE_MAP` 只内置 `internet`、`knowledge_base` 和 `database`;未知远程 MCP 结果不做行业结构猜测,前端统一展示通用 JSON。商业版 EE 另行扩展行业引用类型。
3. **前端渲染**:citations 随 `tool_result` 与 `meta` 事件下发并随消息持久化;`src/frontend/src/utils/citations.ts` 用 `/\[ref:([\w]+-\d+)\]/g` 解析正文标记,`components/citation/CitationBadge.tsx` 渲染为可点击角标,`CitationMarkdownBlock` / `CitationHtmlBlock` 负责正文内嵌展示。
1. **发号回注(后端中间件)**:`core/llm/middlewares.py::CitationAnchorMiddleware` 挂在 AgentScope 2.0 的 `on_acting` 钩子上,在工具结果回给模型前调用 `orchestration/citation_anchor.py` 完成 提取 → 发号 → 回注:为每条可引用条目分配会话内单调唯一的锚点 id(`e1`、`e2`、…,跨工具、跨调用、跨轮不重复,新一轮从该会话历史消息的最大锚点续号),并把 `"cite_id": "e7"` 就地写进结果 JSON(纯文本结果在文末追加 `[cite_id: e7]` 行)。**发号器绑在 agent 实例上**(`attach_allocator()` / `resolve_allocator()`)——编排层与中间件由此共享同一个计数器;ContextVar 只作子智能体链路的兜底,因为 `astream_chat_workflow` 是 async generator,其上下文与 agent 实际执行所在的 task 并不互通。提取按四层降级:工具自声明 `__citations__` → 工具规格注册表(`TOOL_SPECS` 配置,列表路径 + 中英字段别名)→ 通用启发式(唯一字典数组字段)→ 整份结果 1 个锚点;操作型工具(写文件、pin 等,`SKIP_TOOLS`)直接放行。任何异常原样放行、绝不阻断对话。
2. **提示词约定**:系统提示词(`prompts/prompt_text/default/system/40_format.system.md` 的兜底版本,运行时以 DB 激活版本为准)只需一条与工具数量无关的通用规则:把结果里标注的 `cite_id` 原样复制进 `[锚文本](cite:e7)`(或句末 `[来源](cite:e7)`),禁止自行编号。
3. **编排层消费**:每个 `tool_result` 事件经 `collect_citation_dicts()` 按 `tool_id` 从发号器注册表精确取 `CitationItem`(`id` / `tool_name` / `tool_id` / `title` / `url` / `snippet` / `source_type` / `item_index`);发号器缺位(旧对话回放等)时回退 `orchestration/citations.py` 的旧偏移提取。`source_type` 取值:CE 内置 `internet`、`knowledge_base`、`database`;`industry_news`、`ai_news`、`chain_info`、`company_profile` 等行业引用类型由商业版 EE 扩展。
4. **前端渲染**:citations 随 `tool_result` 与 `meta` 事件下发并随消息持久化(落库的就是注号后的结果,回放/分享与生成时编号一致)。`components/citation/CitationMarkdownBlock.tsx` 并行识别三种标记——`[锚文本](cite:eN)`(渲染为带悬浮出处卡片的文字链接)、`[[eN]]`(obsidian 双链容错)、旧格式 `[ref:工具名-序号]`(历史消息,渲染为角标);工具卡片条目上同步显示 `cite_id` 小徽章(`jx-tr-citeTag`),没有专属渲染器的工具由通用列表渲染器兜底成标准卡片。

**工具开发约定**:需要精确控制引用粒度的工具(自研或 MCP),在返回 JSON 里带 `__citations__` 字段——`[{"title": "...", "url": "...", "snippet": "...", "source_type": "..."}, …]`,条目顺序与结果正文对应;中间件优先采用并就地注入 `cite_id`。未自声明的工具按注册表配置或启发式提取,最差整份结果 1 个锚点——**任何工具默认可引用**。详见《[MCP 工具](mcp-tools.md)》的引用声明一节。

## 计划模式(Plan Mode)

Expand Down
35 changes: 35 additions & 0 deletions document/zh-CN/modules/mcp-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,41 @@ mcp_servers/<name>_mcp/

两条铁律:**stdout 保留给 MCP 协议**(业务日志一律走 stderr,server.py 里用 `contextlib.redirect_stdout` 兜底);**对 LLM 生成的畸形参数保持容错**(如 dict 误塞进字符串参数时自动拆包)。

## 工具引用声明(`__citations__`)

平台的[引用系统](chat.md)(证据锚点)会在每个工具结果回给模型前自动提取可引用条目、
分配全会话唯一锚点(`e1`、`e2`、…)并把 `cite_id` 回注进结果——**任何工具默认可引用**,
不写一行引用代码也能工作。但提取粒度取决于后端认不认识你的返回结构,因此开发新工具
(自研工具或 MCP tool)时遵循以下优先级:

1. **结果需要被引用、且希望精确控制粒度 → 返回 JSON 里带 `__citations__` 字段(推荐)**:

```json
{
"result": "……业务数据本体……",
"__citations__": [
{"title": "来源标题", "url": "https://…", "snippet": "关键摘录", "source_type": "internet"},
{"title": "第二个来源"}
]
}
```

- 条目顺序与结果正文对应(`item_index` 按声明顺序记录);
- `title` 必填倾向(缺省回退工具显示名),`url` / `snippet` / `source_type` 可选;
- 中间件(`CitationAnchorMiddleware`)会**优先采用**该声明,就地为每条注入
`"cite_id": "eN"`,模型引用时原样复制。
2. **标准列表结构 → 加一行注册表配置**:结果形如 `{"items": [{"title": …, "content": …}]}`
的列表型工具,在 `orchestration/citation_anchor.py::TOOL_SPECS` 里登记
`items_paths` + 字段别名即可逐条编号,不改工具代码。
3. **什么都不做 → 自动兜底**:通用启发式能识别顶层(或 `result` 下一层)唯一的
字典数组字段并逐条编号;彻底认不出时整份结果作为 1 个锚点。
4. **操作型工具(写文件 / 发布 / 增删改回执)→ 加进 `SKIP_TOOLS`**:这类结果没有
引用价值,登记跳过名单可免去无意义的锚点噪音。

注意:`__citations__` 与 `cite_id` 是**平台层约定**,不是 MCP 协议字段;第三方 MCP
Server 同样适用(返回 JSON 即可)。回注发生在结果进模型上下文与落库之前,前端工具卡片
会把 `cite_id` 渲染成条目徽章,与正文 `[锚文本](cite:eN)` 引用一一对应。

## 后端客户端:连接池与裸名还原

后端基于 AgentScope 2.0 的 `MCPClient` 连接 MCP Server,核心在两个文件:
Expand Down
10 changes: 6 additions & 4 deletions src/backend/api/routes/v1/catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -131,10 +131,12 @@ def _plugin_component_ids(db) -> tuple:
Union of two sources — both are required:
1. **DB install source**: ``AdminSkill/AdminMcpServer.source_plugin`` is non-null —
written dynamically when a user installs a plugin.
2. **Built-in manifest declaration**: ``components`` in
``plugin_bundles/{default,marketplace}/*/plugin.json`` — MCPs of built-in plugins
(e.g. automation / skill-manager) go through ``_ports.py`` → catalog.json and
statically bubble up as first-class entries; the DB has no ``source_plugin`` row
2. **Built-in bundle scan**: skills/MCP provided by
``plugin_bundles/{default,marketplace}/*`` (derived from each bundle's
``skills/*/`` dirs + MCP declarations; the Agent Plugins standard manifest
has no ``components`` list) — MCPs of built-in plugins (e.g. automation /
skill-manager) go through ``_ports.py`` → catalog.json and statically
bubble up as first-class entries; the DB has no ``source_plugin`` row
for them, so source 1 alone cannot remove them.

Filters **display** only; does not affect the enablement resolution of
Expand Down
Loading
Loading