diff --git a/docs/plans/2026-03-21-github-issue-template-design.md b/docs/plans/2026-03-21-github-issue-template-design.md new file mode 100644 index 0000000..822da36 --- /dev/null +++ b/docs/plans/2026-03-21-github-issue-template-design.md @@ -0,0 +1,164 @@ +# GitHub Issue Template Workflow Design + +## 背景 +当前仓库已经具备以下零散能力： +- GitHub / GHE webhook 接入与 payload 归一化 +- task / template / trigger / worker 基础执行链路 +- `安全加密agent`、`des_encrypt`、`github_issue_feedback` 的雏形 +- Git 分支推送与 PR 创建能力 + +但这些能力还没有收敛成一套可稳定执行的 “GitHub issue -> distribution agent -> worker agent -> 回帖 issue” 闭环。现状的主要问题是： +- 模板、触发器、agent、skill 的协议不一致 +- `github_issue_number` 只在 mock 流程中稳定回填，真实 webhook 未闭环 +- `issue distribution agent` 还未成为统一入口 +- 当前 `安全加密agent` 依赖脚本和 prompt 零散拼装，缺少内置模板约束 + +## 真实样本 +2026-03-21 通过 GHE API 拉取 `china/starbucks-asg-api#13` 的内容如下： +- 标题：`安全加密` +- 正文：`安全加密agent，对本项目的phone字段进行安全加密` +- URL：`https://scm.starbucks.com/china/starbucks-asg-api/issues/13` + +该 issue 是本次首个真实验收样本，目标路由结果应为： +- 命中 `github issue template` +- 统一进入 `issue distribution agent` +- 识别为安全加密类需求 +- 分发给 `安全加密agent` +- worker 完成代码提交、推送分支、回帖 task URL 与分支名 + +## 方案选择 +本次采用 “单模板入口 + 固定两阶段链路” 方案。 + +### 方案说明 +1. 新增一个统一模板 `github_issue_template` +2. 模板内固定两个阶段： + - `dispatch_issue` -> `issue distribution agent` + - `process_security_issue` -> `安全加密agent` +3. 所有命中该模板的 GitHub issue 都先进入 distribution stage +4. distribution stage 必须输出结构化分发结果 +5. 当前版本只支持一个 worker agent：`安全加密agent` +6. 当 distribution 识别结果不是安全加密时，先产出明确的“不支持/待扩展”说明，不动态增删 stage + +### 采用该方案的原因 +- 与用户要求一致：统一进入 distribution agent +- 能复用当前引擎，不需要先改动态编排核心 +- 便于未来增加更多 worker agent 时扩展 dispatch 协议 +- 风险可控，适合以 issue #13 为真实样本先落地 + +## 目标能力 +### 1. 统一入口模板 +新增内置模板 `github_issue_template`，作为 GitHub issue 自动任务的标准入口。 + +### 2. 两类 Agent 角色 +- `issue distribution agent` +- `安全加密agent` + +### 3. 分发输出协议 +`issue distribution agent` 必须输出结构化结果，至少包含： +- `selected_agent_role` +- `intent` +- `issue_number` +- `issue_url` +- `repo_full_name` +- `task_title` +- `work_summary` +- `acceptance_criteria` +- `dispatch_reason` + +### 4. worker 执行闭环 +`安全加密agent` 必须严格按对应 skill 执行： +- 基于 `des_encrypt` skill 修改代码 +- 推送远端分支 +- 通过 `github_issue_feedback` skill 回帖 issue + +### 5. 任务可追踪性 +task 需要稳定保留 GitHub issue 关联信息，至少确保： +- `github_issue_number` +- issue URL / repo 信息进入 task 描述或 stage prompt +- task 完成后可拼出 `http://127.0.0.1:3000/tasks/` 形式的任务地址 + +## 关键设计 +### 1. 模板定义 +`github_issue_template` 使用标准 `StageDefinition` 字段： +- `name`: `dispatch_issue` +- `agent_role`: `issue distribution agent` +- `order`: `0` +- `instruction`: 强调读取 GitHub issue 上下文并输出结构化分发结果 + +- `name`: `process_security_issue` +- `agent_role`: `安全加密agent` +- `order`: `1` +- `instruction`: 强调读取 dispatch 输出，仅处理安全加密任务，并回帖 issue + +### 2. Trigger 规则 +GitHub webhook 命中 `github_issue_template` 后： +- 统一创建 task +- 标题使用 issue 标题渲染 +- 描述中保留 issue URL、repo、作者、正文 +- 真实 webhook 路径与 mock webhook 路径都要写入 `github_issue_number` + +### 3. Distribution 行为 +`issue distribution agent` 的职责是识别 issue 意图，不直接改代码。当前支持规则： +- 命中 “安全加密 / encryption / phone 字段加密” 等意图 -> `安全加密agent` +- 其他意图 -> 输出 unsupported 说明，供后续扩 worker agent 时接入 + +### 4. Worker 行为 +`安全加密agent` 的职责是执行，不重新决定路由。它接收的上下文应包含： +- 原始 issue 标题、正文、URL、编号 +- repo 信息 +- distribution 产出的结构化工作单 +- 当前 task_id 与 task_url + +执行完成后必须回帖： +- Git 分支名 +- Silicon Agent task URL + +### 5. Skill 目录与权限 +当前 worker 的技能分布在两个位置： +- 仓库根目录 `skills/des_encrypt/SKILL.md` +- 平台共享技能 `platform/skills/shared/github_issue_feedback/SKILL.md` + +因此需要让角色配置与 skill dir 白名单对齐，确保： +- `issue distribution agent` 能加载 dispatch skill +- `安全加密agent` 能同时加载 `des_encrypt` 与 `github_issue_feedback` + +### 6. 非目标 +本轮不做以下内容： +- 不引入动态 stage 增删 +- 不实现多个 worker agent 的真正分流执行 +- 不要求 issue 回帖后自动关单 +- 不在本轮内重构整套 worker 架构 + +## 失败处理 +- 若 distribution 无法识别意图，task 保持失败或输出明确阻塞原因，不允许假成功 +- 若 worker 未推送成功，不允许回帖成功态 +- 若代码已推送但回帖失败，task 应能从错误日志定位到 comment API 失败原因 + +## 测试策略 +### 1. 单元 / 服务层 +- 内置模板 seed 正确创建 +- 内置 agent seed 正确创建 +- 真实 webhook 创建 task 时能保存 `github_issue_number` +- task 描述中包含 issue 关键上下文 + +### 2. mock webhook +- 构造 GitHub issue payload，命中 `github_issue_template` +- 验证创建出的 task stage 顺序与 agent_role 正确 + +### 3. 真实 issue 样本 +- 使用 GHE API 拉取 `china/starbucks-asg-api#13` +- 以真实 title/body 验证 distribution 识别结果应为 `安全加密agent` + +### 4. 闭环验证 +若当前环境具备仓库访问和推送条件，则继续验证： +- task 被创建 +- worker 进入安全加密 stage +- 分支被记录 +- issue 评论包含 task URL 和 branch + +## 交付结果 +本次设计通过后，实施阶段需要至少交付： +- 一套稳定的 `github_issue_template` +- 一套稳定的 distribution / security worker 协议 +- 一条可用的 GitHub issue 自动触发链路 +- 针对 issue #13 的验证与修复结果 diff --git a/docs/plans/2026-03-21-github-issue-template-workflow.md b/docs/plans/2026-03-21-github-issue-template-workflow.md new file mode 100644 index 0000000..a8bb023 --- /dev/null +++ b/docs/plans/2026-03-21-github-issue-template-workflow.md @@ -0,0 +1,223 @@ +# GitHub Issue Template Workflow Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Build a stable `github_issue_template` workflow that routes every matched GitHub issue through `issue distribution agent`, dispatches security-encryption issues to `安全加密agent`, and posts branch plus Silicon task URL back to the originating issue. + +**Architecture:** Reuse the existing webhook -> trigger -> task -> worker pipeline, but formalize it with one built-in template, seeded agent roles, a fixed two-stage definition, and stable GitHub issue metadata propagation. Keep the first version intentionally static: distribution always runs first, while the security worker remains the only execution agent. + +**Tech Stack:** FastAPI, SQLAlchemy async ORM, Pydantic, existing worker engine, pytest, GitHub Enterprise REST API + +--- + +### Task 1: Lock the spec and plan files into the repo + +**Files:** +- Create: `docs/plans/2026-03-21-github-issue-template-design.md` +- Create: `docs/plans/2026-03-21-github-issue-template-workflow.md` +- Create: `platform/docs/specs/feature-009-GitHubIssue任务分发工作流/01_requirements.md` +- Create: `platform/docs/specs/feature-009-GitHubIssue任务分发工作流/02_interface.md` +- Create: `platform/docs/specs/feature-009-GitHubIssue任务分发工作流/03_implementation.md` + +**Step 1: Verify the new spec files exist** + +Run: `find docs/plans platform/docs/specs/feature-009-GitHubIssue任务分发工作流 -maxdepth 2 -type f | sort` +Expected: all 5 files are listed + +**Step 2: Commit the documentation checkpoint** + +Run: `git add docs/plans/2026-03-21-github-issue-template-design.md docs/plans/2026-03-21-github-issue-template-workflow.md platform/docs/specs/feature-009-GitHubIssue任务分发工作流/01_requirements.md platform/docs/specs/feature-009-GitHubIssue任务分发工作流/02_interface.md platform/docs/specs/feature-009-GitHubIssue任务分发工作流/03_implementation.md && git commit -m "docs: define github issue template workflow"` +Expected: commit created + +### Task 2: Write failing tests for built-in template and seeded agent definitions + +**Files:** +- Modify: `platform/tests/test_template_service.py` +- Modify: `platform/tests/test_agents_api.py` +- Modify: `platform/tests/test_agents.py` +- Modify: `platform/app/services/template_service.py` +- Modify: `platform/app/services/seed_service.py` + +**Step 1: Write the failing template seed test** + +Add a test asserting a built-in template named `github_issue_template` exists and has exactly two ordered stages: +- `dispatch_issue` / `issue distribution agent` +- `process_security_issue` / `安全加密agent` + +**Step 2: Run the focused test to verify it fails** + +Run: `cd platform && pytest tests/test_template_service.py -k github_issue_template -v` +Expected: FAIL because template is not seeded yet + +**Step 3: Write the failing agent seed test** + +Add tests asserting the seeded agent catalog contains: +- `issue distribution agent` +- `安全加密agent` + +And verify their configs include the skill directories and prompt append needed for dispatch / feedback behavior. + +**Step 4: Run the focused agent tests to verify they fail** + +Run: `cd platform && pytest tests/test_agents.py tests/test_agents_api.py -k "issue distribution agent or 安全加密agent" -v` +Expected: FAIL because the roles are not seeded consistently yet + +**Step 5: Implement the minimal seed changes** + +Update the built-in template seed and the built-in agent seed path so the two new roles and the new template are created deterministically. + +**Step 6: Re-run the focused tests** + +Run: `cd platform && pytest tests/test_template_service.py tests/test_agents.py tests/test_agents_api.py -k "github_issue_template or issue distribution agent or 安全加密agent" -v` +Expected: PASS + +**Step 7: Commit** + +Run: `git add platform/app/services/template_service.py platform/app/services/seed_service.py platform/tests/test_template_service.py platform/tests/test_agents.py platform/tests/test_agents_api.py && git commit -m "feat: seed github issue template agents"` + +### Task 3: Write failing tests for GitHub issue metadata propagation + +**Files:** +- Modify: `platform/tests/test_webhook_project.py` +- Modify: `platform/tests/test_mock_webhook.py` +- Modify: `platform/tests/test_trigger_complex.py` +- Modify: `platform/tests/test_task_service.py` +- Modify: `platform/app/api/webhooks/github.py` +- Modify: `platform/app/services/trigger_service.py` +- Modify: `platform/app/services/task_service.py` +- Modify: `platform/app/schemas/task.py` + +**Step 1: Write the failing project-webhook test** + +Add a test asserting a real GitHub issue webhook: +- creates a task +- stores `github_issue_number` +- preserves issue URL and repo context inside task description or structured prompt input + +**Step 2: Write the failing mock-webhook regression test** + +Add a test asserting the same metadata is preserved when using `/mock-webhook`. + +**Step 3: Run the focused webhook tests to verify they fail** + +Run: `cd platform && pytest tests/test_webhook_project.py tests/test_mock_webhook.py -k "issue and github_issue_number" -v` +Expected: FAIL because real webhook flow does not persist all issue metadata yet + +**Step 4: Implement the minimal metadata propagation** + +Update webhook normalization and trigger task creation so `issue_number`, `issue_url`, `repo_full_name`, and issue body flow into task creation consistently. + +**Step 5: Re-run the focused tests** + +Run: `cd platform && pytest tests/test_webhook_project.py tests/test_mock_webhook.py tests/test_task_service.py -k "github_issue_number or issue_url or repo_full_name" -v` +Expected: PASS + +**Step 6: Commit** + +Run: `git add platform/app/api/webhooks/github.py platform/app/services/trigger_service.py platform/app/services/task_service.py platform/app/schemas/task.py platform/tests/test_webhook_project.py platform/tests/test_mock_webhook.py platform/tests/test_trigger_complex.py platform/tests/test_task_service.py && git commit -m "fix: propagate github issue metadata into tasks"` + +### Task 4: Write failing tests for dispatch and worker prompt contracts + +**Files:** +- Modify: `platform/tests/test_prompts.py` +- Modify: `platform/tests/test_worker.py` +- Modify: `platform/app/worker/prompts.py` +- Modify: `platform/app/worker/agents.py` + +**Step 1: Write the failing prompt contract test for distribution** + +Assert that the `dispatch_issue` stage instruction and `issue distribution agent` system prompt require structured dispatch output with `selected_agent_role`, `issue_number`, `repo_full_name`, `work_summary`, and `acceptance_criteria`. + +**Step 2: Write the failing prompt contract test for security worker** + +Assert that `process_security_issue` and `安全加密agent` require: +- strict execution by skill +- git branch push +- GitHub issue feedback with branch and task URL + +**Step 3: Write the failing skill directory / tool exposure test** + +Assert that: +- distribution agent can load shared dispatch skills +- security worker can load both shared feedback skills and the repository-level `des_encrypt` +- security worker has the tool permissions needed for code change and git execution + +**Step 4: Run the focused tests to verify they fail** + +Run: `cd platform && pytest tests/test_prompts.py tests/test_worker.py tests/test_agents.py -k "dispatch_issue or process_security_issue or 安全加密agent" -v` +Expected: FAIL because current prompts and role configuration are incomplete + +**Step 5: Implement the minimal prompt and role updates** + +Update role prompts, stage instructions, and role skill/tool configuration so the contracts become explicit and testable. + +**Step 6: Re-run the focused tests** + +Run: `cd platform && pytest tests/test_prompts.py tests/test_worker.py tests/test_agents.py -k "dispatch_issue or process_security_issue or 安全加密agent" -v` +Expected: PASS + +**Step 7: Commit** + +Run: `git add platform/app/worker/prompts.py platform/app/worker/agents.py platform/tests/test_prompts.py platform/tests/test_worker.py platform/tests/test_agents.py && git commit -m "feat: formalize github issue dispatch contracts"` + +### Task 5: Validate the real issue #13 classification path + +**Files:** +- Modify: `platform/tests/test_integration_service.py` +- Create: `platform/tests/test_github_issue_template_real_sample.py` + +**Step 1: Write a sample-backed test using issue #13 facts** + +Use the known issue facts: +- repo `china/starbucks-asg-api` +- issue number `13` +- title `安全加密` +- body `安全加密agent，对本项目的phone字段进行安全加密` + +Assert the dispatch stage contract would select `安全加密agent`. + +**Step 2: Run the focused test to verify the behavior** + +Run: `cd platform && pytest tests/test_github_issue_template_real_sample.py -v` +Expected: PASS once prompt and metadata contracts are in place + +**Step 3: Commit** + +Run: `git add platform/tests/test_github_issue_template_real_sample.py platform/tests/test_integration_service.py && git commit -m "test: cover github issue template real sample"` + +### Task 6: End-to-end verification in the local environment + +**Files:** +- Modify only if verification exposes a defect in existing implementation + +**Step 1: Run the main targeted backend suite** + +Run: `cd platform && pytest tests/test_template_service.py tests/test_mock_webhook.py tests/test_webhook_project.py tests/test_prompts.py tests/test_worker.py tests/test_agents.py tests/test_github_issue_template_real_sample.py -v` +Expected: PASS + +**Step 2: Start or restart the local services if needed** + +Run: `./skills/start-project-services/scripts/start_services.sh` +Expected: frontend and backend healthy + +**Step 3: Trigger a local mock GitHub issue for the new template** + +Run the project mock-webhook endpoint with a payload matching issue #13 semantics. +Expected: task created with the two expected stages and the correct GitHub metadata + +**Step 4: If credentials and remote repo permissions are valid, run the real issue workflow** + +Expected: +- `issue distribution agent` runs first +- `安全加密agent` receives the dispatch context +- branch name is recorded on the task +- issue comment contains branch + task URL + +**Step 5: If any verification fails, fix with the same TDD loop before closing** + +Run: focused failing pytest target +Expected: PASS after the fix + +**Step 6: Final commit** + +Run: `git add && git commit -m "feat: complete github issue template workflow"` diff --git a/platform/app/api/webhooks/github.py b/platform/app/api/webhooks/github.py index a6a9836..0f0d25f 100644 --- a/platform/app/api/webhooks/github.py +++ b/platform/app/api/webhooks/github.py @@ -7,6 +7,7 @@ from app.config import settings from app.db.session import get_db +from app.services.github_comment_commands import parse_silicon_agent_command from app.services.trigger_service import TriggerService logger = logging.getLogger(__name__) @@ -118,13 +119,13 @@ def _normalize_github_payload(gh_event: str, event_type: str, body: dict) -> dic "title": f"push to {push_branch}", }) - # Issues 事件 elif gh_event == "issues": issue = body.get("issue") or {} labels = [lb.get("name", "") for lb in (issue.get("labels") or [])] base.update({ - "issue_number": issue.get("number", ""), + "issue_number": issue.get("number"), "issue_title": issue.get("title", ""), + "issue_body": issue.get("body", "")[:2000], "issue_url": issue.get("html_url", ""), "issue_author": (issue.get("user") or {}).get("login", ""), "labels": labels, @@ -135,12 +136,20 @@ def _normalize_github_payload(gh_event: str, event_type: str, body: dict) -> dic elif gh_event == "issue_comment": issue = body.get("issue") or {} comment = body.get("comment") or {} + command = parse_silicon_agent_command(comment.get("body", "")) base.update({ - "issue_number": issue.get("number", ""), + "issue_number": issue.get("number"), "issue_title": issue.get("title", ""), - "comment_body": comment.get("body", "")[:200], + "issue_body": issue.get("body", "")[:2000], + "issue_url": issue.get("html_url", ""), + "issue_author": (issue.get("user") or {}).get("login", ""), + "comment_id": comment.get("id", ""), + "comment_url": comment.get("html_url", ""), + "comment_body": comment.get("body", "")[:2000], + "comment_author": (comment.get("user") or {}).get("login", ""), "title": issue.get("title", ""), }) + base.update(command) # 保留原始 body 供模板使用（顶层字段） for k, v in body.items(): diff --git a/platform/app/config.py b/platform/app/config.py index 4074c71..5b9f21d 100644 --- a/platform/app/config.py +++ b/platform/app/config.py @@ -111,6 +111,9 @@ class Settings(BaseSettings): GHE_USERNAME: str = "" GHE_TOKEN: str = "" + # App + APP_BASE_URL: str = "http://127.0.0.1:3000" + # ROI ESTIMATED_HOURS_PER_TASK: float = 8.0 HOURLY_RATE_RMB: float = 150.0 diff --git a/platform/app/models/__init__.py b/platform/app/models/__init__.py index a3c2d49..a5a8cd1 100644 --- a/platform/app/models/__init__.py +++ b/platform/app/models/__init__.py @@ -9,6 +9,7 @@ from app.models.task_log import TaskStageLogModel from app.models.skill_feedback import SkillFeedbackModel from app.models.integration import ProjectIntegrationModel +from app.models.chat_session import ChatSessionModel __all__ = [ "AgentModel", @@ -24,4 +25,5 @@ "TaskStageLogModel", "SkillFeedbackModel", "ProjectIntegrationModel", + "ChatSessionModel", ] diff --git a/platform/app/models/chat_session.py b/platform/app/models/chat_session.py new file mode 100644 index 0000000..19acc4a --- /dev/null +++ b/platform/app/models/chat_session.py @@ -0,0 +1,47 @@ +from __future__ import annotations + +import uuid +from datetime import datetime +from typing import Optional + +from sqlalchemy import JSON, DateTime, ForeignKey, String, func +from sqlalchemy.orm import Mapped, mapped_column, relationship + +from app.db.base import Base + + +class ChatSessionModel(Base): + __tablename__ = "chat_sessions" + + id: Mapped[str] = mapped_column( + String(36), primary_key=True, default=lambda: str(uuid.uuid4()) + ) + title: Mapped[str] = mapped_column(String(200), nullable=False) + status: Mapped[str] = mapped_column( + String(20), nullable=False, default="active", index=True + ) + # The compiled plan so far + plan: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True) + # Array of message objects {"role": "user"|"assistant", "content": "..."} + messages: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True) + + project_id: Mapped[Optional[str]] = mapped_column( + String(36), ForeignKey("projects.id", ondelete="SET NULL"), nullable=True, index=True + ) + # If the session was converted to a task + task_id: Mapped[Optional[str]] = mapped_column( + String(36), ForeignKey("tasks.id", ondelete="SET NULL"), nullable=True + ) + + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + nullable=False, + server_default=func.now(), + onupdate=func.now(), + ) + + project = relationship("ProjectModel", lazy="selectin") + task = relationship("TaskModel", lazy="selectin") diff --git a/platform/app/schemas/chat_session.py b/platform/app/schemas/chat_session.py new file mode 100644 index 0000000..a416956 --- /dev/null +++ b/platform/app/schemas/chat_session.py @@ -0,0 +1,50 @@ +from __future__ import annotations + +from datetime import datetime +from typing import List, Optional + +from pydantic import BaseModel + + +class ChatMessage(BaseModel): + role: str # user, assistant, system + content: str + + +class ChatSessionCreate(BaseModel): + title: str + project_id: Optional[str] = None + + +class ChatSessionMessageRequest(BaseModel): + message: str + + +class ChatSessionApproveRequest(BaseModel): + """ + Approve the plan and convert to a Task. + Optionally passing an overriding final plan or just letting the backend pull it from the session state. + """ + template_id: Optional[str] = None + target_branch: Optional[str] = None + + +class ChatSessionResponse(BaseModel): + id: str + title: str + status: str + project_id: Optional[str] = None + task_id: Optional[str] = None + plan: Optional[dict] = None + messages: Optional[List[ChatMessage]] = None + created_at: datetime + updated_at: datetime + + model_config = {"from_attributes": True} + + +class ChatSessionListResponse(BaseModel): + items: List[ChatSessionResponse] + total: int + page: int + page_size: int diff --git a/platform/app/schemas/task.py b/platform/app/schemas/task.py index 8d8d0d7..091ef1f 100644 --- a/platform/app/schemas/task.py +++ b/platform/app/schemas/task.py @@ -1,6 +1,6 @@ from __future__ import annotations -from datetime import datetime +from datetime import datetime, timezone from typing import List, Optional from pydantic import BaseModel, field_validator @@ -13,6 +13,14 @@ def _validate_auto_target_branch(value: Optional[str]) -> Optional[str]: return None +def _assume_utc_for_naive_datetime(value: Optional[datetime]) -> Optional[datetime]: + if value is None: + return None + if value.tzinfo is None: + return value.replace(tzinfo=timezone.utc) + return value + + class TaskCreateRequest(BaseModel): jira_id: Optional[str] = None title: str @@ -56,6 +64,11 @@ class TaskStageResponse(BaseModel): def _none_to_zero(cls, v: object) -> int: return v if v is not None else 0 + @field_validator("started_at", "completed_at", mode="after") + @classmethod + def _datetime_assume_utc(cls, value: Optional[datetime]) -> Optional[datetime]: + return _assume_utc_for_naive_datetime(value) + model_config = {"from_attributes": True} @@ -90,6 +103,11 @@ def _tokens_none(cls, v: object) -> int: def _cost_none(cls, v: object) -> float: return v if v is not None else 0.0 + @field_validator("created_at", "completed_at", mode="after") + @classmethod + def _datetime_assume_utc(cls, value: Optional[datetime]) -> Optional[datetime]: + return _assume_utc_for_naive_datetime(value) + model_config = {"from_attributes": True} diff --git a/platform/app/services/agent_service.py b/platform/app/services/agent_service.py index 1241b03..412e4f4 100644 --- a/platform/app/services/agent_service.py +++ b/platform/app/services/agent_service.py @@ -29,6 +29,8 @@ ("review", "Review Agent"), ("smoke", "Smoke Test Agent"), ("doc", "Documentation Agent"), + ("dispatch issue", "Issue Distribution"), + ("des encrypt", "Des Encrypt"), ] DEFAULT_AVAILABLE_MODELS = [ @@ -47,6 +49,8 @@ "review": "claude-opus-4-20250514", "smoke": "claude-sonnet-4-20250514", "doc": "claude-sonnet-4-20250514", + "dispatch issue": "claude-sonnet-4-20250514", + "des encrypt": "claude-sonnet-4-20250514", } @@ -65,6 +69,7 @@ async def ensure_agents_exist(self) -> None: await self.session.commit() async def list_agents(self) -> List[AgentStatusResponse]: + await self.ensure_agents_exist() result = await self.session.execute( select(AgentModel).order_by(AgentModel.role) ) diff --git a/platform/app/services/github_comment_commands.py b/platform/app/services/github_comment_commands.py new file mode 100644 index 0000000..27cf116 --- /dev/null +++ b/platform/app/services/github_comment_commands.py @@ -0,0 +1,30 @@ +from __future__ import annotations + +import re + +_COMMAND_PATTERN = re.compile( + r"(?@silicon_agent|/silicon_agent)\b" +) + + +def parse_silicon_agent_command(comment_body: str | None) -> dict[str, str | bool | None]: + """Parse a silicon_agent mention or slash command from an issue comment.""" + text = str(comment_body or "") + match = _COMMAND_PATTERN.search(text) + if match is None: + return { + "silicon_agent_command_triggered": False, + "silicon_agent_command_style": None, + "silicon_agent_command_text": None, + "silicon_agent_command_note": None, + } + + token = match.group("token") + note = f"{text[:match.start()].strip()} {text[match.end():].strip()}".strip() + style = "mention" if token.startswith("@") else "slash" + return { + "silicon_agent_command_triggered": True, + "silicon_agent_command_style": style, + "silicon_agent_command_text": token, + "silicon_agent_command_note": note or None, + } diff --git a/platform/app/services/skill_service.py b/platform/app/services/skill_service.py index 9d35fa3..28a1b21 100644 --- a/platform/app/services/skill_service.py +++ b/platform/app/services/skill_service.py @@ -20,6 +20,24 @@ class SkillService: def __init__(self, session: AsyncSession): self.session = session + async def _hydrate_skill_content(self, skill: SkillModel) -> SkillModel: + if (skill.content or "").strip(): + return skill + + version_result = await self.session.execute( + select(SkillVersionModel) + .where( + SkillVersionModel.skill_id == skill.id, + SkillVersionModel.content.isnot(None), + ) + .order_by(SkillVersionModel.created_at.desc()) + .limit(1) + ) + latest_version = version_result.scalar_one_or_none() + if latest_version and (latest_version.content or "").strip(): + skill.content = latest_version.content + return skill + async def list_skills( self, page: int = 1, @@ -100,6 +118,7 @@ async def get_skill(self, name: str) -> Optional[SkillDetailResponse]: skill = result.scalar_one_or_none() if skill is None: return None + skill = await self._hydrate_skill_content(skill) return SkillDetailResponse.model_validate(skill) async def update_skill(self, name: str, request: SkillUpdateRequest) -> Optional[SkillDetailResponse]: diff --git a/platform/app/services/template_service.py b/platform/app/services/template_service.py index 06948d7..08f2dbd 100644 --- a/platform/app/services/template_service.py +++ b/platform/app/services/template_service.py @@ -85,6 +85,16 @@ "stages": [], "gates": [], }, + { + "name": "github_issue_template", + "display_name": "Github Issue", + "description": "GitHub issue 统一入口模板，先分发再执行", + "stages": [ + {"name": "dispatch_issue", "agent_role": "dispatch issue", "order": 0}, + {"name": "des encrypt", "agent_role": "des encrypt", "order": 1}, + ], + "gates": [], + }, ] @@ -236,6 +246,28 @@ async def seed_builtin_templates(self) -> None: ) self.session.add(template) logger.info("Seeded builtin template: %s", tpl_data["name"]) + continue + + changed = False + expected_stages = json.dumps(tpl_data["stages"], ensure_ascii=False) + expected_gates = json.dumps(tpl_data["gates"], ensure_ascii=False) + if existing.display_name != tpl_data["display_name"]: + existing.display_name = tpl_data["display_name"] + changed = True + if existing.description != tpl_data["description"]: + existing.description = tpl_data["description"] + changed = True + if existing.stages != expected_stages: + existing.stages = expected_stages + changed = True + if existing.gates != expected_gates: + existing.gates = expected_gates + changed = True + if existing.is_builtin is not True: + existing.is_builtin = True + changed = True + if changed: + logger.info("Updated builtin template: %s", tpl_data["name"]) await self.session.commit() @staticmethod diff --git a/platform/app/services/trigger_service.py b/platform/app/services/trigger_service.py index af91bb7..d66d816 100644 --- a/platform/app/services/trigger_service.py +++ b/platform/app/services/trigger_service.py @@ -10,6 +10,7 @@ from app.models.trigger import TriggerEventModel, TriggerRuleModel from app.schemas.task import TaskCreateRequest from app.schemas.trigger import MockWebhookRequest, MockWebhookResponse, TriggerRuleResponse +from app.services.github_comment_commands import parse_silicon_agent_command from app.services.task_service import TaskService logger = logging.getLogger(__name__) @@ -87,7 +88,14 @@ async def process_event( # 3. 创建任务（首条命中，立即返回，不继续评估后续规则） title = _render_template(rule.title_template, payload) - description = _render_template(rule.desc_template or "", payload) or None + rendered_description = _render_template(rule.desc_template or "", payload) or None + description = _build_task_description( + source=source, + event_type=event_type, + payload=payload, + rendered_description=rendered_description, + ) + github_issue_number = _extract_github_issue_number(source, payload) task_service = TaskService(self.session) task = await task_service.create_task(TaskCreateRequest( @@ -95,6 +103,7 @@ async def process_event( description=description, template_id=rule.template_id, project_id=rule.project_id, + github_issue_number=github_issue_number, )) await self._log_event( @@ -444,6 +453,12 @@ def _eval_leaf(node: dict, payload: dict) -> bool: ) return str(author) not in [str(a) for a in (v or [])] + if t == "field_equals": + field = str(node.get("field") or "").strip() + if not field: + return False + return flat.get(field) == v + # 未知类型：放行 return True @@ -498,6 +513,42 @@ def _build_normalized_payload(request: MockWebhookRequest) -> dict: if request.event_type.startswith("issues"): payload["issue"] = issue_or_pr + payload["issue_title"] = request.title + payload["issue_body"] = request.body or "" + payload["issue_author"] = request.author or "mock-user" + elif request.event_type.startswith("issue_comment") or request.event_type.startswith("issue_comment."): + issue_body = "" + issue_url = "" + comment_id = None + comment_url = "" + if request.extra: + issue_body = str(request.extra.get("issue_body") or "") + issue_url = str(request.extra.get("issue_url") or "") + comment_id = request.extra.get("comment_id") + comment_url = str(request.extra.get("comment_url") or "") + + payload["issue"] = { + "title": request.title, + "body": issue_body, + "number": request.number, + "html_url": issue_url, + "user": {"login": request.author or "mock-user"}, + } + payload["comment"] = { + "id": comment_id, + "body": request.body or "", + "html_url": comment_url, + "user": {"login": request.author or "mock-user"}, + } + payload["issue_title"] = request.title + payload["issue_body"] = issue_body + payload["issue_url"] = issue_url + payload["issue_author"] = request.author or "mock-user" + payload["comment_id"] = comment_id + payload["comment_url"] = comment_url + payload["comment_body"] = request.body or "" + payload["comment_author"] = request.author or "mock-user" + payload.update(parse_silicon_agent_command(request.body or "")) elif request.event_type.startswith("pull_request"): issue_or_pr["head"] = {"ref": request.ref or "feature-branch"} issue_or_pr["base"] = {"ref": "main"} @@ -637,3 +688,80 @@ def _passes_filters(filters: dict, payload: dict) -> bool: return False return True + + +def _extract_github_issue_number(source: str, payload: dict) -> int | None: + """Extract a GitHub issue number from normalized or mock payloads.""" + if (source or "").strip().lower() != "github": + return None + + candidates = [ + payload.get("issue_number"), + payload.get("issue", {}).get("number") if isinstance(payload.get("issue"), dict) else None, + payload.get("number"), + ] + for candidate in candidates: + try: + if candidate in (None, ""): + continue + return int(candidate) + except (TypeError, ValueError): + continue + return None + + +def _build_task_description( + *, + source: str, + event_type: str, + payload: dict, + rendered_description: str | None, +) -> str | None: + """Build a task description, falling back to normalized GitHub issue context.""" + description = (rendered_description or "").strip() + if description: + return description + + if (source or "").strip().lower() != "github": + return None + + if "issue" not in (event_type or "").lower(): + return None + + issue_url = str(payload.get("issue_url") or "").strip() + repo_full_name = str(payload.get("repo_full_name") or "").strip() + issue_author = str(payload.get("issue_author") or payload.get("author") or "").strip() + issue_title = str(payload.get("issue_title") or payload.get("title") or "").strip() + issue_body = str(payload.get("issue_body") or "").strip() + comment_body = str(payload.get("comment_body") or "").strip() + comment_url = str(payload.get("comment_url") or "").strip() + comment_author = str(payload.get("comment_author") or "").strip() + command_style = str(payload.get("silicon_agent_command_style") or "").strip() + command_note = str(payload.get("silicon_agent_command_note") or "").strip() + issue_number = _extract_github_issue_number(source, payload) + + parts: list[str] = [] + if issue_number is not None: + parts.append(f"Issue Number: {issue_number}") + if issue_title: + parts.append(f"Issue Title: {issue_title}") + if issue_url: + parts.append(f"Issue URL: {issue_url}") + if repo_full_name: + parts.append(f"Repo: {repo_full_name}") + if issue_author: + parts.append(f"Issue Author: {issue_author}") + if issue_body: + parts.append(f"Body: {issue_body}") + if comment_url: + parts.append(f"Comment URL: {comment_url}") + if comment_author: + parts.append(f"Comment Author: {comment_author}") + if comment_body: + parts.append(f"Comment Body: {comment_body}") + if command_style: + parts.append(f"Command Style: {command_style}") + if command_note: + parts.append(f"Command Note: {command_note}") + + return "\n".join(parts) if parts else None diff --git a/platform/app/worker/agents.py b/platform/app/worker/agents.py index 83baf5e..c729a75 100644 --- a/platform/app/worker/agents.py +++ b/platform/app/worker/agents.py @@ -1,9 +1,12 @@ """Agent pool: create and cache SandboxedAgentRunner instances per (role, task_id).""" from __future__ import annotations +import asyncio import json import logging +import os import tempfile +from contextlib import contextmanager from pathlib import Path from typing import Any @@ -26,6 +29,7 @@ logger = logging.getLogger(__name__) _agents: dict[str, AgentRunner] = {} +_TOOL_ENV_LOCK = asyncio.Lock() # Roles that need more turns for deep exploration / code generation _MAX_TURNS: dict[str, int] = { @@ -33,18 +37,22 @@ "coding": 8, "doc": 5, "test": 8, + "dispatch issue": 5, + "des encrypt": 15, } _DEFAULT_MAX_TURNS = 5 # Per-role tool whitelist (SkillKit built-in: read, write, edit, execute, execute_script, skill) ROLE_TOOLS: dict[str, set[str]] = { "orchestrator": {"read", "execute", "skill"}, - "spec": {"read", "write", "edit", "skill"}, - "coding": {"read", "write", "edit", "execute", "execute_script"}, - "test": {"read", "write", "edit", "execute", "execute_script"}, - "review": {"read", "execute", "skill"}, - "smoke": {"read", "execute", "skill"}, - "doc": {"read", "write", "edit", "skill"}, + "spec": {"read", "write", "edit", "skill"}, + "coding": {"read", "write", "edit", "execute", "execute_script"}, + "test": {"read", "write", "edit", "execute", "execute_script"}, + "review": {"read", "execute", "skill"}, + "smoke": {"read", "execute", "skill"}, + "doc": {"read", "write", "edit", "skill"}, + "dispatch issue": {"read", "execute", "skill"}, + "des encrypt": {"read", "write", "edit", "execute", "execute_script", "skill"}, } _ALL_TOOLS: set[str] = set() _TOOL_ARGUMENT_HINTS: dict[str, str] = {} @@ -56,12 +64,14 @@ _ROLE_SKILL_DIRS: dict[str, list[str]] = { "orchestrator": ["shared", "orchestrator"], - "spec": ["shared", "spec"], - "coding": [], - "test": [], - "review": ["shared", "review"], - "smoke": ["shared", "smoke"], - "doc": ["shared", "doc"], + "spec": ["shared", "spec"], + "coding": [], + "test": [], + "review": ["shared", "review"], + "smoke": ["shared", "smoke"], + "doc": ["shared", "doc"], + "dispatch issue": ["shared"], + "des encrypt": ["shared"], } @@ -305,6 +315,39 @@ def _resolve_max_turns(role: str, override: int | None) -> int: return _MAX_TURNS.get(role, _DEFAULT_MAX_TURNS) +def _build_tool_runtime_env(task_id: str | None) -> dict[str, str]: + env: dict[str, str] = {} + ghe_token = (settings.GHE_TOKEN or "").strip() + ghe_base_url = (settings.GHE_BASE_URL or "").strip() + if ghe_token: + env["GHE_TOKEN"] = ghe_token + if ghe_base_url: + env["GHE_BASE_URL"] = ghe_base_url + if task_id: + env["SILICON_AGENT_TASK_URL"] = f"http://127.0.0.1:3000/tasks/{task_id}" + return env + + +@contextmanager +def _temporary_env(overrides: dict[str, str]): + if not overrides: + yield + return + + previous: dict[str, str | None] = {} + try: + for key, value in overrides.items(): + previous[key] = os.environ.get(key) + os.environ[key] = value + yield + finally: + for key, old_value in previous.items(): + if old_value is None: + os.environ.pop(key, None) + else: + os.environ[key] = old_value + + def _normalize_prompt_append(value: str | None) -> str | None: text = (value or "").strip() return text or None @@ -386,6 +429,7 @@ def __init__(self, *args, default_cwd: str | None = None, self.allowed_tools = allowed_tools or _ALL_TOOLS self._tool_argument_hints = _TOOL_ARGUMENT_HINTS self._gemini_tool_call_signatures: dict[str, str] = {} + self.task_id: str | None = None def _resolve_workspace_path(self, path: str) -> tuple[str, str | None]: """Resolve a possibly-relative path into task workspace safely.""" @@ -460,7 +504,14 @@ async def _wrapped_create(*args, **kwargs): self.client.chat.completions.create = original_create async def _execute_tool_base(self, tool_call, on_output=None) -> str: - return await super()._execute_tool(tool_call, on_output) + tool_name = str(tool_call.get("name") or "").strip().lower() + if tool_name not in {"execute", "execute_script"}: + return await super()._execute_tool(tool_call, on_output) + + runtime_env = _build_tool_runtime_env(getattr(self, "task_id", None)) + async with _TOOL_ENV_LOCK: + with _temporary_env(runtime_env): + return await super()._execute_tool(tool_call, on_output) async def _execute_tool(self, tool_call, on_output=None): return await self._execute_tool_with_policy(tool_call, on_output=on_output) @@ -566,6 +617,7 @@ def _create_runner( default_cwd=str(workdir), allowed_tools=allowed, ) + runner.task_id = task_id configured_model = getattr(runner.config, "model", None) configured_temperature = getattr(runner.config, "temperature", None) configured_max_tokens = getattr(runner.config, "max_tokens", None) diff --git a/platform/app/worker/engine.py b/platform/app/worker/engine.py index 09f12ef..fe829e1 100644 --- a/platform/app/worker/engine.py +++ b/platform/app/worker/engine.py @@ -13,7 +13,6 @@ from datetime import datetime, timezone from pathlib import Path from typing import Any, Dict, List, Optional - from sqlalchemy import select, update from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.orm import selectinload @@ -410,6 +409,34 @@ async def _has_git_worktree_changes(worktree_path: Optional[str]) -> Optional[bo return None +async def _has_git_committed_changes_since_base( + worktree_path: Optional[str], + base_branch: Optional[str], +) -> Optional[bool]: + """Return whether HEAD is ahead of origin/base_branch; None when check cannot be performed.""" + if not worktree_path or not (base_branch or "").strip(): + return None + try: + proc = await asyncio.create_subprocess_shell( + f"git rev-list --count origin/{base_branch.strip()}..HEAD", + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE, + cwd=worktree_path, + ) + stdout, _ = await proc.communicate() + if proc.returncode != 0: + return None + return int(stdout.decode().strip() or "0") > 0 + except Exception: + logger.warning( + "Failed to verify committed git changes for worktree %s against %s", + worktree_path, + base_branch, + exc_info=True, + ) + return None + + async def _ensure_code_stage_has_changes( session: AsyncSession, task: TaskModel, @@ -417,7 +444,8 @@ async def _ensure_code_stage_has_changes( worktree_path: Optional[str], ) -> bool: """Code stage must produce repository changes; otherwise fail fast.""" - if (stage.stage_name or "").lower() != "code": + change_required_stages = {"code", "coding", "des encrypt"} + if (stage.stage_name or "").lower() not in change_required_stages: return True # When worktree mode is disabled/unavailable, skip git diff verification. # Some tasks still complete via sandbox workspace edits without a git worktree. @@ -431,13 +459,22 @@ async def _ensure_code_stage_has_changes( changed = await _has_git_worktree_changes(worktree_path) if changed: return True + if changed is False: + project = getattr(task, "project", None) + committed = await _has_git_committed_changes_since_base( + worktree_path, + getattr(project, "branch", None) if project else None, + ) + if committed: + return True + stage_name = stage.stage_name or "unknown" reason = ( - "Code stage produced no repository file changes." + f"Stage {stage_name} produced no repository file changes." if changed is False - else "Code stage change verification failed (worktree unavailable or git status failed)." + else f"Stage {stage_name} change verification failed (worktree unavailable or git status failed)." ) - logger.error("Task %s code stage has no detectable changes: %s", task.id, reason) + logger.error("Task %s stage %s has no detectable changes: %s", task.id, stage_name, reason) await mark_stage_failed(session, task, stage, reason) from app.worker.agents import close_agents_for_task @@ -2939,7 +2976,7 @@ def _infer_test_target(test_examples: list[str], impl_examples: list[str]) -> st def _build_stage_preflight_summary(stage_name: str, workspace_path: Optional[str]) -> Optional[str]: normalized = (stage_name or "").strip().lower() - if normalized not in {"code", "coding", "test"}: + if normalized not in {"code", "coding", "test", "des encrypt"}: return None if not workspace_path: return None @@ -2997,7 +3034,10 @@ def _build_stage_preflight_summary(stage_name: str, workspace_path: Optional[str validation_command = _infer_validation_command(build_files) lines = [] - if normalized in {"code", "coding"}: + if normalized in {"code", "coding", "des encrypt"}: + lines.append( + "- 当前工作区: 目标仓库已在当前 workspace 根目录检出；直接在这里读写、commit、push，不要再次 git clone 到子目录。" + ) lines.append(_format_preflight_section("构建入口", build_files, limit=2)) lines.append(f"- 推荐修改落点: {_infer_coding_edit_target(source_roots, impl_examples)}") lines.append(_format_preflight_section("最相关实现参考", impl_examples, limit=2)) diff --git a/platform/app/worker/executor.py b/platform/app/worker/executor.py index f5a35b6..3747dd7 100644 --- a/platform/app/worker/executor.py +++ b/platform/app/worker/executor.py @@ -84,6 +84,8 @@ def _build_runtime_overrides( "coding": 6, "doc": 10, "test": 6, + "dispatch issue": 5, + "des encrypt": 15, } _STAGE_MAX_TURN_CAPS: dict[str, int] = { @@ -149,6 +151,10 @@ def _is_signoff_stage(stage_name: str) -> bool: return lowered == "signoff" or "signoff" in lowered or "签收" in (stage_name or "") +def _is_text_only_stage(stage_name: str) -> bool: + return False + + def _output_summary_limit(stage_name: str) -> int: # Cap stage output stored in DB to limit downstream prior-context injection. normalized = (stage_name or "").strip().lower() @@ -204,12 +210,18 @@ def _stage_goal_summary(stage_name: str | None) -> str: return "直接完成最小必要代码修改，并提供最小验证结果。" if normalized == "test": return "直接完成最小、最相关的验证，并明确成功或阻塞结论。" + if normalized == "des encrypt": + return ( + "继续完成加密改造的剩余工作。" + "如果代码文件已创建但 Entity/Mapper 还未修改，请立即修改。" + "如果代码改造已完成，请 git add/commit/push，然后调用 github_issue_feedback skill 回帖。" + ) return "完成当前阶段的最终结果。" def _prefer_restart_continuations(stage_name: str | None) -> bool: normalized = (stage_name or "").strip().lower() - return normalized in {"code", "coding", "test"} + return normalized in {"code", "coding", "test", "des encrypt"} # --------------------------------------------------------------------------- @@ -259,6 +271,12 @@ def _build_continuation_prompt(stage_name: str | None) -> str: "如果验证命令失败，必须直接给出失败命令、关键报错和唯一阻塞点，" "不要再用代码阅读代替测试结论。" ) + if normalized == "des encrypt": + return ( + "请继续完成加密改造。查看上方「已创建/修改的文件」列表，不要重复创建已有文件。" + "如果 Entity/Mapper 还未修改，立即修改；如果代码改造已完成，执行 git add/commit/push。" + "Push 完成后调用 github_issue_feedback skill 回帖。" + ) return "请继续完成上面的输出，从你停下的地方继续。" @@ -277,6 +295,12 @@ def _build_forced_convergence_prompt(stage_name: str | None) -> str: "如果验证命令失败，必须明确给出失败命令、关键报错和唯一阻塞点；" "不要仅凭代码阅读判断测试通过。" ) + if normalized == "des encrypt": + return ( + "轮次即将用完。请立即完成剩余工作：" + "如果代码改造已完成，直接 git add/commit/push 并调用 github_issue_feedback skill 回帖。" + "如果代码改造未完成，只完成最关键的修改，然后 commit/push。" + ) return "请立即收敛到当前阶段的最终结果，不要继续扩展。" @@ -293,7 +317,18 @@ def _build_stage_restart_prompt( stage_name = str(context.get("stage_name") or tracker.stage_name).strip() preflight_summary = str(context.get("preflight_summary") or "").strip() partial_output = _clip_text((output or "").replace("[Max turns reached. Please continue the conversation.]", "").strip(), _RESTART_OUTPUT_CHARS) - tool_digest = _format_tool_digest(tracker.get_completed_tool_runs(), limit=2) + completed_runs = tracker.get_completed_tool_runs() + tool_digest = _format_tool_digest(completed_runs, limit=2) + + # Collect all successfully written files for restart context + written_files: list[str] = [] + for item in completed_runs: + status = str(item.get("status") or "").lower() + command = str(item.get("command") or "") + if status == "success" and command.startswith("write "): + fpath = command[len("write "):].strip() + written_files.append(fpath) + action_prompt = ( _build_forced_convergence_prompt(stage_name) if reason == "forced_convergence" @@ -309,6 +344,8 @@ def _build_stage_restart_prompt( parts.append(_stage_goal_summary(stage_name)) if preflight_summary: parts.append(f"\n## 阶段预扫摘要\n{preflight_summary}") + if written_files: + parts.append("\n## 已创建/修改的文件\n" + "\n".join(f"- {f}" for f in written_files)) if partial_output: parts.append(f"\n## 当前阶段已有部分输出\n{partial_output}") if tool_digest: @@ -693,7 +730,11 @@ async def execute_stage( runtime_overrides = _build_runtime_overrides(agent, stage_model) stage_max_turns = _resolve_stage_max_turns(stage.agent_role, runtime_overrides["max_turns"]) - runner_factory = get_agent_text_only if _is_signoff_stage(stage.stage_name) else get_agent + runner_factory = ( + get_agent_text_only + if _is_signoff_stage(stage.stage_name) or _is_text_only_stage(stage.stage_name) + else get_agent + ) runner = runner_factory( stage.agent_role, task_id, diff --git a/platform/app/worker/prompts.py b/platform/app/worker/prompts.py index 8a739a2..e589732 100644 --- a/platform/app/worker/prompts.py +++ b/platform/app/worker/prompts.py @@ -6,7 +6,7 @@ from pathlib import Path from typing import Dict, List, Optional -_EXECUTION_STAGE_NAMES = {"code", "coding", "test"} +_EXECUTION_STAGE_NAMES = {"code", "coding", "test", "des encrypt"} _EXECUTION_MEMORY_LIMIT = 320 _EXECUTION_REPO_HINT_LIMIT = 720 _EXECUTION_PRIOR_LIMITS = { @@ -29,8 +29,6 @@ # --------------------------------------------------------------------------- _PROMPTS_DIR = Path(__file__).parent / "prompts" - - def _load_prompt(filename: str, fallback: str = "") -> str: """Load a prompt from an external .md file, falling back to *fallback*.""" path = _PROMPTS_DIR / filename @@ -87,6 +85,18 @@ def _load_prompt(filename: str, fallback: str = "") -> str: "你需要生成：API文档、使用说明、变更日志和架构说明。" "文档应清晰、准确、易于理解，面向开发者和使用者。", ), + "dispatch issue": ( + "你是 GitHub Issue 分发 Agent，负责理解 Issue 内容并将任务分发给对应的执行 Agent。\n" + "开始工作前，你必须先调用 `skill` 工具加载 `github_dispatch_issue` skill，然后严格按照 skill 内容执行。\n" + "你只负责分析和分发，不直接修改任何代码。\n" + "输出只包含纯 JSON，不要在 JSON 前后附加任何自然语言叙述或「发往下一阶段」的指令文本。\n" + ), + "des encrypt": ( + "你是安全加密 Agent，负责对数据库的某个字段进行安全加密改造，并在完成后将结果回帖到 GitHub Issue。\n" + "开始工作前，你必须先调用 `skill` 工具分别加载 `des_encrypt` 和 `github_issue_feedback` 两个 skill，然后严格按顺序完成：\n" + "1. **按照 `des_encrypt` skill 执行代码改造**：完成加密代码修改，提交并推送到远端新分支。\n" + "2. **按照 `github_issue_feedback` skill 回帖**：Push 完成后，用 curl 将分支名和任务地址贴回原始 GitHub Issue。\n" + ), } # --------------------------------------------------------------------------- @@ -175,6 +185,21 @@ def _load_prompt(filename: str, fallback: str = "") -> str: "4. 遗留问题清单（如有）\n" "5. 最终签收结论", ), + "dispatch_issue": ( + "先调用 `skill` 工具加载 `github_dispatch_issue`，然后严格按照 skill 内容完成分发。\n" + "只输出 skill 中定义的 JSON Schema 结构化结果（纯 JSON，无 markdown 代码块标记），" + "不要在 JSON 前后附加自然语言总结或对下一阶段的指令。\n" + "不得直接修改任何代码，只负责分析和分发。" + ), + "des encrypt": ( + "接手 dispatch issue 传来的上下文。\n" + "**第一步（必须）**：立即连续调用两次 `skill` 工具，分别加载 `des_encrypt` 和 `github_issue_feedback`。\n" + "然后按顺序完成：\n" + "1. **Coding**：严格按照 `des_encrypt` skill 执行代码改造。目标仓库已在当前 workspace 根目录检出，" + "直接在此读写、commit、push，不要 `git clone` 到子目录。" + "提交前先执行 `git status --short` 确认改动在同一仓库。\n" + "2. **回帖**：Push 完成后，严格按照 `github_issue_feedback` skill，用 curl 将分支名和任务地址回帖到原始 GitHub Issue。\n" + ), } @@ -209,6 +234,14 @@ def _load_prompt(filename: str, fallback: str = "") -> str: "优先复用 test 阶段已经完成的最终验证结果；除非存在明确缺口，不要重复安装依赖、重跑整套测试，" "也不要让宿主环境差异覆盖已在正确环境中验证通过的结论。", ), + "des encrypt": ( + "只完成当前阶段，不要提前执行后续阶段任务。\n" + "当前 task workspace 根目录已经是可提交的目标仓库，请直接在这里修改、提交和推送。\n" + "禁止再次 `git clone` 到子目录，也不要把 read/write/edit/commit 分散到两个不同仓库路径。\n" + "开始提交前必须先在当前 workspace 根目录执行 `git status --short`，确认改动出现在同一个仓库。\n" + "如果 issue 已明确只处理 `phone` 等单一字段，请把改动限制在直接相关的实体、Mapper、必要支撑类和最小验证；不要顺手修改 logback、代码生成器、环境模板或其他无直接关联文件。\n" + "如果没有产生 git 变更，不要伪造完成结果；必须继续定位原因或明确失败点。" + ), } diff --git "a/platform/docs/specs/feature-009-GitHubIssue\344\273\273\345\212\241\345\210\206\345\217\221\345\267\245\344\275\234\346\265\201/01_requirements.md" "b/platform/docs/specs/feature-009-GitHubIssue\344\273\273\345\212\241\345\210\206\345\217\221\345\267\245\344\275\234\346\265\201/01_requirements.md" new file mode 100644 index 0000000..189a88a --- /dev/null +++ "b/platform/docs/specs/feature-009-GitHubIssue\344\273\273\345\212\241\345\210\206\345\217\221\345\267\245\344\275\234\346\265\201/01_requirements.md" @@ -0,0 +1,73 @@ +# feature-009-GitHubIssue任务分发工作流 + +## 1. 背景与目标 +为 Silicon Agent 平台补齐 “GitHub issue 触发任务 -> distribution agent 分发 -> worker agent 执行 -> 回帖 issue” 的标准工作流。当前首个真实落地场景是 GHE 仓库 `china/starbucks-asg-api` 的 issue `#13`，需要把 issue 中的安全加密需求分发给 `des encrypt` 执行。 + +## 2. 真实样本 +2026-03-21 通过 GHE API 获取到： +- 仓库：`china/starbucks-asg-api` +- issue 编号：`13` +- 标题：`安全加密` +- 正文：`安全加密agent，对本项目的phone字段进行安全加密` + +## 3. 用户故事 +1. 作为平台使用者，我希望 GitHub issue 命中 `github issue template` 后，统一先进入 `issue distribution`，由它识别意图并选择执行 agent。 +2. 作为平台维护者，我希望当前版本只接入一个 worker agent：`des encrypt`，但未来可平滑扩展更多 issue worker agent。 +3. 作为 issue 发起人，我希望 worker 执行完成后，GitHub issue 能收到评论，看到生成的 Git 分支和 Silicon Agent task URL。 +4. 作为研发，我希望 task 能保留 issue 号、issue URL、repo 信息，便于排查和回归验证。 +5. 作为仓库协作者，我希望在 issue 评论里通过 `@silicon_agent` 或 `/silicon_agent` 显式触发工作流，而不是只能依赖 issue 创建事件。 + +## 4. 功能范围 +1. 新增内置模板 `github_issue_template`。 +2. 新增或标准化两个 agent 角色： + - `issue distribution` + - `des encrypt` +3. 所有命中该模板的 GitHub issue 都先进入 distribution stage。 +4. distribution stage 基于 issue 内容输出结构化分发结果。 +5. 当前 worker 仅支持把安全加密类 issue 分发给 `des encrypt`。 +6. `des encrypt` 执行完成后，向原始 GitHub issue 回帖： + - Git 分支名 + - Silicon Agent task URL +7. 支持 GitHub issue comment 事件中的 `@silicon_agent` 与 `/silicon_agent` 命令触发，继续复用同一模板与 worker 链路。 + +## 5. 验收标准 +1. GitHub issue 命中 `github_issue_template` 后，task stages 顺序固定为： + - `dispatch_issue` + - `des encrypt` +2. 第一阶段 agent_role 必须是 `issue distribution`。 +3. distribution 产出必须显式包含 `selected_agent_role` 等结构化字段，并能把 issue #13 识别为 `des encrypt`。 +4. 真实 webhook 与 mock webhook 两条路径都必须回填 `github_issue_number`。 +5. task 必须保留 issue URL、repo_full_name、issue body 等关键上下文。 +6. `des encrypt` 完成后必须尝试回帖 issue；评论内容至少包含分支名和 task URL。 +7. 若分支推送失败或评论失败，日志中必须能定位失败原因，不能出现静默成功。 +8. 普通 issue 评论不得触发任务；只有命令评论会命中 trigger。 +9. 当前版本不做评论人权限校验，任何可评论用户均可触发；该行为必须在文档中标记为高风险默认值。 + +## 6. 文件路径 +### 6.1 预计修改文件 +- `app/api/webhooks/github.py` +- `app/services/trigger_service.py` +- `app/services/task_service.py` +- `app/services/template_service.py` +- `app/services/seed_service.py` +- `app/worker/prompts.py` +- `app/worker/agents.py` +- `app/schemas/task.py` +- `tests/test_template_service.py` +- `tests/test_mock_webhook.py` +- `tests/test_webhook_project.py` +- `tests/test_task_service.py` +- `tests/test_prompts.py` +- `tests/test_worker.py` +- `tests/test_agents.py` + +### 6.2 本次 spec 文档 +- `docs/specs/feature-009-GitHubIssue任务分发工作流/01_requirements.md` +- `docs/specs/feature-009-GitHubIssue任务分发工作流/02_interface.md` +- `docs/specs/feature-009-GitHubIssue任务分发工作流/03_implementation.md` + +## 7. 非目标 +1. 本轮不做动态 stage 增删。 +2. 本轮不实现多个 worker agent 的真正并行分发。 +3. 本轮不自动关闭 GitHub issue。 +4. 本轮不重构整套 worker graph。 diff --git "a/platform/docs/specs/feature-009-GitHubIssue\344\273\273\345\212\241\345\210\206\345\217\221\345\267\245\344\275\234\346\265\201/02_interface.md" "b/platform/docs/specs/feature-009-GitHubIssue\344\273\273\345\212\241\345\210\206\345\217\221\345\267\245\344\275\234\346\265\201/02_interface.md" new file mode 100644 index 0000000..ccf245f --- /dev/null +++ "b/platform/docs/specs/feature-009-GitHubIssue\344\273\273\345\212\241\345\210\206\345\217\221\345\267\245\344\275\234\346\265\201/02_interface.md" @@ -0,0 +1,202 @@ +# feature-009-GitHubIssue任务分发工作流 - 接口与数据结构 + +## 1. 相关接口 + +### 1.1 GitHub 项目级 webhook +```http +POST /webhooks/github/{project_id} +``` + +### 1.2 Mock webhook +```http +POST /api/v1/projects/{project_id}/mock-webhook +``` + +### 1.3 查询任务详情 +```http +GET /api/v1/tasks/{task_id} +``` + +### 1.4 查询模板 +```http +GET /api/v1/templates +``` + +## 2. 核心签名 + +### 2.1 GitHub payload 标准化 +```python +# app/api/webhooks/github.py +def _normalize_github_payload(gh_event: str, event_type: str, body: dict) -> dict +``` + +### 2.2 触发器任务创建 +```python +# app/services/trigger_service.py +class TriggerService: + async def process_event( + self, + source: str, + event_type: str, + payload: dict, + project_id: Optional[str] = None, + ) -> Optional[str] +``` + +### 2.3 任务创建 +```python +# app/services/task_service.py +class TaskService: + async def create_task(self, request: TaskCreateRequest) -> TaskDetailResponse +``` + +### 2.4 内置模板 seed +```python +# app/services/template_service.py +class TemplateService: + async def seed_builtin_templates(self) -> None +``` + +## 3. 模板结构 + +### 3.1 github_issue_template +```json +{ + "name": "github_issue_template", + "display_name": "Github Issue", + "description": "GitHub issue 统一入口模板，先分发再执行", + "stages": [ + { + "name": "dispatch_issue", + "agent_role": "issue distribution", + "order": 0, + "instruction": "读取 GitHub issue 上下文并输出结构化分发结果" + }, + { + "name": "des encrypt", + "agent_role": "des encrypt", + "order": 1, + "instruction": "基于 dispatch 产出执行安全加密改造并回帖 GitHub issue" + } + ], + "gates": [] +} +``` + +## 4. Distribution 输出协议 + +### 4.1 结构化字段 +```json +{ + "selected_agent_role": "des encrypt", + "intent": "security_encryption", + "issue_number": 13, + "issue_url": "https://scm.starbucks.com/china/starbucks-asg-api/issues/13", + "repo_full_name": "china/starbucks-asg-api", + "task_title": "处理 GitHub Issue #13: 安全加密", + "work_summary": "对本项目的 phone 字段接入安全加密改造", + "acceptance_criteria": [ + "代码已完成 phone 字段加密接入", + "变更已推送到远端分支", + "Issue 已回帖 task URL 与分支信息" + ], + "dispatch_reason": "issue 标题和正文均明确要求安全加密 agent 处理 phone 字段" +} +``` + +## 5. 任务数据 + +### 5.1 TaskCreateRequest 关键字段 +```python +class TaskCreateRequest(BaseModel): + title: str + description: Optional[str] = None + template_id: Optional[str] = None + project_id: Optional[str] = None + github_issue_number: Optional[int] = None +``` + +### 5.2 TaskDetailResponse 关键字段 +```python +class TaskDetailResponse(BaseModel): + id: str + title: str + description: Optional[str] + github_issue_number: Optional[int] + branch_name: Optional[str] + pr_url: Optional[str] + stages: list[TaskStageResponse] +``` + +## 6. Mock Data + +### 6.1 GitHub issue 标准化 payload +```json +{ + "event_type": "issue_opened", + "repo_full_name": "china/starbucks-asg-api", + "repo_name": "starbucks-asg-api", + "issue_number": 13, + "issue_title": "安全加密", + "issue_body": "安全加密agent，对本项目的phone字段进行安全加密", + "issue_url": "https://scm.starbucks.com/china/starbucks-asg-api/issues/13", + "issue_author": "jowang", + "title": "安全加密", + "author": "jowang", + "labels": [] +} +``` + +### 6.2 GitHub issue comment 命令标准化 payload +```json +{ + "event_type": "issue_comment_created", + "repo_full_name": "china/starbucks-asg-api", + "issue_number": 13, + "issue_title": "安全加密", + "issue_body": "安全加密agent，对本项目的phone字段进行安全加密", + "issue_url": "https://scm.starbucks.com/china/starbucks-asg-api/issues/13", + "comment_id": 1001, + "comment_url": "https://scm.starbucks.com/china/starbucks-asg-api/issues/13#issuecomment-1001", + "comment_body": "@silicon_agent 只处理 phone 字段", + "comment_author": "jowang", + "silicon_agent_command_triggered": true, + "silicon_agent_command_style": "mention", + "silicon_agent_command_text": "@silicon_agent", + "silicon_agent_command_note": "只处理 phone 字段" +} +``` + +### 6.3 任务详情响应 +```json +{ + "id": "task-123", + "title": "处理 GitHub Issue #13: 安全加密", + "description": "Issue URL: https://scm.starbucks.com/china/starbucks-asg-api/issues/13\nRepo: china/starbucks-asg-api\nBody: 安全加密agent，对本项目的phone字段进行安全加密", + "github_issue_number": 13, + "branch_name": "silicon_agent/abc123", + "stages": [ + {"stage_name": "dispatch_issue", "agent_role": "issue distribution"}, + {"stage_name": "des encrypt", "agent_role": "des encrypt"} + ] +} +``` + +### 6.4 GitHub issue comment 触发规则示例 +```json +{ + "source": "github", + "event_type": "issue_comment_created", + "filters": { + "op": "and", + "conditions": [ + { + "type": "field_equals", + "field": "silicon_agent_command_triggered", + "value": true + } + ] + }, + "dedup_key_template": "github:{repo_full_name}:issue_comment:{comment_id}" +} +``` diff --git "a/platform/docs/specs/feature-009-GitHubIssue\344\273\273\345\212\241\345\210\206\345\217\221\345\267\245\344\275\234\346\265\201/03_implementation.md" "b/platform/docs/specs/feature-009-GitHubIssue\344\273\273\345\212\241\345\210\206\345\217\221\345\267\245\344\275\234\346\265\201/03_implementation.md" new file mode 100644 index 0000000..9f7e8b7 --- /dev/null +++ "b/platform/docs/specs/feature-009-GitHubIssue\344\273\273\345\212\241\345\210\206\345\217\221\345\267\245\344\275\234\346\265\201/03_implementation.md" @@ -0,0 +1,114 @@ +# feature-009-GitHubIssue任务分发工作流 - 实现细节 + +## 1. 总体流程 +1. GitHub issue webhook 进入 `/webhooks/github/{project_id}`。 +2. `github.py` 将 issue payload 标准化，包含： + - `issue_number` + - `issue_url` + - `issue_title` + - `issue_body` + - `repo_full_name` +3. GitHub issue comment webhook 进入同一个 `/webhooks/github/{project_id}`，并标准化出： + - `comment_id` + - `comment_body` + - `comment_author` + - `silicon_agent_command_triggered` + - `silicon_agent_command_style` + - `silicon_agent_command_note` +4. `TriggerService.process_event(...)` 匹配到 `github_issue_template` 规则。 +5. `TaskService.create_task(...)` 创建 task，并回填 `github_issue_number`。 +6. 模板自动生成两个 stages： + - `dispatch_issue` + - `des encrypt` +7. worker 执行 `dispatch_issue`，输出结构化分发结果。 +8. worker 执行 `des encrypt`，读取 dispatch 结果后按 `des_encrypt` skill 改代码、推分支、回帖 issue。 + +## 2. 关键改动点 + +### 2.1 模板 seed +在 `app/services/template_service.py` 中新增内置模板： +- `name = "github_issue_template"` +- 固定两阶段 +- 不引入 gate + +### 2.2 Agent seed / 配置 +在 agent seed 路径补齐两个角色： +- `issue distribution` +- `des encrypt` + +要求： +- distribution agent 能加载 shared dispatch skill +- `des encrypt` 能加载 shared feedback skill 与仓库级 `des_encrypt` +- `des encrypt` 拥有执行 git / curl / 改代码所需工具权限 + +### 2.3 webhook 元数据传播 +需要统一真实 webhook 与 mock webhook 的 issue 信息： +- `github_issue_number` +- issue URL +- repo_full_name +- issue body +- comment body +- comment author +- `silicon_agent_command_*` + +其中 `github_issue_number` 不能只在 mock 流程补写，真实 webhook 创建 task 时也必须持久化。 + +### 2.4 trigger filter 扩展 +触发器过滤器需要支持通用字段比较： +- `type = "field_equals"` +- 通过 `_flatten(payload)` 后的字段路径读取值 +- 用于 `silicon_agent_command_triggered == true` 这类 comment command 规则 +### 2.5 prompt 合约 +`app/worker/prompts.py` 中需要把以下语义写死： + +#### distribution stage +- 只能分析和分发，不直接编码 +- 必须输出结构化结果 +- 当前若识别为安全加密类问题，必须选择 `des encrypt` + +#### security worker stage +- 严格依据 `des_encrypt` skill 执行 +- 完成后推送远端分支 +- 使用 `github_issue_feedback` skill 回帖 issue +- 回帖内容至少包含分支名和 task URL + +## 3. issue #13 的预期识别结果 +对于以下输入： +- title: `安全加密` +- body: `安全加密agent，对本项目的phone字段进行安全加密` + +distribution 预期输出： +- `selected_agent_role = "des encrypt"` +- `intent = "security_encryption"` +- `work_summary = "对 phone 字段进行安全加密改造"` + +## 4. 测试实施方式 + +### 4.1 Template / seed +- 验证 `github_issue_template` 被 seed +- 验证 stages 顺序与 role 正确 + +### 4.2 webhook / task +- 验证真实 webhook 创建的 task 含 `github_issue_number` +- 验证 description 中保留 issue URL 与 repo 信息 +- 验证 `issue_comment_created` 的命令评论会创建 task，普通评论不会 +- 验证 description 中保留 comment body、command style、command note + +### 4.3 prompt / contract +- 验证 distribution prompt 明确要求结构化 dispatch 输出 +- 验证 security worker prompt 明确要求回帖 GitHub issue + +### 4.4 真实样本 +- 用 issue #13 的真实 title/body 做一个回归测试样本 + +## 5. 风险与边界 +1. 当前 worker graph 仍是固定 stage 流程，不支持真正动态分配 stage。 +2. `des_encrypt` skill 位于仓库根目录，需确保角色可见性和路径白名单一致。 +3. GHE 评论回帖依赖 `GHE_TOKEN` 可用性与 repo 权限。 +4. 若远端仓库无法推送，闭环验证只能做到“task 创建 + prompt 合同 + mock 验证”。 + +## 6. 回滚策略 +若上线后发现新模板影响现有 webhook： +1. 禁用对应 trigger rule +2. 保留模板但从 UI 或 seed 中移除默认规则绑定 +3. 不影响其他已有模板与非 GitHub 触发器 diff --git a/platform/scripts/seed_github_security_agent.py b/platform/scripts/seed_github_security_agent.py new file mode 100644 index 0000000..537457b --- /dev/null +++ b/platform/scripts/seed_github_security_agent.py @@ -0,0 +1,146 @@ +import asyncio +import uuid +from sqlalchemy import select +from sqlalchemy.ext.asyncio import create_async_engine, async_sessionmaker +import json +import sys + +from app.models.agent import AgentModel +from app.models.skill import SkillModel, SkillVersionModel +from app.models.template import TaskTemplateModel +from app.models.trigger import TriggerRuleModel +from app.config import settings +import os + +async def main(): + # Read SKILL.md from the skills directory + skill_file_path = os.path.join(os.path.dirname(__file__), '../../skills/des_encrypt/SKILL.md') + with open(skill_file_path, 'r', encoding='utf-8') as f: + skill_markdown_content = f.read() + + # Setup DB connection + engine = create_async_engine(settings.DATABASE_URL) + async_session = async_sessionmaker(engine, expire_on_commit=False) + + async with async_session() as session: + # 1. Create or verify 'des_encrypt' skill + result = await session.execute(select(SkillModel).where(SkillModel.name == 'des_encrypt')) + skill = result.scalars().first() + + if not skill: + skill_id = str(uuid.uuid4()) + skill = SkillModel( + id=skill_id, + name="des_encrypt", + display_name="DES Encryption Skill", + description="Provides DES encryption utilities. Useful when writing security-related code.", + layer="L1", + tags=["security", "encryption"], + status="active", + version="1.0.0" + ) + + skill_version = SkillVersionModel( + id=str(uuid.uuid4()), + skill_id=skill_id, + version="1.0.0", + content=skill_markdown_content, + change_summary="Initial commit for DES encryption skill parsed from CLAUDE.md" + ) + session.add(skill) + session.add(skill_version) + print(f"Created skill: des_encrypt ({skill_id})") + else: + print(f"Skill des_encrypt already exists ({skill.id}), updating content.") + result_v = await session.execute(select(SkillVersionModel).where(SkillVersionModel.skill_id == skill.id).order_by(SkillVersionModel.created_at.desc())) + latest_version = result_v.scalars().first() + if latest_version: + latest_version.content = skill_markdown_content + else: + skill_version = SkillVersionModel( + id=str(uuid.uuid4()), + skill_id=skill.id, + version="1.0.0", + content=skill_markdown_content, + change_summary="Update DES encryption skill content" + ) + session.add(skill_version) + + # 2. Create Agent "des encrypt" + result = await session.execute(select(AgentModel).where(AgentModel.role == 'des encrypt')) + agent = result.scalars().first() + + if not agent: + agent = AgentModel( + id=str(uuid.uuid4()), + role="des encrypt", + display_name="Des Encrypt", + status="idle", + model_name=settings.LLM_MODEL, + config={ + "skills": ["des_encrypt", "github-issue", "github-repo-manager"], + "system_prompt": "You are the des encrypt agent. Read GitHub issues about encryption work, update code with the des_encrypt skill, push a remote branch, and then update the issue status." + } + ) + session.add(agent) + print(f"Created agent: {agent.role} ({agent.id})") + else: + print(f"Agent {agent.role} already exists ({agent.id})") + + # 3. Create Task Template for GitHub issues + template_name = "github_security_issue_handler" + result = await session.execute(select(TaskTemplateModel).where(TaskTemplateModel.name == template_name)) + template = result.scalars().first() + + if not template: + template = TaskTemplateModel( + id=str(uuid.uuid4()), + name=template_name, + display_name="GitHub Security Issue Auto-Encryption", + description="Template for processing GitHub issues for security encryption requests.", + is_builtin=False, + stages=json.dumps([ + { + "stage_name": "des encrypt", + "agent_role": "des encrypt" + } + ]), + gates="[]" + ) + session.add(template) + print(f"Created task template: {template.name} ({template.id})") + else: + print(f"Template {template.name} already exists ({template.id})") + + # 4. Create Trigger Rule + trigger_name = "github_issue_security" + result = await session.execute(select(TriggerRuleModel).where(TriggerRuleModel.name == trigger_name)) + trigger = result.scalars().first() + + if not trigger: + trigger = TriggerRuleModel( + id=str(uuid.uuid4()), + name=trigger_name, + source="github", + event_type="issue_created", + filters={ + "title_contains": "encrypt" # Simplified filter, can be customized + }, + template_id=template.id, + title_template="处理 GitHub 安全加密 Issue: {issue_title}", + desc_template="URL: {issue_url}\\nBody: {issue_body}", + dedup_key_template="github:issue:{issue_number}", + dedup_window_hours=24, + enabled=True + ) + session.add(trigger) + print(f"Created trigger: {trigger.name} ({trigger.id})") + else: + print(f"Trigger {trigger.name} already exists ({trigger.id})") + + await session.commit() + print("Database seeding completed securely.") + +if __name__ == "__main__": + # Workaround for running from asyncio script without throwing an error if event loop is running + asyncio.run(main()) diff --git a/platform/skills/shared/des_encrypt/SKILL.md b/platform/skills/shared/des_encrypt/SKILL.md new file mode 100644 index 0000000..4832e70 --- /dev/null +++ b/platform/skills/shared/des_encrypt/SKILL.md @@ -0,0 +1,222 @@ +# DES 安全加密接入 Skill + +为 Spring Boot + MyBatis 项目接入国安 DES (Data Encryption Service) 加密服务，对数据库中的敏感字段（手机号、邮箱等）进行 SM4/GCM 加密存储。 + +--- + +## 使用方式 + +``` +/des-encrypt +``` + +执行后 Claude 会引导你完成以下流程。你只需回答几个问题即可。 + +--- + +## 前置条件 + +- Spring Boot 项目（2.x 或 3.x），使用 MyBatis XML Mapper +- 已从 DES 团队获取：quickapi-client-java JAR 包、密钥 ID、加密服务 IP、证书文件 +- 已明确需要加密的表和字段 + +## GitHub Issue 最小改造模式 + +当该技能由 GitHub Issue 工作流触发，且 issue 只给出明确字段范围（例如“对 `phone` 字段进行安全加密”），但没有提供完整的 DES 基础设施信息时，优先采用**最小改造模式**： + +1. 只围绕 issue 指定的字段落地最小可交付闭环。 +2. 优先修改与该字段直接相关的实体类、Mapper XML、必要的加密支撑类与最小验证代码。 +3. 不要默认扩展到无直接关系的文件，例如日志配置、代码生成器、环境模板、整仓库批量字段改造。 +4. 若缺少 `keyId`、JAR、服务 IP、证书等外部依赖信息，不要停在反问阶段；可以先按仓库现状完成代码骨架和接入点，并把外部依赖列入结果中的待办事项。 +5. 若 issue 没有要求整套 DES 上线，就不要自动生成超出字段范围的大规模基础设施改造。 + +--- + +## 执行流程 + +### Step 1：收集信息 + +请用户提供以下信息（若未提供则逐一询问）： + +1. **需要加密的表和字段**，格式如： + - `表名: 字段1, 字段2` + - 例：`alipay_refund: phone`、`alipay_receipt: phone, email` +2. **密钥 ID**（DES 平台申请的 keyId，如 `o2oomsorder`、`asg-api`） +3. **加密 JAR 包路径**（quickapi-client-java-*.jar 的位置） +4. **加密服务 IP**（至少一个，最多两个双机） +5. **项目构建工具**（Gradle 或 Maven） + +### Step 2：实施改造 + +按以下顺序自动完成代码改造： + +#### 2.1 引入依赖 + +**Gradle：** +```groovy +implementation files('lib/quickapi-client-java-x.x.x-SNAPSHOT-shaded.jar') +``` + +**Maven：** +```xml + + org.quickssl + quickapi-client-java + x.x.x-SNAPSHOT + system + ${pom.basedir}/lib/quickapi-client-java-x.x.x-SNAPSHOT-shaded.jar + +``` + +#### 2.2 创建加密包 `{basePackage}.encryption` + +**EncryptionComponent.java** — 加密服务初始化： +```java +@Slf4j +@Component +@RefreshScope +public class EncryptionComponent { + + @Value("${encryption.server.ip1}") + private String encryptionServerIp1; + @Value("${encryption.server.ip2}") + private String encryptionServerIp2; + + @PostConstruct + public void init() { + log.info("--------加密组件初始化开始--------"); + try { + LoadingKeyCacheWithLocalFile kekDekLoader = new LoadingKeyCacheWithLocalFile() + .setLocalKekPath("/opt/sec-kek/{keyId}.kek") + .setLocalDekPath("/opt/sec-dek/{keyId}.dek"); + CryptoClient.Config config = CryptoClient.Config.newBuilder() + .setSocket(encryptionServerIp1, encryptionServerIp2) + .setAuthority("quickservice") + .setCaCertFile("/opt/sec-cert/cacert.pem") + .setKeyStoreFile("/opt/sec-cert/client.pfx") + .enableKeyCache(10, 100, 2592000) + .setDekCacheLoader(kekDekLoader) + .setKekCacheLoader(kekDekLoader) + .build(); + CryptoHelper.initConfig(config); + } catch (CryptoClient.CryptoException e) { + throw new RuntimeException(e); + } + log.info("--------加密组件初始化完成--------"); + } +} +``` + +**EncryptionUtils.java** — 加解密工具类（3 个静态方法）： +- `encodeData(String plaintext)` — SM4/GCM PB 格式加密，返回 Base64；失败返回原文 +- `deocdeData(String encodeData)` — 解密；先 isEncode 检查，非密文直接返回（兼容明文数据） +- `isEncode(String encodeData)` — 判断是否为 PB 格式密文 + +关键实现： +```java +// 加密：使用 PB 格式（优先），解密时无需传密钥/IV +byte[] pbEnBytes = CryptoHelper.symmEncryptEx(KEYID, SymAlg.QK_SGD_SM4_GCM, plaintext.getBytes()); +return new String(Base64.getEncoder().encode(pbEnBytes), StandardCharsets.UTF_8); + +// 解密：PB 格式可自动还原 +byte[] pbDeBytes = CryptoHelper.symmDecrypt("", Base64.getDecoder().decode(encodeData)); +return new String(pbDeBytes, StandardCharsets.UTF_8); +``` + +**EncryptionFieldHelper.java** — 写入归一化（受 `encryption.switch` 开关控制）： +- 开关 false：不填充加密字段，保持原有行为 +- 开关 true：将明文字段值复制到 `_encrypt` 字段（TypeHandler 负责实际加密） + +```java +@Component +@RefreshScope +public class EncryptionFieldHelper { + @Value("${encryption.switch:false}") + private boolean encryptionSwitch; + + public void normalizeForWrite(Entity entity) { + if (!encryptionSwitch) return; + if (StringUtils.isNotEmpty(entity.getPhone())) { + entity.setPhoneEncrypt(entity.getPhone()); // TypeHandler 负责加密 + } + } +} +``` + +#### 2.3 创建 TypeHandler + +**EncryptionTypeHandler.java** — 继承 `BaseTypeHandler`： +- 写入时自动加密（`EncryptionUtils.encodeData`） +- 读取时自动解密（`EncryptionUtils.deocdeData`） +- **禁止** `@MappedTypes(String.class)`，必须在 Mapper XML 中显式绑定到 `_encrypt` 列 + +#### 2.4 实体类新增字段 + +每个加密字段新增对应的 `{field}Encrypt` 属性。 + +#### 2.5 Mapper XML 改造 + +**ResultMap** — 新增 `_encrypt` 列映射，绑定 TypeHandler： +```xml + +``` + +**SELECT** — column list 追加 `_encrypt` 列，WHERE 条件不变： +```xml +select ..., phone, phone_encrypt, ... from table +``` + +**INSERT/UPDATE** — 追加 `_encrypt` 列，使用 TypeHandler： +```xml + + phone_encrypt = #{phoneEncrypt, typeHandler=...EncryptionTypeHandler}, + +``` + +#### 2.6 Service 层接入 + +在所有 insert/update 调用前，加入 `encryptionFieldHelper.normalizeXxxForWrite(entity)`。 + +#### 2.7 配置文件 + +各环境 bootstrap 配置添加： +```yaml +encryption: + switch: false # 灰度期间默认 false，Nacos 动态切换 + server: + ip1: {加密服务IP1} + ip2: {加密服务IP2} +``` + +### Step 3：生成 DDL + +自动生成 SQL 文件（存放在 `docs/des_encrypt_columns.sql`），格式： +```sql +ALTER TABLE `{table}` + ADD COLUMN `{field}_encrypt` VARCHAR(512) DEFAULT NULL COMMENT '{字段描述}密文' AFTER `{field}`; +``` + +### Step 4：输出待办清单 + +完成后输出后续待办： +1. 确认 KEYID、生产环境 IP、证书/密钥文件部署 +2. 各环境执行 DDL +3. 部署代码（switch=false），验证无回归 +4. 存量数据回刷（使用 DES 回刷工具） +5. Nacos 切换 `encryption.switch=true` + +--- + +## 设计原则 + +- **最小改动**：不引入全局拦截器，不修改现有表结构，只新增列 +- **保留明文列**：迁移期间原字段不动，确保可回滚 +- **单开关控制**：`encryption.switch` 一个开关管读写，Nacos 动态生效 +- **TypeHandler 显式绑定**：仅作用于 `_encrypt` 列，不影响其他 String 字段 +- **加密失败兜底**：`encodeData` 加密异常时返回原文，不阻断业务 + +## 参考实现 + +- OMS 项目：`oms-base-service/order_data_manager/src/main/java/com/freemud/encryption/` +- enc-test 项目：`/Users/jowang/Desktop/enc-test/` diff --git a/platform/skills/shared/github_dispatch_issue/SKILL.md b/platform/skills/shared/github_dispatch_issue/SKILL.md new file mode 100644 index 0000000..9d2ad4b --- /dev/null +++ b/platform/skills/shared/github_dispatch_issue/SKILL.md @@ -0,0 +1,74 @@ +--- +name: github_dispatch_issue +display_name: GitHub Issue Dispatch +description: Understands the Github Issue and dispatches tasks to execution agents. +layer: L1 +tags: ["github", "dispatch", "orchestrator"] +status: active +version: 1.1.0 +--- + +# GitHub Issue Dispatch Skill + +该技能负责分析上游传入的 GitHub Issue 数据，并将其结构化分发给对应的处理 Agent。 + +## 职责 + +1. **理解 Issue**：阅读并理解收到的 Issue 信息（标题、正文、标签、评论指令等）。 +2. **关联项目**：通过 Issue URL 中的仓库路径，确认目标仓库和 `repo_full_name`。 +3. **提取关键信息**：从 Issue 内容中提取需要执行的具体任务信息（如需要加密哪些字段）。 +4. **识别执行 Agent**：根据 Issue 类型匹配对应的执行 Agent（见下方可选列表）。 +5. **输出分发结果**：严格按照下方 JSON Schema 输出结构化分发结果，并附上发往下一阶段的完整处理指令。 + +--- + +## 可选执行 Agent + +| agent_role | 适用场景 | +|---|---| +| `des encrypt` | 数据库敏感字段安全加密改造（如 phone、email、身份证等字段的 SM4/DES 加密） | + +> 当前版本仅支持以上 agent。如 Issue 类型不在列表中，在 `dispatch_reason` 中说明并将 `selected_agent_role` 设为 `unknown`。 + +--- + +## 输出 JSON Schema + +分发结果必须是合法 JSON，包含以下所有字段： + +```json +{ + "selected_agent_role": "string // 选中的执行 agent 角色，如 'des encrypt'", + "intent": "string // 意图分类，如 'security_encryption'", + "issue_number": "integer // Issue 编号，如 13", + "issue_url": "string // Issue 完整 URL，如 'https://scm.example.com/owner/repo/issues/13'", + "repo_full_name": "string // 仓库全名，如 'china/starbucks-asg-api'", + "task_title": "string // 为下一阶段生成的任务标题，简洁描述本次改造", + "work_summary": "string // 具体需要做什么，包含字段名、表名等关键信息", + "acceptance_criteria": "string // 验收标准，下一阶段完成后需满足的条件", + "dispatch_reason": "string // 选择该 agent 的原因说明" +} +``` + +### 示例输出 + +```json +{ + "selected_agent_role": "des encrypt", + "intent": "security_encryption", + "issue_number": 13, + "issue_url": "https://scm.starbucks.com/china/starbucks-asg-api/issues/13", + "repo_full_name": "china/starbucks-asg-api", + "task_title": "安全加密：starbucks-asg-api phone 字段加密改造", + "work_summary": "对 starbucks-asg-api 项目中的 phone 字段进行 SM4/GCM 安全加密改造，使用 des_encrypt skill 标准流程。", + "acceptance_criteria": "phone 字段加密代码完成并推送到远端分支，GitHub Issue 收到包含分支名和任务地址的回帖。", + "dispatch_reason": "Issue 明确要求对 phone 字段进行安全加密，符合 des encrypt agent 的适用场景。" +} +``` + +--- + +## 注意事项 + +- 输出 JSON 后，必须附上发往下一阶段执行 agent 的**完整处理指令**，包括：指定字段名、表名（如已知）、最小改造范围说明。 +- 你只负责分析和分发，**不直接修改任何代码**。 diff --git a/platform/skills/shared/github_issue_feedback/SKILL.md b/platform/skills/shared/github_issue_feedback/SKILL.md new file mode 100644 index 0000000..961a6a8 --- /dev/null +++ b/platform/skills/shared/github_issue_feedback/SKILL.md @@ -0,0 +1,63 @@ +--- +name: github_issue_feedback +display_name: GitHub Issue Feedback +description: Report the processing result back to the GitHub Issue via curl webhook. +layer: L1 +tags: ["github", "feedback"] +status: active +version: 1.1.0 +--- + +# GitHub Issue Feedback Skill + +在按照对应 skill 完成 Coding 并且 Git Commit & Push 分支后，必须使用此技能，通过 REST API 将结果回帖到原始 GitHub Issue。 + +## 职责 + +1. 从前序阶段（dispatch_issue）的 prior output 中提取以下信息： + - `issue_number`：Issue 编号（整数） + - `issue_url`：Issue 完整 URL（用于推断 API base 和 owner/repo） + - `repo_full_name`：仓库全名，格式 `owner/repo` +2. 从当前 task 上下文中获取： + - `$GHE_TOKEN` 或 `$GITHUB_TOKEN`：GitHub API 认证 Token + - `$APP_BASE_URL`：Silicon Agent 前端地址（默认 `http://127.0.0.1:3000`） + - 当前推送的 Git 分支名（通过 `git branch --show-current` 或 git push 输出获取） + - 当前 task ID（在任务上下文或环境变量中） + +## 推断 API Base URL + +从 `issue_url` 中提取 hostname： + +- 若 hostname 是 `github.com`，API base 为 `https://api.github.com` +- 若是 GHE 内网域名（如 `scm.example.com`），API base 为 `https:///api/v3` +- Token 选择：公网 GitHub 用 `$GITHUB_TOKEN`，GHE 用 `$GHE_TOKEN` + +## 评论内容格式 + +``` +任务已完成！ +- Git 分支: <你推送的分支名> +- Silicon Agent 任务地址: <$APP_BASE_URL>/tasks/ +``` + +## 执行命令参考 + +```bash +# 公网 GitHub +curl -s -X POST \ + -H "Authorization: token $GITHUB_TOKEN" \ + -H "Accept: application/vnd.github.v3+json" \ + "https://api.github.com/repos/{OWNER}/{REPO}/issues/{ISSUE_NUMBER}/comments" \ + -d '{"body": "任务已完成！\n- Git 分支: {BRANCH}\n- Silicon Agent 任务地址: {APP_BASE_URL}/tasks/{TASK_ID}"}' + +# GitHub Enterprise (GHE) +curl -s -X POST \ + -H "Authorization: token $GHE_TOKEN" \ + -H "Accept: application/vnd.github.v3+json" \ + "https://{GHE_HOST}/api/v3/repos/{OWNER}/{REPO}/issues/{ISSUE_NUMBER}/comments" \ + -d '{"body": "任务已完成！\n- Git 分支: {BRANCH}\n- Silicon Agent 任务地址: {APP_BASE_URL}/tasks/{TASK_ID}"}' +``` + +> 注意：将 `{OWNER}`, `{REPO}`, `{ISSUE_NUMBER}`, `{BRANCH}`, `{APP_BASE_URL}`, `{TASK_ID}`, `{GHE_HOST}` 替换为实际值后执行。 + +此行为标志着整个 Issue 生命周期的真正交付与闭环。 diff --git a/platform/tests/test_agents.py b/platform/tests/test_agents.py index 7b7df35..183f988 100644 --- a/platform/tests/test_agents.py +++ b/platform/tests/test_agents.py @@ -68,6 +68,28 @@ def test_orchestrator_no_write(): assert "execute_script" not in tools +def test_issue_distribution_tools_are_dispatch_only(): + tools = ROLE_TOOLS["dispatch issue"] + assert tools == {"read", "execute", "skill"} + + +def test_des_encrypt_tools_allow_coding_and_skills(): + tools = ROLE_TOOLS["des encrypt"] + assert {"read", "write", "edit", "execute", "execute_script", "skill"} == tools + + +def test_issue_distribution_uses_shared_skills(): + dirs = agents_mod._get_skill_dirs("dispatch issue") + rendered = [p.name for p in dirs] + assert rendered == ["shared"] + + +def test_des_encrypt_uses_shared_skills(): + dirs = agents_mod._get_skill_dirs("des encrypt") + rendered = [p.name for p in dirs] + assert rendered == ["shared"] + + def test_validate_role_tools_raises_on_unknown(monkeypatch): monkeypatch.setattr(agents_mod, "_ALL_TOOLS", {"read"}) monkeypatch.setattr(agents_mod, "ROLE_TOOLS", {"coding": {"read", "write"}}) diff --git a/platform/tests/test_agents_api.py b/platform/tests/test_agents_api.py index e24773b..0db18d8 100644 --- a/platform/tests/test_agents_api.py +++ b/platform/tests/test_agents_api.py @@ -48,6 +48,16 @@ async def test_list_agents(client, seed_agent): assert "ag-test-coding" in roles +@pytest.mark.asyncio +async def test_list_agents_includes_github_issue_workflow_roles(client): + """GET /api/v1/agents should expose the built-in GitHub issue workflow roles.""" + resp = await client.get("/api/v1/agents") + assert resp.status_code == 200 + roles = [a["role"] for a in resp.json()["agents"]] + assert "dispatch issue" in roles + assert "des encrypt" in roles + + @pytest.mark.asyncio async def test_get_agent(client, seed_agent): """GET /api/v1/agents/{role} returns the agent with correct fields.""" @@ -234,3 +244,13 @@ async def test_agent_session_404(client): """GET /api/v1/agents/nonexistent/session returns 404.""" resp = await client.get("/api/v1/agents/nonexistent/session") assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_get_agent_config_options_include_issue_roles(client): + """Config options should expose defaults for the GitHub issue workflow roles.""" + resp = await client.get("/api/v1/agents/config/options") + assert resp.status_code == 200 + data = resp.json() + assert "dispatch issue" in data["role_defaults"] + assert "des encrypt" in data["role_defaults"] diff --git a/platform/tests/test_engine_worktree_and_workspace.py b/platform/tests/test_engine_worktree_and_workspace.py index c94ba05..ce19616 100644 --- a/platform/tests/test_engine_worktree_and_workspace.py +++ b/platform/tests/test_engine_worktree_and_workspace.py @@ -309,6 +309,8 @@ async def test_finalize_task_resources_commit_push_success(monkeypatch): branch="main", ) task.target_branch = None + task.github_issue_number = 13 + task.branch_name = None session = SimpleNamespace(commit=AsyncMock()) @@ -328,6 +330,7 @@ async def test_finalize_task_resources_commit_push_success(monkeypatch): assert result is True commit_push_mock.assert_awaited_once() + assert task.branch_name == "feat/branch-123" @pytest.mark.asyncio @@ -369,6 +372,42 @@ async def _raising_commit(*a, **kw): fail_task_mock.assert_awaited_once() +@pytest.mark.asyncio +async def test_finalize_task_resources_github_issue_number_does_not_block_success(monkeypatch): + """github_issue_number set → engine no longer posts feedback; task still succeeds.""" + monkeypatch.setattr(engine.settings, "MEMORY_ENABLED", False) + monkeypatch.setattr(engine.settings, "SKILL_FEEDBACK_ENABLED", False) + monkeypatch.setattr(engine, "_emit_system_log", AsyncMock(return_value="log-id")) + monkeypatch.setattr(engine, "_close_started_system_log", AsyncMock()) + monkeypatch.setattr(engine, "_cleanup_runtime_resources", AsyncMock()) + monkeypatch.setattr(engine, "commit_and_push_workspace", AsyncMock(return_value="feat/branch-123")) + monkeypatch.setattr(engine, "create_pr_for_workspace", AsyncMock(return_value=None)) + + task = _make_task(id="tt-finalize-feedback-no-block-1", title="Feedback No Block Test") + task.project = SimpleNamespace( + repo_url="https://github.com/test/repo", + branch="main", + ) + task.target_branch = None + task.github_issue_number = 13 + task.branch_name = None + + session = SimpleNamespace(commit=AsyncMock()) + + result = await engine._finalize_task_resources( + session, # type: ignore[arg-type] + task, # type: ignore[arg-type] + [], + None, + None, None, "/tmp/test_ws", "tmp_cloned", "feat/branch-123", + None, None, + ) + + # Feedback is now agent's responsibility; engine should still succeed + assert result is True + assert task.branch_name == "feat/branch-123" + + # ── 16. _execute_single_stage paths ─────────────────────────────────────── # Section 22: _ensure_code_stage_has_changes additional paths # ═══════════════════════════════════════════════════════════════════════ @@ -429,6 +468,47 @@ async def test_ensure_code_stage_not_code_stage(monkeypatch): git_check.assert_not_awaited() +@pytest.mark.asyncio +async def test_ensure_des_encrypt_no_changes_false(monkeypatch): + """des encrypt stage must also produce repository changes.""" + monkeypatch.setattr(engine, "_has_git_worktree_changes", AsyncMock(return_value=False)) + mark_failed = AsyncMock() + fail_task = AsyncMock() + monkeypatch.setattr(engine, "mark_stage_failed", mark_failed) + monkeypatch.setattr(engine, "_fail_task", fail_task) + + task = _make_task() + stage = _make_stage(stage_name="des encrypt") + session = SimpleNamespace(commit=AsyncMock()) + + with patch("app.worker.agents.close_agents_for_task"): + result = await engine._ensure_code_stage_has_changes(session, task, stage, "/some/path") + + assert result is False + mark_failed.assert_awaited_once() + fail_task.assert_awaited_once() + + +@pytest.mark.asyncio +async def test_ensure_des_encrypt_allows_clean_worktree_when_commit_exists(monkeypatch): + monkeypatch.setattr(engine, "_has_git_worktree_changes", AsyncMock(return_value=False)) + monkeypatch.setattr(engine, "_has_git_committed_changes_since_base", AsyncMock(return_value=True)) + mark_failed = AsyncMock() + fail_task = AsyncMock() + monkeypatch.setattr(engine, "mark_stage_failed", mark_failed) + monkeypatch.setattr(engine, "_fail_task", fail_task) + + task = _make_task(project=SimpleNamespace(branch="master")) + stage = _make_stage(stage_name="des encrypt") + session = SimpleNamespace(commit=AsyncMock()) + + result = await engine._ensure_code_stage_has_changes(session, task, stage, "/some/path") + + assert result is True + mark_failed.assert_not_awaited() + fail_task.assert_not_awaited() + + # ═══════════════════════════════════════════════════════════════════════ # Section 23: _setup_worktree paths # ═══════════════════════════════════════════════════════════════════════ diff --git a/platform/tests/test_executor_stage_logs.py b/platform/tests/test_executor_stage_logs.py index 8b9ae2d..099ae09 100644 --- a/platform/tests/test_executor_stage_logs.py +++ b/platform/tests/test_executor_stage_logs.py @@ -438,6 +438,75 @@ def _text_only_runner( assert text_only_called['value'] is True +@pytest.mark.asyncio +async def test_execute_stage_uses_tool_enabled_runner_for_dispatch_issue(monkeypatch): + session = SimpleNamespace(commit=AsyncMock()) + task = SimpleNamespace( + id='task-dispatch-issue-1', + title='task title', + description='task description', + total_tokens=0, + total_cost_rmb=0.0, + ) + stage = SimpleNamespace( + id='stage-dispatch-issue-1', + stage_name='dispatch_issue', + agent_role='dispatch issue', + status='pending', + started_at=None, + completed_at=None, + duration_seconds=None, + tokens_used=0, + output_summary=None, + ) + + fake_pipeline = _FakePipeline() + tool_runner_called = {'value': False} + + monkeypatch.setattr(executor, 'get_task_log_pipeline', lambda: fake_pipeline) + monkeypatch.setattr(executor, '_get_agent', AsyncMock(return_value=None)) + monkeypatch.setattr(executor, '_safe_broadcast', AsyncMock()) + monkeypatch.setattr(executor, 'build_user_prompt', lambda _ctx: 'dispatch prompt') + + def _tool_runner( + _role, + _task_id, + model=None, + temperature=None, + max_tokens=None, + max_turns=None, + extra_skill_dirs=None, + system_prompt_append=None, + ): + tool_runner_called['value'] = True + return _FakeRunner() + + def _text_only_runner( + _role, + _task_id, + model=None, + temperature=None, + max_tokens=None, + max_turns=None, + extra_skill_dirs=None, + system_prompt_append=None, + ): + raise AssertionError('dispatch_issue should not use text-only runner') + + monkeypatch.setattr(executor, 'get_agent', _tool_runner) + monkeypatch.setattr(executor, 'get_agent_text_only', _text_only_runner) + + result = await executor.execute_stage( + session=session, + task=task, + stage=stage, + prior_outputs=[], + ) + + assert result == 'stage output' + assert tool_runner_called['value'] is True + + def test_apply_runner_workspace_override_replaces_prompt_and_cwd(): runner = SimpleNamespace( default_cwd='/tmp/old-workspace', diff --git a/platform/tests/test_mock_webhook.py b/platform/tests/test_mock_webhook.py index 6352b78..4e5158a 100644 --- a/platform/tests/test_mock_webhook.py +++ b/platform/tests/test_mock_webhook.py @@ -98,6 +98,41 @@ async def seed_rule_with_filter(seed_project): await session.commit() +@pytest_asyncio.fixture +async def seed_issue_comment_rule(seed_project): + """用于 issue comment 命令触发的规则。""" + rule_id = "tt-mock-rule-issue-comment-1" + async with async_session_factory() as session: + rule = TriggerRuleModel( + id=rule_id, + name="mock github issue comment rule", + source="github", + event_type="issue_comment_created", + filters={ + "op": "and", + "conditions": [ + { + "type": "field_equals", + "field": "silicon_agent_command_triggered", + "value": True, + } + ], + }, + title_template="Issue command #{number}: {title}", + dedup_key_template="github:{repo_full_name}:issue_comment:{comment_id}", + project_id=seed_project, + enabled=True, + ) + session.add(rule) + await session.commit() + yield rule_id + async with async_session_factory() as session: + r = await session.get(TriggerRuleModel, rule_id) + if r: + await session.delete(r) + await session.commit() + + # ── Dry-Run Tests ───────────────────────────────────────────────────────────── @@ -157,6 +192,57 @@ async def test_dry_run_filter_skip(self, client, seed_rule_with_filter, seed_pro data = resp.json() assert data["result"] in ("skipped_filter", "skipped_no_rule") + @pytest.mark.asyncio + async def test_dry_run_issue_comment_command_matches( + self, client, seed_issue_comment_rule, seed_project + ): + resp = await client.post( + f"/api/v1/projects/{seed_project}/mock-webhook", + json={ + "source": "github", + "event_type": "issue_comment_created", + "title": "安全加密", + "body": "@silicon_agent 只处理 phone 字段", + "number": 13, + "author": "alice", + "extra": { + "repo_full_name": "china/starbucks-asg-api", + "comment_id": 1001, + "issue_body": "安全加密agent，对本项目的phone字段进行安全加密", + }, + "dry_run": True, + }, + ) + assert resp.status_code == 200 + data = resp.json() + assert data["matched"] is True + assert data["result"] == "would_trigger" + + @pytest.mark.asyncio + async def test_dry_run_issue_comment_without_command_skips( + self, client, seed_issue_comment_rule, seed_project + ): + resp = await client.post( + f"/api/v1/projects/{seed_project}/mock-webhook", + json={ + "source": "github", + "event_type": "issue_comment_created", + "title": "安全加密", + "body": "帮忙看看这个 issue", + "number": 13, + "author": "alice", + "extra": { + "repo_full_name": "china/starbucks-asg-api", + "comment_id": 1002, + }, + "dry_run": True, + }, + ) + assert resp.status_code == 200 + data = resp.json() + assert data["matched"] is False + assert data["result"] == "skipped_filter" + # ── Actual Trigger Tests ────────────────────────────────────────────────────── @@ -280,6 +366,35 @@ def test_github_pr_payload(self): assert payload["pull_request"]["title"] == "Add feature" assert payload["pull_request"]["head"]["ref"] == "refs/heads/feature" + def test_github_issue_comment_payload_extracts_command(self): + from app.schemas.trigger import MockWebhookRequest + from app.services.trigger_service import _build_normalized_payload + + req = MockWebhookRequest( + source="github", + event_type="issue_comment_created", + title="Issue title", + body="@silicon_agent 只处理 phone 字段", + number=13, + author="alice", + extra={ + "issue_body": "原始 issue 描述", + "issue_url": "https://scm.starbucks.com/china/starbucks-asg-api/issues/13", + "comment_id": 1001, + "comment_url": "https://scm.starbucks.com/china/starbucks-asg-api/issues/13#issuecomment-1001", + }, + ) + + payload = _build_normalized_payload(req) + + assert payload["issue"]["title"] == "Issue title" + assert payload["issue"]["number"] == 13 + assert payload["comment"]["body"] == "@silicon_agent 只处理 phone 字段" + assert payload["silicon_agent_command_triggered"] is True + assert payload["silicon_agent_command_style"] == "mention" + assert payload["silicon_agent_command_text"] == "@silicon_agent" + assert payload["silicon_agent_command_note"] == "只处理 phone 字段" + def test_gitlab_payload(self): from app.schemas.trigger import MockWebhookRequest from app.services.trigger_service import _build_normalized_payload diff --git a/platform/tests/test_prompts.py b/platform/tests/test_prompts.py index c8f9ca4..1111271 100644 --- a/platform/tests/test_prompts.py +++ b/platform/tests/test_prompts.py @@ -3,7 +3,12 @@ import pytest -from app.worker.prompts import STAGE_INSTRUCTIONS, StageContext, build_user_prompt +from app.worker.prompts import ( + STAGE_INSTRUCTIONS, + SYSTEM_PROMPTS, + StageContext, + build_user_prompt, +) def _minimal_ctx(**overrides) -> StageContext: @@ -49,6 +54,105 @@ def test_test_guardrail_emphasizes_minimal_validation(): assert "不要只根据代码阅读就判定测试通过" in result +def test_dispatch_issue_prompt_contract(): + ctx = _minimal_ctx( + stage_name="dispatch_issue", + agent_role="dispatch issue", + task_description="Issue URL: https://scm.starbucks.com/china/starbucks-asg-api/issues/13", + ) + result = build_user_prompt(ctx) + assert "GitHub Issue" in SYSTEM_PROMPTS["dispatch issue"] + assert "`skill` 工具" in SYSTEM_PROMPTS["dispatch issue"] + assert "github_dispatch_issue" in SYSTEM_PROMPTS["dispatch issue"] + assert "github_dispatch_issue" in STAGE_INSTRUCTIONS["dispatch_issue"] + assert "不得直接修改任何代码" in STAGE_INSTRUCTIONS["dispatch_issue"] + assert STAGE_INSTRUCTIONS["dispatch_issue"] in result + + +def test_dispatch_issue_prompt_does_not_embed_skill_body(): + """Skill content should NOT be injected into prompt; agent loads via skill tool.""" + ctx = _minimal_ctx( + stage_name="dispatch_issue", + agent_role="dispatch issue", + task_description="Issue URL: https://scm.starbucks.com/china/starbucks-asg-api/issues/13", + ) + result = build_user_prompt(ctx) + assert "## 分发技能" not in result + assert "# GitHub Issue Dispatch Skill" not in result + + +def test_issue_distribution_has_single_canonical_prompt_name(): + assert "dispatch issue" in SYSTEM_PROMPTS + assert "dispatch issue agent" not in SYSTEM_PROMPTS + assert "dispatch agent" not in SYSTEM_PROMPTS + + +def test_des_encrypt_prompt_contract(): + ctx = _minimal_ctx( + stage_name="des encrypt", + agent_role="des encrypt", + task_description="Issue #13 要求对 phone 字段进行安全加密", + prior_outputs=[ + { + "stage": "dispatch_issue", + "output": '{"selected_agent_role":"des encrypt","issue_number":13}', + } + ], + ) + result = build_user_prompt(ctx) + assert "安全加密" in SYSTEM_PROMPTS["des encrypt"] + assert "des_encrypt" in SYSTEM_PROMPTS["des encrypt"] + assert "github_issue_feedback" in SYSTEM_PROMPTS["des encrypt"] + assert STAGE_INSTRUCTIONS["des encrypt"] in result + assert "dispatch_issue" in result + + +def test_des_encrypt_prompt_does_not_embed_skill_body(): + """Skills should be loaded via the `skill` tool, not injected into the prompt.""" + ctx = _minimal_ctx( + stage_name="des encrypt", + agent_role="des encrypt", + task_description="Issue #13 要求对 phone 字段进行安全加密", + ) + result = build_user_prompt(ctx) + assert "## 安全加密技能" not in result + assert "# DES 安全加密接入 Skill" not in result + assert "## GitHub 回帖技能" not in result + assert "# GitHub Issue Feedback Skill" not in result + + +def test_issue_stage_instructions_follow_existing_numbered_style(): + assert "github_dispatch_issue" in STAGE_INSTRUCTIONS["dispatch_issue"] + assert "JSON Schema" in STAGE_INSTRUCTIONS["dispatch_issue"] + assert "des_encrypt" in STAGE_INSTRUCTIONS["des encrypt"] + assert "github_issue_feedback" in STAGE_INSTRUCTIONS["des encrypt"] + assert "1. **Coding**" in STAGE_INSTRUCTIONS["des encrypt"] + assert "2. **回帖**" in STAGE_INSTRUCTIONS["des encrypt"] + + +def test_des_encrypt_prompt_forbids_nested_clone(): + ctx = _minimal_ctx( + stage_name="des encrypt", + agent_role="des encrypt", + preflight_summary="- 当前工作区: 目标仓库已在当前 workspace 根目录检出；直接在这里读写、commit、push。", + ) + result = build_user_prompt(ctx) + assert "git clone" in result + assert "当前 workspace 根目录" in result + + +def test_des_encrypt_prompt_enforces_minimal_issue_scope(): + ctx = _minimal_ctx( + stage_name="des encrypt", + agent_role="des encrypt", + task_description="Issue #13: 仅对 phone 字段进行安全加密", + ) + result = build_user_prompt(ctx) + # Minimal scope enforcement comes from guardrail and des_encrypt skill + assert "单一字段" in result or "最小改造模式" in result + assert "logback" in result or "环境模板" in result + + # --------------------------------------------------------------------------- # With description # --------------------------------------------------------------------------- diff --git a/platform/tests/test_skills_api.py b/platform/tests/test_skills_api.py index 518f7a1..3c60078 100644 --- a/platform/tests/test_skills_api.py +++ b/platform/tests/test_skills_api.py @@ -120,6 +120,47 @@ async def test_get_skill(client, skill_factory): assert data["version"] == "1.0.0" +@pytest.mark.asyncio +async def test_get_skill_falls_back_to_latest_version_content_when_primary_content_empty(client): + name = _unique_name("skill-fallback") + + async with async_session_factory() as session: + skill = SkillModel( + name=name, + display_name="Fallback Skill", + layer="L1", + content=None, + ) + session.add(skill) + await session.flush() + session.add( + SkillVersionModel( + skill_id=skill.id, + version="1.0.0", + content="# Des Encrypt\\n\\nThis content comes from the latest version.", + change_summary="Seeded version content", + ) + ) + await session.commit() + + resp = await client.get(f"/api/v1/skills/{name}") + assert resp.status_code == 200 + data = resp.json() + assert data["content"] == "# Des Encrypt\\n\\nThis content comes from the latest version." + + async with async_session_factory() as session: + result = await session.execute(select(SkillModel).where(SkillModel.name == name)) + skill = result.scalar_one_or_none() + if skill: + ver_result = await session.execute( + select(SkillVersionModel).where(SkillVersionModel.skill_id == skill.id) + ) + for version in ver_result.scalars().all(): + await session.delete(version) + await session.delete(skill) + await session.commit() + + @pytest.mark.asyncio async def test_get_skill_404(client): """GET /api/v1/skills/{name} returns 404 for unknown skill.""" diff --git a/platform/tests/test_tasks_api.py b/platform/tests/test_tasks_api.py index 8558dd0..b3e07e0 100644 --- a/platform/tests/test_tasks_api.py +++ b/platform/tests/test_tasks_api.py @@ -941,3 +941,46 @@ def test_task_detail_response_null_tokens(): }) assert task.total_tokens == 0 assert task.total_cost_rmb == 0.0 + + +def test_task_detail_response_assumes_utc_for_naive_datetimes(): + """Naive task timestamps should be normalized to UTC-aware values for API output.""" + from app.schemas.task import TaskDetailResponse + + task = TaskDetailResponse.model_validate({ + "id": "fake-task", + "title": "Test", + "status": "pending", + "created_at": "2026-03-23T10:41:08", + "completed_at": "2026-03-23T10:45:24.303349", + "total_tokens": 1, + "total_cost_rmb": 0.01, + }) + + assert task.created_at.tzinfo is not None + assert task.completed_at is not None and task.completed_at.tzinfo is not None + dumped = task.model_dump(mode="json") + assert dumped["created_at"].endswith("Z") or dumped["created_at"].endswith("+00:00") + assert dumped["completed_at"].endswith("Z") or dumped["completed_at"].endswith("+00:00") + + +def test_task_stage_response_assumes_utc_for_naive_datetimes(): + """Naive stage timestamps should be normalized to UTC-aware values for API output.""" + from app.schemas.task import TaskStageResponse + + stage = TaskStageResponse.model_validate({ + "id": "stage-1", + "task_id": "task-1", + "stage_name": "des encrypt", + "agent_role": "des encrypt", + "status": "completed", + "started_at": "2026-03-23T10:41:50", + "completed_at": "2026-03-23T10:45:24.303349", + "tokens_used": 100, + }) + + assert stage.started_at is not None and stage.started_at.tzinfo is not None + assert stage.completed_at is not None and stage.completed_at.tzinfo is not None + dumped = stage.model_dump(mode="json") + assert dumped["started_at"].endswith("Z") or dumped["started_at"].endswith("+00:00") + assert dumped["completed_at"].endswith("Z") or dumped["completed_at"].endswith("+00:00") diff --git a/platform/tests/test_template_service.py b/platform/tests/test_template_service.py index ae79ad8..d74f8dc 100644 --- a/platform/tests/test_template_service.py +++ b/platform/tests/test_template_service.py @@ -656,6 +656,96 @@ async def test_seed_builtin_templates_idempotent(): assert len(names_found) == len(set(names_found)) +@pytest.mark.asyncio +async def test_seed_builtin_templates_includes_github_issue_template(): + """github_issue_template should be seeded with distribution + security stages.""" + async with async_session_factory() as session: + svc = TemplateService(session) + await svc.seed_builtin_templates() + + async with async_session_factory() as session: + result = await session.execute( + select(TaskTemplateModel).where( + TaskTemplateModel.name == "github_issue_template" + ) + ) + template = result.scalar_one_or_none() + + assert template is not None + assert template.is_builtin is True + assert template.display_name == "Github Issue" + stages = json.loads(template.stages) + assert stages == [ + { + "name": "dispatch_issue", + "agent_role": "dispatch issue", + "order": 0, + }, + { + "name": "des encrypt", + "agent_role": "des encrypt", + "order": 1, + }, + ] + + +@pytest.mark.asyncio +async def test_seed_builtin_templates_updates_existing_builtin_display_name(): + """Existing builtin templates should be refreshed to the latest seeded display name.""" + async with async_session_factory() as session: + result = await session.execute( + select(TaskTemplateModel).where( + TaskTemplateModel.name == "github_issue_template" + ) + ) + template = result.scalar_one_or_none() + if template is None: + template = TaskTemplateModel( + name="github_issue_template", + display_name="GitHub Issue Template", + description="outdated description", + stages="[]", + gates="[]", + is_builtin=True, + ) + session.add(template) + else: + template.display_name = "GitHub Issue Template" + template.description = "outdated description" + template.stages = "[]" + template.gates = "[]" + template.is_builtin = True + await session.commit() + + async with async_session_factory() as session: + svc = TemplateService(session) + await svc.seed_builtin_templates() + + async with async_session_factory() as session: + result = await session.execute( + select(TaskTemplateModel).where( + TaskTemplateModel.name == "github_issue_template" + ) + ) + template = result.scalar_one() + + assert template.display_name == "Github Issue" + assert template.description == "GitHub issue 统一入口模板，先分发再执行" + assert json.loads(template.stages) == [ + { + "name": "dispatch_issue", + "agent_role": "dispatch issue", + "order": 0, + }, + { + "name": "des encrypt", + "agent_role": "des encrypt", + "order": 1, + }, + ] + assert json.loads(template.gates) == [] + + # ── API-level tests for additional coverage ─────────────────────────────────── diff --git a/platform/tests/test_trigger_complex.py b/platform/tests/test_trigger_complex.py index 03441e3..f8062c2 100644 --- a/platform/tests/test_trigger_complex.py +++ b/platform/tests/test_trigger_complex.py @@ -59,6 +59,31 @@ def test_leaf_unknown_type_passes(self): node = {"type": "nonexistent", "value": "x"} assert _eval_filter_node(node, {}) is True + def test_leaf_field_equals_pass(self): + node = { + "type": "field_equals", + "field": "silicon_agent_command_triggered", + "value": True, + } + assert _eval_filter_node(node, {"silicon_agent_command_triggered": True}) is True + + def test_leaf_field_equals_fail(self): + node = { + "type": "field_equals", + "field": "silicon_agent_command_triggered", + "value": True, + } + assert _eval_filter_node(node, {"silicon_agent_command_triggered": False}) is False + + def test_leaf_field_equals_supports_flattened_paths(self): + node = { + "type": "field_equals", + "field": "silicon_agent.command.triggered", + "value": True, + } + payload = {"silicon_agent": {"command": {"triggered": True}}} + assert _eval_filter_node(node, payload) is True + # AND 节点 def test_and_all_pass(self): node = { diff --git a/platform/tests/test_webhook_project.py b/platform/tests/test_webhook_project.py index acb89e5..7847917 100644 --- a/platform/tests/test_webhook_project.py +++ b/platform/tests/test_webhook_project.py @@ -12,6 +12,7 @@ from app.db.session import async_session_factory from app.models.project import ProjectModel from app.models.integration import ProjectIntegrationModel +from app.models.task import TaskModel from app.models.trigger import TriggerRuleModel, TriggerEventModel from sqlalchemy import delete @@ -155,6 +156,251 @@ async def test_github_project_webhook_no_integration(client): assert resp.status_code == 404 +@pytest.mark.asyncio +async def test_github_issue_project_webhook_persists_issue_metadata(client): + name = _unique_name() + resp = await client.post("/api/v1/projects", json={ + "name": name, + "display_name": f"Display {name}", + }) + assert resp.status_code == 201 + project = resp.json() + pid = project["id"] + + resp = await client.post(f"/api/v1/projects/{pid}/integrations", json={ + "provider": "github", + }) + assert resp.status_code == 201 + integration = resp.json() + secret = integration["webhook_secret"] + + async with async_session_factory() as session: + rule = TriggerRuleModel( + name="test-github-issue-rule", + source="github", + event_type="issue_opened", + project_id=pid, + title_template="Issue: {issue_title}", + template_id=None, + enabled=True, + ) + session.add(rule) + await session.commit() + + payload = { + "action": "opened", + "issue": { + "number": 13, + "title": "安全加密", + "body": "安全加密agent，对本项目的phone字段进行安全加密", + "html_url": "https://scm.starbucks.com/china/starbucks-asg-api/issues/13", + "user": {"login": "jowang"}, + "labels": [], + }, + "repository": {"name": "starbucks-asg-api", "full_name": "china/starbucks-asg-api"}, + "sender": {"login": "jowang"}, + } + body = json.dumps(payload).encode() + signature = _github_signature(body, secret) + + resp = await client.post( + f"/webhooks/github/{pid}", + content=body, + headers={ + "X-GitHub-Event": "issues", + "X-Hub-Signature-256": signature, + "Content-Type": "application/json", + }, + ) + assert resp.status_code == 200 + task_id = resp.json()["task_id"] + assert task_id is not None + + task_resp = await client.get(f"/api/v1/tasks/{task_id}") + assert task_resp.status_code == 200 + task_data = task_resp.json() + assert task_data["github_issue_number"] == 13 + assert "china/starbucks-asg-api" in (task_data["description"] or "") + assert "https://scm.starbucks.com/china/starbucks-asg-api/issues/13" in (task_data["description"] or "") + assert "phone字段进行安全加密" in (task_data["description"] or "") + + async with async_session_factory() as session: + await session.execute( + delete(TriggerEventModel).where(TriggerEventModel.project_id == pid) + ) + await session.execute( + delete(TriggerRuleModel).where(TriggerRuleModel.project_id == pid) + ) + await session.execute( + delete(ProjectIntegrationModel).where( + ProjectIntegrationModel.project_id == pid + ) + ) + task = await session.get(TaskModel, task_id) + if task: + await session.delete(task) + proj = await session.get(ProjectModel, pid) + if proj: + await session.delete(proj) + await session.commit() + + +def test_normalize_github_issue_comment_payload_extracts_silicon_agent_command(): + from app.api.webhooks.github import _normalize_github_payload + + payload = { + "action": "created", + "issue": { + "number": 13, + "title": "安全加密", + "body": "安全加密agent，对本项目的phone字段进行安全加密", + "html_url": "https://scm.starbucks.com/china/starbucks-asg-api/issues/13", + "user": {"login": "issue-owner"}, + }, + "comment": { + "id": 1001, + "body": "/silicon_agent 只处理 phone 字段", + "html_url": "https://scm.starbucks.com/china/starbucks-asg-api/issues/13#issuecomment-1001", + "user": {"login": "commenter"}, + }, + "repository": {"name": "starbucks-asg-api", "full_name": "china/starbucks-asg-api"}, + "sender": {"login": "commenter"}, + } + + normalized = _normalize_github_payload("issue_comment", "issue_comment_created", payload) + + assert normalized["issue_number"] == 13 + assert normalized["issue_title"] == "安全加密" + assert normalized["issue_url"].endswith("/issues/13") + assert normalized["issue_body"] == "安全加密agent，对本项目的phone字段进行安全加密" + assert normalized["issue_author"] == "issue-owner" + assert normalized["comment_id"] == 1001 + assert normalized["comment_author"] == "commenter" + assert normalized["comment_body"] == "/silicon_agent 只处理 phone 字段" + assert normalized["silicon_agent_command_triggered"] is True + assert normalized["silicon_agent_command_style"] == "slash" + assert normalized["silicon_agent_command_text"] == "/silicon_agent" + assert normalized["silicon_agent_command_note"] == "只处理 phone 字段" + + +@pytest.mark.asyncio +async def test_github_issue_comment_project_webhook_triggers_command_rule(client): + name = _unique_name() + resp = await client.post("/api/v1/projects", json={ + "name": name, + "display_name": f"Display {name}", + }) + assert resp.status_code == 201 + project = resp.json() + pid = project["id"] + + resp = await client.post(f"/api/v1/projects/{pid}/integrations", json={ + "provider": "github", + }) + assert resp.status_code == 201 + integration = resp.json() + secret = integration["webhook_secret"] + + async with async_session_factory() as session: + rule = TriggerRuleModel( + name="test-github-issue-comment-rule", + source="github", + event_type="issue_comment_created", + project_id=pid, + title_template="Issue Command: {issue_title}", + dedup_key_template="github:{repo_full_name}:issue_comment:{comment_id}", + filters={ + "op": "and", + "conditions": [ + { + "type": "field_equals", + "field": "silicon_agent_command_triggered", + "value": True, + } + ], + }, + enabled=True, + ) + session.add(rule) + await session.commit() + + payload = { + "action": "created", + "issue": { + "number": 13, + "title": "安全加密", + "body": "安全加密agent，对本项目的phone字段进行安全加密", + "html_url": "https://scm.starbucks.com/china/starbucks-asg-api/issues/13", + "user": {"login": "issue-owner"}, + }, + "comment": { + "id": 1001, + "body": "@silicon_agent 只处理 phone 字段", + "html_url": "https://scm.starbucks.com/china/starbucks-asg-api/issues/13#issuecomment-1001", + "user": {"login": "commenter"}, + }, + "repository": {"name": "starbucks-asg-api", "full_name": "china/starbucks-asg-api"}, + "sender": {"login": "commenter"}, + } + body = json.dumps(payload).encode() + signature = _github_signature(body, secret) + + resp = await client.post( + f"/webhooks/github/{pid}", + content=body, + headers={ + "X-GitHub-Event": "issue_comment", + "X-Hub-Signature-256": signature, + "Content-Type": "application/json", + }, + ) + assert resp.status_code == 200 + task_id = resp.json()["task_id"] + assert task_id is not None + + task_resp = await client.get(f"/api/v1/tasks/{task_id}") + assert task_resp.status_code == 200 + task_data = task_resp.json() + assert task_data["github_issue_number"] == 13 + assert "Issue URL: https://scm.starbucks.com/china/starbucks-asg-api/issues/13" in (task_data["description"] or "") + assert "Body: 安全加密agent，对本项目的phone字段进行安全加密" in (task_data["description"] or "") + assert "Comment Body: @silicon_agent 只处理 phone 字段" in (task_data["description"] or "") + assert "Command Style: mention" in (task_data["description"] or "") + assert "Command Note: 只处理 phone 字段" in (task_data["description"] or "") + + second_resp = await client.post( + f"/webhooks/github/{pid}", + content=body, + headers={ + "X-GitHub-Event": "issue_comment", + "X-Hub-Signature-256": signature, + "Content-Type": "application/json", + }, + ) + assert second_resp.status_code == 200 + assert second_resp.json()["task_id"] is None + + async with async_session_factory() as session: + await session.execute( + delete(TriggerEventModel).where(TriggerEventModel.project_id == pid) + ) + await session.execute( + delete(TriggerRuleModel).where(TriggerRuleModel.project_id == pid) + ) + await session.execute( + delete(ProjectIntegrationModel).where( + ProjectIntegrationModel.project_id == pid + ) + ) + task = await session.get(TaskModel, task_id) + if task: + await session.delete(task) + proj = await session.get(ProjectModel, pid) + if proj: + await session.delete(proj) + await session.commit() + + # ── Jira project-level webhook ──────────────────────────────── diff --git a/platform/tests/test_worker.py b/platform/tests/test_worker.py index 6cc064a..1e28879 100644 --- a/platform/tests/test_worker.py +++ b/platform/tests/test_worker.py @@ -156,6 +156,18 @@ def test_build_stage_preflight_summary_for_test(self, tmp_path: Path): assert "推荐最小验证命令: ./mvnw test" in result assert "HelloControllerTest.java" in result + def test_build_stage_preflight_summary_for_des_encrypt(self, tmp_path: Path): + (tmp_path / "build.gradle").write_text("plugins {}", encoding="utf-8") + (tmp_path / "src/main/java/demo/controller").mkdir(parents=True) + (tmp_path / "src/main/java/demo/controller/HelloController.java").write_text("class X {}", encoding="utf-8") + + result = _build_stage_preflight_summary("des encrypt", str(tmp_path)) + + assert result is not None + assert "当前工作区" in result + assert "不要再次 git clone" in result + assert "推荐修改落点" in result + def test_build_stage_preflight_summary_prioritizes_controller_tests(self, tmp_path: Path): (tmp_path / "build.gradle").write_text("plugins {}", encoding="utf-8") (tmp_path / "src/test/java/demo/sdk").mkdir(parents=True) diff --git a/platform/tests/test_worker_tool_workspace.py b/platform/tests/test_worker_tool_workspace.py index aa21331..80d3a2c 100644 --- a/platform/tests/test_worker_tool_workspace.py +++ b/platform/tests/test_worker_tool_workspace.py @@ -1,10 +1,12 @@ from __future__ import annotations import json +import os from pathlib import Path import pytest +from app.worker import agents as agents_mod from app.worker.agents import SandboxedAgentRunner from app.worker.executor import infer_tool_status @@ -13,6 +15,7 @@ def _make_runner(workspace: Path) -> SandboxedAgentRunner: runner = object.__new__(SandboxedAgentRunner) runner.default_cwd = str(workspace) runner.allowed_tools = {"read", "write", "edit", "execute", "execute_script", "skill"} + runner.task_id = None return runner @@ -136,3 +139,43 @@ def test_infer_tool_status_treats_read_errors_as_failed(): assert infer_tool_status("Error: File not found: package.json") == "failed" assert infer_tool_status("Error (exit 1): command failed") == "failed" assert infer_tool_status("normal output") == "success" + + +def test_build_tool_runtime_env_includes_issue_feedback_context(monkeypatch): + monkeypatch.setattr(agents_mod.settings, "GHE_TOKEN", "secret-token") + monkeypatch.setattr(agents_mod.settings, "GHE_BASE_URL", "https://scm.starbucks.com/api/v3") + + env = agents_mod._build_tool_runtime_env("task-123") + + assert env["GHE_TOKEN"] == "secret-token" + assert env["GHE_BASE_URL"] == "https://scm.starbucks.com/api/v3" + assert env["SILICON_AGENT_TASK_URL"] == "http://127.0.0.1:3000/tasks/task-123" + + +@pytest.mark.asyncio +async def test_execute_script_receives_runtime_env(monkeypatch, tmp_path: Path): + runner = _make_runner(tmp_path) + runner.task_id = "task-xyz" + monkeypatch.setattr(agents_mod.settings, "GHE_TOKEN", "secret-token") + monkeypatch.setattr(agents_mod.settings, "GHE_BASE_URL", "https://scm.starbucks.com/api/v3") + + captured: dict[str, str | None] = {} + + async def _fake_super_execute(self, tool_call, on_output=None): + captured["ghe_token"] = os.environ.get("GHE_TOKEN") + captured["ghe_base_url"] = os.environ.get("GHE_BASE_URL") + captured["task_url"] = os.environ.get("SILICON_AGENT_TASK_URL") + return "ok" + + monkeypatch.setattr(agents_mod._BaseRunner, "_execute_tool", _fake_super_execute, raising=False) + + result = await runner._execute_tool_base( + {"name": "execute_script", "arguments": json.dumps({"script": "env"})} + ) + + assert result == "ok" + assert captured == { + "ghe_token": "secret-token", + "ghe_base_url": "https://scm.starbucks.com/api/v3", + "task_url": "http://127.0.0.1:3000/tasks/task-xyz", + } diff --git a/skills/des_encrypt/SKILL.md b/skills/des_encrypt/SKILL.md new file mode 100644 index 0000000..ce5f0c8 --- /dev/null +++ b/skills/des_encrypt/SKILL.md @@ -0,0 +1,212 @@ +# DES 安全加密接入 Skill + +为 Spring Boot + MyBatis 项目接入国安 DES (Data Encryption Service) 加密服务，对数据库中的敏感字段（手机号、邮箱等）进行 SM4/GCM 加密存储。 + +--- + +## 使用方式 + +``` +/des-encrypt +``` + +执行后 Claude 会引导你完成以下流程。你只需回答几个问题即可。 + +--- + +## 前置条件 + +- Spring Boot 项目（2.x 或 3.x），使用 MyBatis XML Mapper +- 已从 DES 团队获取：quickapi-client-java JAR 包、密钥 ID、加密服务 IP、证书文件 +- 已明确需要加密的表和字段 + +--- + +## 执行流程 + +### Step 1：收集信息 + +请用户提供以下信息（若未提供则逐一询问）： + +1. **需要加密的表和字段**，格式如： + - `表名: 字段1, 字段2` + - 例：`alipay_refund: phone`、`alipay_receipt: phone, email` +2. **密钥 ID**（DES 平台申请的 keyId，如 `o2oomsorder`、`asg-api`） +3. **加密 JAR 包路径**（quickapi-client-java-*.jar 的位置） +4. **加密服务 IP**（至少一个，最多两个双机） +5. **项目构建工具**（Gradle 或 Maven） + +### Step 2：实施改造 + +按以下顺序自动完成代码改造： + +#### 2.1 引入依赖 + +**Gradle：** +```groovy +implementation files('lib/quickapi-client-java-x.x.x-SNAPSHOT-shaded.jar') +``` + +**Maven：** +```xml + + org.quickssl + quickapi-client-java + x.x.x-SNAPSHOT + system + ${pom.basedir}/lib/quickapi-client-java-x.x.x-SNAPSHOT-shaded.jar + +``` + +#### 2.2 创建加密包 `{basePackage}.encryption` + +**EncryptionComponent.java** — 加密服务初始化： +```java +@Slf4j +@Component +@RefreshScope +public class EncryptionComponent { + + @Value("${encryption.server.ip1}") + private String encryptionServerIp1; + @Value("${encryption.server.ip2}") + private String encryptionServerIp2; + + @PostConstruct + public void init() { + log.info("--------加密组件初始化开始--------"); + try { + LoadingKeyCacheWithLocalFile kekDekLoader = new LoadingKeyCacheWithLocalFile() + .setLocalKekPath("/opt/sec-kek/{keyId}.kek") + .setLocalDekPath("/opt/sec-dek/{keyId}.dek"); + CryptoClient.Config config = CryptoClient.Config.newBuilder() + .setSocket(encryptionServerIp1, encryptionServerIp2) + .setAuthority("quickservice") + .setCaCertFile("/opt/sec-cert/cacert.pem") + .setKeyStoreFile("/opt/sec-cert/client.pfx") + .enableKeyCache(10, 100, 2592000) + .setDekCacheLoader(kekDekLoader) + .setKekCacheLoader(kekDekLoader) + .build(); + CryptoHelper.initConfig(config); + } catch (CryptoClient.CryptoException e) { + throw new RuntimeException(e); + } + log.info("--------加密组件初始化完成--------"); + } +} +``` + +**EncryptionUtils.java** — 加解密工具类（3 个静态方法）： +- `encodeData(String plaintext)` — SM4/GCM PB 格式加密，返回 Base64；失败返回原文 +- `deocdeData(String encodeData)` — 解密；先 isEncode 检查，非密文直接返回（兼容明文数据） +- `isEncode(String encodeData)` — 判断是否为 PB 格式密文 + +关键实现： +```java +// 加密：使用 PB 格式（优先），解密时无需传密钥/IV +byte[] pbEnBytes = CryptoHelper.symmEncryptEx(KEYID, SymAlg.QK_SGD_SM4_GCM, plaintext.getBytes()); +return new String(Base64.getEncoder().encode(pbEnBytes), StandardCharsets.UTF_8); + +// 解密：PB 格式可自动还原 +byte[] pbDeBytes = CryptoHelper.symmDecrypt("", Base64.getDecoder().decode(encodeData)); +return new String(pbDeBytes, StandardCharsets.UTF_8); +``` + +**EncryptionFieldHelper.java** — 写入归一化（受 `encryption.switch` 开关控制）： +- 开关 false：不填充加密字段，保持原有行为 +- 开关 true：将明文字段值复制到 `_encrypt` 字段（TypeHandler 负责实际加密） + +```java +@Component +@RefreshScope +public class EncryptionFieldHelper { + @Value("${encryption.switch:false}") + private boolean encryptionSwitch; + + public void normalizeForWrite(Entity entity) { + if (!encryptionSwitch) return; + if (StringUtils.isNotEmpty(entity.getPhone())) { + entity.setPhoneEncrypt(entity.getPhone()); // TypeHandler 负责加密 + } + } +} +``` + +#### 2.3 创建 TypeHandler + +**EncryptionTypeHandler.java** — 继承 `BaseTypeHandler`： +- 写入时自动加密（`EncryptionUtils.encodeData`） +- 读取时自动解密（`EncryptionUtils.deocdeData`） +- **禁止** `@MappedTypes(String.class)`，必须在 Mapper XML 中显式绑定到 `_encrypt` 列 + +#### 2.4 实体类新增字段 + +每个加密字段新增对应的 `{field}Encrypt` 属性。 + +#### 2.5 Mapper XML 改造 + +**ResultMap** — 新增 `_encrypt` 列映射，绑定 TypeHandler： +```xml + +``` + +**SELECT** — column list 追加 `_encrypt` 列，WHERE 条件不变： +```xml +select ..., phone, phone_encrypt, ... from table +``` + +**INSERT/UPDATE** — 追加 `_encrypt` 列，使用 TypeHandler： +```xml + + phone_encrypt = #{phoneEncrypt, typeHandler=...EncryptionTypeHandler}, + +``` + +#### 2.6 Service 层接入 + +在所有 insert/update 调用前，加入 `encryptionFieldHelper.normalizeXxxForWrite(entity)`。 + +#### 2.7 配置文件 + +各环境 bootstrap 配置添加： +```yaml +encryption: + switch: false # 灰度期间默认 false，Nacos 动态切换 + server: + ip1: {加密服务IP1} + ip2: {加密服务IP2} +``` + +### Step 3：生成 DDL + +自动生成 SQL 文件（存放在 `docs/des_encrypt_columns.sql`），格式： +```sql +ALTER TABLE `{table}` + ADD COLUMN `{field}_encrypt` VARCHAR(512) DEFAULT NULL COMMENT '{字段描述}密文' AFTER `{field}`; +``` + +### Step 4：输出待办清单 + +完成后输出后续待办： +1. 确认 KEYID、生产环境 IP、证书/密钥文件部署 +2. 各环境执行 DDL +3. 部署代码（switch=false），验证无回归 +4. 存量数据回刷（使用 DES 回刷工具） +5. Nacos 切换 `encryption.switch=true` + +--- + +## 设计原则 + +- **最小改动**：不引入全局拦截器，不修改现有表结构，只新增列 +- **保留明文列**：迁移期间原字段不动，确保可回滚 +- **单开关控制**：`encryption.switch` 一个开关管读写，Nacos 动态生效 +- **TypeHandler 显式绑定**：仅作用于 `_encrypt` 列，不影响其他 String 字段 +- **加密失败兜底**：`encodeData` 加密异常时返回原文，不阻断业务 + +## 参考实现 + +- OMS 项目：`oms-base-service/order_data_manager/src/main/java/com/freemud/encryption/` +- enc-test 项目：`/Users/jowang/Desktop/enc-test/` diff --git a/skills/start-project-services/scripts/restart_backend.sh b/skills/start-project-services/scripts/restart_backend.sh index 068021e..7458501 100755 --- a/skills/start-project-services/scripts/restart_backend.sh +++ b/skills/start-project-services/scripts/restart_backend.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash set -euo pipefail -ROOT="/Users/abnzhang/Project/silicon_agent" +ROOT="/Users/jowang/Documents/github/silicon_agent" pid8000=$(lsof -tiTCP:8000 -sTCP:LISTEN -n -P || true) if [ -n "${pid8000:-}" ]; then kill -9 $pid8000 || true; fi diff --git a/skills/start-project-services/scripts/start_services.sh b/skills/start-project-services/scripts/start_services.sh index 1824ea9..4036f3d 100755 --- a/skills/start-project-services/scripts/start_services.sh +++ b/skills/start-project-services/scripts/start_services.sh @@ -1,7 +1,13 @@ #!/usr/bin/env bash set -euo pipefail -ROOT="/Users/abnzhang/Project/silicon_agent" +ROOT="/Users/jowang/Documents/github/silicon_agent" +export PATH="/opt/homebrew/bin:$PATH" + +NPM_BIN="$(command -v npm || true)" +if [ -z "${NPM_BIN:-}" ] && [ -x "/opt/homebrew/bin/npm" ]; then + NPM_BIN="/opt/homebrew/bin/npm" +fi # cleanup old listeners pid8000=$(lsof -tiTCP:8000 -sTCP:LISTEN -n -P || true) @@ -15,9 +21,9 @@ nohup ./.venv/bin/python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 >/tm # start frontend cd "$ROOT/web" -nohup env NODE_OPTIONS=--dns-result-order=ipv4first npm run dev -- --host 0.0.0.0 --port 3000 >/tmp/silicon_web_3000.log 2>&1 & +nohup env NODE_OPTIONS=--dns-result-order=ipv4first "${NPM_BIN}" run dev -- --host 0.0.0.0 --port 3000 >/tmp/silicon_web_3000.log 2>&1 & -sleep 1 +sleep 3 echo "== ports ==" lsof -nP -iTCP:8000 -sTCP:LISTEN || true diff --git a/web/src/components/AgentCard.tsx b/web/src/components/AgentCard.tsx index 0c0467c..c0d3137 100644 --- a/web/src/components/AgentCard.tsx +++ b/web/src/components/AgentCard.tsx @@ -15,10 +15,31 @@ const STATUS_BADGE: Record = ({ agent }) => { + if (!agent) { + return ( + +
+ +
+ 未注册 Agent +
+ unknown +
+
+
+ stopped} /> +
+ + 未配置 + + + ); + } + const displayName = ROLE_DISPLAY_NAMES[agent.role] || agent.role; const badgeStatus = STATUS_BADGE[agent.status] || 'default'; diff --git a/web/src/hooks/useWebSocket.ts b/web/src/hooks/useWebSocket.ts index 87ac8ad..fd01260 100644 --- a/web/src/hooks/useWebSocket.ts +++ b/web/src/hooks/useWebSocket.ts @@ -68,6 +68,10 @@ export function useWebSocket() { switch (msg.type) { case 'agent_status': { const p = msg.payload as WSAgentStatusPayload; + if (typeof p?.role !== 'string' || !p.role.trim()) { + console.warn('[WS] Skip agent_status without role', p); + break; + } updateAgent(p.role, { status: p.status as 'running' | 'idle' | 'waiting' | 'error' | 'stopped', current_task_id: p.current_task_id, diff --git a/web/src/pages/CircuitBreaker/index.tsx b/web/src/pages/CircuitBreaker/index.tsx index a6f80c8..0a119c3 100644 --- a/web/src/pages/CircuitBreaker/index.tsx +++ b/web/src/pages/CircuitBreaker/index.tsx @@ -26,7 +26,7 @@ const CircuitBreakerPage: React.FC = () => { try { await Promise.all( Object.keys(agents) - .filter((role) => agents[role].status !== 'stopped') + .filter((role) => agents[role] && agents[role].status !== 'stopped') .map((role) => stopAgent(role)), ); message.success('所有 Agent 已停止'); @@ -43,7 +43,7 @@ const CircuitBreakerPage: React.FC = () => { try { await Promise.all( Object.keys(agents) - .filter((role) => agents[role].status === 'stopped') + .filter((role) => agents[role] && agents[role].status === 'stopped') .map((role) => startAgent(role)), ); message.success('所有 Agent 已启动'); diff --git a/web/src/stores/agentStore.ts b/web/src/stores/agentStore.ts index 935581d..05eee61 100644 --- a/web/src/stores/agentStore.ts +++ b/web/src/stores/agentStore.ts @@ -9,14 +9,27 @@ export interface AgentState { error_message: string | null; } +function createDefaultAgent(role: string): AgentState { + return { + role, + status: 'stopped', + model: '未配置', + current_task_id: null, + current_stage: null, + error_message: null, + }; +} + const DEFAULT_AGENTS: Record = { - orchestrator: { role: 'orchestrator', status: 'stopped', model: '未配置', current_task_id: null, current_stage: null, error_message: null }, - spec: { role: 'spec', status: 'stopped', model: '未配置', current_task_id: null, current_stage: null, error_message: null }, - coding: { role: 'coding', status: 'stopped', model: '未配置', current_task_id: null, current_stage: null, error_message: null }, - test: { role: 'test', status: 'stopped', model: '未配置', current_task_id: null, current_stage: null, error_message: null }, - review: { role: 'review', status: 'stopped', model: '未配置', current_task_id: null, current_stage: null, error_message: null }, - smoke: { role: 'smoke', status: 'stopped', model: '未配置', current_task_id: null, current_stage: null, error_message: null }, - doc: { role: 'doc', status: 'stopped', model: '未配置', current_task_id: null, current_stage: null, error_message: null }, + orchestrator: createDefaultAgent('orchestrator'), + spec: createDefaultAgent('spec'), + coding: createDefaultAgent('coding'), + test: createDefaultAgent('test'), + review: createDefaultAgent('review'), + smoke: createDefaultAgent('smoke'), + doc: createDefaultAgent('doc'), + 'dispatch issue': createDefaultAgent('dispatch issue'), + 'des encrypt': createDefaultAgent('des encrypt'), }; interface AgentStore { @@ -28,11 +41,19 @@ interface AgentStore { export const useAgentStore = create((set) => ({ agents: { ...DEFAULT_AGENTS }, updateAgent: (role, update) => - set((state) => ({ - agents: { - ...state.agents, - [role]: { ...state.agents[role], ...update }, - }, - })), + set((state) => { + const normalizedRole = role?.trim(); + if (!normalizedRole) { + return state; + } + + const current = state.agents[normalizedRole] ?? createDefaultAgent(normalizedRole); + return { + agents: { + ...state.agents, + [normalizedRole]: { ...current, ...update, role: normalizedRole }, + }, + }; + }), setAgents: (agents) => set({ agents }), })); diff --git a/web/src/utils/constants.ts b/web/src/utils/constants.ts index fb37e66..28b9afc 100644 --- a/web/src/utils/constants.ts +++ b/web/src/utils/constants.ts @@ -6,6 +6,8 @@ export const AGENT_ROLES = [ { key: 'review', name: '审计官', color: '#eb2f96' }, { key: 'smoke', name: '巡检官', color: '#13c2c2' }, { key: 'doc', name: '文档官', color: '#fa8c16' }, + { key: 'dispatch issue', name: 'GitHub Issue分发Agent', color: '#2f54eb' }, + { key: 'des encrypt', name: '安全加密Agent', color: '#389e0d' }, ] as const; export const ROLE_DISPLAY_NAMES: Record = { @@ -16,6 +18,8 @@ export const ROLE_DISPLAY_NAMES: Record = { review: '审计官', smoke: '巡检官', doc: '文档官', + 'dispatch issue': 'GitHub Issue分发Agent', + 'des encrypt': '安全加密Agent', }; export const STATUS_COLORS: Record = { diff --git a/web/tests/agentStore.test.ts b/web/tests/agentStore.test.ts new file mode 100644 index 0000000..1f3ff4d --- /dev/null +++ b/web/tests/agentStore.test.ts @@ -0,0 +1,115 @@ +import { beforeEach, describe, expect, it } from 'vitest'; +import { useAgentStore } from '@/stores/agentStore'; + +describe('agentStore', () => { + beforeEach(() => { + useAgentStore.setState({ + agents: { + orchestrator: { + role: 'orchestrator', + status: 'stopped', + model: '未配置', + current_task_id: null, + current_stage: null, + error_message: null, + }, + spec: { + role: 'spec', + status: 'stopped', + model: '未配置', + current_task_id: null, + current_stage: null, + error_message: null, + }, + coding: { + role: 'coding', + status: 'stopped', + model: '未配置', + current_task_id: null, + current_stage: null, + error_message: null, + }, + test: { + role: 'test', + status: 'stopped', + model: '未配置', + current_task_id: null, + current_stage: null, + error_message: null, + }, + review: { + role: 'review', + status: 'stopped', + model: '未配置', + current_task_id: null, + current_stage: null, + error_message: null, + }, + smoke: { + role: 'smoke', + status: 'stopped', + model: '未配置', + current_task_id: null, + current_stage: null, + error_message: null, + }, + doc: { + role: 'doc', + status: 'stopped', + model: '未配置', + current_task_id: null, + current_stage: null, + error_message: null, + }, + 'dispatch issue': { + role: 'dispatch issue', + status: 'stopped', + model: '未配置', + current_task_id: null, + current_stage: null, + error_message: null, + }, + 'des encrypt': { + role: 'des encrypt', + status: 'stopped', + model: '未配置', + current_task_id: null, + current_stage: null, + error_message: null, + }, + }, + }); + }); + + it('ships defaults for github issue roles', () => { + const agents = useAgentStore.getState().agents; + + expect(agents['dispatch issue']).toMatchObject({ + role: 'dispatch issue', + status: 'stopped', + }); + expect(agents['des encrypt']).toMatchObject({ + role: 'des encrypt', + status: 'stopped', + }); + }); + + it('ignores empty role updates', () => { + const before = useAgentStore.getState().agents; + + useAgentStore.getState().updateAgent('', { status: 'running' }); + useAgentStore.getState().updateAgent(' ', { status: 'running' }); + + expect(useAgentStore.getState().agents).toEqual(before); + }); + + it('creates a safe placeholder for unexpected roles', () => { + useAgentStore.getState().updateAgent('legacy agent', { status: 'idle' }); + + expect(useAgentStore.getState().agents['legacy agent']).toMatchObject({ + role: 'legacy agent', + status: 'idle', + model: '未配置', + }); + }); +});