feat(catalog): persist & expose upstream health and evaluation

[papysans]

改动内容

把上游 catalog index.json 的 health(freshness / popularity / source_trust 信号)和 evaluation(六个评分维度 + 总分 + 评测模型)两个块,经 ingest 落库到 capability_items,并在 item 详情 API 暴露。此前这两块在 ingest 阶段被直接丢弃,只有 final_score 以 experience_score 存活。

具体变更

• models:CapabilityItem 新增 health / evaluation 两个 jsonb 列(GORM 自动迁移)
• ingest:以 raw JSON 原样解析(无损,保留上游额外字段,如 health.signals.install_popularity);在 create / update / 仅元数据 三条写路径写入;computeMetadataDelta 比对 health/eval 漂移,保证回填重跑不被跳过;非 object 负载归一化为 {}
• handlers:ItemResponse 新增 health / evaluation(omitempty)
• migrate:抽出 prepareSchema(统一有序入口——pre-migrations 先于 AutoMigrate);ingest-upstream 通过幂等增量 DDL 确保两列存在,--dry-run 下跳过
• tests:ingest 落库 / 回填 / object 契约 + 详情 API 输出

说明

• experience_score 语义不变;所有新字段均为增量、非破坏。
• 前端配套(详情页面板):feat(store): show catalog health & evaluation on item detail zgsm-sangfor/opencode#517
• 部署:常规 migrate 会自动加列;重跑 ingest-upstream 回填存量数据。

Summary by CodeRabbit

Release Notes

• New Features

  • Items now expose health and evaluation data in API responses when available.
  • Health and evaluation information is automatically ingested and stored during catalog updates.
  • Missing health and evaluation data are represented as empty objects in responses.

• Tests

  • Added comprehensive test coverage for health and evaluation data persistence and ingestion workflows.

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This PR adds health and evaluation metadata support to capability items. The data model is extended with two JSONB columns; the catalog ingest pipeline reads and normalizes these blocks from upstream bundles; handlers expose them in item responses; and migrations ensure the schema is prepared with dry-run safety.

Changes

Health and Evaluation Metadata Support

Layer / File(s)	Summary
Data model contract server/internal/models/models.go	CapabilityItem adds Health and Evaluation fields as JSONB, both defaulting to {}.
Migration and schema preparation server/cmd/migrate/main.go	Refactor migration execution to centralize schema prep into prepareSchema (pre-migrations, AutoMigrate, Goose, user identity ensures); add idempotent column-ensure DDL for health/evaluation before ingest, respecting --dry-run mode.
Catalog ingest health and evaluation persistence server/internal/services/catalog_ingest_service.go	Read upstream health/evaluation JSON blocks from bundle index.json; normalize via rawBlockJSON contract (valid objects pass through, invalid/non-object inputs become {}); extend metadata drift detection with semantic JSONB equality; write both on content changes and metadata-only updates; populate Health/Evaluation on insert and update.
Item detail handler and response server/internal/handlers/capability_item.go	ItemResponse struct adds optional Health and Evaluation JSON fields; buildItemResponse populates them from the underlying CapabilityItem.
Handler integration tests server/internal/handlers/capability_item_health_eval_test.go, server/internal/handlers/registry_test.go	Test that item detail endpoint returns health/evaluation objects when populated and omits keys when null/empty (consistent with omitempty tagging); update handler test DB schema.
Catalog ingest test suite server/internal/services/catalog_ingest_health_eval_test.go	Full coverage: create-then-update persistence with upstream field passthrough; missing health/eval results in {} without error; metadata-only backfill regression (existing {} is updated on re-ingest without bumping revision); rawBlockJSON unit contract (normalizes invalid/non-object to {}, preserves objects); end-to-end non-object normalization.
Scan service test schemas server/internal/services/scan_service_test.go	In-memory SQLite test databases add health and evaluation columns with TEXT DEFAULT '{}'.
Generated API documentation server/docs/docs.go, server/docs/swagger.json, server/docs/swagger.yaml	Swagger definitions for CapabilityItem, RecommendedItem, SearchResultItem, ItemResponse, and MyItem extended with evaluation and health object properties.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• XDfield/costrict-web#96: Introduced the upstream catalog ingest pipeline and CatalogIngestService; this PR extends that same ingest flow to persist and expose health/evaluation fields.
• XDfield/costrict-web#28: Both refactor migration execution orchestration in server/cmd/migrate/main.go toward Goose-based staged workflows (pre-migrations, AutoMigrate, Goose runs).
• XDfield/costrict-web#98: Both extend ItemResponse and buildItemResponse to populate additional JSON fields from CapabilityItem into handler responses.

Poem

🐰 Health and eval hop into sight,
JSON blocks dance through ingest's flight,
Schema prepared, migrations run bright,
Handler returns them—oh, what a delight!
Empty objects prevent nil's fright.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 68.75% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title "feat(catalog): persist & expose upstream health and evaluation" directly and clearly describes the main change—adding persistence and exposure of health and evaluation data from upstream catalog sources into the capability items table and API.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/catalog-health-eval-panels

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain modules listed in go.work or their selected dependencies"

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}