fix: retry invalid AI summary generation

[Chenpeel]

针对返回思考内容： #174

Summary

修复 AI 仓库分析中 summary 不符合要求仍被保存的问题。
当模型把提示词要求复述到 summary 中时，不再保存占位内容，而是重新生成符合要求的结果。

Changes

• 增加 summary 校验，拦截“我们被要求”“只输出JSON”“根据仓库信息”“summary/tags/platforms”等提示词复述内容。
• 当 AI 返回合法 JSON 但 summary 不合规时，自动重试生成。
• 多次重试仍不合规时，将分析判为失败。
• 更新设置页默认提示词，明确要求 summary 只能描述仓库功能和用途。

Summary by CodeRabbit

• Bug Fixes

  • Improved repository analysis reliability with retry-based validation and clearer error handling.
  • Strengthened summary sanitization to remove invalid or prompt-like outputs and reject empty/meaningless results.
  • Better parsing and normalization of tags and platforms, with deduplication and count limits.

• Other

  • Tightened prompt instructions to prevent AI from restating meta/prompt phrases in responses.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9615a63a-783e-4a29-9905-122d138ca262

📥 Commits

Reviewing files that changed from the base of the PR and between 2700847 and 5b6d9a8.

📒 Files selected for processing (2)

• src/components/settings/AIConfigPanel.tsx
• src/services/aiService.ts

🚧 Files skipped from review as they are similar to previous changes (2)

• src/components/settings/AIConfigPanel.tsx
• src/services/aiService.ts

---

📝 Walkthrough

Walkthrough

AIService repository analysis now validates AI-generated summaries across up to three retry attempts, using a new sanitization pipeline to reject meta-text and instruction-like content while stricter prompts forbid restatement phrases in both Chinese and English.

Changes

AI Repository Analysis Validation

Layer / File(s)	Summary
Prompt constraint tightening src/components/settings/AIConfigPanel.tsx, src/services/aiService.ts	Added constraint lines forbidding AI restatement of prompt/instruction phrases (e.g., "asked to", "only output JSON", "based on repository information", field names) in both Chinese and English default and analysis prompts.
Result types and retry constant src/services/aiService.ts	Introduced RepositoryAnalysisResult and ParsedAIResponse types, added ANALYSIS_MAX_ATTEMPTS = 3, and updated analyzeRepository return type to the centralized result shape.
Summary sanitization and response parsing src/services/aiService.ts	Added sanitizeSummary to trim/strip quotes and reject empty/too-short/mostly-punctuation or prompt-restatement summaries; reworked parseAIResponse to return isValid/invalidReason, filter/tag-cap, and normalize/deduplicate platforms.
Retry-based analysis orchestration src/services/aiService.ts	Refactored analyzeRepository to loop up to ANALYSIS_MAX_ATTEMPTS, parse and validate each response, retry with a seeded retry prompt when invalid, and throw a language-specific error after exhaustion.

Sequence Diagram

sequenceDiagram
  participant Caller
  participant analyzeRepository
  participant requestText
  participant parseAIResponse
  
  Caller->>analyzeRepository: analyzeRepository()
  
  loop up to ANALYSIS_MAX_ATTEMPTS
    analyzeRepository->>requestText: request AI output (with retry prompt if needed)
    requestText-->>analyzeRepository: raw AI response
    analyzeRepository->>parseAIResponse: parse and validate
    parseAIResponse-->>analyzeRepository: { isValid, summary, tags, platforms, invalidReason }
    
    alt isValid
      analyzeRepository-->>Caller: return { summary, tags, platforms }
    else attempt < MAX
      Note over analyzeRepository: record invalidReason, retry
    else final attempt failed
      analyzeRepository-->>Caller: throw language-specific error
    end
  end

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• AmintaCCCP/GithubStarsManager#175: Shares prompt tightening in AIConfigPanel and overlapping platforms normalization/validation in aiService.
• AmintaCCCP/GithubStarsManager#159: Related work on sanitizing repository/README content before embedding into analysis prompts.
• AmintaCCCP/GithubStarsManager#69: Overlaps in refactoring parseAIResponse and validation of {summary, tags, platforms}.

Poem

🐰🌿 Three attempts to make the summary clean,
Ban the echoed prompts, strip the quote between,
Trim tags, fold platforms, dedupe what’s in view,
I nibble at the noise and hop to something true. 🥕

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: retry invalid AI summary generation' directly describes the main change across the PR - adding validation, retry logic, and sanitization for AI-generated summaries.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/services/aiService.ts`:
- Around line 701-770: sanitizeSummary currently strips known prompt-restatement
prefixes (metaPrefixes) and returns the remainder, which lets parseAIResponse
accept summaries like "We were asked... A CLI backup tool." — instead, treat any
text that had a meta-prefix as invalid so parseAIResponse/analyzeRepository will
retry/fail. Update sanitizeSummary (and its metaPrefixes logic and use of
isMetaText) to detect when cleaning removed a meta-prefix (e.g., compare the
original trimmed raw to the final cleaned or check if any metaPrefixes matched)
and return null in that case rather than returning the salvaged remainder; keep
other validation (length, punctuation-only, hardMetaPatterns) intact.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ec054eaa-75f1-4a29-8af4-c72dca99ef6d

📥 Commits

Reviewing files that changed from the base of the PR and between df55321 and 2700847.

📒 Files selected for processing (2)

• src/components/settings/AIConfigPanel.tsx
• src/services/aiService.ts